Migrating from the Google Cloud Storage API

If your code currently uses the Google Cloud Storage API , the migration to the GCS client library is straightforward, as the client library supports very similar functionality. The one exception is bulk deletes, which are only available in the deprecated Google Cloud Storage API. The client library does not support bulk deletes. Instead, you will need to specify each file to be deleted in a separate call.

Also, the old Google Cloud Storage API wrote file data to App Engine during the process of sending it to GCS, so it was harder to discover or handle errors. With The new client library does all the writing from client-side, so the client has full knowledge of write errors, and can handle them.

The following table compares the functionality provided by the API and the new client library:

Migration Example

In the following tabs you can see a sample that uses the deprecated API ( com.google.appengine.api.files ) and a direct port of the sample in code that instead uses the GCS client library.

Tips for running the code:

• If you are using Maven, add the appengine-gcs-client artifact to the application's pom.xml file. All other dependencies (e.g., for the Google HTTP CLient lbirary) will be included automatically. For more information, see Downloading the GCS Client Library .

• If you are using Eclipse:

  • You will need to include the libraries manually. See Downloading the GCS Client Library for the list of Jars to include. For example, if you just add the appengine-gcs-client library, you will have unresolved references in the code.
  • Add Jars to the war/WEB-INF/lib folder of your Web Application project as well as to the classpath build path of the application project.
  • If you receive exceptions in the deployed application, e.g., java.lang.NoClassDefFoundError: com/google/api/client/http/HttpRequestInitializer , check that you have included all the libraries.

• Specify values for the BUCKETNAME and FILENAME variables in the code.

• Set the bucket permissions so that the sample code can write to the bucket. We recommend you do this by adding the application service account name ( <app-id>@appspot.gserviceaccount.com ) as a user to the bucket, with write permissions. If you get Access Denied message in your deployed code, check the bucket permissions.

package com.google.appengine.demos;

import com.google.appengine.tools.cloudstorage.GcsFileOptions;
import com.google.appengine.tools.cloudstorage.GcsFilename;
import com.google.appengine.tools.cloudstorage.GcsInputChannel;
import com.google.appengine.tools.cloudstorage.GcsOutputChannel;
import com.google.appengine.tools.cloudstorage.GcsService;
import com.google.appengine.tools.cloudstorage.GcsServiceFactory;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.PrintWriter;
import java.nio.ByteBuffer;
import java.nio.channels.Channels;

import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/**
 * Create, Write, Read, and Finalize Cloud Storage objects.
 * Access your app at: http://myapp.appspot.com/
 */
@SuppressWarnings("serial")
public class PortOfFilesAPIGuestbookServlet extends HttpServlet {
   public static final String BUCKETNAME = "ExampleBucketName";
   public static final String FILENAME = "ExampleFileName";

  @Override
  public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
    resp.setContentType("text/plain");
    resp.getWriter().println("Hello, world from java");
    GcsService gcsService = GcsServiceFactory.createGcsService();
    GcsFilename filename = new GcsFilename(BUCKETNAME, FILENAME);
    GcsFileOptions options = new GcsFileOptions.Builder()
        .mimeType("text/html")
        .acl("public-read")
        .addUserMetadata("myfield1", "my field value")
        .build();

    GcsOutputChannel writeChannel = gcsService.createOrReplace(filename, options);
    // You can write to the channel using the standard Java methods.
    // Here we use a PrintWriter:
    PrintWriter writer = new PrintWriter(Channels.newWriter(writeChannel, "UTF8"));
    writer.println("The woods are lovely dark and deep.");
    writer.println("But I have promises to keep.");
    writer.flush();

    // Note that the writeChannel is Serializable, so it is possible to store it somewhere and write
    // more to the file in a separate request. To make the object as small as possible call:
    writeChannel.waitForOutstandingWrites();

    // This time we write to the channel directly
    writeChannel.write(ByteBuffer.wrap("And miles to go before I sleep.".getBytes("UTF8")));

    // If you want partial content saved in case of an exception, close the
    // GcsOutputChannel in a finally block. See the GcsOutputChannel interface
    // javadoc for more information.
    writeChannel.close();
    resp.getWriter().println("Done writing...");

    // At this point, the file is visible to anybody on the Internet through Cloud Storage as:
    // (http://storage.googleapis.com/BUCKETNAME/FILENAME)

    GcsInputChannel readChannel = null;
    BufferedReader reader = null;
    try {
      // We can now read the file through the API:
      readChannel = gcsService.openReadChannel(filename, 0);
      // Again, different standard Java ways of reading from the channel.
      reader = new BufferedReader(Channels.newReader(readChannel, "UTF8"));
      String line;
      // Prints "The woods are lovely, dark, and deep."
      // "But I have promises to keep."
      // "And miles to go before I sleep."
      while ((line = reader.readLine()) != null) {
        resp.getWriter().println("READ:" + line);
      }
    } finally {
      if (reader != null) { reader.close(); }
    }
  }
}

import com.google.appengine.api.files.AppEngineFile;
import com.google.appengine.api.files.FileReadChannel;
import com.google.appengine.api.files.FileService;
import com.google.appengine.api.files.FileServiceFactory;
import com.google.appengine.api.files.FileWriteChannel;
import com.google.appengine.api.files.GSFileOptions.GSFileOptionsBuilder;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.PrintWriter;
import java.nio.ByteBuffer;
import java.nio.channels.Channels;

import javax.servlet.http.*;
/**
 * Copyright 2011 Google Inc. All Rights Reserved.
 * Create, Write, Read, and Finalize Cloud Storage objects.
 * Access your app at: http://myapp.appspot.com/
**/

@SuppressWarnings("serial")
public class DeprecatedAPIGuestbookServlet extends HttpServlet {
   public static final String BUCKETNAME = "YOUR_BUCKET_NAME";
   public static final String FILENAME = "YOUR_FILE_NAME";

  @Override
  public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
    resp.setContentType("text/plain");
    resp.getWriter().println("Hello, world from java");
    FileService fileService = FileServiceFactory.getFileService();
    GSFileOptionsBuilder optionsBuilder = new GSFileOptionsBuilder()
       .setBucket(BUCKETNAME)
       .setKey(FILENAME)
       .setMimeType("text/html")
       .setAcl("public_read")
       .addUserMetadata("myfield1", "my field value");
    AppEngineFile writableFile =
         fileService.createNewGSFile(optionsBuilder.build());
    // Open a channel to write to it
     boolean lock = false;
     FileWriteChannel writeChannel =
         fileService.openWriteChannel(writableFile, lock);
     // Different standard Java ways of writing to the channel
     // are possible. Here we use a PrintWriter:
     PrintWriter out = new PrintWriter(Channels.newWriter(writeChannel, "UTF8"));
     out.println("The woods are lovely dark and deep.");
     out.println("But I have promises to keep.");
     // Close without finalizing and save the file path for writing later
     out.close();
     String path = writableFile.getFullPath();
     // Write more to the file in a separate request:
     writableFile = new AppEngineFile(path);
     // Lock the file because we intend to finalize it and
     // no one else should be able to edit it
     lock = true;
     writeChannel = fileService.openWriteChannel(writableFile, lock);
     // This time we write to the channel directly
     writeChannel.write(ByteBuffer.wrap
               ("And miles to go before I sleep.".getBytes()));

     // Now finalize
     writeChannel.closeFinally();
     resp.getWriter().println("Done writing...");

     // At this point, the file is visible in App Engine as:
     // "/gs/BUCKETNAME/FILENAME"
     // and to anybody on the Internet through Cloud Storage as:
     // (http://storage.googleapis.com/BUCKETNAME/FILENAME)
     // We can now read the file through the API:
     String filename = "/gs/" + BUCKETNAME + "/" + FILENAME;
     AppEngineFile readableFile = new AppEngineFile(filename);
     FileReadChannel readChannel =
         fileService.openReadChannel(readableFile, false);
     // Again, different standard Java ways of reading from the channel.
     BufferedReader reader =
             new BufferedReader(Channels.newReader(readChannel, "UTF8"));
     String line = reader.readLine();
     resp.getWriter().println("READ:" + line);

    // line = "The woods are lovely, dark, and deep."
     readChannel.close();
  }
}

Using Blobstore with GCS Client Library

Developers have been using the deprecated Cloud Storage API to upload files to blobstore, and then serving them as part of their request. The Blobstore API also supports serving files in GCS.

Both of these use cases are supported by the GCS client library, so we recommend that developers switch to the GCS client library instead.

Writing a File over Multiple Requests

The deprecated Cloud Storage API could be used to start writing a file in one request and finish it in another.

This is also supported by the GCS client libary, although less directly. The GcsService object returns a GcsInputChannel and a GcsOutputChannel when you start reading and writing a file respectively. These are serializable. So if you need to upload a file incrementally across many requests it is possible to start writing the file, and then serialize the GcsOutputChannel , write it to datastore, and then in the following request read it back from datastore and continue to use it. If you do this, we recommend calling waitForOutstandingWrites on the GcsOutputChannel immediately prior to serialization as this will reduce the object size.

It is considered invalid to use this serialization to make multiple copies of the object and attempt to write in parallel. This results in undefined behavior.

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 3.0 License , and code samples are licensed under the Apache 2.0 License . For details, see our Site Policies .