Please note that the contents of this offline web site may be out of date. To access the most recent documentation visit the online version .
Note that links that point to online resources are green in color and will open in a new window.
We would love it if you could give us feedback about this material by filling this form (You have to be online to fill it)



Object Versioning


Google Cloud Storage buckets that have Object Versioning turned on maintain an archive of objects, providing a way to un-delete data that you accidentally deleted, or to retrieve older versions of your data. This page describes how Object Versioning works and provides examples for working with it.

Contents

How Object Versioning Works

Google Cloud Storage allows you to enable Object Versioning at the bucket level . Once enabled, a history of modifications (overwrite / delete) of objects is kept for all objects in the bucket. You can list archived versions of an object, restore an object to an older state, or permanently delete a version, as needed. All objects have generation properties associated with them to help you identify the objects and perform safe read-modify-write updates on them as well as perform conditional operations on them. Generation properties are numeric and monotonically increasing, which lets you quickly evaluate the version order. When an object is overwritten or deleted in a bucket which has versioning enabled, a copy of the object is automatically saved with generation properties that identify it. You can turn versioning on or off for a bucket at any time. Turning versioning off leaves existing object versions in place, and simply causes the bucket to stop accumulating new object versions. In this case, if you upload to an existing object, the current version is overwritten instead of creating a new version.

Object Versioning Details

Google Cloud Storage uses two generation properties, that together identify the version of an object. These properties are always present with every version of the object, even if versioning is not enabled. They can be used for conditional updates to enforce ordering of updates. See the Examples section below for details.

Google Cloud Storage marks every object using the following properties:

generation
Identifies the content (data) generation, and is updated when the content of an object is overwritten.
metageneration
Identifies the metadata generation, and is updated every time the metadata (e.g. ACL) for a given content generation is updated. metageneration also gets reset to 1 whenever generation changes. The metageneration has no meaning without the content generation and should only be used in conjunction with it. In other words, it is meaningless to compare metadata generations of two versions that have different data generations.

For more details about working with Object Versioning, see the Tips section below.

Object Versioning Examples

Let's take a look at what happens when you create objects, overwrite object data, or update objects on a versioning enabled bucket. In the following examples, we show you how to work with versioning using the gsutil tool, the Java client library , and the XML API . You can read and write data to a bucket that has versioning enabled with any tool that supports Google Cloud Storage.

Enabling Versioning

The following code sample shows you how to enable versioning on a bucket.

gsutil

Uses the gsutil tool. 

gsutil versioning set on gs://bucket

Enabling versioning for gs://bucket/...

To disable versioning, substitute off for on in the command above.

Java

Uses the Java client library . 

Bucket content = new Bucket().setVersioning(new Versioning().setEnabled(true));
Bucket versionedBucket = storage.buckets().patch("bucket", content).execute();
System.out.println("versioning enabled: " + versionedBucket.getVersioning().get("enabled"));

versioning enabled: true

To disable versioning, substitute false for true in the setEnabled method.

XML API

Uses the XML API . 

PUT /bucket/?versioning HTTP/1.1
Host: storage.googleapis.com
Accept-Encoding: identity
Date: Fri, 15 Feb 2013 00:06:00 GMT
Content-Length: 114

<?xml version="1.0" encoding="UTF-8"?>
<VersioningConfiguration><Status>Enabled</Status></VersioningConfiguration>

HTTP/1.1 200 OK

To disable versioning, use a empty VersioningConfiguration element, <VersioningConfiguration/> .

The following code sample shows you how to verify the current status of versioning on a bucket.

gsutil

Uses the gsutil tool. 

gsutil versioning get gs://bucket

gs://bucket: Enabled

Java

Uses the Java client library . 

Bucket bucket = storage.buckets().get("bucket").execute();
System.out.println("versioning enabled: " + bucket.getVersioning().get("enabled"));

versioning enabled: true

XML API

Uses the XML API . 

GET /bucket/?versioning HTTP/1.1
Host: storage.googleapis.com
Date: Fri, 15 Feb 2013 00:07:43 GMT

HTTP/1.1 200 OK
Server: HTTP Upload Server Built on Feb 6 2013 15:53:54 (1360194834)
Content-Length: 113
Date: Fri, 15 Feb 2013 00:07:43 GMT
Expires: Fri, 15 Feb 2013 00:07:43 GMT

<?xml version='1.0' encoding='UTF-8'?><VersioningConfiguration><Status>Enabled</Status></VersioningConfiguration>

Creating a New Object

When you create a new object, generation and metageneration properties are created for the object regardless of whether versioning on a bucket is enabled or not.

The following code sample shows creating an object for which the generation for the object is 1360887697105000 and metageneration is 1.

gsutil

Uses the gsutil tool. 

echo 'this is foo' > foo
gsutil cp -v foo gs://bucket

Copying file://foo [Content-Type=application/octet-stream]...
Created: gs://bucket/foo#1360887697105000

Java

Uses the Java client library . 

InputStream inputStream = new ByteArrayInputStream("this is foo".getBytes());
InputStreamContent mediaContent = new InputStreamContent("text/plain", inputStream);
StorageObject fooObject = storage.objects().insert("bucket", null, mediaContent).setName("foo").execute();
System.out.println("generation: " + fooObject.getGeneration());

generation: 1360887697105000

XML API

Uses the XML API . 

PUT /bucket/foo HTTP/1.1
Host: storage.googleapis.com
Content-Length: 12
x-goog-acl: public-read
x-goog-api-version: 2

this is foo

HTTP/1.1 200 OK
ETag: "4be8bde80854190cbe801133c6682ecf"
x-goog-generation: 1360887697105000
x-goog-metageneration: 1

Overwriting Object Data

If you overwrite an object in a versioned bucket, that is, you upload a new version of the object, Google Cloud Storage archives the current object, creates the new object and increases the generation, as shown in the following figure.

Object Version Control for Data Update
Object Data Update

When you overwrite the data of an object by uploading over the existing object, the x-goog-generation is updated and x-goog-metageneration is restored back to 1. The following example shows how to overwrite an existing object.

gsutil

Uses the gsutil tool. 

echo 'this is foo but better' > foo
gsutil cp -v foo gs://bucket

Copying file://foo [Content-Type=application/octet-stream]...
Created: gs://bucket/foo#1360887759327000

Java

Uses the Java client library . 

InputStream inputStream = new ByteArrayInputStream("this is foo is better".getBytes());
InputStreamContent mediaContent = new InputStreamContent("text/plain", inputStream);
StorageObject fooObj = storage.objects().insert("bucket", null, mediaContent).setName("foo").execute();
System.out.println("generation: " + fooObj.getGeneration());

generation: 1360887759327000

XML API

Uses the XML API . 

PUT /bucket/foo HTTP/1.1
Host: storage.googleapis.com
Content-Length: 23
x-goog-acl: public-read
x-goog-api-version: 2

this is foo but better

HTTP/1.1 200 OK
ETag: "b859c7443ae764f0cbeba4647e3831d4"
x-goog-generation: 1360887759327000
x-goog-metageneration: 1

If you set the x-goog-if-generation-match header to 0 when uploading an object, Google Cloud Storage only performs the specified request if the object does not currently exist. For example, you can perform a PUT request to create a new object with x-goog-if-generation-match set to 0, and the object will be created only if it doesn't already exist. Otherwise, Google Cloud Storage aborts the update with a status code of 412 Precondition Failed .

The following examples shows how to use the x-goog-if-generation-match header. Note that even though the content of the file to upload is different, the update does not occur.

gsutil

Uses the gsutil tool. 

echo 'another bar' > foo
gsutil -h 'x-goog-if-generation-match:0' cp -v foo gs://bucket

Copying file://foo [Content-Type=application/octet-stream]...
GSResponseError: status=412, code=PreconditionFailed, reason=Precondition Failed.

Java

Uses the Java client library . 

InputStream inputStream = new ByteArrayInputStream("another bar".getBytes());
InputStreamContent mediaContent = new InputStreamContent("text/plain", inputStream);
StorageObject fooObj = storage.objects().insert("bucket", null, mediaContent)
    .setIfGenerationMatch(new BigInteger("0"))
    .setName("foo").execute();

412 Precondition Failed

XML API

Uses the XML API . 

PUT /bucket/foo HTTP/1.1
x-goog-if-generation-match:0
Content-Length: 11

another bar

HTTP/1.1 412 Precondition Failed
<?xml version='1.0' encoding='UTF-8'?>
<Error>
    <Code>PreconditionFailed</Code>
    <Message>At least one of the pre-conditions you specified did not hold.</Message>
</Error>

However, the request will succeed if archived generations of the object exist but there is no current live object. In other words, setting the x-goog-if-generation-match header to 0 will only succeed if attempting a non-versioned read of the object would result in a 404 Not Found (not found) error.

Listing Archived Object Versions

At this point, there are now two generations of the same object: the latest generation of the object, which is the current one with generation 1360887759327000, and one archived object with generation 1360887697105000. If you do not specify that object version should be returned when listing objects, you only get the current object as shown in the following example.

gsutil

Uses the gsutil tool. 

gsutil ls -l gs://bucket

23  2013-02-15T00:22:39.324Z  gs://bucket/foo

Java

Uses the Java client library . 

Storage.Objects.List listObjects = storage.objects().list(settings.getBucket());
Objects objects;
do {
  objects = listObjects.execute();
  for (StorageObject obj : objects.getItems()) {
    System.out.format("Size: %s, Creation Date: %s, Name: %s\n",
        obj.getSize(), obj.getUpdated(), obj.getName());
  }
  listObjects.setPageToken(objects.getNextPageToken());
} while (null != objects.getNextPageToken());

Size: 23, Creation Date: 2013-02-15T00:22:39.324Z, Name: foo

XML API

Uses the XML API . The output is formatted for reading clarity. 

GET /bucket HTTP/1.1

HTTP/1.1 200 OK

<?xml version='1.0' encoding='UTF-8'?>
  <ListBucketResult xmlns='http://doc.s3.amazonaws.com/2006-03-01'>
    <Name>bucket</Name>
    <Prefix></Prefix>
    <Marker></Marker>
    <IsTruncated>false</IsTruncated>
    <Contents>
      <Key>foo</Key>
      <Generation>1360887759327000</Generation>
      <MetaGeneration>1</MetaGeneration>
      <LastModified>2013-02-15T00:22:39.324Z</LastModified>
      <ETag>"b859c7443ae764f0cbeba4647e3831d4"</ETag>
      <Size>23</Size>
      <Owner>
        <ID>00b4903a97a3341225a8da631155093237119370d36b57e9d73bd3475050babb</ID>
        <DisplayName>Example User</DisplayName>
      </Owner>
    </Contents>
  </ListBucketResult>

However, if you specify that versions should be returned when listing objects, then all versions of objects are returned as shown in the following example. Note that how you indicate this depends on how you access Google Cloud Storage. In the XML API example below, you specify the versions query parameter. In the gsutil example, you specify the -a option.

gsutil

Uses the gsutil tool. 

gsutil ls -la gs://bucket

 6  2013-02-15T00:21:37.102Z  gs://bucket/foo#1360887697105000  metageneration=1
23  2013-02-15T00:22:39.324Z  gs://bucket/foo#1360887759327000  metageneration=1

Java

Uses the Java client library . 

Storage.Objects.List listObjects = storage.objects().list(settings.getBucket()).setVersions(true);
Objects objects;
do {
  objects = listObjects.execute();
  for (StorageObject obj : objects.getItems()) {
    System.out.format("Size: %s, Creation Date: %s, Name: %s\n",
        obj.getSize(), obj.getUpdated(), obj.getName());
  }
  listObjects.setPageToken(objects.getNextPageToken());
} while (null != objects.getNextPageToken());

Size: 6, Creation Date: 2013-02-15T00:21:37.102Z, Name: foo
Size: 23, Creation Date: 2013-02-15T00:22:39.324Z, Name: foo

XML API

Uses the XML API . 

GET /bucket?versions HTTP/1.1

HTTP/1.1 200 OK

<?xml version='1.0' encoding='UTF-8'?>
  <ListBucketResult xmlns='http://doc.s3.amazonaws.com/2006-03-01'>
    <Name>bucket</Name>
    <Prefix></Prefix>
    <Marker></Marker>
    <GenerationMarker></GenerationMarker>
    <IsTruncated>false</IsTruncated>
    <Version>
      <Key>foo</Key>
      <Generation>1360887697105000</Generation>
      <MetaGeneration>1</MetaGeneration>
      <IsLatest>false</IsLatest>
      <LastModified>2013-02-15T00:21:37.102Z</LastModified>
      <DeletedTime>2013-02-15T00:22:39.327Z</DeletedTime>
      <ETag>"4be8bde80854190cbe801133c6682ecf"</ETag>
      <Size>12</Size>
      <Owner>
        <ID>00b4903a97a3341225a8da631155093237119370d36b57e9d73bd3475050babb</ID>
        <DisplayName>Example User</DisplayName>
      </Owner>
    </Version>
    <Version>
      <Key>foo</Key>
      <Generation>1360887759327000</Generation>
      <MetaGeneration>1</MetaGeneration>
      <IsLatest>true</IsLatest>
      <LastModified>2013-02-15T00:22:39.324Z</LastModified>
      <DeletedTime></DeletedTime>
      <ETag>"b859c7443ae764f0cbeba4647e3831d4"</ETag>
      <Size>23</Size>
      <Owner>
        <ID>00b4903a97a3341225a8da631155093237119370d36b57e9d73bd3475050babb</ID>
        <DisplayName>Example User</DisplayName>
    </Owner>
    </Version>
  </ListBucketResult>
There are a few differences in the results of the GET request when using the versions query parameter compared to not using it. Specifically, Google Cloud Storage returns the following when you provide a version query parameter in your request:
  • A Version element which contains information about each object.
  • A DeletedTime element which contains the time the object was archived (deleted or overwritten).
  • An IsLatest element which indicates if the specific object is the latest version.
  • Although it is not shown in the listing above, a NextGenerationMarker element is returned if the listing of objects is a partial listing. You could then use this marker with the NextMarker value to list more results. You will get a partial list of results when you have many object versions in a bucket. In this case, the results are truncated and you use the NextGenerationMarker value to pick up where you left off on subsequent requests with a generationmarker query parameter. Use the generationmarker query parameter like you use the marker query parameter to page through a listing for a nonversioned bucket.

Continuing with the example, you can verify that you can get both generations of the object by first specifying no generation, which fetches the current version of the object, and then explicitly specifying the generation of the current object and then the archived generation as shown in the following examples.

gsutil

Uses the gsutil tool. 

gsutil cat gs://bucket/foo

this is foo but better

gsutil cat gs://bucket/foo#1360887759327000

this is foo but better

gsutil cat gs://bucket/foo#1360887697105000

this is foo

Java

Uses the Java client library . 

Storage.Objects.Get fooObj = storage.objects().get("bucket", "foo");
ByteArrayOutputStream out = new ByteArrayOutputStream();
fooObj.executeMediaAndDownloadTo(out);
System.out.println(out);

this is foo but better

Storage.Objects.Get fooObj = storage.objects().get("bucket", "foo");
fooObj.setGeneration(new BigInteger("1360887759327000"));
ByteArrayOutputStream out = new ByteArrayOutputStream();
fooObj.executeMediaAndDownloadTo(out);
System.out.println(out);

this is foo but better

Storage.Objects.Get fooObj = storage.objects().get("bucket", "foo");
fooObj.setGeneration(new BigInteger("1360887697105000"));
ByteArrayOutputStream out = new ByteArrayOutputStream();
fooObj.executeMediaAndDownloadTo(out);
System.out.println(out);

this is foo

XML API

Uses the XML API . 

GET /bucket/foo HTTP/1.1

HTTP/1.1 200 OK
ETag: "b859c7443ae764f0cbeba4647e3831d4"
x-goog-generation: 1360887759327000
x-goog-metageneration: 1

this is foo but better

GET /bucket/foo?generation=1360887759327000 HTTP/1.1

HTTP/1.1 200 OK
ETag: "b859c7443ae764f0cbeba4647e3831d4"
x-goog-generation: 1360887759327000
x-goog-metageneration: 1

this is foo but better

GET /bucket/foo?generation=1360887697105000 HTTP/1.1

HTTP/1.1 200 OK
x-goog-generation: 1360887697105000
x-goog-metageneration: 1

this is foo

Updating Object Metadata

If you update object metadata, for example, update an object's ACL, Google Cloud Storage does not archive a copy of the current object, but simply updates the metadata of the specified generation (or live object if none given) and increases the metadata generation by 1. The generation of the object remains unchanged. To see that, let's update the ACLs on the archived object:

gsutil

Uses the gsutil tool. 

gsutil -h 'x-goog-if-generation-match:1360887697105000' \
  acl set public-read gs://bucket/foo

Setting ACL on gs://bucket/foo...

gsutil -d cat gs://bucket/foo#1360887697105000

reply: 'HTTP/1.1 200 OK\r\n'
header: x-goog-generation: 1360887697105000
header: x-goog-metageneration: 2

this is foo

Java

Uses the Java client library . 

StorageObject content = new StorageObject()
    .setName("foo")
    .setAcl(ImmutableList.of(
        new ObjectAccessControl().setEntity("allUsers").setRole("READER")
        ));
Storage.Objects.Patch fooObj = storage.objects().patch("bucket", "foo", content);
fooObj.setIfGenerationMatch(new BigInteger("1360887697105000"));
StorageObject retObj = fooObj.execute();
System.out.println("generation: " + retObj.generation());
System.out.println("metageneration: " + retObj.getMetageneration());

generation: 1360887697105000
metageneration 2

Storage.Objects.Get fooObj = storage.objects().get("bucket", "foo");
fooObj.setGeneration(new BigInteger("1360887697105000"));
ByteArrayOutputStream out = new ByteArrayOutputStream();
fooObj.executeMediaAndDownloadTo(out);
System.out.println(out);

this is foo

XML API

Uses the XML API . 

PUT /bucket/foo?acl&generation;=1360887697105000 HTTP/1.1
x-goog-acl: public-read

HTTP/1.1 200 OK
x-goog-generation: 1360887697105000
x-goog-metageneration: 2

GET /bucket/foo?generation=1360887697105000 HTTP/1.1

HTTP/1.1 200 OK
ETag: "4be8bde80854190cbe801133c6682ecf"
x-goog-generation: 1360887697105000
x-goog-metageneration: 2

this is foo

And you can see from the example, the metageneration has been changed but the generation has not, since the data has not changed.

Conditional Updates Using Object Versioning

To ensure you are updating the correct ACL (when doing Read-Modify-Write), you should use both x-goog-if-generation-match and x-goog-if-metageneration-match . as shown in the following example. The example shows how to change the ACL on an object if the generation and metadata generation match specified values and then confirm that only the metadata generation changed by getting the object.

gsutil

Uses the gsutil tool. 

gsutil -h 'x-goog-if-generation-match:1360887697105000' \
  -h 'x-goog-if-metageneration-match:2' acl set private gs://bucket/foo

Setting ACL on gs://bucket/foo...

gsutil -d cat gs://bucket/foo#1360887697105000

reply: 'HTTP/1.1 200 OK\r\n'
header: x-goog-generation: 1360887697105000
header: x-goog-metageneration: 3

this is foo

Java

Uses the Java client library . 

StorageObject content = new StorageObject()
    .setName("foo")
    .setAcl(ImmutableList.of(
        new ObjectAccessControl().setEntity("allUsers").setRole("READER")
        ));
Storage.Objects.Patch fooObj = storage.objects().patch("bucket", "foo", content);
fooObj.setIfGenerationMatch(new BigInteger("1360887697105000"));
fooObj.setIfMetagenerationMatch(new BigInteger("2"));
StorageObject retObj = fooObj.execute();
System.out.println("generation: " + retObj.generation());
System.out.println("metageneration: " + retObj.getMetageneration());

generation: 1360887697105000
metageneration 2

Storage.Objects.Get fooObj = storage.objects().get("bucket", "foo");
fooObj.setGeneration(new BigInteger("1360887697105000"));
ByteArrayOutputStream out = new ByteArrayOutputStream();
fooObj.executeMediaAndDownloadTo(out);
System.out.println(out);

this is foo

XML API

Uses the XML API . 

PUT /bucket/foo?acl HTTP/1.1
x-goog-acl: private
x-goog-if-generation=1360887697105000
x-goog-if-metageneration: 2

HTTP/1.1 200 OK
x-goog-generation: 1360887697105000
x-goog-metageneration: 3

GET /bucket/foo?generation=1360887697105000 HTTP/1.1

HTTP/1.1 200 OK
ETag: "4be8bde80854190cbe801133c6682ecf"
x-goog-generation: 1360887697105000
x-goog-metageneration: 3

this is foo

Copying Objects

You can copy versioned objects just like unversioned ones, you just have to specify which generation you're from which to copy. For example, when you use gsutil you specify the generation appended to the URI as shown below. When you use the XML API you specify the generation with a x-goog-copy-source-generation header.

gsutil

Uses the gsutil tool. 

gsutil cp gs://bucket/foo#1360887697105000 gs://bucket/newfoo

Copying gs://bucket/foo#1360887697105000...

Java

Uses the Java client library . 

Storage.Objects.Copy newfooObj = storage.objects()
    .copy("bucket", "foo", "bucket", "newfoo", null)
    .setSourceGeneration(new BigInteger("1360887697105000"));
newfooObj.execute();

XML API

Uses the XML API . 

PUT /bucket/newfoo HTTP/1.1
x-goog-copy-source:/bucket/foo
x-goog-copy-source-generation:1360887697105000
Content-Length: 0

HTTP/1.1 200 OK

<?xml version='1.0' encoding='UTF-8'?>
<CopyObjectResult>
    <LastModified>2013-02-15T00:21:37.102Z</LastModified>
    <ETag>"4be8bde80854190cbe801133c6682ecf"</ETag>
</CopyObjectResult>

Conditional Copies Using Object Versioning

To ensure you are copying from the correct object, you should use x-goog-copy-source-generation and/or and x-goog-copy-source-metageneration . To ensure that you're copying into (or updating) the correct objects metadata, you should use either x-goog-if-generation-match and/or and x-goog-if-metageneration-match .

Tips

This section discusses tips to help you work with Object Versioning more effectively.

Using gsutil

  • The gsutil tool has comprehensive support for working with versioned objects that makes many tasks involving object versioning easier. For example, you can work with versioned objects directly by appending the generation you want to work with to the object name with a "#". For more information, see Object Versioning and Concurrency Control or view the built-in documentation by running: 
    gsutil help versions

Creating, retrieving, and copying versioned objects

Using preconditions

Deleting versioned objects

Back to top

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 3.0 License , and code samples are licensed under the Apache 2.0 License . For details, see our Site Policies .

Authentication required

You need to be signed in with Google+ to do that.

Signing you in...

Google Developers needs your permission to do that.