To support common use cases like setting a Time to Live (TTL) for objects or deleting older versions of objects, Google Cloud Storage offers the object lifecycle management feature.

Contents

1. Introduction
2. Lifecycle Configuration
3. Setting Up Lifecycle Management
4. Object Lifecycle Behavior
5. Access Logs

Introduction

You can define lifecycle management policies to let Google Cloud Storage perform automatic actions on your objects based on certain conditions. Here are some example use cases:

• Delete objects older than 365 days.
• Delete objects created before January 1, 2013.
• Keep only the 3 most recent versions of each object in a bucket with versioning enabled.

Lifecycle Configuration

The lifecycle management policies are specified in the lifecycle configuration, which is assigned to a bucket. The lifecycle configuration contains rules that describe which actions occur on objects when certain conditions are met. For example, the configuration might state that after 365 days, objects in the bucket should be deleted.

The following actions are supported:

• Delete : Delete live and/or archived objects.
  This action can be applied to both versioned and non-versioned objects. In a bucket with versioning enabled, deleting a live object creates an archived object, deleting an archived object deletes the object permanently.
  

The following conditions are supported:

• Age : This condition is satisfied when an object reaches the specified age (in days).
  When you specify the Age condition, you are specifying a Time to Live (TTL) for objects in a bucket with lifecycle management configured. The time when the Age condition is considered to be satisfied is calculated by adding the specified value to the object creation time and rounding it to the next midnight in UTC. For example, if the object creation time is 2013/01/10 10:00 UTC and the Age condition is 10 days, then the condition is satisfied on and after 2013/01/21 00:00 UTC.
  
• CreatedBefore : This condition is satisfied when an object is created before midnight of the specified date in UTC.
• NumberOfNewerVersions : Relevant only for versioned objects. If the value is N , this condition is satisfied when there are at least N versions (including the live version) newer than this version of the object.
  For live objects, the number of newer versions is considered to be 0. For the most recent archived version, the number of newer versions is 1 (or 0 if there is no live object), and so on.
  
• IsLive : Relevant only for versioned objects. If the value is true , this condition matches the live objects; if the value is false , it matches archived objects.

When defining a rule, you can specify any set of conditions for any action. If multiple conditions are specified, an object has to match all of them for the action to be taken. You can also define multiple rules with different conditions for the same action, and if an object matches the condition(s) specified in any rule, the action will be taken.

The lifecycle configuration is specified as an XML document for requests to the XML API , and as a JSON document for requests to the JSON API and with the gsutil tool. Here is the RELAX NG compact syntax schema of the lifecycle configuration XML.

Lifecycle Configuration Examples

Deleting objects older than 365 days in a bucket.

The following configuration document defines a rule to delete all objects older than 365 days in a bucket. This lifecycle configuration is the same as defining a Time to Live (TTL) for each object regardless of when it was added to the bucket.

<?xml version="1.0" ?>
<LifecycleConfiguration>
    <Rule>
        <Action>
            <Delete/>
        </Action>
        <Condition>
            <Age>365</Age>
        </Condition>
    </Rule>
</LifecycleConfiguration>

{
  "lifecycle": {
    "rule":
    [
      {
        "action": {"type": "Delete"},
        "condition": {"age": 365}
      }
    ]
  }
}

{
    "rule":
    [
      {
        "action": {"type": "Delete"},
        "condition": {"age": 365}
      }
    ]
}

Deleting live objects after 30 days, and keeping only the 3 most recent versions of each object in a bucket with versioning enabled.

The following XML document defines a rule to delete any live objects older than 30 days, and another rule to delete archived objects that have at least 3 newer versions (including the live version), which is equivalent to keeping only the 3 most recent versions.

<?xml version="1.0" ?>
<LifecycleConfiguration>
    <Rule>
        <Action>
            <Delete/>
        </Action>
        <Condition>
            <IsLive>true</IsLive>
            <Age>30</Age>
        </Condition>
    </Rule>
    <Rule>
        <Action>
            <Delete/>
        </Action>
        <Condition>
            <IsLive>false</IsLive>
            <NumberOfNewerVersions>3</NumberOfNewerVersions>
        </Condition>
    </Rule>
</LifecycleConfiguration>

{
 "lifecycle": {
  "rule": [
   {
    "action": {
     "type": "Delete"
    },
    "condition": {
     "age": 30,
     "isLive": true
    }
   },
   {
    "action": {
     "type": "Delete"
    },
    "condition": {
     "isLive": false,
     "numNewerVersions": 3
    }
   }
  ]
 }
}

{
  "rule": [
   {
    "action": {
     "type": "Delete"
    },
    "condition": {
     "age": 30,
     "isLive": true
    }
   },
   {
    "action": {
     "type": "Delete"
    },
    "condition": {
     "isLive": false,
     "numNewerVersions": 3
    }
   }
  ]
}

Note: The IsLive condition in the second rule in the above example is optional, because the NumberOfNewerVersions condition already guarantees that the live objects will not be deleted (because a live object has 0 newer versions).

Setting Up Lifecycle Management

Warning: Once an object is deleted, it cannot be undeleted. Take care in setting up your lifecycle policies so that you do not cause more data to be deleted than you intend. It is recommended that you test your lifecycle policy on development data before applying to production, and observe the expiration time metadata to ensure your policy has the effect you intend.

The following steps describe how to set up lifecycle management for a bucket using the gsutil lifecycle command. If you don't have the gsutil tool yet, download and install it.

Enabling Lifecycle Management

To enable lifecycle management for a bucket, first create a file with the lifecycle configuration you would like to specify (see examples above), then use the lifecycle set command to configure it:

gsutil lifecycle set lifecycle_config.json gs://bucket_name

Disabling Lifecycle Management

To disable lifecycle management for a bucket, use the same lifecycle set command as above with an empty lifecycle configuration as shown below:

{}

Checking Lifecycle Configuration

You can check the lifecycle configuration set on a bucket using the lifecycle get command:

gsutil lifecycle get gs://bucket_name

You can also save the lifecycle configuration to a file, which you can then edit and use as input to the lifecycle set command:

gsutil lifecycle get gs://bucket_name > lifecycle_config.json

If lifecycle management is not enabled, the following is returned when using lifecycle get :

gs://bucket_name/ has no lifecycle configuration.

You can also set or retrieve the lifecycle configuration using the XML and JSON APIs. With the XML API, issue a PUT Bucket or GET Bucket request, with the lifecycle query parameter. With the JSON API, issue a PATCH Bucket or GET Bucket request, with the fields=lifecycle query parameter.

Object Lifecycle Behavior

Google Cloud Storage inspects all the objects in a bucket with lifecycle configuration once a day. Any objects that match the condition(s) specified for a rule are queued for the corresponding action and Google Cloud Storage performs the action asynchronously. There may be a lag between when the conditions are satisfied and when the action is taken.

Expiration Time Metadata

If a Delete action is specified for a bucket with the Age condition (and no NumberOfNewerVersions condition), then some objects may be tagged with expiration time metadata. An expiration time tells you when a given object will be deleted if lifecycle configuration does not change, and guarantees that the lifecycle configuration will not cause the object to be deleted prior to the expiration time.

Note that the absence of expiration time metadata does not necessarily mean the object will not be deleted, but rather that not enough information is available to determine when or if it will be deleted. For example, if the object creation time is 2013/01/10 10:00 UTC and the Age condition is set to 10 days, then the object expiration time is 2013/01/21 00:00 UTC. However, the expiration time will not be available for the object if:

• The NumberOfNewerVersions condition is also specified. In this case, older versions of the object may still be deleted if new versions are added.
• The CreatedBefore condition is also specified and set to “2013-01-01”, because the object doesn’t satisfy this condition.

You are not charged for storage after the object expiration time even if the object is not deleted immediately. You can continue to access the object before it is deleted and are responsible for other charges (request, network bandwidth). If the expiration time is not available for an object, the object is charged for storage until the time it is deleted.

You can access an object's expiration time in its metadata if it is available. The REST API returns the object's expiration time in the x-goog-expiration response header.

Access Logs

To find out what lifecycle management actions have been taken, you can enable access logs for your bucket. A value of "GCS Lifecycle Management" in the “cs_user_agent” field in the log entry indicates the action was taken by Google Cloud Storage based on the lifecycle configuration. For more information, see Access Logs & Storage Data .

Back to top

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 3.0 License , and code samples are licensed under the Apache 2.0 License . For details, see our Site Policies .