Synopsis

gsutil compose gs://bucket/obj1 gs://bucket/obj2 ... gs://bucket/composite

Description

The compose command creates a new object whose content is the concatenation of a given sequence of component objects under the same bucket. gsutil uses the content type of the first source object to determine the destination object’s content type. For more information, please see: https://developers.google.com/storage/docs/composite-objects

Note also that the gsutil cp command will automatically split uploads for large files into multiple component objects, upload them in parallel, and compose them into a final object (which will be subject to the component count limit). This will still perform all uploads from a single machine. For extremely large files and/or very low per-machine bandwidth, you may want to split the file and upload it from multiple machines, and later compose these parts of the file manually. See the ‘PARALLEL COMPOSITE UPLOADS’ section under gsutil help cp for details.

Appending simply entails uploading your new data to a temporary object, composing it with the growing append-target, and deleting the temporary object:

$ echo 'new data' | gsutil cp - gs://bucket/data-to-append
$ gsutil compose gs://bucket/append-target gs://bucket/data-to-append \
    gs://bucket/append-target
$ gsutil rm gs://bucket/data-to-append

Note that there is a limit (currently 1024) to the number of components for a given composite object. This means you can append to each object at most 1023 times.