Sign in

1. Products
2. Google Cloud Platform
3. Google Compute Engine

Google Compute Engine

Feedback on this document

gcutil Usage Tips

This document describes some helpful usage tips for gcutil. Note that this is not a comprehensive list. For a comprehensive list of all available gcutil flags and commands, run:

$ gcutil
$ gcutil --help

Contents

Listing available gcutil commands

To list all available gcutil commands, run:

$ gcutil

Similarly, you can list all global gcutil flags by running:

$ gcutil --help

For help on a specific command and list non-global flags for that command:

$ gcutil help <command-name>

Filtering list results

When you list resources, you might want to filter your list results based on a set of criteria. This can help so that you have a shorter list of more relevant results. To do so, use the --filter flag:

--filter="<expression>"

Your <expression> must contain the following:

<field-name> <comparison-string> <literal-string>

<field-name>

The name of the field you want to compare. The field name must be valid for the type of resource being filtered. Only atomic field types are supported (string, number, boolean). Array and object fields are not currently supported.

<comparison-string>

The comparison string, either eq (equals) or ne (not equals).

<literal-string>

The literal string value to filter to. The literal value must be valid for the type of field (string, number, boolean). For string fields, the literal value is interpreted as a regular expression using RE2 syntax . The literal value must match the entire field. For example, when filtering instances, name eq my-instance won't work, but name eq .*my-instance will work.

For example:

$ gcutil listinstances --filter="status ne RUNNING" --project=my-project

The above filter causes gcutil to return only results whose status field does not equal RUNNING . Here is a more complex example:

$ gcutil listinstances --filter="name eq '.*/my-instance-[0-9]+'" --project=my-project

This would list all instances whose name matches the given regular expression.

Formatting list results

gcutil allows you format your list results into one of several display options, including:

• sparse
• json
• csv
• table
• names

For example, if you wanted to just list the names of a certain resource, you can use the names options:

$ gcutil listinstances --format=names --project=<project-id>

Moving instances across zones

gcutil offers a helper function to move instances across zones using the gcutil moveinstances command. The command copies your instance configuration, takes snapshots of any persistent disks attached to the instance, deletes the existing instances, and then recreates the instances in your destination zone.

Important warnings

This particular feature takes a snapshot of all your existing persistent disk data, shuts down your instance, recreates your persistent disks in the new zone, and starts new instances with your instance configurations. It is possible that this process could fail before it completes your move. For this reason, we encourage that you also backup your data before running this command , so that you can keep track of your backup resources in case you need to perform recovery. In particular, you should create snapshots of any persistent disk data you want to keep and also create custom images of the instances you are moving.

Other warnings to keep in mind:

• This command deletes and recreates instances for you, but does not preserve any ephemeral IP addresses or contents of memory. Before executing this command, make sure you have backed up any data you want to keep.

• You should not modify your project while a move is in progress. In particular, any changes that affect quotas, introduce naming conflicts, or deletes required resources, such as snapshots, will cause unpredictable behavior from gcutil.

Prerequisites

You must fulfill the following prerequisites before you use this feature:

1. The desired zone must have enough quota to handle the number of new instances and disks that are created.

   To see how much quota is available in a zone using gcutil, use the gcutil getregion <region> command.

2. If you want to move instances that have persistent disks attached, you must move all instances that are using those persistent disks.

   For example, if instance-1 and instance-2 are both using persistent-disk-1 , you must move both instances together to be able to successfully move the persistent disk. If you attempt to move one instance and not the other, gcutil won't perform the move.

3. The destination zone cannot contain instances with the same names as the instances you want to move.

   The moveinstances feature also copies over instance names to your desired zone. For example, if you move an instance named my-instance from us-central1-a to us-east1-a , gcutil automatically uses the my-instance name for your new instance. However, if your new zone already has an instance with that name, the command quits, and your instances are not moved. Make sure your desired destination zone does not already contain an instance with the same name as an instance you want to move.

If the above preconditions are not met, gcutil won't perform the move at all and your instances won't be modified.

When you run this command, almost all data that is accessible through the API, such as persistent disk data, is copied to your new instances. Server-generated data and data that is not accessible through the API, such as selfLinks, ephemeral IP addresses, and any contents of memory, are not preserved. Generally, startup scripts and storing data on persistent disks are the ideal ways reinstall software and store persistent data.

To move an instance:

$ gcutil --project=<project-id> moveinstances <name-regex> [<name-regex> ... <name-regex>] \
         --source_zone=<zone-name> \
         --destination_zone=<zone-name> \
         [--force] [--keep_snapshots]

Import flags and parameters:

--project=<project-id>

[ Required ] The project ID for this request.

<name-regex>

[ Required ] One or more regular expressions matched to instance names that should be moved to the new zone. If any instance names match any of these regex expression, they will be moved.

--source_zone=<zone-name>

[ Required ] The current zone where the instances live.

--destination_zone=<zone-name>

[ Required ] The new zone where you want your instances to be recreated.

--force

[ Optional ] Overrides the confirmation prompt.

--keep_snapshot

[ Optional ] After a successful move, all generated snapshots from the move are deleted. If you want to keep these snapshots, specify this flag.

Also, as part of the move process, gcutil creates snapshots of any attached persistent disks and recreates these disks in the new zone. After the move is complete, the snapshots are automatically deleted. It is possible that users might experience some charges for these snapshots. For information how snapshots are priced, see the price sheet .

Resuming a move

gcutil generates a log file every time you start the moveinstances command. If the move fails, you can use the log file to resume a move.

$ gcutil --project=<project-id> resumemove <log-path> \
        [--force] [--keep_log_file] [--keep_snapshots]

Important flags and parameters:

--project=<project-id>

[ Required ] The project ID for this request.

--force

[ Optional ] Override the confirmation prompt.

--keep_log_file

[ Optional ] By default, gcutil deletes the log file after the command is completely successfully. If you want to keep the log file, include this flag.

--keep_snapshots

[ Optional ] After a successful move, all generated snapshots from the move are deleted. If you want to keep these snapshots, set this flag.

<log-path>

[ Required ] The path to the log file that gcutil should use. When running gcutil moveinstances , look for the following line:

If the move fails, you can re-attempt it using:

$ gcutil resumemove $HOME/.gcutil.move.20130201231024

Where $HOME/.gcutil.move.<datetime> is the log path.

Checking which user you are authorized as

Use the --just_check_auth flag:

$ gcutil --project=<project-id> auth --just_check_auth
INFO: Authorization succeeded for user <user>

Printing a REST request and its corresponding JSON response

To see the details of the REST request and its corresponding JSON response sent and received by gcutil, use the --dump_request_response flag:

$ gcutil --dump_request_response <command> <command_flags>

To see only the JSON response to your request, use the --print_json flag.

$ gcutil <command> <command_flags> --print_json

Saving and reusing flag values

When you run gcutil commands, there are certain flags that you must always specify. For example, every gcutil command requires a --project flag so that gcutil knows for which project it should execute the command. To make this more convenient, gcutil provides a --cache_flag_values flag which saves all your specified flags for that command into a .gcutil.flags file. To save and use gcutil flag values, run:

$ gcutil <command> <command_flags> --cache_flag_values=True

By default, gcutil creates a .gcutil.flags file in your home directory. If you would like to save your .gcutil.flags file elsewhere, you can specify a file path using the --cached_flags_file flag:

$ gcutil <command> <command_flags> --cache_flag_values=True [--cached_flags_file=<some_file_path>]

Where <some_file_path> is your desired file path. You can specify the file path in a variety of manners:

• ~/<filename>
• ./<filename>
• subdirectory/<filename>

gcutil will always look for an existing .gcutil.flags value in the following manner:

1. If the --cached_flags_value flag is specified with a file path, gcutil the file path provided.
2. If the --cached_flags_value flag is not specified, or is specified only with a file name rather than a file path, gcutil looks for the file name in the current directory. If it doesn't find the file in the current directory, it looks for the file in the parent directories, and lastly, it looks for the file in the home directory if the file is not found in the parent directories. gcutil uses the first instance of the file it finds.

This behavior allows you to create multiple .gcutil.flag values for easy access to more than one project. For example, if you use multiple projects with gcutil, you can save a customized cached flag file in each directory to use for each project and gcutil automatically selects the right cached flag file for you.

Revoking a refresh token

If your refresh tokens are compromised, you can revoke them so that they can no longer be used. To revoke a refresh token:

1. Log into your Google account page.
2. Click Security and then click the Edit button next to Authorizing applications and sites .
3. Click Revoke Access next to Google Cloud SDK .

Resetting an instance

For information on how to reset an instance, review the Instances documentation.

Performing requests in asynchronous mode

To perform a request in asynchronous mode, meaning gcutil returns immediately after posting a request, regardless of whether or not the request has been completed, use the --[no]synchronous_mode flag. This can be useful if you know a request will take a while to complete and you want to be able to perform other tasks. By default, gcutil will perform requests in synchronous mode.

Using gcutil with multiple accounts

You will need a different authorization token for each Google account with which you want to access Google Compute Engine. gcutil provides a --credentials_file flag for this purpose. When specifying a new --credentials_file that does not yet exist, gcutil prompts you through the authorization process again. Sign into the intended Google account in your browser, and authorize gcutil to access your account. For example:

1. Sign into Google with [email protected] on your browser.
2. Run gcutil auth --credentials_file=$HOME/.user1_credentials .
3. Follow the authorization steps.
4. Sign into Google with [email protected] on your browser.
5. Run gcutil auth --credentials_file=$HOME/.user2_credentials .
6. Follow the authorization steps

Isolating your scripts from future changes

If you use applications or scripts that depend on a certain API version, image, or disk location, you might find that your scripts break from time to time as Google Compute Engine releases a new API, releases a new default image, or changes the disk path. If you don't always have the time to resolve these breaking changes as they happen, here are some tips on how to isolate your scripts and applications from changes until you have time to resolve these conflicts:

• Always reference your persistent disks using the disk alias path rather than linking to it using /dev/vd* . The /dev/vd* path can change at anytime and is not a reliable path to use. Use the disk alias path, which always points to the correct location of the disk:

  /dev/disk/by-id/google-*
  

  For more information, see attaching a persistent disk .

• Hard coding an API version or image name in your script:

  • If your scripts or applications depend on a particular API version

    Consider hard coding the API version number using the gcutil --service_version=<api-version-number> flag. If you don’t specify an API version number, Google Compute Engine automatically uses the most current version. As versions change, there might be changes that break your script. To help with this, hard code the API version number so that your script continues to run until you have time to address any conflicts. Deprecated APIs are supported for 3 months from the deprecation date, providing a relatively large time frame to convert your scripts.

  • If your scripts or applications depend on a particular image

    Consider hard coding the image name into your script. The default image does change from time to time, and your script might not work or run correctly if it relies on certain capabilities of an older image that is not available in newer images. To prevent your script from breaking until you have time to resolve conflicts, you can hard code the image name and change it when you have updated your script to use the new images. In gcutil, you can specify an image using the --image=<image-name> flag.

Back to top