Authenticating from Google Compute Engine

You can authenticate from Google Compute Engine to other Google services using OAuth 2.0 authentication. Depending on your use case, you can either perform the standard OAuth 2.0 flow or use service accounts, as described here.

Not what you're looking for? Are you looking for documentation on how to authorize requests to Google Compute Engine?

Overview

When you write applications that need to talk to other Google services, you need to authenticate your application to these services before you can perform any operations or access any data. Often times, you can authenticate these applications using the standard OAuth 2.0 flow, but Google Compute Engine makes this process simpler by providing service accounts .

When you create a Google Compute Engine project, Google Compute Engine also creates a service account for that project. This service account authenticates the project to other Google services but you can also use the service account to authenticate your instances to other APIs. This is ideal for cases when you have applications running within instances that need to programmatically talk between services but don't need access to user data.

For example, let's say you want to write an application that runs within a VM instance. This application saves and retrieves log files to and from Google Cloud Storage but in order for the application to talk to Google Cloud Storage, it needs to authenticate to the Google Cloud Storage API. You can use the standard OAuth 2.0 flow, as briefly described in the table below, to do so or you can save yourself several steps if you set up your instance to use service accounts. Your application can then authenticate seamlessly to Google Cloud Storage and other Google APIs without having to perform the full OAuth 2.0 flow, which is handled by the instance. Here is a table that briefly outlines the steps required for each authentication process:

Standard OAuth 2.0 Flow	Service Accounts
Set up your application to construct an OAuth 2.0 authorization request to the Google OAuth 2.0 authorization server. Log in and consent to the access. The server returns an authorization code. Exchange your authorization code for a token. Use the token to make requests to the API.	Create an instance with access to a service account. Set up your application to query the metadata server with a simple REST request to obtain a token. Use the token to make requests to the API.

If you don't need access to user information, using a service account is a quicker, easier way to set up your application to access other APIs. This document discusses:

Using service accounts with your applications

If you want to run an application from within your instance that needs access other Google services, you can setup your application to use service accounts.
Using service accounts with tools and libraries

When you set up a service account with your instance, it automatically allows tools and libraries within that instance, such as gsutil , to use the service account credentials to access the respective API, without requiring additional authentication on your end. See Using Service Accounts with Tools and Client Libraries for more information.

Service account names

All Google Compute Engine projects can have one Google Compute Engine service account associated with them and multiple Developers Console service accounts . In the API, the Google Compute Engine service account is referred to as the default account, but you can find out the actual service account name by querying the metadata server . Generally, the account name has the format:

[email protected]

Preparing an instance to use service accounts

Before you can use a service account for your application or tools, you must first set up your instance with a service account. To do so, provide the --service_account_scopes=<scope> flag during instance creation. The OAuth2 scope you provide determines the level of access your instance has to that service:

$ gcloud compute instances create INSTANCE --scopes [ACCOUNT=]SCOPE [[ACCOUNT=]SCOPE …]]

Behind the scenes, Google Compute Engine uses these scopes to request a refresh token that it holds on to. Although you cannot directly access the refresh token yourself, you can use the metadata server to request an access token that you can use in requests.

For example, you can create a new instance and authorize it to use a service account to access Google Cloud Storage with full control like so:

$ gcloud compute instances create INSTANCE --scopes https://www.googleapis.com/auth/devstorage.full_control

To specify multiple scopes, so your instance can access multiple services, provide a list of scopes separated by spaces:

$ gcloud compute instances create INSTANCE \
      --scopes https://www.googleapis.com/auth/devstorage.full_control https://www.googleapis.com/auth/compute

Keep in mind the following as you're setting up your instance to use service accounts:

After you have created an instance with a service account and specified scopes, you cannot change or expand the list of scopes.

You should specify all desired scopes at instance creation time.
Each Google service has its own unique scopes and you'll need to find the right scope for your desired service.

Generally, scopes are listed in the API documentation of the service. For example, see Google Cloud Storage Developer's Guide for more specific scope information. You can see a list of all supported scopes in the help for instances create .
Currently, it is only possible to authorize your instance for the service account of the project it belongs to.

It is not possible to specify multiple service accounts or service accounts of other projects.

Service account scope aliases

Usually, OAuth 2.0 scopes are long URIs that can be difficult to remember. For example, the Google Cloud Storage OAuth 2.0 scopes all begin with the URI:

https://www.googleapis.com/auth/devstorage.

For convenience, gcloud compute provides a list of aliases that you can use in place of the longer scope URIs. For example, for full access to Google Cloud Storage , the scope is:

https://www.googleapis.com/auth/devstorage.full_control

with an alias of storage-full . Specify the alias the same way you would specify the normal scope URI:

$ gcloud compute instances create INSTANCE --scopes storage-full

To see a list of available scopes, see the --scopes flag description in the instances create help page, or run:

$ gcloud compute instances create --help

Using service accounts with applications

You can use service accounts with your applications to authenticate to other Google services. To do so:

Make sure your instance has been set up to use a service account

If not, follow the instructions provided above under Preparing an Instance to Use Service Accounts .
Query the metadata server for an access token

After you have created your instance, you need to query the metadata server for an access token. Here's how to do so using curl:
```
$ curl "http://metadata/computeMetadata/v1/instance/service-accounts/default/token" -H "X-Google-Metadata-Request: True"
```
Your token automatically has access to any of the scopes you specified when you initially started the instance. It is not possible to ask for a token outside of the scopes you originally indicated.

The server returns something similar to the following:
```
{
  "access_token":"ya29.AHES6ZRN3-HlhAPya30GnW_bHSb_QtAS08i85nHq39HE3C2LTrCARA",
  "expires_in":3599,
  "token_type":"Bearer"
}
```
Use the access token in requests to the desired API

Use the access_token provided by the metadata server to perform requests to the API. Note that you can't use the access token for a scope that you didn't specify. For example, you can not use the access token above to access the Google Prediction API because the scope was only authorized for the Google Cloud Storage API and the Google Compute Engine API. The following demonstrates how to use the access token in a curl request:
```
curl https://www.googleapis.com/[api_path] -H "Authorization":"Bearer [access_token]"
```
you can also use:
```
curl https://www.googleapis.com/[api_path]?access_token=[access_token]
```
Please read the documentation on the parameters you can set.

As a security feature, access tokens are designed to expire after a short period of time. After an access token reaches 5 minutes of its expiration window, the metadata server no longer caches it. Within the 5 minute expiration window, you can request a new access token to use.

If you need a refresh token , which never expires and is used to request new access tokens, you need to perform the standard OAuth 2.0 flow.

Python example

Here is a basic Python example that demonstrates how to request a token and access the Google Cloud Storage API from within an instance. This sample:

Connects to the metadata server
Provides a scope for access (in this case, the scope is for read-write access to Google Cloud Storage)
Queries the metadata server
Extracts the access token from the response
Uses the access token to make a request to Google Cloud Storage

Note that we didn't need to construct an OAuth 2.0 authorization request to the Google OAuth 2.0 authorization server.

#!/usr/bin/python

import json
import urllib
import httplib2

METADATA_SERVER = 'http://metadata/computeMetadata/v1/instance/service-accounts'
SERVICE_ACCOUNT = 'default'
GOOGLE_STORAGE_PROJECT_NUMBER = '<var>YOUR_GOOGLE_STORAGE_PROJECT_NUMBER</var>'

"""This script grabs a token from the metadata server, and queries the
Google Cloud Storage API to list buckets for the desired project."""

def main():
  token_uri = '%s/%s/token' % (METADATA_SERVER, SERVICE_ACCOUNT)
  http = httplib2.Http()
  resp, content = http.request(token_uri, method='GET', body=None, headers={'X-Google-Metadata-Request': 'True'}) # Make request to metadata server

  if resp.status == 200:
    d = json.loads(content)
    access_token = d['access_token'] # Save the access token

    # Construct the request to Google Cloud Storage
    resp, content = http.request('https://storage.googleapis.com', \
                                  body=None, \
                                  headers={'Authorization': 'OAuth ' + access_token, \
                                           'x-goog-api-version': '2', \
                                           'x-goog-project-id': GOOGLE_STORAGE_PROJECT_NUMBER })

    if resp.status == 200:
       print content
    else:
       print resp.status

  else:
    print resp.status

if __name__ == '__main__':
  main()

Using service accounts with tools and client libraries

By default, tools that are provided in your instance, such as gsutil and gcloud compute , recognize and use a service account if it is available. For gsutil (and access to Google Cloud Storage), use one of the scopes defined at the Google Cloud Storage developer's guide and for gcloud compute , use one of the following scopes:

Scope	Meaning
`https://www.googleapis.com/auth/compute`	Read-write access to Google Compute Engine methods.
`https://www.googleapis.com/auth/compute.readonly`	Read-only access to Google Compute Engine methods.

This only applies to tools that are provided automatically with the instance. If you created new tools, or added custom tools, you need to authorize your application as described in using service accounts with your application .

Checking if an instance uses service accounts

To find out if an instance is using service accounts, you can do either of the following:

Query the metadata server from within that instance

To query the metadata server from within the instance itself, make a request to http://metadata/computeMetadata/v1/instance/service-accounts/ . Here's how you would use curl to do so:
```
user@myinst:~$ curl "http://metadata/computeMetadata/v1/instance/service-accounts/" -H "X-Google-Metadata-Request: True"
```
The command will return output similar to the following if one or more scopes were requested when the instance was created:
```
[email protected]/
default/
```

Use gcloud compute instances describe from your local machine

Run the following command from your local machine:

$ gcloud compute instances describe INSTANCE --format json

The response returned from either method should contain the following:

{
   ...
   "serviceAccounts":[
      {
         "email":"[email protected]",
         "scopes":[
            "https://www.googleapis.com/auth/devstorage.full_control"
         ]
      }
   ]
   ...
}

If your instance is using a Google Compute Engine service account:

The service account name will appear in the response. In this example, the service account [email protected] is associated with the Google Cloud Storage scope.
If your instance isn't using a Google Compute Engine service account:

You will receive an empty serviceAccounts list.

Service account token limits

When you set up an instance to use a service account, Google Compute Engine automatically creates OAuth2 refresh tokens for your service account to use for authenticating to other Google services. Google Compute Engine creates OAuth2 refresh tokens for each VM instance that uses a unique set of {service_account, service_account_scopes} and reuses refresh tokens for each VM instance that uses an existing set of {service_account, service_account_scopes}. For example, if you have three instances:

Instance name	service_account	service_account_scopes
instance1	default	scope1, scope2
instance2	default	scope2, scope1
instance3	default	scope1, scope2, scope3

Google Compute Engine creates two refresh tokens: one for the set {default, scope1, scope2} and one for the set {default, scope1, scope2, scope3}. {{ product_name }} does not create an additional refresh token for instance2 because service_account_scopes are unordered sets, so {scope1, scope2} is equivalent to {scope2, scope1} and can use the same refresh token.

There is a limit to the total number of refresh tokens that your service account can have at any one point in time. Currently, this limit is 600. If this limit is reached, Google Compute Engine will not be able to create an instance which requires a new refresh token, and you will get a SERVICE_ACCOUNT_TOO_MANY_TOKENS error. For example, if you have reached the refresh token limit, and you attempt to create an instance with a new, unique set {default, scope1, scope2, scope3, scope4), the action fails and you will receive the SERVICE_ACCOUNT_TOO_MANY_TOKENS error.

However, if Google Compute Engine already has a refresh token for an existing set, you can create new instances with that same set and it will reuse the same refresh token. For example, if you created an instance4 with the same set as instance1 {default, scope1, scope2}, the instance creation will succeed because you are not creating a new refresh token.

How to free up refresh tokens

If you reach the refresh token limit and cannot create new refresh tokens, you must delete some refresh tokens associated with the service account. To do so, delete Google Compute Engine instances that are using the service account, in order to reduce the number of distinct service_account_scopes sets used by the service account.

Google Developers Console service accounts

Currently, Google offers two different types of service accounts that you can use. We are working on clarifying the terminology here, but until then, here are how the two different types of service accounts differ:

Developers Console Service Accounts

Developers Console service accounts lets you write applications which can authenticate themselves to access a compatible Google API using a JSON Web Token (JWT). The JWT can be used to acquire OAuth2 access tokens that can access Google APIs. You can write applications on Google Compute Engine that uses Developers Console service accounts, by following the Developers Console Service Accounts documentation.
Google Compute Engine service accounts

Google Compute Engine service accounts work without the need to create or manage a JWT. Instead, programs running within your Google Compute Engine instances can automatically acquire OAuth2 access tokens with credentials to access any service in your Google API project and any other services that granted access to that service account.

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 3.0 License , and code samples are licensed under the Apache 2.0 License . For details, see our Site Policies .

Google Compute Engine