With the Task Queue API, applications can perform work outside of a user request, initiated by a user request. If an app needs to execute some background work, it can use the Task Queue API to organize that work into small, discrete units, called tasks . The app adds tasks to task queues to be executed later.
Task Queue concepts
Tasks queues are an efficient and powerful tool for background processing; they allow your application to define tasks, add them to a queue, and then use the queue to process them in aggregate. You name queues and
configure their properties
in a configuration file named
queue.yaml
. A queue can be one of two types—
push
or
pull
—based on your needs.
Push queues function only within the App Engine environment. These queues are the best choice for applications whose tasks work only with App Engine tools and services. With push queues, you simply configure a queue and add tasks to it. App Engine handles the rest. Push queues are easier to implement, but are restricted to use within App Engine. For more information about push queues and examples of how to use them, see Using Push Queues .
In summary, push queues allow you to process tasks within App Engine at a steady rate and App Engine scales computing resources according to the number of tasks in your queue.
The default queue
For convenience, App Engine provides a default push queue for each application (there is no default pull queue). If you do not name a queue for a task, App Engine automatically inserts it into the default queue. You can use this queue immediately without any additional configuration. All modules and versions of the same application share the same default task queue.
The default queue is preconfigured with a throughput rate of 5 task invocations per second. If you want to change the preconfigured settings, simply define a queue named
default
in
queue.yaml
. Code may always insert new tasks into the default queue, but if you wish to disable execution of these tasks, you may do so by clicking the Pause Queue button in the
Task Queues
tab of the Administration Console.
Named queues
While the default queue makes it easy to enqueue tasks with no configuration, you can also create custom queues by defining them in
queue.yaml
. Custom queues allow you to more effectively handle task processing by grouping similar types of tasks. You can control the processing rate—and a number of other settings—based specifically on the type of task in each queue.
All versions of an application share the same named task queues.
For more information about configuring queues in
queue.yaml
, please see
Task Queue Configuration
.
Task queues in the Administration Console
You can manage task queues for an application using the Task Queue tab of the Administration Console . The Task Queue tab lists all of the queues in the application. Clicking on a queue name brings up the Task Queue Details page where you can see all of the tasks scheduled to run in a queue and you can manually delete individual tasks or purge every task from a queue. This is useful if a task in a push queue cannot be completed successfully and is stuck waiting to be retried. You can also pause and resume a queue on this page.
You can view details of individual tasks by clicking the task name from the list of tasks on the Task Queue Details page. This page allows you to debug why a task did not run successfully. You can also see information about the previous run of the task as well as the task's body.
Task concepts
A
task
is a unit of work to be performed by the application. Each task is an object of the
PushTask
class. Each Task object contains an endpoint (with a request handler for the task and an optional data payload that parameterizes the task). You can enqueue push tasks to a queue defined in
queue.yaml
.
Task names
In addition to a task's contents, you can declare a task's name. Once a task with name N is written, any subsequent attempts to insert a task named N fail.
While this is generally true, task names do not provide an absolute guarantee of once-only semantics. In rare cases, multiple calls to create a task of the same name may succeed. It's also possible in exceptional cases for a task to run more than once—even if it was only created once.
All project, queue, and task names must be a combination of one or more digits, letters a–z, underscores, and/or dashes, satisfying the following regular expression:
[0-9a-zA-Z\-\_]+
Task names may be up to 500 characters long.
If a push task is created successfully, it will eventually be deleted (at most seven days after the task successfully executes). Once deleted, its name can be reused.
If a pull task is created successfully, your application needs to delete the task after processing. The system may take up to seven days to recognize that a task has been deleted; during this time, the task name remains unavailable. Attempting to create another task during this time with the same name will result in an "item exists" error. The system offers no method to determine if deleted task names are still in the system. To avoid these issues, we recommend that you let App Engine generate the task name automatically.