Please note that the contents of this offline web site may be out of date. To access the most recent documentation visit the online version .
Note that links that point to online resources are green in color and will open in a new window.
We would love it if you could give us feedback about this material by filling this form (You have to be online to fill it)



Monitoring a MapReduce Job

If you know the ID of your MapReduceJob, a url of the form http://<app-id>.appspot.com/_ah/pipeline/status.html?root=<job-id> displays information about that job.

The url http://<app-id>.appspot.com/_ah/pipeline/list shows all the jobs that are running in an application. Use the filter drop-down menu to display only MapReduce jobs, along with their IDs. You can select a job from this list to display its information.

The information is displayed in a window with three panes.

MapReduce Console

The job outline pane

When a MapReduce job runs it is factored into a hierarchy of subordinate jobs. The job outline pane on the left shows the hierarchy of the jobs. Clicking on the console link for a specific step shows a summary of the activity and resource usage for that step:

MapReduce pipeline

The names of the jobs indicate their function: MapReduceJob$MapJob, MapReduceJob$SortJob, and MapReduceJob$ReduceJob correspond to the map, shuffle, and reduce stages of the MapReduce algorithm. The two MapReduceJob$CleanupPipelineJobs before and after ReduceJob are utility steps that remove the intermediate data files.

The color of a job indicates its status:

  • Gray - waiting, not yet run
  • White - running
  • Green - done
  • Red - failed

Clicking on the name of any job (or its console link) will display more information for that job in the two panes on the right.

The job info pane

The job info pane shows the job's parameters, status, input and/or output, and lists its child nodes:

MapReduce pipeline

The stage status pane

The stage status pane depends on the type of node you’re looking at. For each of the MapReduce stages it shows sharding statistics, and information about any counters used by the job.

MapReduce pipeline

Authentication required

You need to be signed in with Google+ to do that.

Signing you in...

Google Developers needs your permission to do that.