perfdiag - Run performance diagnostic
Synopsis
gsutil perfdiag [-i in.json] [-o out.json] [-n iterations] [-c processes] [-k threads] [-s size] [-t tests] url...
Description
The perfdiag command runs a suite of diagnostic tests for a given Google Storage bucket.
The ‘url’ parameter must name an existing bucket (e.g. gs://foo) to which the user has write permission. Several test files will be uploaded to and downloaded from this bucket. All test files will be deleted at the completion of the diagnostic if it finishes successfully.
gsutil performance can be impacted by many factors at the client, server, and in-between, such as: CPU speed; available memory; the access path to the local disk; network bandwidth; contention and error rates along the path between gsutil and Google; operating system buffering configuration; and firewalls and other network elements. The perfdiag command is provided so that customers can run a known measurement suite when troubleshooting performance problems.
Providing Diagnostic Output To Google Cloud Storage Team
If the Google Cloud Storage Team asks you to run a performance diagnostic please use the following command, and email the output file (output.json) to gs-team @ google . com :
gsutil perfdiag -o output.json gs://your-bucket
Options
| -n | Sets the number of iterations performed when downloading and uploading files during latency and throughput tests. Defaults to 5. |
| -c | Sets the number of processes to use while running throughput experiments. The default value is 1. |
| -k | Sets the number of threads per process to use while running throughput experiments. Each process will receive an equal number of threads. The default value is 1. |
| -s | Sets the size (in bytes) of the test file used to perform read and write throughput tests. The default is 1 MiB. This can also be specified using byte suffixes. Examples: 1M, 500KB, etc. |
| -t |
Sets the list of diagnostic tests to perform. The default is to run all diagnostic tests. Must be a comma-separated list containing one or more of the following:
|
| -m |
Adds metadata to the result JSON file. Multiple -m values can be specified. Example: gsutil perfdiag -m "key1:value1" -m "key2:value2" gs://bucketname/
Each metadata key will be added to the top-level “metadata” dictionary in the output JSON file. |
| -o | Writes the results of the diagnostic to an output file. The output is a JSON file containing system information and performance diagnostic results. The file can be read and reported later using the -i option. |
| -i | Reads the JSON output file created using the -o command and prints a formatted description of the results. |
Measuring Availability
The perfdiag command ignores the boto num_retries configuration parameter. Instead, it always retries on HTTP errors in the 500 range and keeps track of how many 500 errors were encountered during the test. The availability measurement is reported at the end of the test.
Note that HTTP responses are only recorded when the request was made in a single process. When using multiple processes or threads, read and write throughput measurements are performed in an external process, so the availability numbers reported won’t include the throughput measurements.
Note
The perfdiag command collects system information. It collects your IP address, executes DNS queries to Google servers and collects the results, and collects network statistics information from the output of netstat -s. None of this information will be sent to Google unless you choose to send it.