Testing from Other IDEs

This document describes how to create and run tests directly from the command line. You can use the techniques described here if you are developing in an IDE other than Eclipse or if you prefer to work from the command line. This document assumes that you already know how to create a Android application in your programming environment. Before you start this document, you should read the topic Testing Fundamentals , which provides an overview of Android testing.

If you are developing in Eclipse with ADT, you can set up and run your tests directly in Eclipse. For more information, please read Testing from Eclipse with ADT .

Working with Test Projects

You use the android tool to create test projects. You also use android to convert existing test code into an Android test project, or to add the test Ant target to an existing Android test project. These operations are described in more detail in the section Updating a test project . The test target is described in Quick build and run with Ant .

Creating a test project

To create a test project with the android tool, enter:

       android create test-project -m <main_path> -n <project_name> -p <test_path>
      

You must supply all the flags. The following table explains them in detail:

Flag	Value	Description
-m, --main	Path to the project of the application under test, relative to the test package directory.	For example, if the application under test is in source/HelloAndroid , and you want to create the test project in source/HelloAndroidTest , then the value of --main should be ../HelloAndroid . To learn more about choosing the location of test projects, please read Testing Fundamentals .
-n, --name	Name that you want to give the test project.
-p, --path	Directory in which you want to create the new test project.	The android tool creates the test project files and directory structure in this directory. If the directory does not exist, android creates it.

If the operation is successful, android lists to STDOUT the names of the files and directories it has created.

This creates a new test project with the appropriate directories and build files. The directory structure and build file contents are identical to those in a regular Android application project. They are described in detail in the topic Managing Projects .

The operation also creates an AndroidManifest.xml file with instrumentation information. When you run the test, Android uses this information to load the application you are testing and control it with instrumentation.

For example, suppose you create a project in the directory ~/source/HelloAndroid , with the package name com.example.helloandroid , and the activity name HelloAndroid . You can to create the test for this in ~/source/HelloAndroidTest . To do so, you enter:

       $ cd ~/source
$ android create test-project -m ../HelloAndroid -n HelloAndroidTest -p HelloAndroidTest
      

This creates a directory called ~/src/HelloAndroidTest . In the new directory you see the file AndroidManifest.xml . This file contains the following instrumentation-related elements and attributes:

• <application> : to contain the <uses-library> element.
• <uses-library android:name="android.test.runner" : specifies this testing application uses the android.test.runner library.
• <instrumentation> : contains attributes that control Android instrumentation. The attributes are:
  • android:name="android.test.InstrumentationTestRunner" : InstrumentationTestRunner runs test cases. It extends both JUnit test case runner classes and Android instrumentation classes.
  • android:targetPackage="com.example.helloandroid" : specifies that the tests in HelloAndroidTest should be run against the application with the Android package name com.example.helloandroid .
  • android:label="Tests for .HelloAndroid" : specifies a user-readable label for the instrumentation class. By default, the android tool gives it the value "Tests for " plus the name of the main Activity of the application under test.

Updating a test project

You use the android tool when you need to change the path to the project of the application under test. If you are changing an existing test project created in Eclipse with ADT so that you can also build and run it from the command line, you must use the "create" operation. See the section Creating a test project .

Note: If you change the Android package name of the application under test, you must manually change the value of the <android:targetPackage> attribute within the AndroidManifest.xml file of the test package. Running android update test-project does not do this.

To update a test project with the android tool, enter:

       android update test-project -m <main_path> -p <test_path>
      

Flag	Value	Description
-m, --main	The path to the project of the application under test, relative to the test project	For example, if the application under test is in source/HelloAndroid , and the test project is in source/HelloAndroidTest , then the value for --main is ../HelloAndroid .
-p, --path	The of the test project.	For example, if the test project is in source/HelloAndroidTest , then the value for --path is HelloAndroidTest .

If the operation is successful, android lists to STDOUT the names of the files and directories it has created.

Creating a Test Package

Once you have created a test project, you populate it with a test package. The application does not require an Activity , although you can define one if you wish. Although your test package can combine Activities, Android test class extensions, JUnit extensions, or ordinary classes, you should extend one of the Android test classes or JUnit classes, because these provide the best testing features.

If you run your tests with InstrumentationTestRunner (or a related test runner), then it will run all the methods in each class. You can modify this behavior by using the TestSuite class.

To create a test package, start with one of Android's test classes in the Java package android.test . These extend the JUnit TestCase class. With a few exceptions, the Android test classes also provide instrumentation for testing.

For test classes that extend TestCase , you probably want to override the setUp() and tearDown() methods:

• setUp() : This method is invoked before any of the test methods in the class. Use it to set up the environment for the test. You can use setUp() to instantiate a new Intent object with the action ACTION_MAIN . You can then use this intent to start the Activity under test.

  Note: If you override this method, call super.setUp() as the first statement in your code.

• tearDown() : This method is invoked after all the test methods in the class. Use it to do garbage collection and re-setting before moving on to the next set of tests.

  Note: If you override this method, you must call super.tearDown() as the last statement in your code.

Another useful convention is to add the method testPreConditions() to your test class. Use this method to test that the application under test is initialized correctly. If this test fails, you know that that the initial conditions were in error. When this happens, further test results are suspect, regardless of whether or not the tests succeeded.

To learn more about creating test packages, see the topic Testing Fundamentals , which provides an overview of Android testing. If you prefer to follow a tutorial, try the Activity Testing tutorial, which leads you through the creation of tests for an actual Android application.

Running Tests

You run tests from the command line, either with Ant or with an Android Debug Bridge (adb) shell.

Quick build and run with Ant

You can use Ant to run all the tests in your test project, using the target test , which is created automatically when you create a test project with the android tool.

This target re-builds your main project and test project if necessary, installs the test application to the current AVD or device, and then runs all the test classes in the test application. The results are directed to STDOUT .

You can update an existing test project to use this feature. To do this, use the android tool with the update test-project option. This is described in the section Updating a test project .

Running tests on a device or emulator

When you run tests from the command line with Android Debug Bridge (adb) , you get more options for choosing the tests to run than with any other method. You can select individual test methods, filter tests according to their annotation, or specify testing options. Since the test run is controlled entirely from a command line, you can customize your testing with shell scripts in various ways.

To run a test from the command line, you run adb shell to start a command-line shell on your device or emulator, and then in the shell run the am instrument command. You control am and your tests with command-line flags.

As a shortcut, you can start an adb shell, call am instrument , and specify command-line flags all on one input line. The shell opens on the device or emulator, runs your tests, produces output, and then returns to the command line on your computer.

To run a test with am instrument :

1. If necessary, rebuild your main application and test package.
2. Install your test package and main application Android package files ( .apk files) to your current Android device or emulator
3. At the command line, enter:

            $ adb shell am instrument -w <test_package_name>/<runner_class>
           

   where <test_package_name> is the Android package name of your test application, and <runner_class> is the name of the Android test runner class you are using. The Android package name is the value of the package attribute of the manifest element in the manifest file ( AndroidManifest.xml ) of your test package. The Android test runner class is usually InstrumentationTestRunner .

   Your test results appear in STDOUT .

This operation starts an adb shell, then runs am instrument with the specified parameters. This particular form of the command will run all of the tests in your test package. You can control this behavior with flags that you pass to am instrument . These flags are described in the next section.

Using the am instrument Command

The general syntax of the am instrument command is:

       am instrument [flags] <test_package>/<runner_class>
      

The main input parameters to am instrument are described in the following table:

Parameter	Value	Description
<test_package>	The Android package name of the test package.	The value of the package attribute of the manifest element in the test package's manifest file.
<runner_class>	The class name of the instrumented test runner you are using.	This is usually InstrumentationTestRunner .

The flags for am instrument are described in the following table:

Flag	Value	Description
-w	(none)	Forces am instrument to wait until the instrumentation terminates before terminating itself. The net effect is to keep the shell open until the tests have finished. This flag is not required, but if you do not use it, you will not see the results of your tests.
-r	(none)	Outputs results in raw format. Use this flag when you want to collect performance measurements, so that they are not formatted as test results. This flag is designed for use with the flag -e perf true (documented in the section Instrument options ).
-e	<test_options>	Provides testing options as key-value pairs. The am instrument tool passes these to the specified instrumentation class via its onCreate() method. You can specify multiple occurrences of -e <test_options> . The keys and values are described in the section am instrument options . The only instrumentation class that uses these key-value pairs is InstrumentationTestRunner (or a subclass). Using them with any other class has no effect.

am instrument options

The am instrument tool passes testing options to InstrumentationTestRunner or a subclass in the form of key-value pairs, using the -e flag, with this syntax:

       -e <key> <value>
      

Some keys accept multiple values. You specify multiple values in a comma-separated list. For example, this invocation of InstrumentationTestRunner provides multiple values for the package key:

       $ adb shell am instrument -w -e package com.android.test.package1,com.android.test.package2 \
> com.android.test/android.test.InstrumentationTestRunner
      

The following table describes the key-value pairs and their result. Please review the Usage Notes following the table.

Key	Value	Description
package	<Java_package_name>	The fully-qualified Java package name for one of the packages in the test application. Any test case class that uses this package name is executed. Notice that this is not an Android package name; a test package has a single Android package name but may have several Java packages within it.
class	<class_name>	The fully-qualified Java class name for one of the test case classes. Only this test case class is executed.
	<class_name> # method name	A fully-qualified test case class name, and one of its methods. Only this method is executed. Note the hash mark (#) between the class name and the method name.
func	true	Runs all test classes that extend InstrumentationTestCase .
unit	true	Runs all test classes that do not extend either InstrumentationTestCase or PerformanceTestCase .
size	[ small | medium | large ]	Runs a test method annotated by size. The annotations are @SmallTest , @MediumTest , and @LargeTest .
perf	true	Runs all test classes that implement PerformanceTestCase . When you use this option, also specify the -r flag for am instrument , so that the output is kept in raw format and not re-formatted as test results.
debug	true	Runs tests in debug mode.
log	true	Loads and logs all specified tests, but does not run them. The test information appears in STDOUT . Use this to verify combinations of other filters and test specifications.
emma	true	Runs an EMMA code coverage analysis and writes the output to /data//coverage.ec on the device. To override the file location, use the coverageFile key that is described in the following entry. Note: This option requires an EMMA-instrumented build of the test application, which you can generate with the coverage target.
coverageFile	<filename>	Overrides the default location of the EMMA coverage file on the device. Specify this value as a path and filename in UNIX format. The default filename is described in the entry for the emma key.

-e Flag Usage Notes

• am instrument invokes onCreate(Bundle) with a Bundle containing the key-value pairs.
• The package key takes precedence over the class key. If you specifiy a package, and then separately specify a class within that package, Android will run all the tests in the package and ignore the class key.
• The func key and unit key are mutually exclusive.

Usage examples

The following sections provide examples of using am instrument to run tests. They are based on the following structure:

• The test package has the Android package name com.android.demo.app.tests
• There are three test classes:
  • UnitTests , which contains the methods testPermissions and testSaveState .
  • FunctionTests , which contains the methods testCamera , testXVGA , and testHardKeyboard .
  • IntegrationTests , which contains the method testActivityProvider .
• The test runner is InstrumentationTestRunner .

Running the entire test package

To run all of the test classes in the test package, enter:

       $ adb shell am instrument -w com.android.demo.app.tests/android.test.InstrumentationTestRunner
      

Running all tests in a test case class

To run all of the tests in the class UnitTests , enter:

       $ adb shell am instrument -w  \
> -e class com.android.demo.app.tests.UnitTests \
> com.android.demo.app.tests/android.test.InstrumentationTestRunner
      

am instrument gets the value of the -e flag, detects the class keyword, and runs all the methods in the UnitTests class.

Selecting a subset of tests

To run all of the tests in UnitTests , and the testCamera method in FunctionTests , enter:

       $ adb shell am instrument -w \
> -e class com.android.demo.app.tests.UnitTests,com.android.demo.app.tests.FunctionTests#testCamera \
> com.android.demo.app.tests/android.test.InstrumentationTestRunner
      

You can find more examples of the command in the documentation for InstrumentationTestRunner .