Sauce Runner for Virtual Devices lets you run tests using the native testing frameworks like Espresso with virtual devices in the Sauce Labs testing cloud. This topic describes the required and optional command parameters you can use to set up your test runs.
Command Reference
Command | Required | Description | Examples |
---|
-f
--test-framework
| X | The name of the test framework to use. Espresso is the only current supported option. |
-f espresso
--test-framework=espresso
|
-u
--user
| X | Your Sauce Labs username. You can also use the environment variable SAUCE_USERNAME to provide your login information. The command line argument will take precedence over the environment variable. |
-u test_user
--user=test_user
export $SAUCE_USERNAME=test_user
|
-k
--api-key
| X | Your Sauce Labs API key, which you can find under User Settings in the Sauce Labs interface. You can also use the environment variable SAUCE_ACCESS_KEY to provide your login information. The command line argument will take precedence over the environment variable. |
-k aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
--api-key=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
export $SAUCE_ACCESS_KEY=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
|
-a
--app
| X | The local path or publicly accessible URL to the location of the application you want to test. |
-a ./helloworld.apk
--app='https://the.bestapp.ai/helloworld.apk'
|
-t
--test-app
| X | The local path or publicly accessible URL to the location of the test package you want to use. |
-t ./app-debug-AndroidTest.apk
--test-app='https://the.bestapp.ai/app-debug-AndroidTest.apk'
|
-d
--devices
| X | The type of device you want to use with your test. You can specify two or more device arguments to run tests on multiple devices in parallel, and each device will execute the full test suite. You specify the type of device to use by setting the required deviceName and platformVersion property. Property | Required | Description |
---|
deviceName | Yes | The name of the device to use. You can use the Sauce Labs Platform Configurator to look up the Appium deviceName for supported devices. | platformVersion | Yes | The operating system version of the device you want to use. Supported values depend on the device.You can use the Sauce Labs Platform Configurator to look up the Appium platformVersion for the device. | locale | No | Locale of the device. | orientation | No | Orientation of the device. Supported values are: portrait (default)landscape
|
|
# Test on one device
-d 'deviceName=Google Pixel GoogleAPI Emulator,platformVersion=7.0'
# Test on two devices
--devices='deviceName=LG Nexus 4 GoogleAPI Emulator,platformVersion=4.4' \
--devices='deviceName=Google Pixel GoogleAPI Emulator,platformVersion=7.0'
|
-e
--exclude-tests
-i
--include-tests
|
| Optional parameter to run a subset of tests. You can provide a test filter to either exclude or include tests. By default the full test suite is executed. Description | Filter |
---|
Filter one test method | class com.example.MyClass#testLogin | Filter two test methods | class com.example.MyClass#testLogin,
com.example.MyClass#testOrder
| Filter a test class | class com.example.MyClass | Filter a package | package com.example.testPackage | Filter by size. Tests should be annotated with SmallTest , MediumTest or LargeTest. | size small|medium|large | Filter by annotation | annotation com.example.MyAnnotation |
|
# Run only one test method
--include-tests='class com.example.MyClass#testLogin'
# Run all but one test class
--exclude-tests='class com.example.MyClass'
# Run only the large tests
--include-tests='size large'
|
-n
--tunnel-identifier
|
| Optional parameter to specify a Sauce Connect Proxy tunnel to use with the tests. |
-n dev_tunnel
--tunnel-identifier=dev_tunnel
|
-v
--verbose
|
| Optional parameter to set the verbosity of console output. Valid options as DEBUG , INFO , WARN and ERROR |
|
-h
--help
|
| Print this command line reference to the console. |
|
--version |
| Version information for Sauce Runner. |
|
Exit Status Codes
Sauce Runner for Virtual Devices returns the following status codes based on test execution results.
Status Code | Description |
---|
0 | All the tests passed on all devices. |
1 | This status code can mean multiple things, it is important to refer to the logs to identify the problem. - One or more tests failed during execution
- User error like an invalid path to test files or invalid arguments
- Sauce Labs infrastructure error while executing the test
|
Example Using All Required Parameters for Espresso
./sauce-runner-virtual \
-u test_user \
-k 1234-1235 \
-f espresso \
-a ./helloworld.apk \
-t ./espresso-test-suite.apk \
-d 'deviceName=Samsung Galaxy S8 HD GoogleAPI Emulator,platformVersion=7.0' \
-d 'deviceName=Google Pixel GoogleAPI Emulator,platformVersion=7.1'
This example tests the application helloworld.apk
simultaneously on two emulators, Galaxy S8 and Pixel, using the Espresso test suite espresso-test-suite.apk
.
Sauce-Runner-Virtual installs helloworld.apk
and espresso-test-suite.apk
on Sauce emulators and launches the Espresso test suite on both emulators at the same time. Sauce-Runner-Virtual exits when all the tests have completed. Sauce-Runner-Virtual exits with status code zero if all the tests passed, otherwise it exits with status code 1.