With Sauce Runner for Real Devices, you can run tests using the Espresso and XCUITest frameworks, run tests in parallel across multiple devices, and run subsets of tests against specific devices. This topic describes the options you can use with the runner. You can set the options as environment variables that can be referenced in your testing scripts, or pass them as command line parameters, which will take precedence over options set as environment variables. You can also create a runner configuration file with the options and commands for running your tests.
Only Available in TestObject
At the moment, the Sauce Runner is only available for our Legacy Real Device Cloud Platform, check the Real Device Testing in Sauce Labs Feature Preview for updates on when the Sauce Runner will be available.
What You'll Need
- A Sauce Labs account and access to your account credentials
- A native app (both debug and non-debug app)
- Have Sauce Runner downloaded and installed
General Usage
JAVA_HOME=$(/usr/libexec/java_home --version 8) java -jar runner.jar <command> <command options>
Commands
| Command | Description |
|---|---|
xcuitest | Defines xcuitest as the test framework to use for your native iOS tests |
espresso | Defines espresso as the test framework to use for your native Android tests |
config | Defines a configuration YAML file where the runner executes based on the parameters set in the file. Please note, if you decide to use the config command you can no longer use any of the command options below. For more information please read the following wiki page: Creating a Sauce Runner for Real Devices Configuration File. |
Options
| Required | Option | Description |
|---|---|---|
| X | --apikey | The API key for your Sauce Labs real device cloud account. |
| X | --app | The path to the *.ipa or *.apk file of the application under test or the id number of an already uploaded app. In your command line, refer to the location where you have downloaded the runner.jar file or run the command from the folder from where you downloaded the runner. |
| X | --test | The path to the *.ipa or *.apk file of the test. |
| X | --datacenter | The Data Center endpoint to use in your tests. For Espresso and XCUITest, you'll need to choose from either the US or EU data centers. For more information, see Data Center Endpoints. If you don't specify a device or devices for your test, one will be assigned to your tests based on the type of application you're testing against. |
--device | For static allocation of a device, provide the ID for the type of device to use in your tests, such as 'iPhone_5_real'. You can find the IDs for devices on the Live Testing page in the Sauce Labs web interface. Under the Real Devices tab, search for the type of device you want to use, and then click the Details link in the device description to see the device ID. See Examples of Device Specifications for the Real Device Runner for more information. | |
--devices | The list of devices, allocated dynamically or through static description of the device ID, to use in your tests. See Examples of Device Specifications for the Real Device Runner for more information. | |
--testname | Set a custom test name to appear on the UI, Default is | |
--tunnelIdentifier | If you are using Sauce Connect Proxy, provide the identifier of the tunnel you want to use. | |
--checkFrequency | Interval in seconds to check test results. Default is 30. | |
--timeout | Test timeout in minutes. Test duration cannot exceed 60 minutes. Defaults to 60. | |
--xmlFolder | The folder for the JUnit XML output. | |
--url | Provide the URL of an alternative REST endpoint to use. See Data Center Endpoints. | |
--platformVersion | For dynamic allocation of a device, provide an operating system version to use. For example, use '9' to allocate a device running major version 9 and arbitrary versions of the OS, or '9.3.3' for a specific version. See Examples of Device Specifications for the Real Device Runner for more information. | |
--privateDevicesOnly | If set, only private devices will be queried. | |
| If set, only phones will be queried. | |
--tabletOnly | If set, only tablets will be queried. | |
| For dynamic allocation of a device, provide the device name you would like to dynamically allocate. For example, use | |
XCUITest only | Provide a comma separated list of test cases or test classes. If you want to run all tests of a class, provide only the classname. If you want to run a specific method of a class, provide the class name and method name separated with a '
--testsToRun ClassA,ClassB/methodC | |
Espresso only | If set, the instrumentation will start with Test Orchestrator version 1.1.1 in use. NOTE: with Test Orchestrator, it is in most cases recommended to also add the " | |
Espresso only | Provide a list of test options to Espresso. The key-value pairs supported by espresso are documented here: https://developer.android.com/studio/test/command-line#AMOptionsSyntax. In the example, the test would execute all test methods defined in the class Example: Execute all tests in class TestClassA --e class com.example.android.TestClassA Example: Execute a specific test in class TestClassB --e class com.example.android.TestClassB#methodName |
Using Sauce Runner with a Proxy
If you need Sauce Runner to connect to the internet through a proxy server, use the -D command to specify a direct domain connection to your proxy server and port. The parameters http.proxyUser and http.proxyPassword are optional and can be used if the proxy needs authentication:
java -Dhttp.proxyHost=<your proxy server> -Dhttp.proxyPort=<the port to use> -Dhttp.proxyUser=<the username to use> -Dhttp.proxyPassword=<the password to use>
CLI Examples
XCUITest Example
XCUITest Example with All Required Parameters and Setting the Data Center Option to US
java -jar runner.jar xcuitest --test DummyTestingApp-Runner.ipa --app DummyTestingApp.ipa --apikey <apikey> --datacenter US
Espresso Example
Espresso Example with all Required Parameters and Setting the Data Center Option to US
java -jar runner.jar espresso --test DummyTestingApp-Runner.apk --app DummyTestingApp.apk --apikey <apikey> --datacenter US
Overview
Content Tools
Activity
Tasks