The Sauce Labs Cookbook

The Sauce Labs Cookbook

Sauce Headless

Front End Performance Testing

Insights

External Resources

More Info

Page tree
Skip to end of metadata
Go to start of metadata

Sauce Runner for Virtual Devices lets you run tests using the native testing frameworks like Espresso with virtual devices in the Sauce Labs testing cloud. This topic describes the required and optional command parameters you can use to set up your test runs. 

Command Reference

Command

Required

Description

Examples

-f

--test-framework

X

The name of the test framework to use. Espresso is the only current supported option.

-f espresso
--test-framework=espresso

-u

--user

X

Your Sauce Labs username. You can also use the environment variable SAUCE_USERNAME to provide your login information. The command line argument will take precedence over the environment variable.

-u test_user
--user=test_user
export $SAUCE_USERNAME=test_user

-k

--api-key

X

Your Sauce Labs API key, which you can find under User Settings in the Sauce Labs interface. You can also use the environment variable SAUCE_ACCESS_KEY to provide your login information. The command line argument will take precedence over the environment variable.

-k aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
--api-key=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
export $SAUCE_ACCESS_KEY=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee

-a

--app

X

The local path or publicly accessible URL to the location of the application you want to test.

-a ./helloworld.apk
--app='https://the.bestapp.ai/helloworld.apk'

-t

--test-app

X

The local path or publicly accessible URL to the location of the test package you want to use.

-t ./app-debug-AndroidTest.apk
--test-app='https://the.bestapp.ai/app-debug-AndroidTest.apk'

-d

--devices

X

The type of device you want to use with your test. You can specify two or more device arguments to run tests on multiple devices in parallel, and each device will execute the full test suite. You specify the type of device to use by setting the required deviceName and platformVersion property.

PropertyRequiredDescription
deviceNameYesThe name of the device to use. You can use the Sauce Labs Platform Configurator to look up the Appium deviceName for supported devices.
platformVersionYesThe operating system version of the device you want to use. Supported values depend on the device.You can use the Sauce Labs Platform Configurator to look up the Appium platformVersion for the device.
localeNoLocale of the device.
orientationNo

Orientation of the device. Supported values are:

  • portrait (default)
  • landscape
 

# Test on one device
-d 'deviceName=Google Pixel GoogleAPI Emulator,platformVersion=7.0'
 
# Test on two devices
--devices='deviceName=LG Nexus 4 GoogleAPI Emulator,platformVersion=4.4' \
--devices='deviceName=Google Pixel GoogleAPI Emulator,platformVersion=7.0'

-e

--exclude-tests

-i

--include-tests


Optional parameter to run a subset of tests. You can provide a test filter to either exclude or include tests. By default the full test suite is executed.

DescriptionFilter
Filter one test methodclass com.example.MyClass#testLogin
Filter two test methods

class com.example.MyClass#testLogin,

com.example.MyClass#testOrder

Filter a test classclass com.example.MyClass
Filter a packagepackage com.example.testPackage

Filter by size. Tests should be annotated with SmallTest, MediumTest or LargeTest.

Cannot be used for exclusion.

size small|medium|large
Filter by annotationannotation com.example.MyAnnotation
 


# Run only one test method
--include-tests='class com.example.MyClass#testLogin'
 
# Run all but one test class
--exclude-tests='class com.example.MyClass'
 
# Run only the large tests
--include-tests='size large'

-n

--tunnel-identifier


Optional parameter to specify a Sauce Connect Proxy tunnel to use with the tests.
-n dev_tunnel
--tunnel-identifier=dev_tunnel
--data-center

Optional parameter to specify a Sauce Labs datacenter

Options:

  • us-west-1
  • eu-central-1
--data-center eu-central-1
--skip-download-junit-reports
Optional parameter to skip the download of the JUnit files at the end of the test suite.



-v

--verbose


Optional parameter to set the verbosity of console output. Valid options as DEBUG, INFO, WARN and ERROR

-h

--help


Print this command line reference to the console.
--version
Version information for Sauce Runner.

Exit Status Codes

Sauce Runner for Virtual Devices returns the following status codes based on test execution results.  

Status CodeDescription
0All the tests passed on all devices.
1

This status code can mean multiple things, it is important to refer to the logs to identify the problem.

  • One or more tests failed during execution
  • User error like an invalid path to test files or invalid arguments
  • Sauce Labs infrastructure error while executing the test

Example Using All Required Parameters for Espresso

./sauce-runner-virtual \
   -u test_user \
   -k 1234-1235 \
   -f espresso \
   -a ./helloworld.apk \
   -t ./espresso-test-suite.apk \
   -d 'deviceName=Samsung Galaxy S8 HD GoogleAPI Emulator,platformVersion=7.0' \
   -d 'deviceName=Google Pixel GoogleAPI Emulator,platformVersion=7.1'

This example tests the application helloworld.apk simultaneously on two emulators, Galaxy S8 and Pixel, using the Espresso test suite espresso-test-suite.apk.

Sauce-Runner-Virtual installs helloworld.apk and espresso-test-suite.apk on Sauce emulators and launches the Espresso test suite on both emulators at the same time. Sauce-Runner-Virtual exits when all the tests have completed. Sauce-Runner-Virtual exits with status code zero if all the tests passed, otherwise it exits with status code 1.