The Sauce Labs Cookbook

Sauce Headless

Front End Performance Testing

Insights

External Resources

More Info


Page tree
Skip to end of metadata
Go to start of metadata

The Sauce Labs Training repository on GitHub contains multiple examples of Espresso test configurations with real devices and emulators on the Sauce Labs cloud.

Espresso with Android Real Devices

This repo presents an example of executing Espresso test cases on Sauce Labs Real Devices as part of the native framework capability.

Table of Contents

  1. Prerequisites
  2. Setup
  3. Execution
  4. Examples
    1. Example 1 - Minimal configuration for all tests
    2. Example 2 - Minimal configuration for 2 tests
    3. Example 3 - Run each test on its own device
    4. Example 4 - Run in parallel on hard-coded devices
    5. Example 5 - Parallel execution using dynamic devices
    6. Example 6 - Parallel execution by platform version
    7. Example 7 - Single file test annotation
    8. Example 8 - Multi file test annotation
    9. Example 9 - Single execution by using an already uploaded app

Prerequisites

The test runner used by Sauce Labs to execute the Espresso tests is a downloadable Java jar file. The current release is provided in this release but can also be downloaded from the Sauce Labs Wiki.

The scripts in this repo were developed on macOS and expect JDK 8 or higher to be installed and available.

Setup

A prebuilt native Android application and it's Espresso test cases are included in this repo. Source to this application can be found on GitHub in this folder. The two test-classes can be found here:

The application is bundled in SauceLabs.Mobile.Sample.Espresso.App.apk and the tests cases are bundled in SauceLabs.Mobile.Sample.Espresso.Tests.apk.

Screenshots will automatically be uploaded to Sauce Labs and are added by Spoon and the spoon-gradle-plugin. See also the above mentioned classes for the implementation of Spoon.

Running on Sauce Labs

When you start running Espresso tests for the first time on Sauce Labs, you will need to create a new project. This only needs to be done once!

The SauceLabs.Mobile.Sample.Espresso.App.apk file needs to be uploaded to Sauce Labs and an "App Project" needs to be created to reference the app from our test runner. Follow these steps to upload the APK and create an App Project:

  1. Sign into Sauce Labs at https://app.saucelabs.com

  2. Navigate to SAUCE APPS → Legacy RDC → + New App → Android/IOS App.

  3. Follow the prompts to upload the SauceLabs.Mobile.Sample.Espresso.App.apk file.

  4. The App name will be the name of the project. Give it a descriptive name and replace spaces with for example dashes.

  5. Once created, click on the new Espresso-Swag-Labs-Mobile-App project.

  6. Navigate to AUTOMATED TESTING → Espresso/Robotium → Setup Instructions.

  7. Copy the value of thetestobject_api_keyand set it as an environment variable in the terminal you are executing your tests from:

    # For OSX/Linux
    export ESPRESSO_API_KEY='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

Execution

There are 2 ways of executing Espresso tests on Sauce Labs Real Devices:

  1. Through the command line, see this Sauce Labs Wiki page
  2. Through a configuration file, see this Sauce Labs Wiki page

This project has been set up in such a way that it will use yaml-configuration files, which will be explained in the Examples section.

Executing a test from this repository is a matter of running run-runner.sh and passing in the example number (1 ... 9):

$ ./run-runner.sh 1

Examples

There are nine examples provided, starting from the most basic and moving up to more advanced approaches. The mindset behind the progression is the desire to shard multiple tests cases across multiple devices using parallel execution to ultimately reduce the amount of time needed to execute all tests.

Example 1 - Minimal configuration for all tests

runner-ex1.yml provides a minimum configuration needed to run the Espresso tests. In this example, all test cases are executed on a single, available device in sequential order.

Example 2 - Minimal configuration for 2 tests

runner-ex2.yml provides a minimum configuration needed to run the Espresso tests. In this example, there are two test cases executed on a single, available device in sequential order.

Example 3 - Run each test on its own device

runner-ex3.yml breaks apart the execution of the two test cases such that they can run in parallel on separate devices that are available (ie., not in use) in the pool.

Example 4 - Run in parallel on hard-coded devices

runner-ex4.yml modifies the second example by specifying which device in the pool to execute each test on. Still uses parallel execution.

Example 5 - Parallel execution using dynamic devices

runner-ex5.yml uses the deviceNameQuery capability to look for available devices using wildcard names. This example demonstrates the ability to run a specified test(s) on a pool devices that are configured the same but have different names for parallel processing.

Example 6 - Parallel execution by platform version

runner-ex6.yml uses only the platformVersion field to select an available device. Demonstrates the ability to pick a specific version of Android from the pool of devices for executing tests in parallel.

Example 7 - Single file test annotation

runner-ex7.yml uses Espresso test annotation. Demonstrates the ability to run four tests based on the @ErrorFlow-annotation in a single class, see here.

Example 8 - Multi file test annotation

runner-ex8.yml uses Espresso test annotation. Demonstrates the ability to run two tests based on the @HappyFlow-annotation which are found in two classes, see here and here.

Example 9 - Single execution by using an already uploaded app

runner-ex9.yml uses only uses the appId of an already uploaded app. This can be retrieved by going to:

  1. Sign into Sauce Labs at https://app.saucelabs.com
  2. Navigate to SAUCE APPS → Legacy RDC → Your App Project
  3. When you are on the dashboard go to All Versions → Select a number from the app

Espresso with Android Emulators

This repo presents an example of executing Espresso test cases on Sauce Labs Emulators as part of the native framework capability.

Table of Contents

  1. Prerequisites
  2. Setup
  3. Execution
  4. Examples
    1. Example 1 - Configuration for all tests
    2. Example 2 - Configuration for 2 tests
    3. Example 3 - Run one test on 2 devices
    4. Example 4 - Exclude 1 test

Prerequisites

The test runner used by Sauce Labs to execute the Espresso tests is a downloadable file. The current release is provided in this release but can also be downloaded from the Sauce Labs Wiki.

The scripts in this repo were developed on macOS and expect JDK 8 or higher to be installed and available.

Setup

A prebuilt native Android application and its Espresso test cases are included in this repo. Source to this application can be found on GitHub.

The application is bundled in SauceLabs.Mobile.Sample.Espresso.App.apk and the tests cases are bundled in SauceLabs.Mobile.Sample.Espresso.Tests.apk.

Make sure that you've added the following to your environment variables:

# For OSX/Linux
export SAUCE_USERNAME=********
export SAUCE_ACCESS_KEY=*******

Execution

Executing a test from this repository is a matter of running runner-ex{number}.sh where {number} is the number of the example (1 ... 4).

If you run this for the first time on your Mac you might get a warning that the sauce-runner-virtual is not safe, then go to System Preferences > Security and Privacy > Allow Anyway.

Accept blocked runner

Examples

There are six examples provided starting from the most basic and moving up to more advanced approaches.

Example 1 - Configuration for all tests

runner-ex1.sh: In this example, all test cases are executed on a single emulator in sequential order.

Example 2 - Configuration for 2 tests

runner-ex2.sh: In this example, there are two test cases executed on a single device in sequential order.

Example 3 - Run one test on 2 devices

runner-ex3.sh: In this example, there is one test which is executed on 2 devices.

Example 4 - Exclude 1 test

runner-ex4.sh: In this example, all test cases are executed on a single emulator in sequential order except for 1.

Example 5 - Single file test annotation

runner-ex5.sh: Demonstrates the ability to run four tests based on the @ErrorFlow-annotation in a single class, see here.

Example 6 - Multi file test annotation

runner-ex6.sh: Demonstrates the ability to run two tests based on the @HappyFlow-annotation which are found in two classes, see here and here.