The Sauce Labs Cookbook

Sauce Headless

Front End Performance Testing

Insights

External Resources

More Info


Page tree
Skip to end of metadata
Go to start of metadata

The Sauce Labs real device cloud provides you with the ability to run either live or automated tests across hundreds of mobile device/OS combinations, in either a private or public real device cloud.

See the following sections for more information: 

What You'll Need

  • Before you can upload your app and leverage real devices, ensure that you meet the project support and requirements. See Automated Mobile App Testing Admin Guide for further details.
  • A Sauce Labs account.
  • The mobile app you wish to test. If you don't have an app and would like to try out our test functionality, feel free to download and use our Sauce Labs demo app.

Uploading and Accessing Your Apps on Real Devices

Select your preferred testing platform for uploading your app in order to view the instructions.

Installing Mobile Apps from a Remote Location

There may be situations where you want to install an app from a downloadable remote location (e.g., AWS S3 bucket, a GitHub repository). The app is completely removed from the real device after the test completes, providing an added layer of security for your app. 

Please review the following guidelines below before uploading your app:

  1. Make sure your app meets the prerequisite requirements for Android and iOS Mobile App Testing.
  2. Upload your app to the hosting location.
  3. Ensure Sauce Labs has READ access to the app URL.
  4. In your Appium test script, enter the URL for the app as the "app" desired capability. Below is an example Java snippet:

    Example Java Snippet
    caps.setCapability("app", "https://github.com/saucelabs/sample-app-mobile/releases/download/2.3.0/Android.SauceLabs.Mobile.Sample.app.2.3.0.apk?raw=true");

Installing your App on Private Devices

In some cases you may need to upload / install your app to a private device and also prevent the device from broad internet access while under test. The steps to achieve this are:

  • Upload your app to an internal git repository, or private hosting solution with the necessary permissions (e.g. Amazon S3 with a strict bucket policy).
  • Ensure the hosted app URL is available to the machine running the automated test.
  • Ensure that you've enabled the Require Sauce Connect/VPN setting in your organization's security settings.

Each Session is a "Fresh" Install

You will not be able to access information about different versions of your app because each session includes a "fresh" installation of your app. 

For Automated Appium Testing Only

You can only install remote apps for automated Appium testing. Espresso and Robotium automated tests are not supported. Live Testing is also not supported.

Uploading Mobile Apps with the Sauce Labs REST API

Below are some examples of how to use the Sauce Labs REST API to upload your mobile app to our App Storage and get your real device project started. See also: See Mobile App Testing API.

REST API Authentication

For APIs and authorization credentials, use the Sauce Labs Storage REST API (app.saucelabs.com).

A recommended best practice is to set your credentials as environment variables like so:

SAUCE_USERNAME='valid.username'
SAUCE_ACCESS_KEY='valid.key'

For specific instructions on how to set environment variables visit, the following links:

Supported Use Cases for Sauce Labs Real Device Testing

  • Execute Appium tests against a private real device hosted in the U.S., using your Sauce Labs username and access key

  • Use Sauce Storage, for Appium testing, as you usually do for emulators and simulators tests

  • Analyze Appium test executions, on Sauce Labs similar to the way you do it for desktop, emulators and simulators

  • Consume Real Device Cloud (RDC) API similar to the way you do for emulators and simulators (with applicable RDC settings)

Example Test Scripts

Unified Platform Appium Testing Example
private URL createUrl() throws MalformedURLException {
    return new URL("https://$SAUCE_USERNAME:$SAUCE_ACCESS_KEY@ondemand.us-west-1.saucelabs.com/wd/hub");
}
 
@BeforeEach
void setUp() throws MalformedURLException {
    DesiredCapabilities desiredCapabilities = new DesiredCapabilities();
    desiredCapabilities.setCapability("platformName", "iOS");
    IOSDriver driver = new IOSDriver(createUrl(), desiredCapabilities);
} 

App Storage and Data Center Endpoints

Below are some examples of how to use the Sauce Labs REST API to upload your mobile app to Sauce Storage. For details related to authorization credentials, see Data Center Endpoints

To connect to the real device cloud in your automated Appium tests, you'll need to use include either the EU or US storage endpoint in your test script. This example how to upload an app to App Storage in the US-West Data Center:

Mac OSX / Linux Example
$ curl -F "payload=@/Users/$SAUCE_USERNAME/Downloads/$FILE_NAME.ipa" -F "name=$FILE_NAME.ipa" -u "$SAUCE_USERNAME:$SAUCE_ACCESS_KEY"  'https://api.us-west-1.saucelabs.com/v1/storage/upload'

NOTE: For more detailed information on how to access the app in your automated test builds and use the Storage API, see App Storage.

Here are more examples using the EU and US endpoints:

US Data Center

Mac OSX / Linux Example
$ curl -u "$SAUCE_USERNAME:$SAUCE_ACCESS_KEY" -X POST -w "%{http_code}\n" \
-H "Content-Type: application/octet-stream" \
"https://saucelabs.com/rest/v1/storage/$SAUCE_USERNAME/Android.SauceLabs.Mobile.Sample.app.x.x.x.apk?overwrite=true" \
--data-binary @/path/to/Android.SauceLabs.Mobile.Sample.app.x.x.x.apk
Windows Example
> curl -u "%SAUCE_USERNAME%:%SAUCE_ACCESS_KEY% -X POST -w "%{http_code}\n" \
-H "Content-Type: application/octet-stream" \
"https://saucelabs.com/rest/v1/storage/%SAUCE_USERNAME%/Android.SauceLabs.Mobile.Sample.app.x.x.x.apk?overwrite=true" \
--data-binary @\path\to\Android.SauceLabs.Mobile.Sample.app.x.x.x.apk

EU Data Center

Mac OSX / Linux Example
$ curl -u "$SAUCE_USERNAME:$SAUCE_ACCESS_KEY" -X POST -w "%{http_code}\n" \
-H "Content-Type: application/octet-stream" \
"https://eu-central-1.saucelabs.com/rest/v1/storage/$SAUCE_USERNAME/Android.SauceLabs.Mobile.Sample.app.x.x.x.apk?overwrite=true" \
--data-binary @/path/to/Android.SauceLabs.Mobile.Sample.app.x.x.x.apk
Windows Example
> curl -u "%SAUCE_USERNAME%:%SAUCE_ACCESS_KEY%" -X POST -w "%{http_code}\n" \
-H "Content-Type: application/octet-stream" \
"https://eu-central-1.saucelabs.com/rest/v1/storage/%SAUCE_USERNAME%/Android.SauceLabs.Mobile.Sample.app.x.x.x.apk?overwrite=true" \
--data-binary @\path\to\Android.SauceLabs.Mobile.Sample.app.x.x.x.apk

Uploading Mobile Apps to TestObject

Legacy RDC only

This information applies only to TestObject, our legacy real device platform. For more information, see Legacy Real Device Platform Resources.

 For APIs and authorization credentials, use the Legacy Real Device Cloud REST API (app.testobject.com) and see Data Center Endpoints

Creating a Real Device Project on TestObject

When you create a project, you'll need to provide information about the website or app you want to test, and the device settings you want to use in your tests. You can also create versions of the project to reflect changes in the app or website throughout your development process. 

  1. Log into Sauce Labs, then click SAUCE APPS > Legacy RDC.
  2. Click the New App button.


  3. Choose the type of app you'd like to create. You can choose from an Android/iOS App project, a Mobile Website test project, or an Install Remote app (see App Center Integration for more info). For this example, we'll select Android/iOS App.


  4. Click Choose File, browse to the file containing your app, and upload it. 


    To view and download sample apps for iOS and Android, head to the Sauce Labs sample app page on GitHub

  5. Input your app name and version, then click Save.


  6. Edit the Device Settings as needed, then click Save.

Accessing Setup Instructions (Appium Test Automation Framework)

  1. Log into Sauce Labs, then click SAUCE APPS > Legacy RDC > select your app > AUTOMATED TESTING > Appium.


  2. Click Setup Instructions.


  3. Once you've created your app project, the setup instruction will prompt you to input required test script information:
    • API Key as desired capability


    • Data Center URL to be used as your WebDriver Hub URL


    • If you want to test on a specific mobile device – static device allocation – you'll need to provide the ID for that device. If you'd like to test on a broader array of devices – dynamic device allocation – you can specify your criteria via WebDriver desired capabilities (e.g., all Android devices with OS 9).


      To read more about Dynamic vs. Static Device Allocation, see Dynamic Device Allocation.

Versioning Real Device Projects on TestObject

Once you've created a real device project, you can create versions of it. Each of these versions will be stored in your project, and you can run tests against the current Active version, or a previous version. 

  1. Log into your account.
  2. In the Apps dashboard, select your project.

    When the project loads, you'll see an Active Versions panel. This provides information about the current version of the project.

    You can also:
    1. Click Upload New Version to add a new version of the application if this is a mobile application project.

    2. Click  Upload New URL to change the URL for a mobile website project.
    3. Click All Versions to view and select a version of your project to run your tests against. This will also change the current Active Version of the project. 


TestObject API Examples

US Data Center

Mac OSX / Linux Example
$ curl -u "$TEST_OBJECT_USERNAME:$TEST_OBJECT_API_KEY" -X POST \
"https://app.testobject.com:443/api/storage/upload" -H \
"Content-Type: application/octet-stream" --data-binary @/path/to/Android.SauceLabs.Mobile.Sample.app.x.x.x.apk
Windows Example
> curl -u "%TEST_OBJECT_USERNAME%:%TEST_OBJECT_API_KEY%" -X POST \
"https://app.testobject.com:443/api/storage/upload" -H \
"Content-Type: application/octet-stream" --data-binary @\path\to\Android.SauceLabs.Mobile.Sample.app.x.x.x.apk

EU Data Center

Mac OSX / Linux Example
$ curl -u "$TEST_OBJECT_USERNAME:$TEST_OBJECT_API_KEY" -X POST \
"https://app.testobject.com:443/api/storage/upload" -H \
"Content-Type: application/octet-stream" --data-binary @/path/to/Android.SauceLabs.Mobile.Sample.app.x.x.x.apk
Windows Example
> curl -u "%TEST_OBJECT_USERNAME%:%TEST_OBJECT_API_KEY%" -X POST \
"https://app.testobject.com:443/api/storage/upload" -H \
"Content-Type: application/octet-stream" --data-binary @\path\to\Android.SauceLabs.Mobile.Sample.app.x.x.x.apk




Configuring Appium Tests for Real Devices

Different Behaviors from Local Appium Server Testing

Some Appium capabilities behave differently when running Appium tests in the real device cloud instead of against a local Appium server, including:

  • All the emulator-only capabilities will not work.
  • The app capability is always overwritten, and points to the app file you uploaded to our system.
  • The noReset capability only works if device caching is enabled. By default, noReset is set to false.
  • Some Appium capabilities might not be supported. You can find a list in the last section of this topic.

Different setups might have slightly different ways of handling capabilities and/or different requirements. You should also check the examples for your specific setup to make sure you are providing all of the required capabilities.

Here are some tips for configuring Appium for mobile native app tests:

Setting appiumVersion

If you omit the appiumVersion in your test configuration, your test will be running with our default Appium version. We recommend that you specify one of the newer Appium versions that provides a more extended API and fixes to known bugs. 

Checking the Appium Version for Your Test

  1. Log in to Sauce Labs.
  2. Find and select the test that you ran using Appium to view the Test Details page.
  3. Click the Metadata tab.
  4. Look for the Logs row and select Appium Log.
    The first line should indicate the Appium version. For example, 2014-05-05T17:45:07.541Z - info: Welcome to Appium v1.3.6.

Setting Browser Name

When testing a native mobile app, the value for browserName is an empty string, as in caps.setCapability("browserName"""); 

Setting the Location of Your Mobile App

If the app you want to test has been uploaded to a location other than Sauce Storage, you need to specify this location for app, and make sure that this location is accessible to Sauce Labs browsers. For example, caps.setCapability("app","sauce-storage:mapp.zip");

Setting automationName for Android Apps

If you're testing a native mobile app against Android versions 4.0 - 4.1, or a hybrid mobile against Android versions 4.0 - 4.2, you need to set the capability "automationName","selendroid". These Android versions are only supported via Appium’s bundled version of Selendroid, which utilizes Instrumentation. Later versions of Android are supported via Appium’s own UiAutomator library.

Appium Capabilities for Sauce Labs Real Devices

Below are some examples of how to configure Appium tests for Real Devices. Visit our sample test frameworks GitHub repository for more detailed language-specific examples.

For Example Purposes Only

The code in these scripts is provided on an "AS-IS” basis without warranty of any kind, either express or implied, including without limitation any implied warranties of condition, uninterrupted use, merchantability, fitness for a particular purpose, or non-infringement. Your tests and testing environments may require you to modify these scripts. Issues regarding these scripts should be submitted through GitHub. These scripts are not maintained by Sauce Labs Support.

Using Regular Expressions for deviceName

The examples below use regular expressions for setting the deviceName, because a matching device must be present in your account in order for the test to run.

For example:

  • "iPhone.*""iPhone .*" will allocate any iPhone. "
  • .*nexus.*" will allocate any device with the word "nexus" in its display name.
  • "iPhone [67]" or "iPhone [6-7]" will allocate either "iPhone 7" or "iPhone 6" device.
  • "iPhone [67]S" or "iPhone [6-7]S" will allocate either "iPhone 7S" or "iPhone 6S" device. "iPhone 7.*" will allocate "iPhone 7" or "iPhone 7S", or any device that starts with the display name "iPhone 7"

Regular expressions are case insensitive, for example, "iphone .*S" and "IPHONe .*s" are the same.

Required Appium Capabilities for Real Devices

Your Appium scripts for real device testing must include these capabilities. Check out the example script in Dynamic Allocation Examples for examples of these capabilities and their values. 

CapabilityData TypeSummary
username
StringThe Sauce Labs "Username" you use to authenticate your tests. You can find this under Account > User Settings.
accessKey
StringThe Sauce Labs "Access Key" you use to authenticate your tests. You can find this under Account > User Settings.
Appium EndpointString

This capability refers the specific region/data-center that your desired device is located. There are currently two options for real devices:

app
String

This capability refers to the location of your mobile application. Please refer to the following page to learn more about how to upload your application to real devices.

Please note that if you're running a mobile browser test, this capability can be left blank.

platformName
String

The type of mobile platform to use in your tests, for example "Android" or "iOS".

Values are Not Case Sensitive

The values for capabilities are case-insensitive, so "android" is the same as "Android", and "ios" is the same as "iOS"

platformVersion
StringThe platform version to use in your tests, for example "4" or "4.1". This is a substring match. You can specify both major versions and incremental versions of an operating system. If you set only a major version, for example 4, you will also have access to all devices running incremental versions, such as "4.1"," 4.2", "4.2.1", "4.4.4". This also extends to minor and point versions. If you specify "11.4" it will match "11.4.0", "11.4.1", etc.
deviceName

If you only want to test on a particular type of device, such as a "Samsung S7" or an "iPhone", set deviceName to the display name of the device as shown in the Real Devices menu of Sauce Labs.

Using Regular Expressions for dynamic allocation of deviceName

You can use regular expressions for setting the deviceName dynamically. For example:

"iPhone.*", "iPhone .*" will allocate any iPhone. "

.*nexus.*" will allocate any device with the word "nexus" in its display name.

"iPhone [67]" or "iPhone [6-7]" will allocate either "iPhone 7" or "iPhone 6" device.

"iPhone [67]S" or "iPhone [6-7]S" will allocate either "iPhone 7S" or "iPhone 6S" device. "iPhone 7.*" will allocate "iPhone 7" or "iPhone 7S", or any device that starts with the display name "iPhone 7"

Regular expressions are case insensitive, for example, "iphone .*S" and "IPHONe .*s" are the same.

To learn more please visit: Dynamic Device Allocation.

Optional Appium Capabilities for Real Devices

CapabilityData TypeSetting
appiumVersion
String

The version of Appium you want to use in your tests. If you leave this capability blank, or omit it completely, Sauce Labs will defer to the default Appium version installed on our platform.

name
StringA name for your test, to make it easier to find individual tests.
build
StringIf you're running several tests within a test suite, you can add a build tag to group these tests together.
tag
StringTags are useful for collaborating with teams and also for filtering results on your Sauce Labs Test Results and Archive pages.
newCommandTimeout
Float

Automated tests time out by default after 60 seconds of inactivity. You can increase this to 90 seconds with the newCommandTimeout desired capability.

commandTimeout
Float

Custom timeout in milliseconds for the WebDriverAgent backend. The maximum allowed timeout is 600 seconds.

When creating an Appium session on the real device cloud for any iOS device, it is possible to set the desired capability commandTimeouts. It will set a timeout for the WebDriverAgent backend, which will prevent automated test sessions from being blocked when the Appium WebDriverAgent backend freezes unexpectedly. Sauce Labs sets a default of 60 seconds for this capability.

tabletOnly
BooleanYou can use the tabletOnly desired capability to select only tablet devices by setting this capability to "true".
phoneOnly
BooleanYou can use the  phoneOnly desired capabilities to select only phone devices by setting this capability to "true".
privateDevicesOnly
BooleanIf you have access to both private and public devices, you can request allocation of private devices only by setting this capability to "true".
automationName
String

You can pass an automation value to use native automation frameworks (supported for Android native apps only). For Example:

UiAutomator2 Example
capabilities.setCapability("automationName", "uiautomator2");
cacheId
String
Specifying the desired capability cacheId (a randomized String) will allocate the device to you for 10 seconds after a test finishes. You can then re-use the same device in your next test. However the value for cacheId must be the same for all test methods that you want to run on the cached device. In addition, the application and project ID used for the tests must remain the same, along with the values for these capabilities:
  • deviceName
  • platformName
  • platformVersion
  • tabletOnly
  • phoneOnly
  • privateDevicesOnly
  • automationName
  • autoGrantPermissions
  • appiumVersion

Device Management Awareness

We recommend reviewing Device Management for Real Devices to learn more about how Sauce Labs manages device allocation, device caching, and device cleanup.

noReset
BooleanYou can also use the cacheId capability in conjunction with the standard noReset Appium capability. In the default case, where noReset is set to false, your application will be uninstalled and reinstalled after every test. If noReset is set to true, the application you are testing won't be reinstalled after every test run. This might save you further time, and is suitable for test setups that require the application's state to be reset between tests. Note that then cacheId is set, no device cleaning will take place in between sessions, regardless of noReset value.
carrierConnectivityOnly
BooleanYou can use the carrierConnectivityOnly desired capability to allocate only devices that are connected to a carrier network.
recordDeviceVitals
BooleanDevice vitals are a collection of the mobile device performance data taken in real time during test execution. Vitals includes CPU utilization, memory consumption, network usage for both wifi and carrier connectivity where applicable, file operation and more. Measuring device vitals during test execution provides insights for analyzing app performance during operation.
crosswalkApplication
BooleanAs described in Appium Issue 4597 and ChromeDriver Issue 2375613002, mobile tests using Crosswalk will fail because because of attempts to connect to the wrong socket on the device. Sauce Labs has developed a patched version of ChromeDriver that will work with Crosswalk. You can specify to use this patched version with the crosswalkApplication desired capability.
autoGrantPermissions
BooleanBy default, applications are installed on devices in the Sauce Labs real device cloud with autoGrantPermissions capability set to true. As long as the API number of the device is equal to 23 or higher, you can disable this by explicitly setting autoGrantPermissions to false.
enableAnimations
Boolean

By default, animations are disabled on real devices. You can enable animations for private devices only with the enableAnimations capability.

NOTE: The enableAnimations capability will only work if privateDevicesOnly is set to true.

Unsupported Appium Capabilities

Some Appium capabilities are not yet supported for real devices. If you have any questions or concerns about unsupported capabilities, please contact your Customer Success Manager, refer to the Real Devices FAQ page, or submit a help desk ticket to: support@saucelabs.com.

CapabilityPlatformNotes
installApp
iOS, AndroidManaged by RDC differently, but cannot be used inside an Appium test as part of the routine.
removeApp
iOS, AndroidManaged by RDC differently, but cannot be used inside an Appium test as part of the routine.
Edit Timezone
iOS, AndroidAppium does not provide a capability to edit the timezone of a device in automated testing on real devices. See Appium Capabilities for Emulators and Simulators for information about timezone capabilities in virtual device testing.

Override Settings Capabilities

These are custom capabilities developed by Sauce Labs to override settings that are enabled during app configuration. 

CapabilityPlatformDescription
resigningEnabled
iOS, AndroidEnables the resigning (iOS) or instrumentation (Android) of apps on the Sauce Labs side, which enables the usage of the other capabilities listed in this table.
sauceLabsImageInjectionEnabled
iOS, AndroidEnables the image injection feature.
sauceLabsBypassScreenshotRestriction
AndroidBypasses the restriction on taking screenshots for secure screens (i.e., secure text entry).
allowTouchIdEnroll
iOSEnables the interception of biometric input, allowing the test to simulate Touch ID interactions (not a Sauce Labs-specific capability).
groupFolderRedirectEnabled
iOS

Enables the use of the app's private app container directory instead of the shared app group container directory.
For testing on the Real Device Cloud, the app gets resigned, which is why the shared directory is not accessible.

systemAlertsDelayEnabled
iOSDelays system alerts, such as alerts asking for permission to access the camera, to prevent app crashes at startup.


Examples of Appium Capabilities for Real Devices

Below are examples of an iPhone project using iOS version 12.2.

DesiredCapabilities caps = DesiredCapabilities();
	caps.setCapability("username", "SAUCE_USERNAME");
	caps.setCapability("accessKey", "SAUCE_ACCESS_KEY");
	caps.setCapability("deviceName","iPhone .*");
	caps.setCapability("deviceOrientation", "portrait");
	caps.setCapability("platformVersion","12.2");
	caps.setCapability("platformName", "iOS");
	caps.setCapability("browserName", "");
	caps.setCapability("app", "storage:filename=<file-name>");
caps = {}
caps['username'] = "SAUCE_USERNAME"
caps['accessKey'] = "SAUCE_ACCESS_KEY"
caps['browserName'] = ""
caps['deviceName'] = "iPhone .*"
caps['deviceOrientation'] = "portrait"
caps['platformVersion'] = "12.2"
caps['platformName'] = "iOS"
caps['app'] = "storage:filename=<file-name>"
caps = {};
caps['username'] = 'SAUCE_USERNAME';
caps['accessKey'] = 'SAUCE_ACCESS_KEY';
caps['browserName'] = '';
caps['deviceName'] = 'iPhone .*';
caps['deviceOrientation'] = 'portrait';
caps['platformVersion'] = '12.2';
caps['platformName'] = 'iOS';
caps['app'] = 'storage:filename=<file-name>';
caps = Selenium::WebDriver::Remote::Capabilities()
caps['username'] = 'SAUCE_USERNAME'
caps['accessKey'] = 'SAUCE_ACCESS_KEY'
caps['deviceName'] = 'iPhone .*'
caps['deviceOrientation'] = 'portrait'
caps['platformVersion'] = '12.2'
caps['platformName'] = 'iOS'
caps['browserName'] = ''
caps['app'] = 'storage:filename=<file-name>'
DesiredCapabilities caps = new DesiredCapabilities();
	caps.SetCapability("username", "SAUCE_USERNAME");
	caps.SetCapability("accessKey", "SAUCE_ACCESS_KEY");
	caps.SetCapability("deviceName", "iPhone .*");
	caps.SetCapability("deviceOrientation", "portrait");
	caps.SetCapability("platformVersion", "12.2");
	caps.SetCapability("platformName", "iOS");
	caps.SetCapability("browserName", "");
	caps.SetCapability("app", "storage:filename=<file-name>");

Below are examples of an Samsung Galaxy project using Android version 8.1.

DesiredCapabilities caps = DesiredCapabilities();
	caps.setCapability("username", "SAUCE_USERNAME");
	caps.setCapability("accessKey", "SAUCE_ACCESS_KEY");
	caps.setCapability("deviceName","Samsung.*Galaxy.*");
	caps.setCapability("deviceOrientation", "portrait");
	caps.setCapability("browserName", "");
	caps.setCapability("platformVersion","8.1");
	caps.setCapability("platformName","Android");
	caps.setCapability("app", "sauce-storage:<upload_filename>");
caps = {}
caps['username'] = "SAUCE_USERNAME"
caps['accessKey'] = "SAUCE_ACCESS_KEY"
caps['deviceName'] = "Samsung.*Galaxy.*"
caps['deviceOrientation'] = "portrait"
caps['platformVersion'] = "8.1"
caps['platformName'] = "Android"
caps['app'] = "sauce-storage:<upload_filename>"
caps = {};
caps['username'] = 'SAUCE_USERNAME';
caps['accessKey'] = 'SAUCE_ACCESS_KEY';
caps['deviceName'] = 'Samsung.*Galaxy.*';
caps['deviceOrientation'] = 'portrait';
caps['browserName'] = '';
caps['platformVersion'] = '8.1';
caps['platformName'] = 'Android';
caps['app'] = 'sauce-storage:<upload_filename>';
caps = Selenium::WebDriver::Remote::Capabilities()
caps['username'] = 'SAUCE_USERNAME'
caps['accessKey'] = 'SAUCE_ACCESS_KEY'
caps['deviceName'] = 'Samsung.*Galaxy.*'
caps['deviceOrientation'] = 'portrait'
caps['browserName'] = ''
caps['platformVersion'] = '8.1'
caps['platformName'] = 'Android'
caps['app'] = 'sauce-storage:<upload_filename>'
DesiredCapabilities caps = new DesiredCapabilities();
	caps.SetCapability("username", "SAUCE_USERNAME");
	caps.SetCapability("accessKey", "SAUCE_ACCESS_KEY");
	caps.SetCapability("deviceName", "Samsung.*Galaxy.*");
	caps.SetCapability("deviceOrientation", "portrait");
	caps.SetCapability("browserName", "");
	caps.SetCapability("platformVersion", "8.1");
	caps.SetCapability("platformName", "Android");
	caps.SetCapability("app", "sauce-storage:<upload_filename>");

Configuring Tests for Native Mobile Apps vs. Hybrid Mobile Apps

iPhone 6 Real Device

{
deviceName:'iPhone 6 Device',
platformName:'iOS',
platformVersion:'8.4',
app:'sauce-storage:SampleIOSApp.ipa',
"appium-version":"1.4.16"
}

Samsung Galaxy S5 Real Device

{
deviceName:'Samsung Galaxy S5 Device',
platformVersion:'4.4',
platformName:'Android',
"appActivity": ".ContactManager",
"appPackage": "com.example.android.contactmanager",
app:"http://saucelabs.com/example_files/ContactManager.apk",
"appium-version":"1.4.16"
}

Samsung Galaxy S4

{
deviceName:'Samsung Galaxy S4 Device',
platformVersion:'4.4',
platformName:'Android',
"appActivity": ".ContactManager",
"appPackage": "com.example.android.contactmanager",
app:"http://saucelabs.com/example_files/ContactManager.apk",
"appium-version":"1.4.16"
}

Example Appium Scripts for Real Device Tests

These Appium scripts for iOS and Android mobile app tests on real devices can help streamline your testing process. 

iOS / Real Devices

These examples use the pytest test framework to run tests on real devices. Feel free to clone these scripts directly from GitHub, and follow the instructions in the README file.

Python Config

This script initializes the test fixtures, as well as the prerequisite and post-requisite test tasks.

 conftest.py

Not found

Could not find the given branch refs/remotes/origin/master

Test Objects

These scripts represents the individual tests. Click below to see the script:

 test_invalid_login.py

Not found

Could not find the given branch refs/remotes/origin/master

 test_valid_login.py

Not found

Could not find the given branch refs/remotes/origin/master

Android / Real Devices

These examples use the pytest test framework to run tests on real devices. Feel free to clone these scripts directly from GitHub, and follow the instructions in the README file.

Python Config

This script initializes the test fixtures, as well as the prerequisite and post-requisite test tasks.

 conftest.py

Not found

Could not find the given branch refs/remotes/origin/master

Test Objects

These scripts represents the individual tests. Click below to see the script:

 test_invalid_login.py

Not found

Could not find the given branch refs/remotes/origin/master

 test_valid_login.py

Not found

Could not find the given branch refs/remotes/origin/master


Using TestObject (Legacy Real Device Platform)

These examples are specifically for use with TestObject, our legacy real device cloud platform (SAUCE APPS > Legacy RDC). 

Examples of an iPhone project using iOS version 12.2.

DesiredCapabilities caps = DesiredCapabilities();
	caps.setCapability("testobject_api_key", "project_api_key");
	caps.setCapability("testobject_app_id", "1");
	caps.setCapability("deviceName","iPhone .*");
	caps.setCapability("deviceOrientation", "portrait");
	caps.setCapability("platformVersion","12.2");
	caps.setCapability("platformName", "iOS");
	caps.setCapability("browserName", "");
caps['browserName'] = ""
caps['testobject_api_key'] = "project_api_key"
caps['testobject_app_id'] = "1"
caps['deviceName'] = "iPhone .*"
caps['deviceOrientation'] = "portrait"
caps['platformVersion'] = "12.2"
caps['platformName'] = "iOS"
caps['browserName'] = '';
caps['testobject_api_key'] = "project_api_key";
caps['testobject_app_id'] = "1";
caps['deviceName'] = 'iPhone .*';
caps['deviceOrientation'] = 'portrait';
caps['platformVersion'] = '12.2';
caps['platformName'] = 'iOS';
caps = Selenium::WebDriver::Remote::Capabilities()
caps['testobject_api_key'] = 'project_api_key'
caps['testobject_app_id'] = '1'
caps['deviceName'] = 'iPhone .*'
caps['deviceOrientation'] = 'portrait'
caps['platformVersion'] = '12.2'
caps['platformName'] = 'iOS'
caps['browserName'] = ''
DesiredCapabilities caps = new DesiredCapabilities();
	caps.SetCapability("deviceName", "iPhone .*");
	caps.SetCapability("testobject_api_key", "project_api_key");
	caps.SetCapability("testobject_app_id", "1");
	caps.SetCapability("deviceOrientation", "portrait");
	caps.SetCapability("platformVersion", "12.2");
	caps.SetCapability("platformName", "iOS");
	caps.SetCapability("browserName", "");

Below are examples of an Samsung Galaxy project using Android version 8.1

DesiredCapabilities caps = DesiredCapabilities();
	caps.setCapability("testobject_api_key", "project_api_key");
	caps.setCapability("testobject_app_id", "1");
	caps.setCapability("deviceName","Samsung.*Galaxy.*");
	caps.setCapability("deviceOrientation", "portrait");
	caps.setCapability("browserName", "");
	caps.setCapability("platformVersion","8.1");
	caps.setCapability("platformName","Android");
caps = {}
caps['testobject_api_key'] = "project_api_key"
caps['testobject_app_id'] = "1"
caps['deviceName'] = "Samsung.*Galaxy.*"
caps['deviceOrientation'] = "portrait"
caps['platformVersion'] = "8.1"
caps['platformName'] = "Android"
caps = {};
caps['testobject_api_key'] = 'project_api_key';
caps['testobject_app_id'] = '1';
caps['deviceName'] = 'Samsung.*Galaxy.*';
caps['deviceOrientation'] = 'portrait';
caps['browserName'] = '';
caps['platformVersion'] = '8.1';
caps['platformName'] = 'Android';
caps = Selenium::WebDriver::Remote::Capabilities()
caps['testobject_api_key'] = 'project_api_key'
caps['testobject_app_id'] = '1'
caps['deviceName'] = 'Samsung.*Galaxy.*'
caps['deviceOrientation'] = 'portrait'
caps['browserName'] = ''
caps['platformVersion'] = '8.1'
caps['platformName'] = 'Android'
DesiredCapabilities caps = new DesiredCapabilities();
	caps.SetCapability("testobject_api_key", "project_api_key");
	caps.SetCapability("testobject_app_id", "1");
	caps.SetCapability("deviceName", "Samsung.*Galaxy.*");
	caps.SetCapability("deviceOrientation", "portrait");
	caps.SetCapability("browserName", "");
	caps.SetCapability("platformVersion", "8.1");
	caps.SetCapability("platformName", "Android");

Best Practices and Reporting Test Results

Now that you've been able to get a test running on Sauce, there are a few other modifications you can make to your tests. These example scripts show how to:

You can find more information under Best Practices for Running Tests and Setting Test Status to Pass or Fail.

Features specific to our real device cloud include:

  • Support for multiple frameworks (i.e., Appium, Robotium, Espresso, and XCUITest)
  • IPSec VPN secure connections to private clouds
  • Devices cleaning process between sessions to ensure maximum privacy
  • Carrier Network Connectivity (devices with SIM cards)

As a reminder – from the Sauce Labs platform, you'll have access both our virtual device cloud (emulators, simulators) and real device cloud. This allows you to use the same features – APIs, endpoints, reporting, secure tunnels, analytics, and more – for both clouds.

Monitoring Real Device Performance for Appium Tests

By including the desired capability recordDeviceVitals in your Appium test script, you can collect performance statistics for the real devices used in your tests, including CPU, Network, Memory, and Thread performance. This topic describes how to set up the desired capability in your tests, and collect the statistics when the test completes. 

Setting recordDeviceVitals

Include this setting as part of the desired capabilities in your Appium test script:

capabilities.setCapability("recordDeviceVitals", true);

Collecting Performance Metrics

  1. When your test completes, log into the Real Device Testing web interface, and select the app that you used in the test from your dashboard. 
  2. Click Automated Testing > Appium.
  3. Select the Test Results for your test. 
  4. Under Mobile Vitals, click the link to download the CSV file containing your test performance metrics. 

Performance Metrics for Android Devices

The CSV file will contain these performance metrics for Android devices. 

Metric

Description

unix_epoch_milliseconds 

Unix epoch timestamp in milliseconds, which can be matched to events within your automated tests
device_local_time Device’s local date and time
cpu_totalSystem-wide CPU usage in percentage across all CPU cores. 4 cores at max use would be shown as a value of 400 %
cpu_userCPU usage for user processes in percentage across all CPU cores. 4 cores at max use would be shown as a value of 400 %
cpu_kernelAndroid OS CPU usage in percentage across all CPU cores. 4 cores at max use would be shown as a value of 400 %
n_processors Amount of available CPU cores. Use this to divide the cpu values into per core
n_threadsTotal threads in use by the app
memory_size_kb Total memory currently used by device in kilobytes
memory_resident_kbMemory currently in use by application in kilobytes
memory_shared_kbAnonymous shared memory currently in use by system shared between application(s) and system

network_wifi_receive_b

Data in bytes received over wifi connection
network_wifi_sent_bData in bytes sent over wifi connection
network_mobile_receive_bData in bytes received from the mobile carrier network
network_mobile_sent_b Data in bytes sent over mobile carrier network

Performance Metrics for iOS Devices

The CSV file will contain these performance metrics for iOS devices.

Metric

Description

Metric

Description

Unix_epoch_milliseconds

Unix epoch timestamp in milliseconds, which can be matched to events within your automated tests
device_local_time Device’s local date and time
cpu_totalSystem-wide CPU usage in percentage. Values between 0 and 100%.
cpu_userUser processes CPU usage in percentage. Values between 0 and 100%.
cpu_processApp under test CPU usage in percentage. Values between 0 and 100%.
n_threadsTotal threads in use by the process
process_memory_used_b Memory used in bytes by the app that is under test
net_bytes_inTotal data in bytes received
net_bytes_outTotal data in bytes sent
net_packets_inTotal packets received
net_packets_outTotal packets sent
disk_write_opsDisk write operations during time period
disk_bytes_writtenBytes written to disk during time period

Additional Resources