The Sauce Labs Cookbook

The Sauce Labs Cookbook

Sauce Headless

Front End Performance Testing

Insights

External Resources

More Info

Page tree
Skip to end of metadata
Go to start of metadata

You can manage your tests more effectively from your Dashboard and Archives with annotations. The Sauce Labs REST API includes an update_job method that you can use to set a name, tags, pass/fail status, and custom data for your test after it runs. To automate test annotation with this method, you'll want to create a simple set of functions to perform the put request for you. We've developed a Java library to do just that, with examples for Python and Ruby on GitHub.

Adding Pass/Fail Status and Build Numbers to Test Results with Frameworks

In addition to using the REST API to set these annotations once your test completes, you can use one of the Sauce Labs test framework examples to set these and other annotations for you automatically as part of the test execution.

 See the following sections for more information:

Desired Capabilities for Annotation

You can set the following test configuration options to keep track of your jobs:

OptionDescriptionKeyValue TypeExample

Test Names

Used to record test names for jobs and make it easier to find individual tests
name
string

"name" : "my example name"

Build Numbers

Used to associate jobs with a build number or app version, displayed on the Dashboard and Archives viewsbuildstring"build": "build-1234"

Tagging

User-defined tags for grouping and filtering jobs in the Dashboard and Archivestagslist"tags": ["tag1","tag2","tag3"]

Pass/Fail Status

Selenium and Appium handle sending commands to control a browser or app, but don't report to the server whether a test passed or failed. To record pass/fail status on Sauce, set the passed flag on the job.

Since you can't know in advance whether a test passed or failed, this flag can't be set in the initial configuration.

passedboolean"passed": "true"

Custom Data

User-defined custom data that will accept any valid JSON object, limited to 64KB in size.customDataobject
"customData": {"release": "1.0", 
                "commit": "0k392a9dkjr", 
                "staging": true, 
                "execution_number": 5, 
                "server": "test.customer.com"}

Examples

Setting Job Info with cURL for OS X/Linux
curl -X PUT \
-s -d '{"passed": true}' \ 
-u YOUR_USERNAME:YOUR_ACCESS_KEY \
https://saucelabs.com/rest/v1/YOUR_USERNAME/jobs/YOUR_JOB_ID 
Setting Job Info with cURL for Windows
curl -X PUT 
-s -d "{\"passed\": true}"
-u YOUR_USERNAME:YOUR_ACCESS_KEY 
https://saucelabs.com/rest/v1/YOUR_USERNAME/jobs/YOUR_JOB_ID
Setting Job Info Using JSON
{
      "name": "my job name",
      "passed": true,
      "public": "public",
      "tags": ["tag1", "tag2", "tag3"],
      "build": 234,
      "customData": {
          "release": "1.0",
          "server": "test.customer.com"
      }
  } 

Video: Annotating Tests Using the Sauce Labs REST API

This Tech Tip video shows you how to annotate your tests with meaningful information using the Sauce Labs REST API.

More Information