You can manage your tests more effectively from your Dashboard and Archives with annotations. The Sauce Labs REST API includes an update_job method that you can use to set a name, tags, pass/fail status, and custom data for your test after it runs. To automate test annotation with this method, you'll want to create a simple set of functions to perform the put
request for you. We've developed a Java library to do just that, with examples for Python and Ruby on GitHub.
Adding Pass/Fail Status and Build Numbers to Test Results with Frameworks
In addition to using the REST API to set these annotations once your test completes, you can use one of the Sauce Labs test framework examples to set these and other annotations for you automatically as part of the test execution.
See the following sections for more information:
Desired Capabilities for Annotation
You can set the following test configuration options to keep track of your jobs:
Option | Description | Key | Value Type | Example |
---|---|---|---|---|
Test Names | Used to record test names for jobs and make it easier to find individual tests | name | string |
|
Build Numbers | Used to associate jobs with a build number or app version, displayed on the Dashboard and Archives views | build | string | "build": "build-1234" |
Tagging | User-defined tags for grouping and filtering jobs in the Dashboard and Archives | tags | list | "tags": ["tag1","tag2","tag3"] |
Pass/Fail Status | Selenium and Appium handle sending commands to control a browser or app, but don't report to the server whether a test passed or failed. To record pass/fail status on Sauce, set the Since you can't know in advance whether a test passed or failed, this flag can't be set in the initial configuration. | passed | boolean | "passed": "true" |
Custom Data | User-defined custom data that will accept any valid JSON object, limited to 64KB in size. | customData | object | "customData": {"release": "1.0", "commit": "0k392a9dkjr", "staging": true, "execution_number": 5, "server": "test.customer.com"} |
Examples
curl -X PUT \ -s -d '{"passed": true}' \ -u YOUR_USERNAME:YOUR_ACCESS_KEY \ https://saucelabs.com/rest/v1/YOUR_USERNAME/jobs/YOUR_JOB_ID
curl -X PUT -s -d "{\"passed\": true}" -u YOUR_USERNAME:YOUR_ACCESS_KEY https://saucelabs.com/rest/v1/YOUR_USERNAME/jobs/YOUR_JOB_ID
{ "name": "my job name", "passed": true, "public": "public", "tags": ["tag1", "tag2", "tag3"], "build": 234, "customData": { "release": "1.0", "server": "test.customer.com" } }
Video: Annotating Tests Using the Sauce Labs REST API
This Tech Tip video shows you how to annotate your tests with meaningful information using the Sauce Labs REST API.