The Sauce Labs Cookbook

Sauce Headless

Front End Performance Testing

Insights

External Resources

More Info


Page tree
Skip to end of metadata
Go to start of metadata

Sauce Headless is a lightweight infrastructure solution that allows you to get fast pass/fail data on early pipeline component tests, sanity checks, and pull request tests at scale. It's a container-based architecture for the virtual machines that host our headless browsers.

For debugging, you'll get logs and screenshots of each test result. Sauce Headless provides headless Chrome and Firefox in the three most recent versions and the most recent version of Linux. 

There are a few key differences between setting up Sauce Headless tests versus setting up other Sauce Labs automated web app tests, as described below.

See the following sections for more information:

What You'll Need

ENTERPRISE PLANS ONLY

Should you decide to run Sauce Headless tests through Sauce Connect Proxy tunnels, you'll need to have the latest version.

Setting Up Headless Testing on Sauce Labs

WebDriver Capabilities for Headless Testing

Set the desired capabilities in your test to use headless testing, as shown in this example:

 Headless Testing Desired Capabilities
saucelabs_headless_chrome": {
      "request_timeout_options": {
        "timeout": 900000
      },
      "globals": {
        "waitForConditionTimeout": 60000,
        "waitForConditionPollInterval": 100
      },
      "use_ssl": true,
      "selenium_port": 443,
      "selenium_host": "us-east1.headless.saucelabs.com",
      "desiredCapabilities": {
        "browserName": "chrome",
        "version": "latest-2",
        "platform": "Linux",
        "javascriptEnabled": true
      }
    },

Language Bindings

Select a desired programming language and test framework to see working samples of headless tests running on Sauce Labs. 


Test Frameworks

 Click here to see the full example.

SampleHeadlessSauceTest.java
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.net.MalformedURLException;
import java.net.URL;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.MutableCapabilities;
import org.openqa.selenium.chrome.ChromeOptions;

import org.testng.Assert;
import org.testng.ITestResult;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.Test;

public class SampleHeadlessSauceTest {
    private WebDriver driver;

    @Test
    public void main() throws MalformedURLException {
        String sauceUserName = System.getenv("SAUCE_USERNAME");
        String sauceAccessKey = System.getenv("SAUCE_ACCESS_KEY");
        String URL = "https://ondemand.us-east-1.saucelabs.com/wd/hub";

        ChromeOptions chromeOpts = new ChromeOptions();
        chromeOpts.setExperimentalOption("w3c", true);

        MutableCapabilities sauceOptions = new MutableCapabilities();
        sauceOptions.setCapability("username", sauceUserName);
        sauceOptions.setCapability("accessKey", sauceAccessKey);
        sauceOptions.setCapability("seleniumVersion", "3.141.59");
        sauceOptions.setCapability("name", "headless-chrome-test-java");
        sauceOptions.setCapability("build", "Sample Headless Tests");

        MutableCapabilities caps = new MutableCapabilities();
        caps.setCapability("goog:chromeOptions", chromeOpts);
        caps.setCapability("browserName", "chrome");
        caps.setCapability("browserVersion", "latest");
        caps.setCapability("platformName", "Linux");
        caps.setCapability("sauce:options", sauceOptions);

        driver = new RemoteWebDriver(new URL(URL), caps);

        /* Goes to Sauce Lab's demo page and prints title */

        driver.get("https://www.saucedemo.com");
        System.out.println("title of page is: " + driver.getTitle());
        Assert.assertEquals(driver.getTitle(), "Swag Labs" );
    }
    /* Sends results to SauceLabs.com */
    @AfterMethod
    public void cleanUpAfterTestMethod(ITestResult result) {
        ((JavascriptExecutor)driver).executeScript("sauce:job-result=" + (result.isSuccess() ? "passed" : "failed"));
        driver.quit();
    }
}

TestNG Example Walkthrough

  1. Download or clone the sample script from GitHub.

  2. Ensure you have the prerequisite software.

  3. Resolve dependencies with Maven:

    $ mvn dependency:resolve
  4. Export your Sauce Labs Username and Access Key:

    Sauce Labs Username and Access Key Example:
    export SAUCE_USERNAME=my-sauce-username
    export SAUCE_ACCESS_KEY=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx2cy8f4
  5. Run the following commands:

    $ cd headless/
    $ mvn clean test -Dtest=SampleHeadlessSauceTest 

    You should see the following output (or something similar) in the console:

    -------------------------------------------------------
     T E S T S
    -------------------------------------------------------
    Running SampleHeadlessSauceTest
    Configuring TestNG with: org.apache.maven.surefire.testng.conf.TestNG652Configurator@1ff8b8f
    May 31, 2019 11:46:23 AM org.openqa.selenium.remote.ProtocolHandshake createSession
    INFO: Detected dialect: W3C
    title of page is: Swag Labs
    Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.082 sec
    
    Results :
    
    Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
    
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time:  17.308 s                                



JUnit Jupiter Example Walkthrough

Coming Soon!

 Click here to see the full example.

test_demo.py
import pytest
from os import environ

from selenium import webdriver
from selenium.common.exceptions import WebDriverException
from selenium.webdriver.remote.remote_connection import RemoteConnection

import urllib3

urllib3.disable_warnings()

browsers = [
    {
        "platform": "Linux",
        "browserName": "firefox",
        "version": "latest"

    }, {
        "platform": "Linux",
        "browserName": "firefox",
        "version": "latest-1"

    }, {
        "platform": "Linux",
        "browserName": "firefox",
        "version": "latest"

    }, {
        "platform": "Linux",
        "browserName": "chrome",
        "version": "latest-1",
    }]


def pytest_generate_tests(metafunc):
    if 'driver' in metafunc.fixturenames:
        metafunc.parametrize('browser_config',
                             browsers,
                             ids=_generate_param_ids('broswerConfig', browsers),
                             scope='function')


def _generate_param_ids(name, values):
    return [("<%s:%s>" % (name, value)).replace('.', '_') for value in values]


@pytest.yield_fixture(scope='function')
def driver(request, browser_config):
    desired_caps = dict()
    desired_caps.update(browser_config)
    test_name = request.node.name
    build_tag = environ.get('BUILD_TAG', "local run")
    username = environ.get('SAUCE_USERNAME', None)
    access_key = environ.get('SAUCE_ACCESS_KEY', None)

    # for headless testing you must use the east coast data center endpoint
    selenium_endpoint = "https://ondemand.us-east-1.saucelabs.com/wd/hub"
    desired_caps['build'] = build_tag
    desired_caps['name'] = test_name
    desired_caps['username'] = username
    desired_caps['accessKey'] = access_key

    executor = RemoteConnection(selenium_endpoint, resolve_ip=False)
    browser = webdriver.Remote(executor, desired_capabilities=desired_caps)

    yield browser
    # Teardown starts here
    # report results
    # use the test result to send the pass/fail status to Sauce Labs
    browser.quit()

@pytest.mark.usefixtures("driver")
def test_valid_crentials_login(driver):
    driver.get('http://www.saucedemo.com')

    driver.find_element_by_id('user-name').send_keys('locked_out_user')
    driver.find_element_by_id('password').send_keys('secret_sauce')
    driver.find_element_by_css_selector('.btn_action').click()

    assert driver.find_element_by_css_selector('.error-button').is_displayed()


@pytest.mark.usefixtures("driver")
def test_valid_crentials_login(driver):
    driver.get('http://www.saucedemo.com')

    driver.find_element_by_id('user-name').send_keys('standard_user')
    driver.find_element_by_id('password').send_keys('secret_sauce')
    driver.find_element_by_css_selector('.btn_action').click()

    assert "/inventory.html" in driver.current_url

Pytest Example Walkthrough

  1. Download or clone the sample script from GitHub.

  2. Ensure you have the prerequisite software.

  3. Install the following modules:

    pip install pytest pytest-xdist
  4. Export your Sauce Labs Username and Access Key:

    Sauce Labs Username and Access Key Example:
    export SAUCE_USERNAME=my-sauce-username
    export SAUCE_ACCESS_KEY=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx2cy8f4
  5. Run the following command:

    pytest -n8 test_demo.py

    You should see the following output (or something similar) in the console:

    platform darwin -- Python 3.7.1, pytest-4.4.0, py-1.7.0, pluggy-0.12.0
    rootdir: /path/to/python/headless-examples
    plugins: forked-1.0.2, xdist-1.28.0
    gw0 [4] / gw1 [4] / gw2 [4] / gw3 [4] / gw4 [4] / gw5 [4] / gw6 [4] / gw7 [4]
    ....                                                                    
 Click here to see the full example.

headless-chrome-test.js
/* This code sample uses the headless feature of Sauce Labs, this feature is for invoiced customers only */
const promise = require('selenium-webdriver');
let expect = require('chai').expect;
let webdriver = require('selenium-webdriver');

/* promise manager for selenium-webdriver is going to be deprecated */
/* you must use async/await as an alternative */
promise.USE_PROMISE_MANAGER = false;

let username = process.env.SAUCE_USERNAME,
    accessKey = process.env.SAUCE_ACCESS_KEY,
    appURL = "https://www.saucedemo.com",
    tags = ["sauceDemo", "async", "node", "webdriverjs", "headless" ],
    driver;

describe ('headless chrome test', function() {
    this.timeout(40000);
    beforeEach(async function () {
        driver = await new webdriver.Builder().withCapabilities({
            'browserName': 'chrome',
            'platformName': 'linux',
            'browserVersion': 'latest',
            'goog:chromeOptions' : { 'w3c' : true },
            'sauce:options': {
                'username': username,
                'accessKey': accessKey,
                'seleniumVersion': '3.141.59',
                'build': 'Sample Headless Tests',
                'name': 'headless-chrome-test-js',
                'maxDuration': 3600,
                'idleTimeout': 1000,
                'tags': tags
            }}).usingServer("https://ondemand.us-east-1.saucelabs.com/wd/hub")
            .build();
        await driver.getSession().then(function (sessionid) {
            driver.sessionID = sessionid.id_;
        });
    });

    afterEach(async function() {
        await driver.executeScript("sauce:job-result=" + (this.currentTest.state));
        await driver.quit();
    });

    it('get-title-test', async function() {
        await driver.get(appURL);
        const title = await driver.getTitle();
        console.log('Page Title is: ' + title);
        expect(title).equals('Swag Labs');
    });
});

Mocha-Chai Example Walkthrough

  1. Download or clone the sample script from GitHub.

  2. Ensure you have the prerequisite software.

  3. Navigate to the headless-examples directory and install node package dependencies:

    $ cd headless-examples/
    $ npm install
  4. Export your Sauce Labs Username and Access Key:

    Sauce Labs Username and Access Key Example:
    export SAUCE_USERNAME=my-sauce-username
    export SAUCE_ACCESS_KEY=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx2cy8f4
  5. Run the following command:

    $ npm test test/headless-chrome-test.js

    You should see the following output (or something similar) in the console:

     mocha "test/headless-chrome-test.js"
    
    
    
      headless chrome test
    Page Title is: Swag Labs
        ✓ get-title-test (2594ms)
    
    
      1 passing (13s)                                                              

WebdriverIO Example Walkthrough

Coming Soon!

Rspec Example Walkthrough

Coming Soon!

NUnit Example Walkthrough

Coming Soon!

Data Center Endpoints for Sauce Headless

The containers used for Sauce Headless testing are hosted in the Sauce Labs East Coast data center, which is entirely separate from our data centers in the West Coast and in the EU. You'll need to connect to the US-East Data Center to access the web UI, Selenium endpoint, and Sauce Connect Proxy endpoint for headless testing.

The US-East IP addresses should be reachable from your network. If there's an issue, see the Whitelisting for Restricted Networks section of System and Network Requirements for Sauce Connect Proxy.

NOTE: Cross-Browser VMs are not available at this time in US-East.

Logging into Multiple Data Centers

If you need to be logged into both the US-East and US-West-1 data centers in our web interface, we advise against doing it in the same browser (different tabs). It may cause you to be logged out of one of the data centers or result in other inconsistent interface behavior.

  • If you do want to use the same browser, we recommend logging into the web interfaces from separate browsers or by using an incognito/private browser window. If you have issues with your username for logging into either of the web interfaces and don't want to use an incognito browser, try clearing all Sauce-related cookies from your browser cache. 

Using the Sauce Headless Web UI

Information about your Headless testing jobs is accessible by logging into the headless testing web interface. 

  • Direct Web Interface Link: https://app.us-east-1.saucelabs.com

  • Select Headless US-East from the Sauce Labs dropdown menu

If you want to use a Sauce Connect Proxy tunnel for your Sauce Headless tests, you'll need to start it from here in the UI.

Using Sauce Connect Proxy for Headless Testing

To test a website that's on your local machine or or behind a corporate firewall, we recommend looking at using Sauce Connect Proxy.

To use Sauce Connect Proxy in conjunction with your Sauce Headless tests, be sure you're using the latest version. You'll need to start a new, separate tunnel from the one used for the Virtual and Real Device Cloud, by connecting to the Sauce Headless-specific endpoint (see Data Center Endpoints).

Using Sauce Headless with Other Sauce Labs Solutions

Sauce Headless will exist as its own separate instance at release. It will integrate with the larger Sauce Labs platform at a later date.

Headless will have many of the same features, including, but not limited to:

  • Ability to view individual tests results, as well as automated builds

  • Screenshots, logs, and metadata for debugging 

  • Team Management

  • SSO

  • Sauce Connect support

Video recordings and Extended Debugging features are available in our Cross Browser Testing Platform. 

Video Tutorial: Running Headless Tests

This video shows you how to configure your early pipeline tests to run in Sauce Headless. It also includes advice on which of your tests might be a good candidate for this lightweight and cost-effective testing solution.