The Sauce Labs Cookbook

Sauce Headless

Front End Performance Testing

Insights

External Resources

More Info


Page tree
Skip to end of metadata
Go to start of metadata

Sauce Headless is a container-based, lightweight, and scalable infrastructure solution for testing web apps on headless Chrome and Firefox browsers in our virtual device cloud. Test early in your development cycle on headless browsers to get fast pass/fail data on early pipeline component tests, sanity checks, and pull request tests.

There are a few key differences between setting up Sauce Headless tests versus setting up other Sauce Labs automated web app tests, as described below.

See the following sections for more information:

What You'll Need

ENTERPRISE PLANS ONLY

The latest version of Sauce Connect Proxy, if you're using it to run Headless tests.

Setting Up Headless Testing on Sauce Labs

Sauce Headless provides headless Chrome and Firefox in the three most recent versions and the most recent version of Linux. 

WebDriver Capabilities

Set the desired capabilities in your test to use headless testing, as shown in this example:

saucelabs_headless_chrome": {
      "request_timeout_options": {
        "timeout": 900000
      },
      "globals": {
        "waitForConditionTimeout": 60000,
        "waitForConditionPollInterval": 100
      },
      "use_ssl": true,
      "selenium_port": 443,
      "selenium_host": "us-east1.headless.saucelabs.com",
      "desiredCapabilities": {
        "browserName": "chrome",
        "version": "latest-2",
        "platform": "Linux",
        "javascriptEnabled": true
      }
    },

Language Bindings

Select a desired programming language and test framework to see working samples of headless tests running on Sauce Labs. 

Test Frameworks

 Click here to see the full example.

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.net.MalformedURLException;
import java.net.URL;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.MutableCapabilities;
import org.openqa.selenium.chrome.ChromeOptions;

import org.testng.Assert;
import org.testng.ITestResult;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.Test;

public class SampleHeadlessSauceTest {
    private WebDriver driver;

    @Test
    public void main() throws MalformedURLException {
        String sauceUserName = System.getenv("SAUCE_USERNAME");
        String sauceAccessKey = System.getenv("SAUCE_ACCESS_KEY");
        String URL = "https://ondemand.us-east-1.saucelabs.com/wd/hub";

        ChromeOptions chromeOpts = new ChromeOptions();
        chromeOpts.setExperimentalOption("w3c", true);

        MutableCapabilities sauceOptions = new MutableCapabilities();
        sauceOptions.setCapability("username", sauceUserName);
        sauceOptions.setCapability("accessKey", sauceAccessKey);
        sauceOptions.setCapability("seleniumVersion", "3.141.59");
        sauceOptions.setCapability("name", "headless-chrome-test-java");
        sauceOptions.setCapability("build", "Sample Headless Tests");

        MutableCapabilities caps = new MutableCapabilities();
        caps.setCapability("goog:chromeOptions", chromeOpts);
        caps.setCapability("browserName", "chrome");
        caps.setCapability("browserVersion", "latest");
        caps.setCapability("platformName", "Linux");
        caps.setCapability("sauce:options", sauceOptions);

        driver = new RemoteWebDriver(new URL(URL), caps);

        /* Goes to Sauce Lab's demo page and prints title */

        driver.get("https://www.saucedemo.com");
        System.out.println("title of page is: " + driver.getTitle());
        Assert.assertEquals(driver.getTitle(), "Swag Labs" );
    }
    /* Sends results to SauceLabs.com */
    @AfterMethod
    public void cleanUpAfterTestMethod(ITestResult result) {
        ((JavascriptExecutor)driver).executeScript("sauce:job-result=" + (result.isSuccess() ? "passed" : "failed"));
        driver.quit();
    }
}

TestNG Example Walkthrough

  1. Download or clone the sample script from GitHub.

  2. Ensure you have the prerequisite software.

  3. Resolve dependencies with Maven:

    $ mvn dependency:resolve
  4. Export your Sauce Labs Username and Access Key:

    export SAUCE_USERNAME=my-sauce-username
    export SAUCE_ACCESS_KEY=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx2cy8f4
  5. Run the following command:

    $ cd headless/
    $ mvn clean test -Dtest=SampleHeadlessSauceTest

    You should see the following output (or something similar) in the console:

    -------------------------------------------------------
     T E S T S
    -------------------------------------------------------
    Running SampleHeadlessSauceTest
    Configuring TestNG with: org.apache.maven.surefire.testng.conf.TestNG652Configurator@1ff8b8f
    May 31, 2019 11:46:23 AM org.openqa.selenium.remote.ProtocolHandshake createSession
    INFO: Detected dialect: W3C
    title of page is: Swag Labs
    Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.082 sec
     
    Results :
     
    Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
     
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time:  17.308 s
 Click here to see the full example.

import pytest
from os import environ

from selenium import webdriver
from selenium.common.exceptions import WebDriverException
from selenium.webdriver.remote.remote_connection import RemoteConnection

import urllib3

urllib3.disable_warnings()

browsers = [
    {
        "platform": "Linux",
        "browserName": "firefox",
        "version": "latest"

    }, {
        "platform": "Linux",
        "browserName": "firefox",
        "version": "latest-1"

    }, {
        "platform": "Linux",
        "browserName": "firefox",
        "version": "latest"

    }, {
        "platform": "Linux",
        "browserName": "chrome",
        "version": "latest-1",
    }]


def pytest_generate_tests(metafunc):
    if 'driver' in metafunc.fixturenames:
        metafunc.parametrize('browser_config',
                             browsers,
                             ids=_generate_param_ids('broswerConfig', browsers),
                             scope='function')


def _generate_param_ids(name, values):
    return [("<%s:%s>" % (name, value)).replace('.', '_') for value in values]


@pytest.yield_fixture(scope='function')
def driver(request, browser_config):
    desired_caps = dict()
    desired_caps.update(browser_config)
    test_name = request.node.name
    build_tag = environ.get('BUILD_TAG', "local run")
    username = environ.get('SAUCE_USERNAME', None)
    access_key = environ.get('SAUCE_ACCESS_KEY', None)

    # for headless testing you must use the east coast data center endpoint
    selenium_endpoint = "https://ondemand.us-east-1.saucelabs.com/wd/hub"
    desired_caps['build'] = build_tag
    desired_caps['name'] = test_name
    desired_caps['username'] = username
    desired_caps['accessKey'] = access_key

    executor = RemoteConnection(selenium_endpoint, resolve_ip=False)
    browser = webdriver.Remote(executor, desired_capabilities=desired_caps)

    yield browser
    # Teardown starts here
    # report results
    # use the test result to send the pass/fail status to Sauce Labs
    browser.quit()

@pytest.mark.usefixtures("driver")
def test_valid_crentials_login(driver):
    driver.get('http://www.saucedemo.com')

    driver.find_element_by_id('user-name').send_keys('locked_out_user')
    driver.find_element_by_id('password').send_keys('secret_sauce')
    driver.find_element_by_css_selector('.btn_action').click()

    assert driver.find_element_by_css_selector('.error-button').is_displayed()


@pytest.mark.usefixtures("driver")
def test_valid_crentials_login(driver):
    driver.get('http://www.saucedemo.com')

    driver.find_element_by_id('user-name').send_keys('standard_user')
    driver.find_element_by_id('password').send_keys('secret_sauce')
    driver.find_element_by_css_selector('.btn_action').click()

    assert "/inventory.html" in driver.current_url

Pytest Example Walkthrough

  1. Download or clone the sample script from GitHub.

  2. Ensure you have the prerequisite software.

  3. Install the following modules:

    pip install pytest pytest-xdist
  4. Export your Sauce Labs Username and Access Key:

    export SAUCE_USERNAME=my-sauce-username
    export SAUCE_ACCESS_KEY=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx2cy8f4
  5. Run the following command:

    pytest -n8 test_demo.py

    You should see the following output (or something similar) in the console:

    platform darwin -- Python 3.7.1, pytest-4.4.0, py-1.7.0, pluggy-0.12.0
    rootdir: /path/to/python/headless-examples
    plugins: forked-1.0.2, xdist-1.28.0
    gw0 [4] / gw1 [4] / gw2 [4] / gw3 [4] / gw4 [4] / gw5 [4] / gw6 [4] / gw7 [4]
    ....                                                                    
 Click here to see the full example.

const {config} = require('./wdio.shared.conf');
const build = `Sauce Labs Headless Desktop build-${ new Date().getTime() }`;

// =====================
// Sauce specific config
// =====================
// See https://webdriver.io/docs/sauce-service.html for more information
config.user = process.env.SAUCE_USERNAME;
config.key = process.env.SAUCE_ACCESS_KEY;
config.headless = true;

// ===================================================================================
// Capabilities
// You can find more about constructing the capabilities for real device testing here
// https://wiki.saucelabs.com/display/DOCS/Platform+Configurator#/
// ===================================================================================
config.capabilities = [
    {
        browserName: 'chrome',
        platformName: 'linux',
        browserVersion: 'latest',
        'sauce:options': {
            build,
        },
    },
    {
        browserName: 'chrome',
        platformName: 'linux',
        browserVersion: 'latest-1',
        'sauce:options': {
            build,
        },
    },
    {
        browserName: 'chrome',
        platformName: 'linux',
        browserVersion: 'latest-2',
        'sauce:options': {
            build,
        },
    },
    {
        browserName: 'firefox',
        platformName: 'linux',
        browserVersion: 'latest',
        'sauce:options': {
            build,
        },
    },
    {
        browserName: 'firefox',
        platformName: 'linux',
        browserVersion: 'latest-1',
        'sauce:options': {
            build,
        },
    },
    {
        browserName: 'firefox',
        platformName: 'linux',
        browserVersion: 'latest-2',
        'sauce:options': {
            build,
        },
    },
];

config.services = config.services.concat('sauce');

exports.config = config;

WebdriverIO Example Walkthrough

  1. Download or clone the sample script from GitHub.

  2. Ensure you have the prerequisite software.

  3. Navigate to the headless-examples directory and install node package dependencies:

    $ cd headless-examples/
    $ npm install
  4. Export your Sauce Labs Username and Access Key:

    export SAUCE_USERNAME=my-sauce-username
    export SAUCE_ACCESS_KEY=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx2cy8f4
  5. Run the following command:

    $ npm test test/headless-chrome-test.js

    You should see the following output (or something similar) in the console:

     mocha "test/headless-chrome-test.js"
    
    
    
      headless chrome test
    Page Title is: Swag Labs
        ✓ get-title-test (2594ms)
    
    
      1 passing (13s)                                                              


 Click here to see the full example.

# frozen_string_literal: true

require 'selenium-webdriver'
require 'sauce_whisk'

RSpec.configure do |config|
  config.before do |example|
    url = 'https://ondemand.us-west-1.saucelabs.com:443/wd/hub'
    SauceWhisk.data_center = :US_WEST

    if ENV['PLATFORM_NAME'] == 'linux' # Then Headless
      url = 'https://ondemand.us-east-1.saucelabs.com:443/wd/hub'
      SauceWhisk.data_center = :US_EAST
    end

    browser_name = ENV['BROWSER_NAME'] || 'chrome'

    options = {browser_name: browser_name,
               platform_name: ENV['PLATFORM_NAME'] || 'Windows 10',
               browser_version: ENV['BROWSER_VERSION'] || 'latest',
               'sauce:options': {name: example.full_description,
                                 build: build_name,
                                 username: ENV['SAUCE_USERNAME'],
                                 access_key: ENV['SAUCE_ACCESS_KEY']}}

    capabilities = Selenium::WebDriver::Remote::Capabilities.send(browser_name, options)
    @driver = Selenium::WebDriver.for(:remote,
                                      url: url,
                                      desired_capabilities: capabilities)
  end

  config.after do |example|
    SauceWhisk::Jobs.change_status(@driver.session_id, !example.exception)
    @driver.quit
  end

  #
  # Note that this build name is specifically for Circle CI execution
  # Most CI tools have ENV variables that can be structured to provide useful build names
  #
  def build_name
    if ENV['CIRCLE_JOB']
      "#{ENV['CIRCLE_JOB']}: #{ENV['CIRCLE_BUILD_NUM']}"
    elsif ENV['TRAVIS_REPO_SLUG']
      "#{ENV['TRAVIS_REPO_SLUG'][%r{[^/]+$}]}: #{ENV['TRAVIS_JOB_NUMBER']}"
    elsif ENV['SAUCE_START_TIME']
      ENV['SAUCE_START_TIME']
    else
      "Ruby-RSpec-Selenium: Local-#{Time.now.to_i}"
    end
  end
end

More examples: Sauce Labs Training on GitHub: Ruby examples.

Rspec Example Walkthrough

  1. Setup Gemfile.

    • Install bundler so that we can easily update and resolve dependencies:

      $ gem install bundler
      
      
  2. Execute the bundle Command.

    • Use the following to install the required gems:

      $ bundle install --path .bundle
  3. Run the Test Scripts.

 Click here to see the full example.

���using System;
using System.Collections.Generic;
using NUnit.Framework;
using NUnit.Framework.Interfaces;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Remote;

namespace Selenium3.Nunit.Scripts.SimpleExamples
{
    [TestFixture]
    [Category("SimpleTest")]
    public class SimpleHeadlessTest
    {
        IWebDriver _driver;

        [Test]
        public void ChromeHeadlessW3C()
        {
            var sauceOptions = new Dictionary<string, object>
            {
                ["username"] = Environment.GetEnvironmentVariable("SAUCE_USERNAME", EnvironmentVariableTarget.User),
                ["accessKey"] = Environment.GetEnvironmentVariable("SAUCE_ACCESS_KEY", EnvironmentVariableTarget.User)
            };

            var chromeOptions = new ChromeOptions
            {
                BrowserVersion = "latest",
                PlatformName = "Linux",
                UseSpecCompliantProtocol = true
            };
            sauceOptions.Add("name", TestContext.CurrentContext.Test.Name);
            chromeOptions.AddAdditionalCapability("sauce:options", sauceOptions, true);

            _driver = new RemoteWebDriver(new Uri("https://ondemand.us-east-1.saucelabs.com/wd/hub"),
                chromeOptions.ToCapabilities(), TimeSpan.FromSeconds(600));
            _driver.Navigate().GoToUrl("https://www.saucedemo.com");
            Assert.Pass();
        }

        [TearDown]
        public void CleanUpAfterEveryTestMethod()
        {
            var passed = TestContext.CurrentContext.Result.Outcome.Status == TestStatus.Passed;
            ((IJavaScriptExecutor)_driver).ExecuteScript("sauce:job-result=" + (passed ? "passed" : "failed"));
            _driver?.Quit();
        }
    }
}

NUnit Example Walkthrough

Refer to the instructions at Sauce Labs Training on GitHub: C# demo scripts.

Data Center Endpoints for Sauce Headless

The containers used for Sauce Headless testing are hosted in the Sauce Labs East Coast data center, which is entirely separate from our data centers in the West Coast and in the EU. You'll need to connect to the US-East Data Center to access the web UI, Selenium endpoint, and Sauce Connect Proxy endpoint for headless testing.

The US-East IP addresses should be reachable from your network. If there's an issue, see the Whitelisting for Restricted Networks section of System and Network Requirements for Sauce Connect Proxy.

NOTE: Cross-Browser VMs are not available at this time in US-East.

Using the Sauce Headless Web UI

Information about your Headless testing jobs is accessible by logging into the headless testing web interface. 

If you want to use a Sauce Connect Proxy tunnel for your Sauce Headless tests, you'll need to start it from here in the UI.

Sauce Connect Proxy for Headless Testing

If you're testing website that's on your local machine or behind a corporate firewall, we recommend using Sauce Connect Proxy.

To use it in conjunction with your Sauce Headless tests, be sure you have the latest version on your machine. You'll need to start a new, separate tunnel from the one used for the Virtual and Real Device Cloud, by connecting to the Sauce Headless-specific endpoint (see Data Center Endpoints).

Video Tutorial: Running Headless Tests

This video shows you how to configure your early pipeline tests to run in Sauce Headless. It also includes advice on which of your tests might be a good candidate for this lightweight and cost-effective testing solution.