The Sauce Labs Cookbook

The Sauce Labs Cookbook

Sauce Headless

Front End Performance Testing

Insights

External Resources

More Info

Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 9 Next »

With Sauce Labs you can run automated tests of your web apps against a wide variety of device/operating system/browser combinations.  This tutorial will show you how to get started with testing with Python on Sauce. 

Prerequisites

Code Example

This Python example script executes a simple test to open a browser, navigate to the Sauce Labs demo web app, and then close the browser. It also incorporate a number of best practices, such as using environment variables for authentication credentials. You can clone this script directly from the Sauce Labs Python training repo on GitHub and modify it as you want to test your authentication credentials, set desired capabilities, and run tests against your own web app. 


 Click here to view the example script

test_module4_pytest.py (source from Sauce Labs Training Python) 
import pytest
import os
from selenium import webdriver
from _pytest.runner import runtestprotocol

@pytest.fixture
def driver(request):
    sauce_username = os.environ["SAUCE_USERNAME"]
    sauce_access_key = os.environ["SAUCE_ACCESS_KEY"]
    remote_url = "https://ondemand.saucelabs.com:443/wd/hub"

    sauceOptions = {
        'screenResolution': '1280x768',
        'seleniumVersion': '3.141.59',
        # best practices involve setting a build number for version control
        'build': 'Onboarding Sample App - Python + Pytest',
        'name': '4-best-practices',
        'username': sauce_username,
        'accessKey': sauce_access_key,
        # tags to filter test reporting.
        'tags': ['instant-sauce', 'pytest', 'module4'],
        # setting sauce-runner specific parameters such as timeouts helps
        # manage test execution speed.
        'maxDuration': 1800,
        'commandTimeout': 300,
        'idleTimeout': 1000,
    }

    chromeOpts = {
        'platformName': 'Windows 10',
        'browserName': 'chrome',
        'browserVersion': 'latest',
        'goog:chromeOptions': {'w3c': True},
        'sauce:options': sauceOptions
    }

    browser = webdriver.Remote(command_executor=remote_url, desired_capabilities=chromeOpts)
    yield browser
    browser.quit()


def test_should_open_chrome(driver):
    driver.get("https://www.saucedemo.com")
    actual_title = driver.title
    expected_title = "Swag Labs"
    assert expected_title == actual_title

Wait!

The implicitly wait method tells the browser to wait a set amount of time (in seconds) for elements to appear on the page before giving up. Without it, slow loading DOMs could cause our tests to fail when they might otherwise pass. Using implicit waits is one of our recommended best practices.

Analyzing the Code

If you look at the code closely, you'll see that basics for setting up a test to run on sauce are very straightforward, and really only require two elements.

It's very easy to get your existing Python tests up and running on Sauce. 

Set Sauce URL and Access Credentials

If you wanted to run a Selenium test on Selenium locally, you would initiate a driver for the browser you want to test against like this.

Running a Local Python Test
driver = webdriver.Firefox() 

To run a test on a remote service like Sauce Labs, all you need to change is your driver definition, make sure that the sauce_endpoint variable includes your Sauce USERNAME and ACCESSKEY, and then pass it two parameters: command_executor, which points to the Sauce cloud and uses your Sauce Labs authentication to log in, and desired_capabilties, which specifies the browsers and operating systems to run the tests against.

Running a Python Test Remotely on Sauce
# this is how you set up a test to run on Sauce Labs
desired_cap = {
    'platform': "Mac OS X 10.9",
    'browserName': "chrome",
    'version': "31",
}
username = os.environ['SAUCE_USERNAME']
access_key = os.environ['SAUCE_ACCESS_KEY']
driver = webdriver.Remote(
    command_executor='https://{}:{}@ondemand.saucelabs.com/wd/hub'.format(username, access_key),
    desired_capabilities=desired_cap)

To find your Sauce Labs access key:

  1. Sign in to https://saucelabs.com with the username you set up when you created your account. 
    You will use this same username in your test script. 
  2. To find your access key:
    1. Click your name in the Account Profile menu in the upper-right corner.
    2. Click User Settings.
    3. Scroll down to Access Key and click Show.
    4. Enter the password you use to sign in to Sauce Labs to view the access key.
    5. Click the Copy icon.

PAGE DEPRECATED: Updated content located here.

Set Desired Capabilities

Once you have your tests set up to run in the Sauce cloud, you need to define the platform, browser, and version you want the test to run against, which is where desired capabilities come into play. 

desired_cap = {
    'platform': "Mac OS X 10.9",
    'browserName': "chrome",
    'version': "31",

You can manually enter the values you want for platform, browsername, and version, or you can use the handy Platform Configurator to generate the desired_caps values for any combination of platform, browser, and browser version. 

Running the Example Code

  1. Copy the example code and save it into a file called first_test.py.
    Make sure your username and access key are included in the URL passed through to the command_executor.
  2. Open terminal and navigate to the directory where the file is located.

  3. Execute the test:

    python first_test.py

Check your dashboard and you will see that your test has just run on Sauce!

Running Tests Against Local Applications

If your test application is not publicly available, you will need to use Sauce Connect so that Sauce can reach it. 

Sauce Connect is a tunneling app that allows you to execute tests securely when testing behind firewalls or on localhost. For more detailed information, please visit see the Sauce Connect docs. 

Reporting to the Sauce Labs Dashboard

"Wait," you might be asking, "My test says 'Complete' but what happens if it fails?"

Unfortunately, Sauce has no way to determine whether your test passed or failed automatically, since it is determined entirely by your business logic. You can, however, tell Sauce about the results of our tests automatically using the Sauce python client and adding these lines to your test. 

# this authenticates you 
from sauceclient import SauceClient
sauce_client = SauceClient("YOUR_SAUCE_USERNAME", "YOUR_SAUCE_ACCESSKEY")

# this belongs in your test logic
sauce_client.jobs.update_job(driver.session_id, passed=True) 

Running Tests in Parallel

Now that you’re running tests on Sauce, you may wonder how you can run your tests faster. One way to increase speed is by running tests in parallel across multiple virtual machines.

You can run your tests in parallel at two levels, and you can run your tests in parallel across multiple browsers. For example, if you have 10 tests and want to run on five browsers, this would be parallelism of five. You can also run tests across browsers and each test in parallel. Using our previous example this would be more like 50 parallel tests. Doing this requires that your tests are written in a way that they do not collide with one another, as described in our Best Practice topics avoiding external test dependencies and avoiding dependencies between tests.

Check out Using Frameworks to Run Tests in Parallel for more information, and examples of framework setups for Java, Python, and other programming languages. 

 

 Parallel Testing with py.test

First install py.test:

pip install selenium sauceclient pytest pytest-xdist

Now, let's use the platform configurator to add two platforms that we want to test on. Replace "desired_cap" in our example script above with an array of configs like so:

browsers = [{
    "platform": "Windows 10",
    "browserName": "internet explorer",
    "version": "11"
}, {
    "platform": "OS X 10.11",
    "browserName": "safari",
    "version": "8.1"
}]

Just add and remove browsers here as you see fit.

Then create a decorator and add your tests like in this example below. You can actually just add your test cases to this script:

import os
import unittest
import sys
import new
from selenium import webdriver
from sauceclient import SauceClient

browsers = [{
    "platform": "Windows 10",
    "browserName": "internet explorer",
    "version": "11"
}, {
    "platform": "OS X 10.11",
    "browserName": "safari",
    "version": "8.1"
}]

username = 'philg'
access_key = '45753ef0-4aac-44a3-82e7-b3ac81af8bca'

# This decorator is required to iterate over browsers
def on_platforms(platforms):
    def decorator(base_class):
        module = sys.modules[base_class.__module__].__dict__
        for i, platform in enumerate(platforms):
            d = dict(base_class.__dict__)
            d['desired_capabilities'] = platform
            name = "%s_%s" % (base_class.__name__, i + 1)
            module[name] = new.classobj(name, (base_class,), d)
    return decorator

@on_platforms(browsers)
class FirstSampleTest(unittest.TestCase):

    # setUp runs before each test case
    def setUp(self):
        self.desired_capabilities['name'] = self.id()
        self.driver = webdriver.Remote(
           command_executor="http://%s:%s@ondemand.saucelabs.com:80/wd/hub" % (username, access_key),
           desired_capabilities=self.desired_capabilities)

    # verify google title
    def test_google(self):
        self.driver.get("http://www.google.com")
        assert ("Google" in self.driver.title), "Unable to load google page"

    # type 'Sauce Labs' into google search box and submit
    def test_google_search(self):
        self.driver.get("http://www.google.com")
        elem = self.driver.find_element_by_name("q")
        elem.send_keys("Sauce Labs")
        elem.submit()

    # tearDown runs after each test case
    def tearDown(self):
        self.driver.quit()
        sauce_client = SauceClient(username, access_key)
        status = (sys.exc_info() == (None, None, None))
        sauce_client.jobs.update_job(self.driver.session_id, passed=status)

Since we are running on two platforms, use -n2 like so to run your tests in parallel:

py.test -n10 first_test.py

Visit your dashboard and you should see four tests running at once! Use the py.test docs to learn more about how to properly use py.test for running tests in parallel.

 Parallel Testing with nose

First, install nose:

pip install selenium sauceclient nose==1.1.0

Now, let's use the platform configurator to add two platforms that we want to test on. Replace "desired_cap" in our example script above with an array of configs like so:

browsers = [{
    "platform": "Windows 10",
    "browserName": "internet explorer",
    "version": "11"
}, {
    "platform": "OS X 10.11",
    "browserName": "safari",
    "version": "8.1"
}]

Putting it all together, here is a working script that will run tests in parallel on Sauce with the browsers we have assigned above:

import os
import sys
import inspect
from nose.tools import with_setup
from selenium import webdriver
from sauceclient import SauceClient

browsers = [{
    "platform": "Windows 10",
    "browserName": "internet explorer",
    "version": "11"
}, {
    "platform": "OS X 10.11",
    "browserName": "safari",
    "version": "8.1"
}]

username = 'philg'
access_key = '45753ef0-4aac-44a3-82e7-b3ac81af8bca'

def launchBrowser(caps):
    caps['name'] = inspect.stack()[1][3]
    return webdriver.Remote(
            command_executor = "http://%s:%s@ondemand.saucelabs.com:80/wd/hub" % (username, access_key),
            desired_capabilities = caps);

def teardown_func():
    global driver
    driver.quit()
    sauce_client = SauceClient(username, access_key)
    status = sys.exc_info() == (None, None, None)
    sauce_client.jobs.update_job(driver.session_id, passed=status)

# Will generate a test for each browser and os configuration
def test_generator_verify_google():
    for browser in browsers:
        yield verify_google, browser

@with_setup(None, teardown_func)
def verify_google(browser):
    global driver
    driver = launchBrowser(browser)
    driver.get("http://www.google.com")
    assert ("Google" in driver.title), "Unable to load google page"
    elem = driver.find_element_by_name("q")
    elem.send_keys("Sauce Labs")
    elem.submit()

Now run with the following command: 

nosetests --processes=8 --process-timeout=120

Note: This script references the sauceclient and selenium so make sure you have those installed.

Visit your dashboard and you should see four tests running at once! Use the nose docs to learn more about how to properly use nose for running tests in parallel.

  • No labels