The Sauce Labs Cookbook

Sauce Headless

Front End Performance Testing

Insights

External Resources

More Info


Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Include Page
_sauce description_sauce description
 This tutorial will show you how to get started with testing with Python on Sauce. 
setup_example_intro
_setup_example_intro

Include Page
_example_only
_example_only

Table of Contents

Prerequisites

Include Page
_python_requirements
_python_requirements

...

Include Page
_pythonsetup_codeexample_examplescript
_pythonsetup_codeexample_example

Tip
titleWait!

The implicitly wait method tells the browser to wait a set amount of time (in seconds) for elements to appear on the page before giving up. Without it, slow loading DOMs could cause our tests to fail when they might otherwise pass. Using implicit waits is one of our recommended best practices.

Analyzing the Code

If you look at the code closely, you'll see that basics for setting up a test to run on sauce are very straightforward, and really only require two elements.

...

Running the Example Code

  1. Copy the example code and save it into a file called first_test.py.
    Make sure your username and access key are included in the URL passed through to the command_executor.
  2. Open terminal and navigate to the directory where the file is located.
  3. Execute the test:

    Code Block
    python first_test.py

Check your dashboard and you will see that your test has just run on Sauce!

Running Tests Against Local Applications

If your test application is not publicly available, you will need to use Sauce Connect so that Sauce can reach it. 

Sauce Connect is a tunneling app that allows you to execute tests securely when testing behind firewalls or on localhost. For more detailed information, please visit see the Sauce Connect docs

Reporting to the Sauce Labs Dashboard

 Recording Pass/Failure Results

"Wait," you might be asking, "My test says 'Complete' but what happens if it fails?"

Unfortunately, Sauce has no way to determine whether your test passed or failed automatically, since it is determined entirely by your business logic. We can, however, tell Sauce about the results of our tests automatically using the Sauce python client:

pip install sauceclient

Then add this to your test:

# this authenticates you 
from sauceclient import SauceClient
sauce_client = SauceClient("philg", "45753ef0-4aac-44a3-82e7-b3ac81af8bca")

# this belongs in your test logic
sauce_client.jobs.update_job(driver.session_id, passed=True)

 Setting a Build Number

Now, you may want to associate this with a build id in your Continuous Integration pipeline. To do this, just include a build number in your desired capabilities:

desired_cap = {
    'platform': "Mac OS X 10.9",
    'browserName': "chrome",
    'version': "31",
    'build': "build-1234",
}

This is important for organizing tests in our new dashboard. Please see our docs for more information here

 Tagging Your Tests

You can also create custom tags for your tests that you can use to search on the archives page:

desired_cap = {
    'platform': "Mac OS X 10.9",
    'browserName': "chrome",
    'version': "31",
    'build': "build-1234",
    'tags': [ "tag1", "tag2", "tag3" ]
}

 Running Tests in Parallel

Now that you're running tests on Sauce, you may wonder how you can run your tests more quickly. Running tests in parallel is the answer! To do this we will need to use a third party test runner. The most popular third party test runners for Python are py.test and nose.

 Parallel Testing with py.test

First install py.test:

pip install selenium sauceclient pytest pytest-xdist

Now, let's use the platform configurator to add two platforms that we want to test on. Replace "desired_cap" in our example script above with an array of configs like so:

browsers = [{
    "platform": "Windows 10",
    "browserName": "internet explorer",
    "version": "11"
}, {
    "platform": "OS X 10.11",
    "browserName": "safari",
    "version": "8.1"
}]

Just add and remove browsers here as you see fit.

Then create a decorator and add your tests like in this example below. You can actually just add your test cases to this script:

import os
import unittest
import sys
import new
from selenium import webdriver
from sauceclient import SauceClient

browsers = [{
    "platform": "Windows 10",
    "browserName": "internet explorer",
    "version": "11"
}, {
    "platform": "OS X 10.11",
    "browserName": "safari",
    "version": "8.1"
}]

username = 'philg'
access_key = '45753ef0-4aac-44a3-82e7-b3ac81af8bca'

# This decorator is required to iterate over browsers
def on_platforms(platforms):
    def decorator(base_class):
        module = sys.modules[base_class.__module__].__dict__
        for i, platform in enumerate(platforms):
            d = dict(base_class.__dict__)
            d['desired_capabilities'] = platform
            name = "%s_%s" % (base_class.__name__, i + 1)
            module[name] = new.classobj(name, (base_class,), d)
    return decorator

@on_platforms(browsers)
class FirstSampleTest(unittest.TestCase):

    # setUp runs before each test case
    def setUp(self):
        self.desired_capabilities['name'] = self.id()
        self.driver = webdriver.Remote(
           command_executor="http://%s:%s@ondemand.saucelabs.com:80/wd/hub" % (username, access_key),
           desired_capabilities=self.desired_capabilities)

    # verify google title
    def test_google(self):
        self.driver.get("http://www.google.com")
        assert ("Google" in self.driver.title), "Unable to load google page"

    # type 'Sauce Labs' into google search box and submit
    def test_google_search(self):
        self.driver.get("http://www.google.com")
        elem = self.driver.find_element_by_name("q")
        elem.send_keys("Sauce Labs")
        elem.submit()

    # tearDown runs after each test case
    def tearDown(self):
        self.driver.quit()
        sauce_client = SauceClient(username, access_key)
        status = (sys.exc_info() == (None, None, None))
        sauce_client.jobs.update_job(self.driver.session_id, passed=status)

Since we are running on two platforms, use -n2 like so to run your tests in parallel:

py.test -n10 first_test.py

Visit your dashboard and you should see four tests running at once! Use the py.test docs to learn more about how to properly use py.test for running tests in parallel.

 Parallel Testing with nose

First, install nose:

pip install selenium sauceclient nose==1.1.0

Now, let's use the platform configurator to add two platforms that we want to test on. Replace "desired_cap" in our example script above with an array of configs like so:

browsers = [{
    "platform": "Windows 10",
    "browserName": "internet explorer",
    "version": "11"
}, {
    "platform": "OS X 10.11",
    "browserName": "safari",
    "version": "8.1"
}]

Putting it all together, here is a working script that will run tests in parallel on Sauce with the browsers we have assigned above:

import os
import sys
import inspect
from nose.tools import with_setup
from selenium import webdriver
from sauceclient import SauceClient

browsers = [{
    "platform": "Windows 10",
    "browserName": "internet explorer",
    "version": "11"
}, {
    "platform": "OS X 10.11",
    "browserName": "safari",
    "version": "8.1"
}]

username = 'philg'
access_key = '45753ef0-4aac-44a3-82e7-b3ac81af8bca'

def launchBrowser(caps):
    caps['name'] = inspect.stack()[1][3]
    return webdriver.Remote(
            command_executor = "http://%s:%s@ondemand.saucelabs.com:80/wd/hub" % (username, access_key),
            desired_capabilities = caps);

def teardown_func():
    global driver
    driver.quit()
    sauce_client = SauceClient(username, access_key)
    status = sys.exc_info() == (None, None, None)
    sauce_client.jobs.update_job(driver.session_id, passed=status)

# Will generate a test for each browser and os configuration
def test_generator_verify_google():
    for browser in browsers:
        yield verify_google, browser

@with_setup(None, teardown_func)
def verify_google(browser):
    global driver
    driver = launchBrowser(browser)
    driver.get("http://www.google.com")
    assert ("Google" in driver.title), "Unable to load google page"
    elem = driver.find_element_by_name("q")
    elem.send_keys("Sauce Labs")
    elem.submit()

Now run with the following command: 

nosetests --processes=8 --process-timeout=120

Note: This script references the sauceclient and selenium so make sure you have those installed.

Visit your dashboard and you should see four tests running at once! Use the nose docs to learn more about how to properly use nose for running tests in parallel.

script

You can clone this script from the saucelabs-training repository on GitHub: https://github.com/saucelabs-training/demo-python

Info

There are examples using both pytest and unittest frameworks, change directories to the relevant module before running your tests:

Testing with a Proxy

If you're trying to run this script from behind a VPN or a corporate proxy, you must use either IPSec or Sauce Connect Proxy. Once you've downloaded and installed the relevant software, add the following capability to the test script:

Code Block
'tunnelIdentifier': '<tunnel_id>',

Running the Test

  1. Navigate to the root project directory and use pip to install the latest Selenium library for use in the script:

    Code Block
    $ pip install -r requirements.txt
  2. Set your Sauce Labs Credentials as envrionment variables, indicated by the following lines in the script:

    View Git file
    pathon-boarding-modules/pytest-examples/test_module4_pytest.py
    lastline9
    repository-id36
    firstline8
    branchrefs/remotes/origin/master
  3. Depending on which framework you're using, your commands may be different to run the tests. Use any of the following command based on the chosen framework:

    pytest:

    Code Block
    pytest on-boarding-modules/pytest-examples/test_module4_pytest.py

    unittest:

    Code Block
     python -m unittest on-boarding-modules/unittest-examples/test_module4_unittest.py