Sauce Labs is a cloud platform for executing automated and manual mobile and web tests. Sauce Labs supports running automated tests with Selenium WebDriver (for web applications) and Appium (for native and mobile web applications). This tutorial will show you how to get started with testing with Python on Sauce.
Before getting started, you should read the Best Practices for Running Tests with Sauce Labs.
You will need to install the Selenium WebDriver client driver to your local Python environment
You can either download the driver from the link, or use
pipto install it.
You should also install the Sauce Python client, which provides features for reporting job information to the Sauce Labs dashboard.
This simple Python test script tests the Google front page. Despite its simplicity, it contains everything you need to know in order to run an automated test on Sauce Labs.
Analyzing the Code
If you look at the code closely, you'll see that basics for setting up a test to run on sauce are very straightforward, and really only require two elements.
If you wanted to run Selenium locally, you might initiate a driver for the browser that you want to test on like so:
If you wanted to run on Sauce, you would instead use
webdriver.Remote(), and then pass it two paramaters:
command_executor, which points to the Sauce cloud and uses your Sauce Labs authentication to log in, and
desired_capabilties, which specifies the browsers and operating systems to run the tests against.
You can use the Platform Configurator to specify the desired capabilities for any browser/platform combination you want.
Running the Test
- Copy the example code and save it into a file called
Make sure your username and access key are included in the URL passed through to the command_executor.
- Open a command line terminal and navigate to the directory where the file is located.
Execute the test:
Check your dashboard and you will see that your test has just run on Sauce!
Running Local Tests
Developing apps on
localhost is quick and efficient. The drawback is that
localhost is not a publicly-accessible address on the Internet, so the browsers in the Sauce Labs cloud can't load and test your app. The solution is to use Sauce Connect. Sauce Connect uses a secure tunnel protocol that gives specific Sauce machines access to your local network. You can also use it to test applications that are inside your corporate firewall. Sauce Connect is not required to run tests on Sauce Labs, only in situations where the application or website you want to test is on your local machine or behind a firewall.
Running Tests in Parallel
Now that you’re running tests on Sauce, you may wonder how you can run your tests faster. One way to increase speed is by running tests in parallel across multiple virtual machines.
You can run your tests in parallel at two levels, and you can run your tests in parallel across multiple browsers. For example, if you have 10 tests and want to run on five browsers, this would be parallelism of five. You can also run tests across browsers and each test in parallel. Using our previous example this would be more like 50 parallel tests. Doing this requires that your tests are written in a way that they do not collide with one another. For more on this see the Selenium WebDriver - Running Your Tests in Parallel blog.
Before you start running tests in parallel, you should review the Best Practices for Running Tests with Sauce Labs, especially the topics on avoiding external test dependencies and avoiding dependencies between tests.
Parallel Tests and Concurrency Limits
The number of tests you can run in parallel is determined by the concurrency limit associated with your account. You can check this in you Sauce Labs dashboard under Concurrent VMs.
See the topics under Running Tests in Parallel with Python for more information and examples of setting up popular Python testing frameworks to run tests in parallel on Sauce
Reporting on Test Results
"Wait," you might be asking, "My test says 'Complete' but what happens if it fails?"
Unfortunately, Sauce has no way to determine whether your test passed or failed automatically, since it is determined entirely by your business logic. You can, however, tell Sauce about the results of our tests automatically using the Sauce python client and adding these lines to your test.
You should also follow our recommended best practice of adding build numbers, tags, and other identifying information to your tests so you can easily find and manage them in your test results and archives pages.