Sauce Labs is a cloud platform for executing automated and manual mobile and web tests. Sauce Labs supports running automated tests with Selenium WebDriver (for web applications) and Appium (for native and mobile web applications). This topic will provide you with an example of how to get your Ruby tests up and running on Sauce.
The code in this topic is presented as an example only, since your tests and testing environments may require specialized scripting. This information should be taken only as an illustration of how you could set up your tests with Sauce Labs, and is not directly supported by Sauce.
This code examples shows a very simple test to see if the title of Google's home page has changed, but it includes everything you need to set up and run an automated test on Sauce.
Running the Example Test
- Copy the example code and save it into a file called
- Open a command line terminal and navigate to the file location.
Voila! You have just run your first test on Sauce, which you can verify by checking your Dashboard.
A closer look at the code sample will help you understand how you could use it as the basis for setting up your own tests from scratch, or how you can set up existing tests to run in the Sauce cloud.
To run an existing test on Sauce, all you need to change is your
driver definition, and make sure that the
sauce_endpoint variable includes your Sauce
Once you have your tests set up to run in the Sauce cloud, you need to define the platform, browser, and version you want the test to run against, which is where
caps come into play.
You can manually enter the values you want for
:version, or you can use the handy Platform Configurator to generate the
caps values for any combination of platform, browser, and browser version.
Running Local Tests
localhostis quick and efficient. The drawback is that
localhostis not a publicly-accessible address on the Internet, so the browsers in the Sauce Labs cloud can't load and test your app. The solution is to use Sauce Connect. Sauce Connect is an application that creates a secure tunnel connection between the Sauce Labs virtual machine that runs your test and your local network. You can also use Sauce Connect to test applications and websites that are located within your corporate firewall. Sauce Connect is not required to run tests on Sauce Labs, only in situations where the application or website you want to test is not publicly accessible.
Running Tests in Parallel
You can run your tests in parallel at two levels, and you can run your tests in parallel across multiple browsers. For example, if you have 10 tests and want to run on five browsers, this would be parallelism of five. You can also run tests across browsers and each test in parallel. Using our previous example this would be more like 50 parallel tests. Doing this requires that your tests are written in a way that they do not collide with one another. For more on this see the Selenium WebDriver - Running Your Tests in Parallel blog.
Before you start running tests in parallel, you should review the Best Practices for Running Tests with Sauce Labs, especially the topics on avoiding external test dependencies and avoiding dependencies between tests.
Match Thread Count to Concurrency Limit
You should match your thread count to your concurrency limit, which is shown in the My Account section of your user profile information on the Sauce Labs dashboard.
ParallelTestsgems into your Gemfile, then run
- Create a
specdirectory in your project's root directory.
specdirectory, create a file called
sauce_driver.rb, which will encapsulate the behaviour we need to create Selenium drivers with Sauce Labs.
Now you need to make RSpec create a new driver for each spec. The cleanest way to do this is by defining an
aroundhook, which will be run, naturally, around every spec.
Create a file in the
This file will be
require`d by every spec you create, as this is where, by convention, any setup code is placed.
On line 5 you set up a new block of code to be run
aroundevery example tagged with
:run_on_sauce => true. This lets you have non-Selenium tests that don't spin up Selenium sessions. You have to make sure you include
:run_on_sauceon all your Selenium tests, though!
On line 6 you use the
SauceDriverclass you set up earlier to create a new Selenium driver, 'pointed' at Sauce Labs. Then, on line 8, you run the example. On lines 9 - 11 we call
quiton the Selenium session; This is done in an
ensureblock so that each test always closes off its driver, even if something goes wrong.
Now you have RSpec set up to create drivers and close them down. Your next question might be, "How do I use these drivers?" Super simple. Because
driver is an instance variable, and the
around block runs in the context of the spec, it can use the driver directly. Consider this example spec.
On line 1 you require
spec_helper. Then, on line 4 you add
:run_on_sauce => true to make sure the
around block runs. You can use the created driver,
@driver, without any further setup, and you don't have to do anything to close it off either. This means you're much less likely to forget to do so when you write more tests.
Now, parallel testing! If you go ahead and create some more specs like this one (you can copy
google_spec.rb into other files in the
spec directory, just make sure the filenames end in
spec.rb), then run the specs from your project root directory with:
You should be able to log into Sauce Labs and see your tests running in parallel. If your account has more than two concurrency slots (meaning, you have a paid account), you can increase the number after
-n to match your concurrency, and
parallel_tests will spin up more tests at once.
Reporting Test Results
The Job ID is the simplest part of the process. The ID assigned to each job by Sauce Labs is the same as the Session ID for the corresponding Selenium session, which you can pull off the driver like so:
Using the REST API is best done with the sauce_whisk gem, which assumes you've set your Sauce username and access key as the
SAUCE_ACCESS_KEY environment variables.
This example shows how you would modify the example RSpec
spec_helper.rb file to check the status of the test, and pass that to the REST API.
That's it! Your tests will now be reporting their status to the Sauce Labs REST API.