Sauce Labs is a cloud platform for executing automated and manual mobile and web tests. Sauce Labs supports running automated tests with Selenium WebDriver (for web applications) and Appium (for native and mobile web applications). This topic will provide you with an example of how to get your PHP tests up and running on Sauce.
The code in this topic is presented as an example only, since your tests and testing environments may require specialized scripting. This information should be taken only as an illustration of how you could set up your tests with Sauce Labs, and is not directly supported by Sauce.
You need to have these components installed to set up testing on Sauce with PHP:
- You must have PHP set up on your system before you start testing.
Mac OS X comes with PHP installed, as long as you're running version 5.3 or higher, you're ready to go! If you're a Windows user, you can find instructions for setting up PHP on Windows at PHP.net.
- Windows users must also enable the PHP curl library and OpenSSL support to get the best performance from your PHP setup.
For more information, see Installing Extensions for Windows on the php.net website as well as these setup topics:
- We also recommend using Sausage as your PHP testing framework.
Sausage contains classes and libraries that are designed to work with the Sauce Labs API, and it provides a convenient user interface and other features for managing your test results. It also includes a demo test,
WebDriverDemo.php, that you can run to see how Sauce works. For more information about how to set up and use Sausage, see Setting Up Sausage for Windows and Setting Up Sausage for OS X and Linux.
Running the Example PHP Test
WebDriverDemo.phpexample test that's included with Sausage to make sure everything works.
This starts the PHPUnit test runner and gives it the name of an example test suite that Sausage downloaded. After a few moments you should see that PHPUnit has started. You might not see any output instantaneously, but eventually you will see a series of dots inching across the screen.
Each of these dots represents a test that successfully passed. If a test had an error or if it failed, PHPUnit prints an
E or an
While the tests are running, navigate to your Sauce Labs tests page. From there you'll be able to see each test as it queues, runs, and finishes. You'll notice that each test has a name -- that information is sent automatically by Sausage at the beginning of each test. Sausage also automatically notifies Sauce of the status of your tests when they complete.
Right now each test runs one at a time because PHPUnit currently doesn't support running multiple tests in parallel, however we've developed a way to do that which we'll describe in one of the later tutorials. For now, take advantage of the serial nature of the tests and click on a running test on your tests page. You'll jump to a detail view where, if you caught the test while it was running, you'll be able to watch Selenium controlling the browser. When the test finishes, the page updates with a video of the test and a list of the various Selenium commands that were sent.
If you don't catch a test while it's running, you can click the test's link on the Sauce Labs tests page to see the test's details and video.
Finding Your Username and Access Key
You can find your Sauce Labs username and access key in the User Profile > User Settings section of your Sauce Labs dashboard. You should also check out our topic on setting your username and access key as environment variables.
Running Local Tests
localhostis quick and efficient. The drawback is that
localhostis not a publicly-accessible address on the Internet, so the browsers in the Sauce Labs cloud can't load and test your app. The solution is to use Sauce Connect. Sauce Connect is an application that creates a secure tunnel connection between the Sauce Labs virtual machine that runs your test and your local network. You can also use Sauce Connect to test applications and websites that are located within your corporate firewall. Sauce Connect is not required to run tests on Sauce Labs, only in situations where the application or website you want to test is not publicly accessible.
Running Tests in Parallel
You can run your tests in parallel at two levels, and you can run your tests in parallel across multiple browsers. For example, if you have 10 tests and want to run on five browsers, this would be parallelism of five. You can also run tests across browsers and each test in parallel. Using our previous example this would be more like 50 parallel tests. Doing this requires that your tests are written in a way that they do not collide with one another. For more on this see the Selenium WebDriver - Running Your Tests in Parallel blog.
Before you start running tests in parallel, you should review the Best Practices for Running Tests with Sauce Labs, especially the topics on avoiding external test dependencies and avoiding dependencies between tests.
Match Thread Count to Concurrency Limit
You should match your thread count to your concurrency limit, which is shown in the My Account section of your user profile information on the Sauce Labs dashboard.
Paratest is a command line tool for running PHPUnit that comes bundled with Sausage that you can use to run PHP tests in parallel on Sauce using the PHPUnit test framework.
These examples, for Mac OS X/Linux and Windows, show the commands used to specify the path to the test file you want to run, and to simultaneously run two instances of PHPUnit. In these examples, the path to the
WebDriverDemo.php demo test is shown, while the
-p 2 arguments set the number of instances to run.
Reporting Tests to the Sauce Labs Dashboard
By default, Sauce Labs doesn't know how to display the name of your test. Sausage comes up with a good name (
TestClass::testFunction) and reports it with your test so it's easy to find on your dashboard. Similarly, Sauce has no way to know if a particular test passed or failed. Sausage catches any failed assertions and reports the status of the test to Sauce after it's complete. Upon test failure Sausage will generate an authorized link to the failed job report on the Sauce Labs website, to facilitate reporting to people who need to know the details of the test. The job remains private (unless you change the status yourself), but others can follow the link without needing to log in with your credentials. See the topic Building Links to Test Results for more information.
You should also follow our recommended best practice of adding build numbers, tags, and other identifying information to your tests so you can easily find and manage them in your test results and archives pages, and associate tests with build numbers in your continuous integration pipeline.