The Sauce Labs Cookbook

Sauce Headless

Front End Performance Testing

Insights

External Resources

More Info


Page tree
Skip to end of metadata
Go to start of metadata

 BETA 

Failure Analysis gives you insight into where you can immediately improve your test execution on Sauce Labs. With a single dashboard view within the Sauce UI, Failure Analysis uses machine learning to view pass/fail data and Selenium and Appium command logs to unearth common failures, and how they are affecting the test suite as a whole. It then presents a report with tabs that aggregate patterns that are indicative of failure.

Failure Analysis currently in Beta

If you're interested in trying Failure Analysis, please send inquiries to: failureanalysis@saucelabs.com

What You'll Need

Common Business Use Cases

Failure Analysis was designed to help you optimize test efficiency and pass rates. It's also a key feature for finding and troubleshooting test failures that occur for similar or duplicate reasons. The two key advantages to using Failure Analysis are:

  • Improves developer efficiency, and helps them quickly uncover and triage the most pervasive errors

  • Validates investment in test automation as to quickly deliver an improved end-user experience, while also mitigating risks of poor quality

How it Works

Failure Analysis leverages your test data and identifies potential failure patterns based on aggregate test errors. More specifically, the tool:

  • Identifies tests experiencing failures

  • Aggregates failures on test names

  • Identifies common failure patterns

  • Ranks and prioritizes patterns by most pervasive impact

For example, in the screenshot below, there's a failed build where each test contains a bad, or outdated, web element locator. Failure Analysis detects any failure patterns and attributes a percentage to show how pervasive this failure is within this particular build.

To see the specifics of each failure pattern, go to Insights > Failure Analysisor select Failure Patterns when viewing data about your build. In the screenshot below, you see the specific data regarding your test failure, the amount of tests impacted, and the percentage of total failures.


A best practice is to add context to your Selenium commands with the JavaScript executor. The executor adds more context-sensitive metadata to indicate specific test steps and potentially why your tests fail. Read the following page for more information: Providing Context for Selenium Commands with the JavaScript Executor.

Additional Resources