Failure Analysis gives you insight into where you can immediately improve your test execution on Sauce Labs. With a single dashboard view within the Sauce UI, Failure Analysis uses machine learning to view pass/fail data and Selenium and Appium command logs to unearth common failures, and how they are affecting the test suite as a whole. It then presents a report with tabs that aggregate patterns that are indicative of failure.
Failure Analysis currently in Beta
If you're interested in trying Failure Analysis, please send inquiries to: firstname.lastname@example.org
What You'll Need
- A Sauce Labs Account
- Access to the beta feature
- Your Selenium/Appium test status set to pass or fail
Common Business Use Cases
Failure Analysis was designed to help you optimize test efficiency and pass rates. It's also a key feature for finding and troubleshooting test failures that occur for similar or duplicate reasons. The two key advantages to using Failure Analysis are:
Improves developer efficiency, and helps them quickly uncover and triage the most pervasive errors
Validates investment in test automation as to quickly deliver an improved end-user experience, while also mitigating risks of poor quality
How it Works
Failure Analysis leverages your test data and identifies potential failure patterns based on aggregate test errors. More specifically, the tool:
Identifies tests experiencing failures
Aggregates failures on test names
Identifies common failure patterns
Ranks and prioritizes patterns by most pervasive impact
For example, in the screenshot below, there's a failed build where each test contains a bad, or outdated, web element locator. Failure Analysis detects any failure patterns and attributes a percentage to show how pervasive this failure is within this particular build.
To see the specifics of each failure pattern, go to Insights > Failure Analysis, or select Failure Patterns when viewing data about your build. In the screenshot below, you see the specific data regarding your test failure, the amount of tests impacted, and the percentage of total failures.
- Failure Analysis FAQs
- Guide to Sauce Insights
- Please send all inquiries to: email@example.com
- Join #failure_analysis_beta Slack channel