Front End Performance Testing FAQs

How are performance metrics collected for my tests?

Sauce Labs collect history for performance metrics based on the test name and platform combination. The browser version is currently ignored.    

Can everyone in my organization view the front end application performance stats?

Yes, anyone who has access to the Test Details page can access the performance metrics captured during the test execution process. 

Why aren't performance metrics captured for all of the URLs that were accessed during the test?

We are currently only able to capture performance for full page loads, which means that unless your test explicitly navigates to a page, for example via the navigate command, or a click command the performance metrics won't be captured. 

Specifically, the issue might be more obvious when the test navigates around a web application built on top of modern JS frameworks (Angular, React, Vue.js) since they change the URL in the address base but the page load event is rarely be called past the initial page load. 

Why is "Some data weren't captured during this test run" message displayed? 

There are cases our system wasn't able to capture all of the performance metrics for the given page load for several reasons

• The page loaded too quickly and some metrics weren't captured (e.g. when pages load in less than 1s) 
• The onload command was called by the browser before any content appeared on the page (common with some type of Single Page Applications)
• There were several back to back redirects (common for times when the page has to go through multiple SSO login screens)
• The tests ended too quickly before performance logs were completely uploaded

Which browsers are supported?

As of now, performance metrics can only be captured on Google Chrome. We a looking into adding support for other browsers in the future. 

How are baselines defined? 

Baselines are determined by calculating a confidence interval over the prior runs of the same test name for each URL visited during the test. The confidence interval is calculated to estimate the range of values which are expected in future runs, based on past observed performance. Observations outside of the baseline range are statistically unlikely to be observed in the absence of some fundamental change in the application’s performance.