The Sauce Labs Cookbook

Sauce Headless

Front End Performance Testing

Insights

External Resources

More Info


Page tree
Skip to end of metadata
Go to start of metadata

The approach for Jankiness measurement is slightly different than the one for Page Load Performance. To get a sense of the smoothness of the page we need to interact with it. A common user interaction where jankiness is exposed to users is when one is scrolling in the application. In order to make the perceived performance testable we created a custom command that you can use independently from the framework you use. The command executes a set of automation actions:

  1. It jumps to the top of the back
  2. It then scrolls slowly from the top to the bottom for ~5 seconds
  3. It captures the browser activity for the time it was scrolling through the app
  4. Based on this browser activity it generates the metrics you can use to assert the performance

Enterprise Plans Only

This feature is available for Enterprise plans only. For more information about other benefits that are included with Enterprise plans, check out our Pricing page.

Chrome Only

This feature is available only for testing web applications with the Google Chrome browser.

Custom Command

The custom command is a WebDriver extension that can be used with any framework like Selenium or WebdriverIO. If the framework doesn't support it natively you can always use the JS Executor to call the command:

result = driver.execute('sauce:jankinessCheck')

The Metrics

The result object contains the url that you are currently visiting, the overall performance score, and detailed metric information:

Jankiness Metric Example
{
    "url": "https://googlechrome.github.io/devtools-samples/jank/",
    "score": 0.930758765052368,
    "loaderId": "f7c79740-c355-11e9-8093-5738bb1d1eb8",
    "timestamp": 1566310960826,
    "value": {
        "diagnostics": {
            "fpsScore": 1,
            "layoutUpdateScore": 1,
            "memoryUsageScore": 0.9875818052726453,
            "idleDurationScore": 0.8086081810921014
        },
        "metrics": {
            "averageFPS": 60.07786876377374,
            "forcedReflowWarningCounts": 0,
            "memoryUsageDiff": 13197064,
            "scriptingTime": 411.6740007996559,
            "renderingTime": 352.8389992415905,
            "paintingTime": 110.0319998562336,
            "idleTime": 4131.311000347137,
            "otherTime": 103.3069997727871,
            "scrollTime": 5109.163000017405
        }
    }
}

We recommend asserting against the overall Jankiness Score as it offers a general idea of how smooth the website behaved. You can then assert against the value to ensure it doesn't go below a minimum value:

Jankiness Test Example using WebdriverIO
/**
 * WebdriverIO example
 */
const assert = require("assert")


describe("my performance test", () => {
    it("should provide a smooth experience", () => {
        browser.url("https://saucelabs.com")

        const result = browser.execute("sauce:jankinessCheck")
        assert.true(result.score > 0.93, "App performance was janky")
    })
})

You can find an example script to play around with in our samples repository.

  • No labels