The Sauce Labs Cookbook

Sauce Headless

Front End Performance Testing

Analytics

External Resources

More Info


Page tree
Skip to end of metadata
Go to start of metadata

Speedo has been developed in a way so that it organically fits into any continuous integration and delivery pipeline. By using our published Docker image, you can easily integrate it into JenkinsBamboo or CircleCI. The following example shows a regular Jenkins pipeline with linting, unit, functional and performance tests using the CLI tool

Enterprise Plans Only

This feature is available for Enterprise plans only. For more information about other benefits that are included with Enterprise plans, check out our Pricing page.

Chrome Only

This feature is available only for testing web applications with the Google Chrome browser.

See the following sections for more information: 


Jenkins Pipelines Example

Note The example below is using a "sauce" attribute as a way to pass the Sauce Labs credentials into the environment and is optional

node('linux') {
    stage('Checkout') {
        echo 'Code checkout'
        git <repo link>
    }
    
    stage('Functional Tests ') {
        sauce('xxx') {
             echo 'Running tests'
             sh 'cd  && npm install && npm update && npm test'
        }
    }
    
    stage('Performance Test') {
        sauce('xxx') {
            echo 'Running Performance Tests'
            sh 'speedo run https://www.saucedemo.com'
        }
    }
}

Jenkins Pipelines with Docker Example

If you run your CI/CD pipeline based on Docker (e.g. Jenkins Pipelines) you can also use Speedo as a Docker container. Just define the container name and have the speedo command available in the stage, e.g.:

pipeline {
    agent none
    stages {
        stage('Code Checkout') {
            agent { node 'intrev'}
            steps {
                echo 'Code checkout'
                git <URL>
            }
        }
        stage('Functional Tests') {
            agent { node 'intrev'}
            steps {
                sauce('xxxx') {
                    echo 'Running tests'
                    sh 'cd WebDriver.io/ && npm install && npm update && npm test'
                }
            }
        }
        stage('Performance Test') {
            agent {
                docker {
                    image 'saucelabs/speedo'
                    args '--entrypoint=""'
                }
            }
            steps {
                sauce('xxxx') {
                    echo 'Running Performance Tests'
                    sh 'speedo run https://www.saucelabs.com'
                }
            }
        }
    }
}

GitLabs Pipelines with Docker Example

variables:
  SPEEDO_IMAGE: saucelabs/speedo

stages:
  - lint
  - test
  - performance
  - deploy

# ...

# run performance tests
performance:
  stage: performance
  image: $SPEEDO_IMAGE
  script:
    - speedo run https://google.com -u $SAUCE_USERNAME -k $SAUCE_ACCESS_KEY -b $BUILD_NUMBER

# ...

Accessing Performance Data

Speedo automatically downloads all performance metrics from the test to make them accessible to you. You can also download the generated tracelog by passing in --traceLogs as parameter and after the test inspect the JavaScript execution of the page load in detail. By setting a --logDir parameter you control where all these artifacts are stored:

# create directory for logs
$ mkdir ./performanceLogs


# run Speedo
$ speedo run https://json.org --logDir ./performanceLogs
✔ Start performance test run with user cb-onboarding on page https://json.org...
✔ Run performance test...
✔ Wait for job to finish...
✔ Download performance logs...
✔ Updating job status...
📃 Stored performance logs in /path/to/project/performanceLogs

Performance Results
===================
score: 100/100
estimatedInputLatency: 16ms
timeToFirstByte: 579ms
domContentLoaded: 1s
firstVisualChange: 1.1s
firstPaint: 27ms
firstContentfulPaint: 27ms
firstMeaningfulPaint: 27ms
lastVisualChange: 2.4s
firstInteractive: 1s
load: 2.5s
speedIndex: 1.1s

Result: pass ✅

⚙️  Runtime settings:
- Network Throttling: Good 3G
- CPU Throttling: 4x
- CPU/Memory Power: 932 (desktop-class device)
- User Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 7 Build/MOB30X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3765.0 Safari/537.36

👀 Check out job at https://app.saucelabs.com/performance/911b52e982074befbc37a5769dc64253/0
📔 Check out Lighthouse Report at https://eds.us-west-1.saucelabs.com/911b52e982074befbc37a5769dc64253/performance/B550FACF42068708DBAA0146F3324E76/lhr.html

After the test you can access the data and for example push it to an analysis service or run additional checks on it:

cat ./performanceLogs/performance.json
[
    {
        "sessionId": "911b52e982074befbc37a5769dc64253",
        "url": "https://json.org/",
        "orderIndex": 0,
        "loaderId": "B550FACF42068708DBAA0146F3324E76",
        "metrics": {
            "rtt": 37,
            "load": 2524,
            "score": 0.9999563115602864,
            "maxRtt": 37,
            "numFonts": 1,
            "numTasks": 9,
            "fontSize": 33452,
            "firstPaint": 27,
            "fontCount": 1,
            "imageSize": 173067,
            "numScripts": 0,
            "otherSize": 1300,
            "speedIndex": 1130,
            "throughput": 1368654.9157853597,
            "imageCount": 7,
            "numRequests": 12,
            "otherCount": 2,
            "firstCpuIdle": 1044,
            "requestsSize": 234436,
            "documentSize": 26617,
            "requestsCount": 11,
            "totalTaskTime": 14181,
            "documentCount": 1,
            "numStylesheets": 0,
            "timeToFirstByte": 579,
            "totalByteWeight": 218386,
            "domContentLoaded": 1044,
            "firstInteractive": 1044,
            "lastVisualChange": 2386,
            "maxServerLatency": 549,
            "numTasksOver10ms": 2,
            "numTasksOver25ms": 1,
            "numTasksOver50ms": 1,
            "firstVisualChange": 1104,
            "numTasksOver100ms": 1,
            "numTasksOver500ms": 1,
            "totalBlockingTime": 297,
            "firstContentfulPaint": 27,
            "firstMeaningfulPaint": 27,
            "estimatedInputLatency": 16,
            "mainDocumentTransferSize": 325
        }
    }
]
  • No labels