The Sauce Labs Cookbook

Sauce Headless

Front End Performance Testing

Insights

External Resources

More Info


Page tree
Skip to end of metadata
Go to start of metadata

This is a glossary of terms commonly used in the context of Sauce Labs testing.

Glossary of Sauce Labs Terminology

A

Analytics

See: Insights.

Appium

An open source mobile UI automation framework that uses the Selenium WebDriver protocol to control interaction with native apps, mobile web apps, and hybrid apps in your tests. Appium acts as a wrapper that translates Selenium WebDriver commands into iOS and Android commands. With Sauce Labs, you can use Appium to test mobile apps on emulators, simulators, and real devices. More information: Appium Bootcamp by Dave Haeffner and Matthew Edwards.

See also: selenium, webdriver.

Application Under Test (AUT)

A web or mobile app in the test phase of the software development cycle.

See also: software development lifecycle.

Automated Testing

A testing method where you use separate software to control the execution of tests on your own software and compare your actual test results to your expected results. You can use frameworks like Appium and Selenium to control the execution of automated tests on your web and mobile apps.

B

Build:

1: A suite of individual Sauce Labs tests on various parts (e.g., page objects) of a website or app using any platform/browser combination, bundled together in the same session. A build is defined when you add the same build number to the code for tests in that suite. More information: Best Practice: Use Build IDs, Tags, and Names to Identify Your Tests.

2 : The process by which source code is compiled and converted into an executable or binary pre-release version of your software program. Builds are often comprised of multiple smaller builds.

C

Camera Image Injection

A Sauce Labs feature that enables you to mimic camera behavior when testing applications on the Real Device Cloud by letting you upload an image (in .jpeg .jpg, or .png format) from your computer or another location and presenting it to the application as if it was read by the device camera.

CI/CD Pipeline

An end-to-end software development process that supports continuous integration and continuous deployment throughout the software development lifecycle (building, testing, and deploying software).

See also: CI/CD platform, software development lifecycle.

CI/CD Platform

A pipeline-driven software platform that automates the CI/CD pipeline process at scale. You can configure your CI/CD platform to run tests on Sauce Labs using one of our platform-specific proprietary plug-ins. More information: Using Sauce Labs with Continuous Integration Platforms.

See also: CI/CD pipeline, continuous integration, continuous deployment

Colliding Tunnels

A Sauce Connect Proxy test scenario where two or more tunnels are launched with the same tunnel identifier name. By default, duplicate tunnels are removed from your test unless you append your code with the command line --no-remove-colliding-tunnels. More information: Sauce Connect Proxy Tunnel Types: Ephemeral, Long-Running, Combination.

See also: sauce connect proxy, tunnel identifier

Concurrency Limit

The maximum number of total Sauce Labs tests -- both automated and manual -- that you can run simultaneously across all user accounts within your organization. Concurrency limits vary according to pricing plan. Once you and/or your teammates have used all concurrency slots, additional tests will not launch until an existing test has finished. More information: Understanding Concurrency Limits and Team Accounts.

Continuous Deployment (CD)

A software development practice where code that has passed all required tests is immediately and automatically deployed into production.

See also: CI/CD pipeline.

Continuous Integration (CI)

A software development practice where all code changes are regularly committed to a shared repository and re-tested to collect and act on feedback.

See also: CI/CD pipeline.

Continuous Testing

The process of continuously executing automated tests throughout your software development lifecycle, allowing you to collect and act on feedback.

See also: CI/CD pipeline.

Continuous Testing Benchmark

See: Sauce Labs Continuous Testing Benchmark.

Cross-Browser Compatibility

The consistency of your web or mobile app's user experience across multiple combinations of browsers, devices, and operating systems.

Cross-Browser Testing

A method of testing where you verify the consistency of your web or mobile app when accessed through multiple combinations of browsers, devices, and operating systems. With Sauce Labs automated testing, you can test thousands of these combinations in parallel. More information: Sauce Labs Cross-Browser Testing.

D

Data Center (DC)

A network that houses the set of Sauce Labs services relevant to your license type and your company's needs (i.e., geographic location, real vs. virtual device, and optional other services). To run a Sauce Labs test, you must connect to one or more data centers by including the appropriate endpoint URL(s) in your test script. More information: Data Center Endpoints.

Desired Capabilities (CAPS)

A section of code required in automated test scripts to specify test parameters (e.g., OS, browser, API, device) used to configure the environment for your Selenium, Appium, and Sauce Labs tests. More information: Platform ConfiguratorTest Configuration Options.

See also: platform configurator

E

Emulator

A virtual machine used to mimic the software, operating system, and certain device features (e.g., camera, touch ID, GPS) of the Android mobile application that you're testing in Sauce Labs. Can be used to test multiple browser/device combinations and use cases.

See also: simulator, real device testing.

Enterprise

1 : The Sauce Labs subscription plan that offers the largest amount of testing bandwidth and premium benefits.
2 : A Sauce Labs customer subscribed to our enterprise plan, which offers a dedicated account team and premium support. For more information, speak to your Customer Success Manager or refer to saucelabs.com. More information: Sauce Labs Pricing.

F

Failure Analysis

A Sauce Labs Insights tool that analyzes failures that occur during test runs and reveals any common root causes so that you can debug as quickly as possible. More information: Using Failure Analysis.

See also: Insights.

Failure Pattern

A Sauce Labs Failure Analysis metric that shows a specific, recurring error that's causing test and build failures. You can see the amount of tests impacted and the percentage of total failures attributed to each error.

See also: failure analysis.

Framework

The UI automation library and test runner combination that you use for testing. You can tailor your framework to meet your situation and test goals.

See also: UI automation library, test runner.

Free Trial

A period for prospective customers to explore the full functionality of the Sauce Labs platform for free. Includes automated cross-browser testing, live testing, and access to real devices for mobile testing. More information: Sauce Labs Free Trial.

Front-End Performance Testing

A method of performance testing that enables you to check UI functionality like forms, graphs, and menus, as well as associated JavaScript. Sauce Labs offers a front-end performance testing tool called Sauce Performance, which you can integrate with your existing CI/CD workflows. Please note that Sauce Performance refers to front-end testing only (i.e., tools like Google Lighthouse, GTmetrix that measure how quickly the user can see and interact with your website); it doesn't have back-end load testing functionality (i.e., JMeter, Gatling). More information: Getting Started with Sauce Front-End PerformanceSauce Labs White Paper: Best Practices for Front-End Performance Testing

See also: performance testing.

Functional Testing

A method of testing that validates some functionality or feature of your application. The output of these tests should generally be a simple "pass" or "fail" – either your functionality worked as expected, or it didn't. 

See also: non-functional testing.

H

Headless Browser

A browser or browser simulation without a UI. It's considered by developers to be a lightweight and scalable option if you want to test and collect pass/fail data earlier in the development lifecycle. Available only for Chrome and Firefox.

See also: sauce headless testing.

Hybrid App

A mobile app written in platform-agnostic web technologies like HTML5, CSS, and JavaScript. Hybrid apps run inside a native container and leverage the device’s browser engine to render the HTML and process the JavaScript locally.

I

Image Injection

See: camera image injection.

Insights

A Sauce Labs analytics tool that tracks and reports how your tests are performing over time, allowing you to quickly identify and remediate risk, improve productivity, and create digital confidence in your entire organization. More information: Insights.

Invoice Customer

See: enterprise customer.

IPSecVPN

A protocol used to establish a secure VPN connection between applications hosted on an internal server and the Sauce Labs virtual machines or real devices used for testing. More information: IPSec VPN.

L

Live Testing (LT)

A type of software testing where you'd execute test cases manually, without using any automation tools. More information: Live Testing of Web Apps with Sauce Labs.

See also: manual testing.

logfile

A file where various Sauce Labs processes record events that occur during testing. Access to different logfiles depends on the process that generated them.

M

Managed Customer

See: enterprise customer

Manual Testing

See: live testing.

Mobile App

See: native app, hybrid app.

N

Native App

A mobile software app written in a programming language specific to the platform it is being developed for: either iOS or Android. More information: Live Testing for Native Mobile Apps on Real DevicesMobile App Testing with Native Frameworks on Virtual and Real Devices with Sauce Labs.

See also: hybrid app.

Non-Functional Testing

A type of software testing that validates behavioral, measurable aspects of the software (e.g., performance, compatibility, user experience). Functional testing determines if your software meets its business requirements, whereas non-functional testing determines how it operates. When running non-functional tests in Sauce Labs, you can use custom extensions for WebDriver that will allow you test the performance of your website under specific network conditions and collect network and application-related metrics.

See also: functional testing.

O

OnDemand Service

A prime facility and feature of cloud computing services that allows users to provision raw cloud resources at run time, when and where needed.

Organization Admin

The Sauce Labs account admin role that can manage permissions levels for all users, oversee Sauce Labs test settings and activity for their organization, create Teams and Team Admins, designate other Organization Admins, and set concurrency allocations among different Teams. More information: Account and Team Management.

See also: team adminteam management

P

Parallel Testing

1 : The practice of running multiple tests simultaneously.
2 : When signing up for a self-service license, this is equivalent to your account's concurrency settings. More information: System and Network Requirements for Sauce Connect ProxyUsing Frameworks to Run Tests in Parallel.

See also: concurrency limit.

Parallelization 

See: parallel testing.

Performance Metrics

The data that developers and QA teams use to capture and address performance regressions early in the development cycle. More information: Sauce Connect Proxy Performance Metrics.

Performance Testing

A type of non-functional software testing that ensures your software responds as expected on the front end and meets your requirements under expected workload. Sauce Labs supports front-end performance testing; see front-end performance testing for more information.

See also: front-end performance testing.

pidfile

A text file generated by Sauce Connect Proxy that records your tunnel's process identification number (pid) to confirm that the process is still running. Unless otherwise specified, the file will be cleaned up on exit or overwritten at startup. If needed, you can terminate a job mid-test by sending a kill signal to the pidfile. More information: How to Start and Stop Sauce Connect Tunnels (Startup and Teardown)Sauce Connect Proxy Command-Line Quick Reference Guide.

Platform Configurator

A Sauce Labs tool where you can select your desired capabilities and generate code snippets to copy and paste into your automated testing scripts. More information: Platform Configurator

See also: desired capabilities.

Proxy Auto-Configuration File (PAC)

An optional file you can use in your Sauce Connect Proxy tests to define how web browsers and other user agents automatically choose the appropriate proxy server for fetching a given URL. To use a PAC file, include the --pac <url> command-line in your code. More information: Sauce Connect Proxy Setup with Additional Proxies.

R

RDC on Sauce

The Sauce Labs effort to introduce real device testing (APIs, device endpoints, test data, and UI elements) to the Sauce Labs domain. This is one facet of the broader Unified Platform initiative. 

See also: unified platform.

Real Device Cloud (RDC)

A Sauce Labs cloud service that provides an infrastructure to test your web, hybrid, and native mobile apps on real mobile devices on the secured public cloud or a private set of real mobile devices. You can run tests on thousands of browser, operating system, and device combinations, simultaneously. More information: Automated Testing with Real DevicesLive Mobile Application Testing with Sauce LabsSauce Connect Proxy Setup for Real Device Cloud (RDC), Sauce Labs Pricing.

See also: rdc on sauce.

Real Device Testing

An automated web or mobile app test performed on real, physical devices hosted on the Sauce Labs Real Device Cloud. Real device tests yield accurate results on user interactions and display how your app will appear in real life. More information: Real Device Testing Admin GuideSauce Connect Proxy Setup for Real Device Cloud, Sauce Labs website: Real Device Cloud.

See also: real device cloud.

RemoteWebDriver

A remote instance of WebDriver that you must instantiate during a test to connect with the Selenium server via Sauce Labs. Afterwards, you can use the RemoteWebDriver to control the browser of your choice.

See also: webdriver, webdriverIO.

Restricted Domain

A Sauce Labs feature that allows organization admins to block their internal users from accessing the public-facing Sauce Labs website to prevent anyone from enrolling in a Free Trial. Only accounts originating from the domain you designate will be allowed to access Sauce Labs. This feature is being deprecated as customers migrate to our latest Team Management features.

S

Sauce Connect Host

The machine in your network on which the Sauce Connect Proxy application is running, with a direct connection to the internet. More information: Sauce Connect Proxy Setup and Configuration.

Sauce Connect Proxy

A built-in HTTP proxy server that opens a secure tunnel connection for testing between a Sauce Labs virtual machine or real device and a website or mobile app hosted on your local computer ("localhost") or behind a corporate firewall. Sauce Connect Proxy securely connects Sauce Labs and your application or website under test.

Sauce Connect Proxy Setup, Additional Proxy

A Sauce Connect Proxy network configuration for users with an existing internal network proxy or proxies through which outbound communication is routed from their network to the public internet. More information: Sauce Connect Proxy Setup with Additional Proxies.

Sauce Connect Proxy Setup, Basic

A Sauce Connect Proxy configuration for users with a network configuration requiring a proxy to open up communication between Sauce Labs and their web or mobile app in testing that is hosted on a local machine or behind a firewall. More information: Basic Sauce Connect Proxy Setup.

Sauce Connect Proxy Setup, High Availability (HA)

A Sauce Connect Proxy configuration that allows you to run a high number of tunnels individually or collectively as a tunnel pool. From an end user or test runner perspective, a pool functions the same as a single Sauce Connect instance. More information: High Availability Sauce Connect Proxy Setup.

See also: tunnel pool.

Sauce Connect Proxy Startup

The process of configuring and launching a Sauce Connect Proxy tunnel to run your tests. More information: How to Start and Stop Sauce Connect Tunnels (Startup and Teardown).

Sauce Connect Proxy Teardown

The process of gracefully shutting down and decommissioning a Sauce Connect Proxy tunnel. More information: How to Start and Stop Sauce Connect Tunnels (Startup and Teardown).

Sauce Control (saucectl)

The Sauce Labs command terminal, which can be used as an alternative to the Sauce Labs interface.

Sauce Headless Testing

An environment to execute tests against headless Chrome and Firefox on Linux containers, allowing developers to receive fast feedback on early pipeline component, sanity and pull request tests. Basic pass/fail data (screenshots, logs, and metadata) is provided to ensure that bugs are identified and fixed earlier in the development lifecycle. Considered a complementary solution to the Sauce Cross Browser Testing platform. More information: Getting Started with Sauce Headless.

Sauce Labs Access Key

A randomly generated string of alphanumeric characters assigned to your Sauce Labs account that you must include in your test scripts along with your Sauce Labs username to authenticate your request and allow access to the resources on your Sauce Labs account. Also known as Access Key in the Sauce Labs application and SAUCE_ACCESS_KEY as an environment variable. More information: Environment Variables Used by Sauce Connect ProxySauce Labs User Settings.

Sauce Labs Continuous Testing Benchmark

A Sauce Labs white paper – compiled periodically – that leverages anonymized Insights metrics from millions of user tests performed on the Sauce Labs cloud platform and identifies best testing practices and areas of improvement (e.g., test quality, test run time). Organizations can use this data to measure how their collective continuous testing efforts stack up with those of their peers. To request a copy, reach out to your Customer Success Manager.

Sauce Labs User Name

An ID name that you define when you create your Sauce Labs account. You must include this (along with your Sauce Labs Access Key) in your test scripts to authenticate your request and allow access to the resources on your Sauce Labs account. Also known as Username in the Sauce Labs application and SAUCE_USERNAME as an environment variable. More information: Environment Variables Used by Sauce Connect ProxySauce Labs User Settings.

Sauce Testrunner Toolkit (STT)

A containerized testing solution for developers that simplifies user setup, speeds up test execution time, unifies test results, and supports new open source frameworks like Puppeteer, Playwright, Cypress and TestCafe for running end-to-end web tests.

It extends Sauce Labs capabilities to support tests run using native JavaScript test frameworks, so that test results and test analytics can be viewed within Sauce Labs for better debugging.

Selenium

A portable framework for testing web applications.

Selenium Grid

A part of the Selenium suite that specializes in running multiple tests across different browsers, operating systems, and machines in parallel. Selenium Grid has two versions: Grid 1 (older) and Grid 2 (newer).

Selenium IDE

An integrated development environment for Selenium scripts – implemented as an extension for Chrome and Firefox – that allows you to record, playback, and debug tests in the browser. More information: Selenium Projects.

Selenium Relay

A listener for Selenium commands built into Sauce Connect Proxy that enables inbound and outbound test traffic to be sent through an encrypted tunnel. More information: Using the Selenium Relay with Sauce Connect ProxySauce Connect Proxy Command-Line Quick Reference Guide.

Selenium Server

A server required to run older Selenium RC tests or WebDriver tests in remote machines through the Selenium Grid.

See also: webdriver.

Self-Service

1 : A Sauce Labs customer who has purchased an online subscription plan.
2 : A plan offered by Sauce Labs whereby customers can purchase by credit card and manage their subscription online.
More information: Sauce Labs Pricing

See also: unmanaged customer

Simulator

A virtual machine environment used to mimic the overall behavior (i.e., software variables, configurations) of the iOS mobile app you're testing in Sauce Labs.

See also: emulator, real device testing.

Site Under Test (SUT)

A website in the test phase of the development cycle, following the planning, coding, and building phases. When testing in Sauce Connect Proxy, the Site Under Test will be on the same local network as the Sauce Connect Host machine.

See also: application under test, CI/CD pipeline.

Software Development Lifecycle (SDLC)

An end-to-end process used to develop, plan, design, build, test, and deploy software to production.

SSL Bumping

A feature of Sauce Connect Proxy that automatically re-signs self-signed and invalid SSL certificates, which are not trusted by stock browsers like those installed on the Sauce Labs infrastructure. With SSL Bumping, your tests will not be interrupted with security warnings that can't be dismissed by Selenium. More information: Sauce Connect Proxy and SSL Certificate Bumping.

Stress Testing

A test method where you purposely put your system under extreme conditions – above and beyond your requirements – to identify the breaking point and determine if and when your system fails gracefully.

systemd

A Linux service management tool that facilitates Sauce Connect Proxy tunnel monitoring, system startup and shutdown. More information: Monitoring Sauce Connect Proxy with Service Management Tools.

T

Team Admin

A Sauce Labs user role with permission to add and manage Team Members as well as configure test settings (e.g., concurrency) for their own team.

See also: organization admin, team member.

Team Management

A Sauce Labs feature available to enterprise users that provides organizations with the ability to create a hierarchy of Organization Admins, Team Admins, and Team Members, and manage user access to the system and concurrency settings.

See also: organization admin, team admin, team member.

Team Member

A Sauce Labs user with permission to edit their own user info, run tests, and view tests run by teammates. Depending on their Organization Admin's Team Job Sharing settings, they can also view jobs that were run by members of other teams.

See also: organization admin, team admin, team management.

Test Runner

A library or tool for writing and/or executing code for automated tests; often part of a UI test framework.

See also: framework, UI automation library.

Testing Annotation

The practice of adding test information such as names, tags, pass/fail status to your completed Sauce Labs tests, making it more manageable to search and sort your previous work. You can add annotations using tools such as the Sauce Labs REST API, Selenium JavaScript Executor, or one of our test framework examples that add annotations automatically. More information: Sauce Labs Sample Test Frameworks (GitHub)

Testing Minutes

The number of minutes allotted to a Sauce Labs account for its subscription. 

Transparent Proxy

A server that sits between your computer and the Internet and redirects your requests and responses without modifying them. If your organization has one, please refer to Sauce Connect Proxy Setup with Additional Proxies. More information: Sauce Connect Proxy Setup with Additional Proxies.

Tunnel

A secure connection between your network and Sauce Labs through which you can run a test, test suite, or build. To establish a tunnel, you must download and configure Sauce Connect Proxy. 

See also: sauce connect proxy.

Tunnel Identifier

The Sauce Connect Proxy test configuration option that allows you to assign a name of your tunnel(s), giving you more control and monitoring capability over the tunnel. If you launch a tunnel without identifying it, your test traffic will default to running through that unnamed tunnel. More information: Using Sauce Connect Tunnel Identifiers.

See also: colliding tunnels

Tunnel Pool

A set of tunnels that share the same tunnel identifier and function as a single tunnel in high availability mode.

See also: sauce connect proxy setup, high availability.

Tunnel Virtual Machine (Tunnel VM)

The virtual machine that hosts Sauce Connect Proxy on the Sauce Labs side.

U

UI Automation Library

A library or tool used for writing, running and providing functionality for browser-based tests in a particular setting. Some examples are WebdriverIO (JavaScript), Cucumber (Ruby/JavaScript/C#/Java), XCUITest (iOS mobile only).

See also: framework.

Unified Platform

The Sauce Labs initiative to deliver a fully integrated, unified test experience across all of our products and solutions, with one single login. We're combining virtual device and real device testing assets, functional and non-functional (visual) testing methodologies under the Sauce Labs domain. See also: RDC on Sauce.

Unmanaged Customer

See: self-service

Upstart

See: systemd.

V

Virtual Device Cloud (VDC)

A Sauce Labs cloud service that provides an infrastructure to virtually test your desktop websites and mobile device apps on thousands of browser, operating system, and device combinations. More information: Sauce Labs Pricing.

See also: real device cloud.

Virtual Cloud

See: virtual device cloud.

Virtual Machine (VM)

A virtual software development environment that functions like an isolated, actual computer, with its own CPU, memory, network interface, and storage. A Sauce Labs virtual machine runs on a Sauce Labs server and appears on your host machine as a process in a window. You can run multiple virtual machines at the same time. 

Visual Testing, Component

A Sauce Labs visual testing method that enables you to test individual UI components in isolation. If you’re using a component library like Storybook, Vue, Angular or React, you can will use your existing stories as visual test cases and run them against our automated visual testing service.

Visual Testing, End-to-End (E2E)

A Sauce Labs automated visual testing method that integrates with your WebDriver tests and code, enabling you to combine functional and visual regression UI testing across different browsers and resolutions in the same run.

W

W3C WebDriver Protocol

A platform- and language-neutral wire protocol that enables out-of-process programs to remotely instruct the behavior of web browsers. With Sauce Labs and all major browser vendors now supporting the W3C standard, automated Selenium tests will run with more stability and consistency between different browsers and devices. More information: W3C Capabilities Support.

WebDriver

Selenium's built-in library/API that drives web browser interactions in your automated tests. Supports all major browsers and programming languages. More information: Selenium Projects.

See also: appium, UI automation library, webdriverIO

WebdriverIO

A custom UI test automation library, written in JavaScript on Node.js, that is used for Selenium's W3C WebDriver API. It is not directly affiliated with the Selenium project.

See also: UI automation library, webdriver.