Running parallel Cypress tests in TeamCity using docker

The Problem

Not that long ago I stumbled across the problem of running Cypress tests in parallel to speed up the testing process. Unfortunately, our project setup required me to run those tests on a single TeamCity agent. The problem was that Cypress does not support parallel builds on one machine, and using available parallelization libraries for Cypress made more problems than it solved. Also some tests required to run one after another so I needed a manual way to assign tests to a specific thread instead of randomly splitting tests between multiple threads.

The Solution

Since I was familiar with docker I knew that it can manage resources pretty well so my idea was to spread the tests across docker containers and run them in parallel so docker will utilize the full potential of TeamCity Agent CPU instead of running Cypress tests on a single thread. Turns out this solution cut test times in half which was a pretty satisfying result. To specify which test should run in which container i have used cypress-grep

Requirements

1. Docker

2. Cypress

3. Cypress Grep

4. TeamCity working with your git project

Steps

1. Setup project with requirements

2. Create cypress.config.js

    const { defineConfig } = require("cypress");
   
    module.exports = defineConfig({
      projectId: "test-project",
      e2e: {
        setupNodeEvents(on, config) {
          //setup cypress grep
          require("@cypress/grep/src/plugin")(config);
          config.env.grepOmitFiltered = true;
          config.env.grepFilterSpecs = true;
          config.env.grepTags = process.env.GREP_TAGS;
          return config;
        },
      },
    });
   

3. Create a cypress.dockerfile

    FROM cypress/included:12.8.1
   
    COPY package.json package.json
   
    RUN npm i
   
    COPY cypress.config.js cypress.config.js
    COPY cypress/ cypress/
   

4. Add cypress run script to package.json

     "scripts": {
          "e2e-run": "cypress run --browser chrome --headless"
    }
   

5. Create docker-compose.yml

    version: '3'
    services:
      test1:
        image: <your_docker_registry_host>/cypress-parallel:1.0.0
        environment:
         GREP_TAGS: "Container1"
        command: ["/usr/local/bin/npm", "run", "e2e-run"]
      test2:
        image: <your_docker_registry_host>/cypress-parallel:1.0.0
        environment:
         GREP_TAGS: "Container2"
        command: ["/usr/local/bin/npm", "run", "e2e-run"]
   

6. Tag Your cypress tests with tag coresponding to container You want to run it in

    describe("First Container Test Spec", { tags: "Container1" }, () => {}
    describe("Second Container Test Spec", { tags: "Container2" }, () => {}
   

7. Prepare bash script cypress-build-image.sh to run in teamcity

    #!/bin/sh
   
    # DOCKER_REGISTRY_HOST, DOCKER_REGISTRY_USERNAME, DOCKER_REGISTRY_PASSWORD : all must be set as environmental variables in teamcity
   
    set -eu
    #create temorary directory
    mkdir -p tmp
    rm -rf tmp/*
    # copy required files to temp directory
    cp -R ./cypress-build-image.sh tmp/
    cp -R ./cypress.dockerfile tmp/
    cp -R ./package.json tmp/
    cp -R ./cypress tmp/cypress
    cp -R ./cypress.config.js tmp/
    #login to docker regitry 
    docker login -u "${DOCKER_REGISTRY_USERNAME}" -p "${DOCKER_REGISTRY_PASSWORD}" "${DOCKER_REGISTRY_HOST}"
    #pull cypress image
    docker pull cypress/included:12.8.1
   
    _image_name=cypress-parallel:1.0.0
   
    docker build -f tmp/cypress.dockerfile -t ${_image_name} tmp/
   
    docker image push ${_image_name}
    #
    docker-compose up
    # code below will ensure that if one of containers crashes the process will exit with error code in teamcity
    EXITCODES=$(docker-compose ps -q | xargs docker inspect -f '{{ .State.ExitCode }}' | grep -v '^0' | wc -l | tr -d ' ')
    if [ ${EXITCODES} -ne 0 ]; then
      echo "!!!!!!!!!!!!!!!!Some tests failed!!!!!!!!!!!!!!!!!!"
      exit ${EXITCODES}
    else
      echo "!!!!!!!!!!!!!!!All tests passed!!!!!!!!!!!!!!!!!!"
    fi
    #shut down docker-compose
    docker-compose down
    #remove temp directory
    rm -rf tmp/
   

8. Add files to git, remember to git update-index --chmod=+x cypress-build-image otherwise the script won't start in TeamCity

9. Git Commit and Push

10. Add Configuration in TeamCity.

11. In "Parameters" tab add Environmental variables DOCKER_REGISTRY_HOST, DOCKER_REGISTRY_USERNAME, DOCKER_REGISTRY_PASSWORD with your propper credentials

12. Add build step to created config

    

13. Run the configuration

14. Spend the time you saved on test runs on reading my other articles :)

    You can run as many containers as you want but it looks like the best results are obtained when the number of containers is similar to number of threads on your Team City agent