Setting up Cypress tests for a Node.js app on Azure DevOps

Introduction

This shows how to set up Cypress(cypress.io) tests towards a Node.js application on an Azure DevOps pipeline, based on how the team did this for the first time, step by step.

The steps were done using shared team knowledge.

Overview of steps

• Goal 1: Run Cypress test towards a Node.js app
• Goal 2: Run Cypress tests on an Azure DevOps pipeline
• Goal 3: Send test results to a Cypress Dashboard
• Goal 4: Configure pipeline to fail if tests fail

Goal 1: Run Cypress test towards a Node.js app

Background: Selection of Cypress as the test framework

Cypress was the test automation framework chosen for this project due to it seeming like a good match for the project based on a previous trial, seeing this used successfully for another project, and experience with other test frameworks.

The project is a 20 week, 10 sprint delivery for revamping web application used for both back office administration and customer facing administration. The database will be migrated to the cloud, an API will be built and a new GUI will be created. The team consists of an architect, backend / middleware developers, a front end developer, a tester (myself) and project management.

Try a quick Cypress pilot locally

The front end developer created a Vue app and some Cypress tests.

The tester (myself) setup several different types of Cypress tests to test different elements in a page for a standard web application, based upon the Cypress getting started manual.

In this way we did not depend upon each other and could share different learnings.

We them demo’d this to each other, and were keen to continue with Cypress.

Commands for running the tests on the command line in the node application

Here are the commands for running the tests in UI and headless mode.

UI mode is how the tests will be run locally when building and testing new features:

npm run test:e2e

Headless mode is how the tests will be run on the pipeline:

npm run test:e2ehl

Add the shortcuts “test:e2e” and “test:e2ehl” to package.json

Snip:

{
 "name": "app-ui",
 "version": "0.1.0",
 "private": true,
 "scripts": {
 "serve": "vue-cli-service serve",
 "build": "vue-cli-service build",
 "test:unit": "vue-cli-service test:unit",
 "test:e2e": "vue-cli-service test:e2e",
 "test:e2ehl": "vue-cli-service test:e2e --headless",
 "lint": "vue-cli-service lint"
 }

Test the shortcut commands locally.

Run tests in UI mode locally, with shortcut:

npm run test:e2e

Run tests in headless mode locally, with shortcut:

npm run test:e2ehl

Goal 2: Run Cypress tests on an Azure DevOps pipeline

Pre-requisite: The Azure DevOps space was set up with the UI repository

In short, create and configure a pipeline to run the tests in headless mode

Create pipeline

In Azure DevOps

• *Create pipeline > new pipeline > select Azure > select the repository > select “Node.js with Vue” > create branch and commit message with appropriate links to work item
• The pipeline will start running, take a peek and follow along with the steps to see what it does out of the box

Configure pipeline to run the tests in headless mode

Update azure-pipelines.yml to run the tests in headless mode:

Snip azure-pipelines.yml :

# Node.js with Vue

# Build a Node.js project that uses Vue.

# Add steps that analyze code, save build artifacts, deploy, and more:

# https://docs.microsoft.com/azure/devops/pipelines/languages/javascript

trigger:
 - master

pool:
 vmImage: 'ubuntu-latest'

steps:
 - task: NodeTool@0
   inputs:
     versionSpec: '10.x'
   displayName: 'Install Node.js'

 - script: |
    npm install
    npm run test:e2ehl
    npm run build
   displayName: 'npm install, test and build'

After pushing the update to the repository, verify the logs to be sure the test ran.

Snip:

My First Test
✓ Visits the app root url (982ms)

Goal 3: Send test results to a Cypress Dashboard

Cypress has a Dashboard feature which shows test run results in a simple and elegant way.

Pre-requisites

• A Cypress dashboard was created and tested locally
• It was then shared with some team members for testing
• Note that Cypress Dashboard is a paid feature depending on usage

Update package.json with the new command to send test results to the dashboard

Snip:

{..…
 "build": "vue-cli-service build",
 "test:unit": "vue-cli-service test:unit",
 "test:e2e": "vue-cli-service test:e2e",
 "test:e2ehl": "vue-cli-service test:e2e --headless",
 "test:e2e-pipeline": "vue-cli-service test:e2e --headless --record --key the-key-goes-in-here",
 "lint": "vue-cli-service lint"
 }

Update azure-pipelines.yml to run this command instead

Snip:

- script: |
     npm install
     npm run test:e2e-pipeline
     npm run build

Now when the pipeline is run it is possible to click through to the Dashboard. For example, clicking on a result shows links to test test output, a video of the test run in the case of failure, and so on.

Goal 4: Configure pipeline to fail if tests fail

Currently the azure-pipeline.yml has install, test and build in one step which is a quick proof of concept of tests running, however this results in the pipeline showing success even if Cypress tests fail.

Therefore splitting up the steps is needed as follows.

Snip of azure-pipeline.yml:

 - task: Npm@1
   inputs:
     command: 'install'
 - task: Npm@1
   inputs:
     command: 'custom'
     customCommand: 'run test:e2e-pipeline'
   displayName: 'Test'
 - task: Npm@1
   inputs:
     command: 'custom'
     customCommand: 'run build'
   displayName: 'Build'

Trigger the pipeline where there is one known failing test. If one does not exist, write one for the purpose of testing the pipeline. The pipeline now shows this error message:

The full azure-pipeline.yml is now

# Node.js with Vue
# Build a Node.js project that uses Vue.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/javascript
trigger:
    - dev
    - master
    
pool:
  vmImage: 'ubuntu-latest'
  
steps:
  - task: NodeTool@0
    inputs:
      versionSpec: '10.x'
    displayName: 'Install Node.js'
  
  - task: Npm@1
    inputs:
      command: 'install'
  - task: Npm@1
    inputs:
      command: 'custom'
      customCommand: 'run test:e2e-pipeline'
    displayName: 'Test'
  - task: Npm@1
    inputs:
      command: 'custom'
      customCommand: 'run build'
    displayName: 'Build'

A keen reader will note that the tests are also now triggered on both dev and master branches, instead of just master as in the initial steps, because the pipeline evolves along with the project.

Conclusion

So far Cypress, Node.js with Vue and the Azure DevOps pipeline are working well together.

In addition, the tools are a good match for the project.

Finally, the main success factor has been a great team working together to share knowledge and experience.

That’s it - credit goes to the team for this setup!

Test your application using Cypress, GitHub, and Azure DevOps

 Summary: 

1. create project with product curd operations. 

2. write cypress tests to test fucntionality.

3. Execute tests locally .

4. Execute test using Azure dev ops.

5. Execute tests using Git Ations. 

Very interesting comparision of Azure devops and Git actionaa and writing simple tests using cypress.

Testing is a critical activity that allows development teams to get early feedback about the correctness of their code and eliminate more issues in the early stages of the development cycle. Today’s article will talk about front-end testing with Cypress and its integration with Azure DevOps and GitHub.

What is Cypress?

Cypress is a free, open-source testing tool that runs on a Node.js process and allows a developer to write performance End-to-end, Integration, and unit tests using Mocha’s syntax in JavaScript.

It has two main components that come default in its installation:

• Cypress Test Runner: Runner that executes your actual test cases.
• Dashboard: Service that tracks and provides insight about how your tests ran.

How it works?

When you run a test using Cypress test runner, a Node.js process starts and both your application and the test code are embedded into two different iFrames in a browser managed by Cypress. The test’s code communicates with a Node.js process via WebSockets and acts as a proxy by intercepting every HTTP request from the application and then allows Cypress to mock the responses quickly.

Process Explorer

The ability to mock responses isn’t the only benefit of this architecture. By running on along your application, Cypress can access each DOM (document object model) element, the window element, and even when your test needs to perform a command like ‘click a button’, will send the command to the DOM element by using a DOM event directly instead of relying on out-of-process communication with a WebDriver.

Folder structure

After adding a new project, Cypress will automatically create the following folder structure:

.
├── cypress
│   ├── fixtures
│   │   └── data.json
│   ├── integration
│   │   └── test.js
│   ├── plugins
│   │   └── index.js
│   └── support
│       ├── commands.js
│       └── index.js
└── cypress.json

Let’s take a minute to analyze each of them:

• Fixtures: Store static data that your tests can use.
• Integration: Contains all test files.
• Plugins: Special file that executes before the project is loaded and before the browser launches. During your test, execution that modifies or extends Cypress’s internal behavior.
• Support: This folder contains files with reusable behavior, like custom commands, that run before every spec file.
• Cypress.json: This file allows you to modify the Cypress’ default behavior by supplying specific configurations like base URL, timeouts, etc.

Installation

If you are developing an application based on Node.js, you can easily add Cypress as a dependency to your package.json by executing the following command on our project root directory.

npm install cypress --save-dev

In case you are using Yarn as a package manager, the command is the following:

yarn add cypress --dev

If you don’t want to add any dependency to your project or your project is based on programming languages different from Node.js, for example, Python and Go; you can still enjoy the Cypress testing tool by running the Test Runner inside a Docker container while running the website on the host outside the container. After starting your application on the host machine, run the Docker image with the following parameters:

DISPLAY=$IP:0
docker run 
  -it \
  -v $PWD:/e2e \
  -w /e2e \
  --entrypoint cypress \
  cypress/included:3.2.0 open --project . \
  --config baseUrl=http://host.docker.internal:2222

By doing so, the test runner will point back at the host machine, and you can test your application.

Setup you local environment

Before moving forward with its integration with Azure DevOps and GitHub, clone the repository from the following link.

GitHub - GTRekter/Training: Training template

Training template. Contribute to GTRekter/Training development by creating an account on GitHub.

github.com

I mainly use this repository for demo purposes. In its 0_Application folder, you can find a .NET 5 API developed using C# and a front-end application based on React.js.

.
├── 0. Application
│   ├── Training.API
│   │   ├── Controllers
│   ├── Training.Models
│   │   ├── Managers
│   │   ├── Store
│   └── Training.Web
│       ├── public
│       └── src
│           ├── components
│           ├── images
│           └── services
├── 1. Node.js and Docker
│   └── template
├── 2. k6 and testing
│   ├── Training.LoadTest
│   └── Training.Pipelines
└── 3. Unit testing
    ├── Training.Cypress
    │   ├── fixtures
    │   ├── integration
    │   ├── plugins
    │   └── support
    └── Training.MSTest

Writing tests

First, you need to create the Cypress folder structure in your project.

.
└── cypress
    └── integration

Inside the integration, the folder creates a new JavaScript file called products and paste the following code.

describe('Testing Product CRUD operations', () => {
    beforeEach(() => {
        cy.visit('http://localhost:3000/products')
    })
    it('Adds a new product', () => {
        // Browse to add product
        cy.get('nav a').eq(1).click()
        cy.get('.dropdown-menu.show a').eq(1).click()
        // Add product
        cy.get('input[name="name"]').type('Test 01')
        cy.get('input[name="price"]').type('10')
        cy.get('button[type=submit]').click()
        // Check that there are at least one element
        cy.get('nav a').eq(1).click()
        cy.get('.dropdown-menu.show a').eq(0).click()
        cy.get('table[data-element-id="products"] tbody tr').should('have.length', 1)
    })
    it('Delete a product', () => {
        cy.get('nav a').eq(1).click()
        cy.get('.dropdown-menu.show a').eq(0).click()
        cy.get('table[data-element-id="products"] tbody tr button').eq(0).contains('Delete').click()
        cy.get('table[data-element-id="products"] tbody tr').should('have.length', 0)
    })
})

This integration test is composed of two steps. Before starting the execution of each test, the test runner will browse to the URL http://localhost:3000/products. Then it will sequentially execute the listed commands. In the first step, it browses the add product page by clicking on the relative link on the navbar, then it will fill the input, and finally will submit the form. The test will finish with the assertion that the products table will contain exactly one element. Instead, the second step will simply click on the first button that has Delete as text and check that the list is empty.

Cypress test runner execution

Integration with Azure DevOps

It is time to create your build pipeline.

• From the dashboard, select Pipelines.
• Click the New pipeline button.

Create build pipeline

• Select GitHub and the repository where the source code resides.

• First, instruct the service to trigger the pipeline when there are changes in the path Training/0_Application/Training.Web of the main branch.

trigger:
   branches:
      include:
         - main
   paths:
      include:
         - Training/0_Application/Training.Web/*
variables:
  - name: projectPath
    value: './0_Application/Training.Web/'

• Install NodeJS and the dependencies needed to run the application.

- task: NodeTool@0
     displayName: 'Use Node 12.x'
     inputs:
       versionSpec: 12.x- script: |
       yarn install --verbose
     workingDirectory: ${{variables.projectPath}}
     displayName: 'Install project dependencies'
  
   - script: |
       yarn build
     workingDirectory: ${{variables.projectPath}}
     displayName: 'Build application'

• Then, run the process in the background by using the command (Yarn run start&).

- script: |
       (yarn run start&)
     workingDirectory: ${{variables.projectPath}}
     displayName: 'Start application'

• Finally, run the integration tests and publish the results.

- script: |
       ./node_modules/.bin/cypress run --browser chrome
     workingDirectory: ${{variables.projectPath}}
     displayName: 'Run cypress tests'
- task: PublishTestResults@2
     displayName: 'Publish test results'
     inputs:
       testResultsFormat: 'JUnit'
       testResultsFiles: '*.xml'
       searchFolder: '$(System.DefaultWorkingDirectory)/cypress/reports/junit'
       mergeTestResults: true
       testRunTitle: 'Cypress tests'

The final YAML script will look similar to this:

trigger:
   branches:
      include:
         - main
   paths:
      include:
         - Training/0_Application/Training.Web/*
variables:
  - name: projectPath
    value: './0_Application/Training.Web/'steps:
   - task: NodeTool@0
     displayName: 'Use Node 12.x'
     inputs:
       versionSpec: 12.x- script: |
       yarn install --verbose
     workingDirectory: ${{variables.projectPath}}
     displayName: 'Install project dependencies'
  
   - script: |
       yarn build
     workingDirectory: ${{variables.projectPath}}
     displayName: 'Build application'- script: |
       (yarn run start&)
     workingDirectory: ${{variables.projectPath}}
     displayName: 'Start application'- script: |
       ./node_modules/.bin/cypress run --browser chrome
     workingDirectory: ${{variables.projectPath}}
     displayName: 'Run cypress tests'
     continueOnError: true- task: PublishTestResults@2
     displayName: 'Publish test results'
     inputs:
       testResultsFormat: 'JUnit'
       testResultsFiles: '*.xml'
       searchFolder: '$(System.DefaultWorkingDirectory)/cypress/reports/junit'
       mergeTestResults: true
       testRunTitle: 'Cypress tests'

It’s now time to push your change to your repository and run the pipeline.

Execution using Azure DevOps

Integration with GitHub Actions

First, you need to create a new GitHub Action in your repository.

• Browse to GitHub, log in with your account and select your repository.
• From the navigation bar, select Actions.
• Click the New Workflow button.

GitHub Actions

• GitHub will then propose many templates, but in our case, we need to start from scratch. For this reason, click the link set up a workflow yourself.

Setup a workflow from scratch

• First, you need to specify what will trigger your action. In this case, I want to trigger the action every time there is a change in the directory 0_Application/Training.Web/ of the main branch.

on:
  push:
    paths:
      - 0_Application/Training.Web/**
    branches:
      - main

• To make the workflow easier to manage, I will set an environment variable with the location of my application.

env:
  WORKING_DIRECTORY: './0_Application/Training.Web/'

• It’s now time to create the core of your workflow, the jobs that the agent will execute. Because the cypress task only runs on Linux, you need to instruct GitHub to use an agent installed on an Ubuntu machine.

jobs:
  build:
    runs-on: ubuntu-latest

• Then, you can move forward by checking out your source code and installing Node.js and all of the dependencies specified in your package.json.

- uses: actions/checkout@v2
    
    - name: Set up Node.js version
      uses: actions/setup-node@v1
      with:
        node-version: '14.x'
        
    - name: yarn install, build, and test
      run: |
        yarn install
        yarn build 
      working-directory: ${{ env.WORKING_DIRECTORY }}

• It’s now time to perform some testing. To do so, I’m going to use different cypress-io/github-action@v2 actions for each browser.

Important: Because my application’s package.json is in a subfolder of the repository, I needed to add the parameter working-directory in each of my cypress-io/github-action@v2 task.

- name: Cypress run on chrome
  uses: cypress-io/github-action@v2
  with:
      browser: chrome      
      start: yarn start
      wait-on: 'http://localhost:3000'
      working-directory: ${{ env.WORKING_DIRECTORY }}
    
- name: Cypress run on firefox
  uses: cypress-io/github-action@v2
  with:
      browser: firefox      
      start: yarn start
      wait-on: 'http://localhost:3000'
      working-directory: ${{ env.WORKING_DIRECTORY }}
        
- name: Cypress run on edge
  uses: cypress-io/github-action@v2
  with:
      browser: edge
      start: yarn start
      wait-on: 'http://localhost:3000'
      working-directory: ${{ env.WORKING_DIRECTORY }}

The final YAML script will look similar to this:

name: Cypress testing
on:
  push:
    paths:
      - 0_Application/Training.Web/**
    branches:
      - main
env:
  WORKING_DIRECTORY: './0_Application/Training.Web/'
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    
    - name: Set up Node.js version
      uses: actions/setup-node@v1
      with:
        node-version: '14.x'
        
    - name: yarn install, build, and test
      run: |
        yarn install
        yarn build 
      working-directory: ${{ env.WORKING_DIRECTORY }}

    - name: Cypress run on chrome
      uses: cypress-io/github-action@v2
      with:
          browser: chrome      
          start: yarn start
          wait-on: 'http://localhost:3000'
          working-directory: ${{ env.WORKING_DIRECTORY }}
      
    - name: Cypress run on firefox
      uses: cypress-io/github-action@v2
      with:
          browser: firefox
          start: yarn start
          wait-on: 'http://localhost:3000'
          working-directory: ${{ env.WORKING_DIRECTORY }}
          
    - name: Cypress run on edge
      uses: cypress-io/github-action@v2
      with:
          browser: edge
          start: yarn start
          wait-on: 'http://localhost:3000'
          working-directory: ${{ env.WORKING_DIRECTORY }}

It’s now time to push your change to your repository and start the workflow.

Execution using GitHub Action

After completing each step of your test, you will be able to see the execution time (in case it succeeds), the exception (in case it fails), and a summary at the end of both of the tests and GitHub Action.

Test succeeded

Test failed

Summary at the end of the GitHub Action

References:

• Cypress GitHub Action: cypress-io/cypress: Fast, easy and reliable testing for anything that runs in a browser. (github.com)