Azure DevOps CI/CD Pipeline — JAVA

 

Here, For the beginners I explained about “How to implement Azure DevOps Pipeline”

Image Copied from Azure

Introduction

Using tools like Jenkins does not mean you follow CI/CD.

Continuous Integration (CI) is a development practice. In which developers integrate code into a shared repository frequently, preferably several times a day. An automated build or automated tests should verify each integration build.

Mostly, we follow below flow for CI/CD.

Above image copied from “Release It! Design and Deploy by Michael Nygard”

In Continuous integration — For individual commits — the Azure pipeline will build a container image (prefer docker image), then test it and push it to container registry.

Create Azure Devops Project

Create new repository

*** Refer to the last section for cloning a repository from your Github account.

|
`-- microservice1
    |-- Dockerfile
    |-- pom.xml
    `-- src
        |-- main
        |   |-- java
        |   |   `-- edu
        |   |       `-- sampleapplication
        |   |           `-- MS1
        |   |               |-- ApiController.java
        |   |               `-- Ms1Application.java
        |   `-- resources
        |       |-- application.properties
        |       `-- logback-spring.xml
        `-- test
            `-- java
                `-- edu
                    `-- sampleapplication
                        `-- MS1
                            `-- Ms1ApplicationTests.java

Build pipeline

Do click “Set up Build button” for building a pipeline.

Here, I am selecting an option to build a docker image and push it to the Azure container registry.

Azure pipeline —

Default pipeline YAML file will contain stages like build with the steps or tasks to push the docker image to the container registry.

# Docker
# Build and push an image to Azure Container Registry
# https://docs.microsoft.com/azure/devops/pipelines/languages/dockertrigger:
- masterresources:
- repo: selfvariables:
  # Container registry service connection established during pipeline creation
  dockerRegistryServiceConnection: '8eb2472c-86b9-4669-871b-9b9872670821'
  imageRepository: 'springazuredemo'
  containerRegistry: 'ritresh.azurecr.io'
  dockerfilePath: '$(Build.SourcesDirectory)/microservice1/Dockerfile'
  tag: '$(Build.BuildId)'
  
  # Agent VM image name
  vmImageName: 'ubuntu-latest'stages:
- stage: Build
  displayName: Build and push stage
  jobs:  
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)
    steps:
    - task: Docker@2
      displayName: Build and push an image to container registry
      inputs:
        command: buildAndPush
        repository: $(imageRepository)
        dockerfile: $(dockerfilePath)
        containerRegistry: $(dockerRegistryServiceConnection)
        tags: |
          $(tag)

We need to add maven tasks to build the spring boot application as default conf won’t work for our set up.

# Docker
# Build and push an image to Azure Container Registry
# https://docs.microsoft.com/azure/devops/pipelines/languages/dockertrigger:
- masterresources:
- repo: selfvariables:
  # Container registry service connection established during pipeline creation
  dockerRegistryServiceConnection: 'c1a4b0b1-bbda-4066-810a-a3b646af911f'
  imageRepository: 'springazuredemo'
  containerRegistry: 'ritresh.azurecr.io'
  dockerfilePath: '$(Build.SourcesDirectory)/microservice1/Dockerfile'
  tag: '$(Build.BuildId)'
  
  # Agent VM image name
  vmImageName: 'ubuntu-latest'stages:
- stage: Build
  displayName: Build and push stage
  jobs:  
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)
    steps:
    - task: Maven@3
      inputs:
        mavenPomFile: 'microservice1/pom.xml'
        mavenOptions: '-Xmx3072m'
        javaHomeOption: 'JDKVersion'
        jdkVersionOption: '1.8'
        jdkArchitectureOption: 'x64'
        publishJUnitResults: false
        goals: 'clean install'
    - task: Docker@2
      displayName: Build and push an image to container registry
      inputs:
        command: buildAndPush
        repository: $(imageRepository)
        dockerfile: $(dockerfilePath)
        containerRegistry: $(dockerRegistryServiceConnection)
        tags: |
          $(tag)

Azure container registry —

Let’s check the Azure container registry, here we could see the newly build docker image under springazuredemo repository.

Lets Run it on local environment

:study ritgirdh$ docker login ritresh.azurecr.io
Username: ritresh
Password: 
Login Succeeded
:study ritgirdh$ docker pull ritresh.azurecr.io/springazuredemo:17
17: Pulling from springazuredemo
e7c96db7181b: Already exists 
f910a506b6cb: Already exists 
c2274a1a0e27: Already exists 
8b7b20a1fb06: Pull complete 
Digest: sha256:13f46fce6f2910c6fe84e9ff8459ab1a2973740df442717c99c79a9016fe1caa
Status: Downloaded newer image for ritresh.azurecr.io/springazuredemo:17
ritresh.azurecr.io/springazuredemo:17
WKMIN1307242:study ritgirdh$ docker run -p5555:8080 ritresh.azurecr.io/springazuredemo:17  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.0.0.RELEASE)2020-05-06 11:55:45.969  INFO 1 --- [           main] e.sampleapplication.MS1.Ms1Application   : Starting Ms1Application v1.0.0-SNAPSHOT on 8c5ac374947d with PID 1 (/ms1.jar started by root in /)
2020-05-06 11:55:45.983  INFO 1 --- [           main] e.sampleapplication.MS1.Ms1Application   : No active profile set, falling back to default profiles: default
2020-05-06 11:55:46.086  INFO 1 --- [           main] ConfigServletWebServerApplicationContext : Refreshing org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@38cccef: startup date [Wed May 06 11:55:46 GMT 2020]; root of context hierarchy
2020-05-06 11:55:48.285  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2020-05-06 11:55:48.353  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2020-05-06 11:55:48.353  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet Engine: Apache Tomcat/8.5.28
2020-05-06 11:55:48.381  INFO 1 --- [ost-startStop-1] o.a.catalina.core.AprLifecycleListener   : The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib]
2020-05-06 11:55:48.579  INFO 1 --- [ost-startStop-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2020-05-06 11:55:48.579  INFO 1 --- [ost-startStop-1] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 2499 ms
2020-05-06 11:55:49.257  INFO 1 --- [ost-startStop-1] o.s.b.w.servlet.ServletRegistrationBean  : Servlet dispatcherServlet mapped to [/]
2020-05-06 11:55:49.265  INFO 1 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'characterEncodingFilter' to: [/*]
2020-05-06 11:55:49.265  INFO 1 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'hiddenHttpMethodFilter' to: [/*]
2020-05-06 11:55:49.265  INFO 1 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'httpPutFormContentFilter' to: [/*]
2020-05-06 11:55:49.266  INFO 1 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'requestContextFilter' to: [/*]
2020-05-06 11:55:49.266  INFO 1 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'httpTraceFilter' to: [/*]
2020-05-06 11:55:49.266  INFO 1 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'webMvcMetricsFilter' to: [/*]
2020-05-06 11:55:49.792  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerAdapter : Looking for @ControllerAdvice: org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@38cccef: startup date [Wed May 06 11:55:46 GMT 2020]; root of context hierarchy
2020-05-06 11:55:49.923  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/message]}" onto java.lang.String edu.sampleapplication.MS1.ApiController.getMessage()
2020-05-06 11:55:49.925  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/v1],methods=[GET]}" onto public java.lang.String edu.sampleapplication.MS1.ApiController.test()
2020-05-06 11:55:49.929  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error],produces=[text/html]}" onto public org.springframework.web.servlet.ModelAndView org.springframework.boot.autoconfigure.web.servlet.error.BasicErrorController.errorHtml(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse)
2020-05-06 11:55:49.930  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error]}" onto public org.springframework.http.ResponseEntity<java.util.Map<java.lang.String, java.lang.Object>> org.springframework.boot.autoconfigure.web.servlet.error.BasicErrorController.error(javax.servlet.http.HttpServletRequest)
2020-05-06 11:55:50.015  INFO 1 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/webjars/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2020-05-06 11:55:50.015  INFO 1 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2020-05-06 11:55:50.095  INFO 1 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**/favicon.ico] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2020-05-06 11:55:50.938  INFO 1 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
2020-05-06 11:55:50.946  INFO 1 --- [           main] ConfigServletWebServerApplicationContext : Refreshing org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@6c779568: startup date [Wed May 06 11:55:50 GMT 2020]; parent: org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@38cccef
2020-05-06 11:55:51.012  INFO 1 --- [           main] o.s.b.f.s.DefaultListableBeanFactory     : Overriding bean definition for bean 'handlerExceptionResolver' with a different definition: replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.web.servlet.config.annotation.DelegatingWebMvcConfiguration; factoryMethodName=handlerExceptionResolver; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/web/servlet/config/annotation/DelegatingWebMvcConfiguration.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.actuate.autoconfigure.web.servlet.WebMvcEndpointChildContextConfiguration; factoryMethodName=compositeHandlerExceptionResolver; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/actuate/autoconfigure/web/servlet/WebMvcEndpointChildContextConfiguration.class]]
2020-05-06 11:55:51.097  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8001 (http)
2020-05-06 11:55:51.099  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2020-05-06 11:55:51.100  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet Engine: Apache Tomcat/8.5.28
2020-05-06 11:55:51.122  INFO 1 --- [ost-startStop-1] o.a.c.c.C.[Tomcat-1].[localhost].[/]     : Initializing Spring embedded WebApplicationContext
2020-05-06 11:55:51.123  INFO 1 --- [ost-startStop-1] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 177 ms
2020-05-06 11:55:51.180  INFO 1 --- [ost-startStop-1] o.s.b.w.servlet.ServletRegistrationBean  : Servlet dispatcherServlet mapped to [/]
2020-05-06 11:55:51.256  INFO 1 --- [           main] s.b.a.e.w.s.WebMvcEndpointHandlerMapping : Mapped "{[/actuator/health],methods=[GET],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.String>)
2020-05-06 11:55:51.258  INFO 1 --- [           main] s.b.a.e.w.s.WebMvcEndpointHandlerMapping : Mapped "{[/actuator/info],methods=[GET],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.String>)
2020-05-06 11:55:51.260  INFO 1 --- [           main] s.b.a.e.w.s.WebMvcEndpointHandlerMapping : Mapped "{[/actuator],methods=[GET],produces=[application/vnd.spring-boot.actuator.v2+json || application/json]}" onto protected java.util.Map<java.lang.String, java.util.Map<java.lang.String, org.springframework.boot.actuate.endpoint.web.Link>> org.springframework.boot.actuate.endpoint.web.servlet.WebMvcEndpointHandlerMapping.links(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse)
2020-05-06 11:55:51.334  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error]}" onto public java.util.Map<java.lang.String, java.lang.Object> org.springframework.boot.actuate.autoconfigure.web.servlet.ManagementErrorEndpoint.invoke(org.springframework.web.context.request.ServletWebRequest)
2020-05-06 11:55:51.342  INFO 1 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/webjars/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2020-05-06 11:55:51.343  INFO 1 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2020-05-06 11:55:51.377  INFO 1 --- [           main] s.w.s.m.m.a.RequestMappingHandlerAdapter : Looking for @ControllerAdvice: org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@6c779568: startup date [Wed May 06 11:55:50 GMT 2020]; parent: org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@38cccef
2020-05-06 11:55:51.585  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8001 (http) with context path ''
2020-05-06 11:55:51.615  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2020-05-06 11:55:51.619  INFO 1 --- [           main] e.sampleapplication.MS1.Ms1Application   : Started Ms1Application in 6.497 seconds (JVM running for 7.807)

Lets test it on local —

curl -ivk localhost:5555/v1   
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 5555 (#0)
> GET /v1 HTTP/1.1
> Host: localhost:5555
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 
HTTP/1.1 200 
< Content-Type: text/plain;charset=UTF-8
Content-Type: text/plain;charset=UTF-8
< Content-Length: 13
Content-Length: 13
< Date: Wed, 06 May 2020 11:58:35 GMT
Date: Wed, 06 May 2020 11:58:35 GMT< 
* Connection #0 to host localhost left intact
OKHello world

Clone existing repo from Github

Here, I am cloning existing github repo https://github.com/RitreshGirdhar/SpringBoot-Docker-Ansible. And for this demo I have deleted some of the files related to ansible which don’t belong to this Azure-devops demo.

The only purpose of this article is to demonstrate the purpose of Continuous Integration.

We could Configure the Azure pipeline for every Merge request or pull request.

It will build the application and run the default test cases. Once the application gets build successfully, it will containerize the application and push it to the specified container registry.

Later, the pipeline would deploy the application on some azure server or some other service via ansible or native commands.

Setting up super fast Cypress tests on GitHub Actions

If you've been keeping track of The Array release posts you know that we prioritize shipping things fast and often. Just as important to us is being sure that we are not going to break things unnecessarily for our users as we add new features and speedups.

What we have found that works really well is nothing terribly novel by itself: a solid foundation of unit tests, end to end tests (integration tests), and CI/CD that for automation and gatekeeper keeping master clean.

Unit & Integration tests

In our Django codebase you'll find good number of Django tests that help keep us honest as we hack away at the backend of PostHog that keeps track of all the 1's and 0's that our customers depend on for making product decisions. These are our frontline defenders that let us know that something might be up before we even get to the point of creating a PR.

For this we do lean heavily on the standard Django test runner.

If you are interested in learning more on testing with Django check out Django's great docs on testing.

These tests only get you so far though. You know that the backend is going to behave well after you land the changes that you've made, but what if you accidentally changed something that breaks the frontend in weird and unexpected ways?

Enter Cypress

According to Cypress's GitHub repo it is a fast, easy and reliable testing for anything that runs in a browser. What does that mean exactly though?

It lets you programmatically interact with your application by querying the DOM and running actions against any selected elements. You can see that in a few of our Cypress test definitions

Tracking the elements

To keep our tested elements clear, manageable, and reusable upon refactor, we take advantage of the element attributes that html and react specifically recognize. Cypress has an amazing built in inspector on their test-runner that allows you to identify elements that you would like to add tests to.

While the tool works great, we found that occasionally the heavily nested components and classes would create selectors that were inflexible.

With the data-attr tag, we just need to keep track of the tag when updating/changing the components we're using without needing to rely on the inspector to find the precise selector for the test!

<LineGraph
  data-attr="trend-line-graph"
  {...props}
/>

Example of our integration test for our Funnel user experience:

describe('Funnels', () => {
  //boilerplate to make sure we are on the funnel page of the app
  beforeEach(() => {
    cy.get('[data-attr=menu-item-funnels]').click()
  })

  // Test to make sure that the funnel page actually loaded
  it('Funnels loaded', () => {
    cy.get('h1').should('contain', 'Funnels')
  })

  // Test that when we select a funnel then we can edit that funnel
  it('Click on a funnel', () => {
    cy.get('[data-attr=funnel-link-0]').click()
    cy.get('[data-attr=funnel-editor]').should('exist')
  })

  // Test that we can create a new funnel when we click 'create funnel' button
  it('Go to new funnel screen', () => {
    cy.get('[data-attr=create-funnel]').click()
    cy.get('[data-attr=funnel-editor]').should('exist')
  })

  // Test that we can create a new funnel end to end
  it('Add 1 action to funnel', () => {
    cy.get('[data-attr=create-funnel]').click()
    cy.get('[data-attr=funnel-editor]').should('exist')

    cy.get('[data-attr=edit-funnel-input]').type('Test funnel')
    cy.get('[data-attr=add-action-event-button]').click()
    cy.get('[data-attr=trend-element-subject-0]').click()
    cy.contains('Pageviews').click()

    cy.get('[data-attr=save-funnel-button]').click()

    cy.get('[data-attr=funnel-viz]').should('exist')
  })
})

I personally love this syntax. It feels super readable to me and reminds me a bit of the best parts of jQuery.

GitHub Actions

So that's all well and cool, but what about making sure that in a fit of intense focus and momentum we don't inadvertently push a breaking change to master? We need someone or something to act as a gatekeeper to keep us from from shooting ourselves in the foot. We need CI.

We could use Travis, or Jekins, or CircleCI… but as you may have noticed we keep almost everything about PostHog in GitHub, from our product roadmap, issues, this blog, everything is in GitHub. So it made sense to us to keep our CI in GitHub if we could. We decided to give GitHub Actions a test. So far, we have loved it.

GitHub actions are basically a workflow you can trigger from events that occure on your GitHub repo. We trigger ours on the creation of a pull request. We also require that our actions all return 👍  before you can merge your PR into master. Thus, we keep master clean.

To make sure that things are only improving with our modifications, we first re-run our Django unit and integration tests just to make sure that in our customers final environment things are still going to behave as expected. We need to be sure that there was nothing unique about your dev environment that could have fooled the tests into a false sense of awesome. You can check out how we set this up here Django github actions

The second round of poking we do with our app is we hit it with Cypress tests that we discussed earlier. These boot up our app and click through workflows just as a user would, asserting along the way that things look and behave as we would expect. You can check out how we've setup our Cypress action here

Caching

We ran up upon an issue though. Installing python dependencies, javascript dependencies, building our frontend app, booting up a chromium browser… this all takes a lot of time. We are impatient. We want instant gratifiction, at least when it comes to our code. Most of this stuff doesn't even change between commits on a PR anyways. Why are we spending valuable time and resources towards having things be repulled and rebuilt? That's where we ended up using one of the best features of GitHub Actions: the cache step.

Using the cache step we can cache the results of pulling python dependencies or javascript dependencies. This saves a chunk of time if you have ever messed around with watching yarn sort out the deps for a large frontend project. Check it out:

How we manage caching the cache for pip:

- uses: actions/cache@v1
  name: Cache pip dependencies
  id: pip-cache
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
    restore-keys: |
      ${{ runner.os }}-pip-
- name: Install python dependencies
  run: |
    python -m pip install --upgrade pip
    python -m pip install $(grep -ivE "psycopg2" requirements.txt) --no-cache-dir --compile
    python -m pip install psycopg2-binary --no-cache-dir --compile

Note that there is no if block to determine whether to use the cache or not when we pip install the dependencies. This is because pip is smart enough to use the rehydrated cache if it sees it, if it doesnt see it it will just go out to the internet to grab what it needs.

Yarn is a bit more involved only because we grab the location of the cache directory first and use that output as an input to the caching step

Yarn dependency caching

- name: Get yarn cache directory path
    id: yarn-dep-cache-dir-path
    run: echo "::set-output name=dir::$(yarn cache dir)"
- uses: actions/cache@v1
    name: Setup Yarn dep cache
    id: yarn-dep-cache
    with:
        path: ${{ steps.yarn-dep-cache-dir-path.outputs.dir }}
        key: ${{ runner.os }}-yarn-dep-${{ hashFiles('**/yarn.lock') }}
        restore-keys: |
            ${{ runner.os }}-yarn-dep-
- name: Yarn install deps
    run: |
        yarn install --frozen-lockfile
    if: steps.yarn-dep-cache.outputs.cache-hit != 'true'

That last line with the if block tells GitHub to not run yarn install if the cache exists. This saves us a ton of time if nothing has changed

On top of that, let's say you are making changes to only the API. There's no reason why you should be rebuiling the frontend each time the tests are run. So we go ahead and cache that between runs as well.

Frontend app build cache

- uses: actions/cache@v1
    name: Setup Yarn build cache
    id: yarn-build-cache
    with:
        path: frontend/dist
        key: ${{ runner.os }}-yarn-build-${{ hashFiles('frontend/src/') }}
        restore-keys: |
            ${{ runner.os }}-yarn-build-
- name: Yarn build
    run: |
        yarn build
    if: steps.yarn-build-cache.outputs.cache-hit != 'true'

Now you are catching if the cache exists so we can skip building the frontend altogether since it's been rehydrated from the last run. Nifty!

Throw more computers at it!

One of the best thing about Cypress is that you can grow with it. It would be a real pain if you invested all of this time into building out tests just to have your test suite take 60 minutes to run. Luckily both GitHub actions and Cypress have a solution to that!

Run it in parallel!

matrix:
  # run 4 copies of the current job in parallel
  containers: [1, 2, 3, 4]

Configure Cypress step to coordinate with Cypress SaaS

- name: Cypress run
  uses: cypress-io/github-action@v1
  with:
    config-file: cypress.json
    record: true
    parallel: true
    group: 'PostHog Frontend'
  env:
    # pass the Dashboard record key as an environment variable
    CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
    # Recommended: pass the GitHub token lets this action correctly
    # determine the unique run id necessary to re-run the checks
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Depending on the count of tests and the frequency you are running your suite this might cost you some money having to upgrade your account on Cypress.io but their free tier is pretty generous and they do have OSS plans that are free.

This all has cut the time it takes for GitHub to stamp our pull requests from >10 minutes to ~5 minutes and that's with our relatively small set of tests.

As we grow functionality within PostHog all of this will only become more important so that we don't end up with a 30 minute end to end test blocking you from landing that really killer new feature. Sweet.

👀 at errors

The final bit here is what happens if the tests are failing?

If this is all happening in a browser up in the cloud how do we capture what went wrong? We need that to figure out how to fix it. Luckily, again, Cypress and GitHub actions has a solution: artifacts.

Artifacts allow us to take the screenshots that Cypress takes when things go wrong, zip them up, and make them available on the dashboard for the actions that are being run.

Capturing Cypress screenshots

- name: Archive test screenshots
  uses: actions/upload-artifact@v1
  with:
    name: screenshots
    path: cypress/screenshots
  if: ${{ failure() }}

As you can tell by the if block here, we only upload the artifacts if there is a problem. That's because we already know what the app will look like when things go right… hopefully 😜

Roadmap

There is one thing that we don't capture in our current test suite: Performance!

We have customers who upload hundreds of telemetry events a second. If we introduce a regression that dings performance this could cause an outage for them where they lose data which is arguably worse than a regression on the frontend.

Our plan here use GitHub actions to standup an instance of our infrastructure and hammer it with sythentic event telemetry and compare that against a baseline from prior performance tests. If the test runtime changes materially we will block the pull request from being merged in to guard master from having a potentially breaking change. Stay tuned for a post on automated performance testing.

The pitch™

Hey! You made it this far. If you see yourself working on challenging issues at a high paced startup with a really rad group of people. You are in luck! We are looking for people like you!