This article will talk about parallel jobs in Azure Pipelines and the guidelines to determine how many jobs you need in your organization.
What is a Job?
Each pipeline in Azure DevOps starts with a trigger and is composed of one or more stages. Each stage contains one or more jobs that run multiple tasks on an agent.
With this relationship in mind, we can define a job as a set of steps that run sequentially on an agent (computing infrastructure with an installed agent software that runs a job).
Are parallel Jobs free?
By default, Azure DevOps gives one parallel job to every organization for free. However, there’s a slight difference in the free tier offering according to the types of agents being used.
Microsoft-hosted agents
- Private projects: The build time is limited to 1800 minutes per month and it can only run one pipeline job at a time.
- Public projects: There is no build time limit and it can run up to 10 parallel jobs at a time.
Self-Hosted agents
- Private projects: One self-hosted parallel job at a time. You can also get an additional self-hosted parallel job for each user with an active Visual Studio Enterprise subscriber account in your organization
- Public projects: an unlimited number of parallel jobs at a time
If you want to know more about self-hosted agents, I recommend that you read my previous article in which I explain how I created a self-hosted agent on my Raspberry Pi.
Buy parallel jobs
To buy more parallel jobs you must follow these steps:
- Sign in to your Azure DevOps organization
- Select Organization settings
- Select Parallel jobs under Pipelines, and then select either Purchase parallel jobs for Microsoft-hosted jobs or Change for self-hosted jobs.
- Enter the number of jobs that you want to purchase
- Click Save
Create a pipeline with parallel jobs
In this demo, I’m going to create a pipeline with just one stage. This stage will execute two jobs in parallel, and only when both these tasks are completed, will the pipeline move forward with a final task.
trigger:
- mainpool:
vmImage: 'ubuntu-latest'jobs:
- job: ParallelA
steps:
- bash: sleep 25
displayName: 'Sleep for 25 seconds'
- bash: echo Hello from parallel job A
displayName: 'Run a one-line script'- job: ParallelB
steps:
- bash: sleep 20
displayName: 'Sleep for 20seconds'
- bash: echo Hello from parallel job B
displayName: 'Run a one-line script'- job: Final
dependsOn:
- ParallelA
- ParallelB
steps:
- script: echo Hello from final job
displayName: 'Run a one-line script'
The dependsOn key defines the jobs that must be completed before starting the execution of the final job. As you can see in the image below, the first two jobs start concurrently looking for an available agent (this doesn’t mean that they will run simultaneously) while the final agent is on standby.
Do you need a parallel job?
As the number of users in your organization increases, so does the frequency at which the pipelines are being used. For this reason, to avoid users ending up in a queue waiting for other jobs to complete, Microsoft recommends running one parallel job for every four to five users in your organization.
Other factors that might influence the number of parallel jobs to use are:
- You should have a parallel job for each team that requires a CI build.
- If a CI build trigger applies to multiple branches, You should have a parallel job for each active branch.
- If you have multiple applications in the same organization or server, you should have a parallel job to deploy each application simultaneously.
Resources
- Specify jobs in your pipeline: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml
No comments:
Post a Comment