Azure pipeline

Links

• Azure Pipelines documentation
• Key concepts for new Azure Pipelines users
• Define approvals and checks

Definitions

Term	Definition
Artifact	item from a git repository or a build pipeline
Stages	steps to deploy and validate a software a stage contains jobs
Job	subparts of a stage, contains tasks to execute listed as steps a job runs on a agent or can be manually ran
Tasks / Steps	subparts of a job, those are the concreate action to execute
Release pipeline definition	steps to execute to get the artefacts, install and validate the software template to run a release
Release agent	the one who execute the tasks defined in the release pipeline definition it is the same agent as the one used for the build

Steps of a release

1. Build the software with a pipeline
   • validate product quality (unit tests, SonarCloud)
2. Deploy the software
   • validate runtime stability (compare telemetry with previous version)
3. Release the feature
   • validate feature usage

Predefined variables

Name	Value
System.DefaultWorkingDirectory	E:\Agent\Release_INT_2\_work\24\s
System.ArtifactsDirectory	E:\Agent\Release_INT_2\_work\24\a
Pipeline.Workspace	E:\Agent\Release_INT_2\_work\24

Trigger pipeline for PR

1. Azure DevOps → MyProject → Project Settings (bottom left gear)
2. Repos → Repositories → MyRepo
3. Policies → Branch Policies → master
4. Build Validation → +

YAML

• Specify jobs in your pipeline
• Template types & usage
• Approvals in environments with multi-stage YAML pipelines
• YAML schema reference

Create a new pipeline

From an existing YAML file

• Pipelines → New pipeline
• Azure Repo Git YAML → Select your repo → Existing Azure pipelines YAML file

Exemple

# do not trigger the pipeline on events trigger: none parameters: - name: myStepList type: stepList default: - bash: echo "We are on $(MyVar)" # If you have a single stage, you can omit the stages keyword and directly specify the jobs keyword stages: - stage: CI displayName: CI stage # remove implicite depency on the previous stage, the stages will run in parallel dependsOn: [] jobs: - job: ci_job pool: MyPool # If you have a single stage and a single job, you can omit the stages and jobs keywords and directly specify the steps keyword steps: - powershell: Write-Host "We are on CI" enabled: false # disable the task - stage: PROD displayName: PROD stage dependsOn: [] jobs: # use deployment job with environment and strategy to use the approval mechanism - deployment: prod_job environment: PROD strategy: runOnce: deploy: steps: ${{ parameters.myStepList }}

Trigger

If you specify no push trigger, pushes to any branch trigger a build.

trigger: batch: true # batch changes if true; start a new build for every push if false (default) branches: include: # branch names which will trigger a build - master - release/* exclude: temp # branch names which will not paths: include: MyProject/* # file paths which must match to trigger a build exclude: temp # file paths which will not trigger a build trigger: none # will disable CI builds entirely

Stage

stages: - stage: stage_name # (A-Z, a-z, 0-9, and underscore) # friendly name to display in the UI displayName: 'Stage name' variables: # ... condition: # ... jobs: [ job | templateReference]

Job

jobs: - job: job_name # (A-Z, a-z, 0-9, and underscore) # friendly name to display in the UI displayName: 'Job name' variables: # ... pool: PoolName # wait the dependencies to run and succeed before starting current job dependsOn: job1 dependsOn: - job1 - job2 condition: # ... # what to clean up before the job runs workspace: clean: outputs | resources | all steps: [ script | bash | pwsh | powershell | checkout | task | templateReference ]

Deployment job

jobs: - deployment: job_name # [A-Za-z0-9_]+ # same properties as normal job # ... environment: CI # target environment name CI.MyResource # and optionally a resource-name to record the deployment history; format: <environment-name>.<resource-name> pool: PoolName strategy: runOnce: preDeploy: # Used to run steps that initialize resources before application deployment starts. steps: # ... deploy: # Used to run steps that deploy your application. steps: # ... routeTraffic: # Used to run steps that serve the traffic to the updated version. steps: # ... postRouteTraffic: # Used to run the steps after the traffic is routed. # Typically, these tasks monitor the health of the updated version for defined interval. steps: # ... on: failure: # Used to run steps for rollback actions. steps: # ... success: # Used to run steps for clean-up. steps: # ...

Task

steps: # step - task: TaskToCall@Version # VSBuild@1 name: StepName # A-Z, a-z, 0-9, and underscore displayName: 'My Step' timeoutInMinutes: 120 inputs: key: 'value' enabled: boolean # runs a script in Bash - bash: | which bash echo Hello $(whoami) # friendly name displayed in the UI displayName: Multiline Bash script # runs a script in PowerShell Core - pwsh: Write-Host Hello $(name) # runs a script in Windows PowerShell - powershell: Write-Host Hello $(name) # checkout git repo - checkout: self | none | repo name # self represents the repo where the initial Pipelines YAML file was found clean: boolean # if true, run `execute git clean -ffdx && git reset --hard HEAD` before fetching. Default: false fetchDepth: number # the depth of commits to ask Git to fetch

• Shallow fetch
• Checkout

PowerShell task

- powershell: 'cmd' # 4 blank characters before the command - powershell: | cmd1 cmd2 # pwsh runs PowerShell Core, which must be installed on the agent or container. - pwsh: 'cmd' - task: PowerShell@2 displayName: 'PowerShell Script' inputs: targetType: filePath filePath: 'MyScript.ps1' arguments: '-Arg1 "Arg1"' - task: PowerShell@2 displayName: 'PowerShell Script' inputs: targetType: inline script: | cmd1 cmd2

Publish pipeline artifact

steps: - publish: MyProject\bin\Release # path to the folder to publish artifact: MyProject # artifact name - task: PublishPipelineArtifact@1 inputs: targetPath: MyProject\bin\Release # path to the folder to publish artifactName: MyProject parallel: true # Select whether to copy files in parallel using multiple threads. Default: false parallelCount: 8 # number of threads used to publish a package. Default: 8

Download pipeline artifact

steps: # to stop artifacts from being downloaded automatically - download: none - download: current # options: current (default), specific artifact: ArtifactName # equivalent to - task: DownloadPipelineArtifact@2 inputs: artifact: ArtifactName # artifact from another pipeline - task: DownloadPipelineArtifact@2 inputs: buildType: 'specific' # current (default), specific project: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' # Required when buildType = specific pipeline: '0000' # Required when source = specific buildVersionToDownload: 'specific' buildId: '000000' # Required when source = specific && RunVersion = specific artifact: ArtifactName downloadPath: $(System.DefaultWorkingDirectory) # default: $(Pipeline.Workspace). relative path is concat to $(System.DefaultWorkingDirectory)

• The Download Pipeline Artifact task can download both build artifacts (published with the Publish Build Artifacts task) and pipeline artifacts.
• By default, files are downloaded to $(Pipeline.Workspace)/{artifact name} (E:\Agent\Release_INT_2\_work\1\ArtefactName)
• Downloading artifacts
• Get project GUID: https://dev.azure.com/{organization}/_apis/projects
• Get the pipeline definition ID: https://dev.azure.com/{organization}/{project}/_apis/build/definitions?name={pipeline name}
• Get the build ID: https://dev.azure.com/{organization}/{project}/_apis/build/builds?definitions={pipeline definition ID}&buildNumber=x.x.xxxx.xxxxx
• Get Artifact name: https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}/artifacts
• Publish and download artifacts in Azure Pipelines

Download Build Artifacts

- task: DownloadBuildArtifacts@0 inputs: buildType: 'specific' # current (default), specific project: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' # Required when buildType = specific pipeline: '0000' # Required when source = specific buildVersionToDownload: 'specific' buildId: '000000' # Required when source = specific && RunVersion = specific artifactName: ArtifactName downloadPath: '$(System.ArtifactsDirectory)'

• By default, files are downloaded to $(System.ArtifactsDirectory)/{artifact}

Windows Machine File Copy

- task: WindowsMachineFileCopy@2 displayName: 'Copy Files' inputs: sourcePath: '$(sourcePath)' machineNames: '$(machineNames=' adminUserName: '$(adminUserName)' adminPassword: '$(adminPassword)' targetPath: '$(targetPath)' additionalArguments: '/r:1 /w:10' # if a file copy fails, wait 10 seconds and retry 1 time cleanTargetBeforeCopy: true

PowerShell on Target Machines

- task: PowerShellOnTargetMachines@3 inputs: machines: '$(machines)' userName: '$(userName)' userPassword: '$(userPassword)' scriptType: 'filePath' # inline by default scriptPath: '$(scriptPath)' scriptArguments: '-Arg1 $(Arg1Value)' communicationProtocol: http - task: PowerShellOnTargetMachines@3 inputs: machines: '$(machines)' userName: '$(userName)' userPassword: '$(userPassword)' scriptType: 'inline' # inline by default inlineScript: | echo 'Hello !' communicationProtocol: http

NuGet

- task: NuGetCommand@2 inputs: command: restore restoreSolution: MySolution.sln # relative path to a solution or project file

Visual Studio Build task

- task: VSBuild@1 inputs: solution: MyProject\MyProject.csproj # relative path to a solution or project file platform: AnyCPU # options: Win32, x86, x64 or any cpu. For a project use AnyCPU configuration: release # options: debug or release clean: false # Set to false to make this an incremental build, it might reduce the build time. Make sure not to clean the repo too. msbuildArchitecture: x64 # options: x86 (default), x64

FileTransform

Replace tokens with variable values in XML or JSON configuration files.

- job: variables: Name1: Value1 # replace ... by Value1 <setting name="Name1"><value>...</value></setting> - task: FileTransform@1 inputs: folderPath: $(Pipeline.Workspace)\MyArtefact fileType: xml targetFiles: MyProject.exe.config

MagicChunk

Modify a XML, JSON or YAML file.

- task: MagicChunks@2 inputs: sourcePath: $(System.DefaultWorkingDirectory)\Folder\App.config transformations: | { "root/node[@attribute1='value']/@attribute2": "true" } - task: MagicChunks@2 inputs: sourcePath: $(System.DefaultWorkingDirectory)\Folder\App.config transformationType: file transformationsFile: $(System.DefaultWorkingDirectory)\Folder\Transformations.json

Transformations.json

{ "root/node[@attribute1='value']/@attribute2": "true" }

Pool

# if you use a private pool and don't need to specify demands pool: PoolName pool: name: PoolName demands: - myCustomCapability # check for existence of capability - agent.os -equals Darwin # check for specific string in capability

Variables

# scope: root, stage, job variables: name1: 'value1' # name [\w\d._]+ name2: '${{ variables.name1 }} - value2' # reuse the value defined for name1 variables: - name: variable_name value: 'value' # groups are used to make variables inside that group available across multiple pipelines - group: group_name steps: - powershell: 'echo $(variable_name)' # pipeline variable - powershell: 'echo $(System.StageName)' # system variable - powershell: 'echo $(group_variable_name)' # variable in the group_name group variables - powershell: 'echo $env:VARIABLE_NAME' # environment variable - powershell: 'echo $(variable_name)' # macro syntax, processed during runtime before a task runs - powershell: 'echo ${{ variables.variable_name }}' # template expression syntax, processed at compile time, before runtime starts - powershell: 'echo $[variables.variable_name]' # Runtime expression syntax, processed during runtime, designed for use with conditions and expressions # Set a job-scoped variable from a script # the variable is available to downstream steps within the same job - powershell: echo "##vso[task.setvariable variable=variable_name]variable value" - powershell: echo $(variable_name) # Set a stage-scoped variable from a script jobs: - job: A steps: - powershell: echo "##vso[task.setvariable variable=variable_name;isOutput=true]variable value" name: task_name - powershell: echo $(task_name.variable_name) - job: B dependsOn: A variables: var_from_job_A: $[ dependencies.A.outputs['task_name.variable_name'] ] # expressions require single quotes # Set an inter-stage-scoped variable from a script stages: - stage: A jobs: - job: A1 steps: - powershell: echo "##vso[task.setvariable variable=variable_name;isOutput=true]variable value" name: task_name - stage: B dependsOn: A variables: var_from_stage_A: $[ stageDependencies.A.A1.outputs['task_name.variable_name'] ]

• Use predefined variables
• Create a variable group

Runtime parameters

Ces paramètres sont sélectionnable à chaque lancement de la pipeline.

parameters: - name: my_parameter displayName: My parameter type: string default: Value1 values: - Value1 - Value2

Repositories

• Supported source repositories
• Resources in YAML
• Multi-repo checkout

By default current pipeline repo is checked out.

resources: repositories: - repository: RepoId # A-Z, a-z, 0-9, and underscore name: ProjectName/RepoName # repository name (format depends on `type`) type: git ref: master steps: - checkout: self # checkout in $(Build.SourcesDirectory)\CurrentRepoId (C:\agent\_work\1\s\CurrentRepoId) - checkout: RepoId # checkout in $(Build.SourcesDirectory)\RepoId (C:\agent\_work\1\s\RepoId)

Single repository: If you have a single checkout step in your job, (or no checkout step which is equivalent to checkout: self), your source code is checked out into $(Build.SourcesDirectory). Ex: (C:\agent\_work\1\s) Multiple repositories: If you have multiple checkout steps in your job, your source code is checked out into directories named after the repositories as a subfolder of $(Build.SourcesDirectory). Ex: C:\agent\_work\1\s\MyRepo

Condition and dependency

It's as if you specified condition: succeeded()

Condition	Description
succeeded()	Only when all previous dependencies have succeeded. This is the default if there is not a condition set in the YAML.
succeededOrFailed()	Even if a previous dependency has failed, unless the run was canceled.
always()	Even if a previous dependency has failed, even if the run was canceled.
failed()	Only when a previous dependency has failed.

# scope: stage, job, task condition: eq(variables.Var1, 'Value1') # parameter condition: eq('${{ parameters.Param1 }}', 'Value1') # and, or, eq, ne condition: or(eq(variables.Var1, 'Value1'), ne(variables.Var2, 'Value2')) # in condition: in(variables['Build.Reason'], 'IndividualCI', 'BatchedCI') # with dependencies: wait Job1 and Job2 to run dependsOn: - Job1 - Job2 # test Job1 and Job2 have succeeded and test Var1 = Value1 condition: and(succeeded(), eq(variables.Var1, 'Value1')) # test Job2 has succeeded and test Var1 = Value1 condition: and(succeeded('Job2'), eq(variables.Var1, 'Value1')) # test Job1 and Job2 have succeeded or skipped condition: | and ( in(dependencies.Job1.result, 'Succeeded', 'SucceededWithIssues', 'Skipped'), in(dependencies.Job2.result, 'Succeeded', 'SucceededWithIssues', 'Skipped') ) # stage dependency - stage: Stage2 # on Stage1 condition: in(dependencies.Stage1.result, 'Succeeded', 'SucceededWithIssues', 'Skipped') condition: eq(dependencies.Stage1.result, '') # Stage1 has not been selected # on Stage1.Job1.Task1.Var1 output variable condition: and(succeeded(), eq(dependencies.Stage1.outputs['Job1.Task1.Var1'], 'true')) # job dependency jobs: - job: Job2 # on Stage1.Job1 condition: in(stageDependencies.Stage1.Job1.result, 'Succeeded', 'SucceededWithIssues', 'Skipped') # on Stage1.Job1.Task1.Var1 output variable condition: eq(stageDependencies.Stage1.Job1.outputs['Task1.Var1'], 'true') # on current stage Job3 condition: in(dependencies.Job3.result, 'Succeeded', 'SucceededWithIssues', 'Skipped') # on current stage Job3.Task1.Var1 output variable condition: eq(dependencies.Job3.outputs['Task1.Var1'], 'true')

You can specify conditions under which a step, job, or stage will run.

By default, each stage in a pipeline depends on the one just before it in the YAML file.

• dependency

if statement

# conditional insertion variables: ${{ if eq(variables['Build.SourceBranchName'], 'master') }}: var_name: value steps: - ${{ if eq(parameters.param_name, 'value') }}: - task: - task: inputs: ${{ if eq(variables['Build.SourceBranchName'], 'master') }}: input_name: 'value'

Template

Insert a template

Reuse stages, jobs ans tasks.

stages-template.yml

stages: - stage: Stage1 jobs: - job: Job1 steps: - script: echo 'stages template'

jobs-template.yml

jobs: - job: Job1 steps: - script: echo 'jobs template'

tasks-template.yml

steps: - script: echo 'tasks template'

my-pipeline.yml

stages: - template: stages-template.yml - stage: Stage2 jobs: - template: jobs-template.yml - stage: Stage3 jobs: - job: ExcecuteTasks steps: - template: tasks-template.yml

Extend from a template

Copy the template in the pipeline.

my-template.yml

resources: repositories: - repository: MyRepo type: git name: MyProject/MyRepo steps: - script: echo "My template"

my-pipepline.yml

extends: template: my-template.yml

Parameters

tasks-template.yml

parameters: parameterName: 'parameter value' steps: - script: echo ${{ parameters.parameterName}}

my-pipepline.yml

variables: var: 'value' steps: - template: tasks-template.yml parameters: parameterName: 'value' parameterName: ${{ variables.var }} # $(var) doesn't work

Condition on a template

It is not possible to set a condition on a template.

Solution 1: pass a parameter to the template.

steps-template.yml

parameters: enabled: true steps: - script: echo 'Task 1' condition: ${{ parameters.enabled }}

my-pipepline.yml

steps: - template: steps-template.yml parameters: enabled: false

Solution 2: use if.

my-pipepline.yml

steps: - ${{ if eq($(MyVar), 'value') }}: - template: steps-template.yml

Errors

Environment xxx could not be found

1. Pipelines → Environments → Select the env xxx → 3 dots button on top right → Security
2. Add the user with User role