Azure pipeline

De Banane Atomic
Aller à la navigationAller à la recherche

Links

Definitions

Term Definition
Artifact item from a git repository or a build pipeline
Stages steps to deploy and validate a software
a stage contains jobs
Job subparts of a stage, contains tasks to execute listed as steps
a job runs on a agent or can be manually ran
Tasks / Steps subparts of a job, those are the concreate action to execute
Release pipeline definition steps to execute to get the artefacts, install and validate the software
template to run a release
Release agent the one who execute the tasks defined in the release pipeline definition
it is the same agent as the one used for the build

Steps of a release

  1. Build the software with a pipeline
    • validate product quality (unit tests, SonarCloud)
  2. Deploy the software
    • validate runtime stability (compare telemetry with previous version)
  3. Release the feature
    • validate feature usage

Release variables

Predefined variables

Name Value
System.DefaultWorkingDirectory E:\Agent\Release_INT_2\_work\24\s
System.ArtifactsDirectory E:\Agent\Release_INT_2\_work\24\a
Pipeline.Workspace E:\Agent\Release_INT_2\_work\24

Trigger pipeline for PR

  1. Azure DevOps → MyProject → Project Settings (bottom left gear)
  2. Repos → Repositories → MyRepo
  3. Policies → Branch Policies → master
  4. Build Validation → +

Artifact

YAML

Create a new pipeline

From an existing YAML file

  • Pipelines → New pipeline
  • Azure Repo Git YAML → Select your repo → Existing Azure pipelines YAML file

Exemple

Yaml.svg
# do not trigger the pipeline on events
trigger: none

parameters:
- name: myStepList
  type: stepList
  default:
    - bash: echo "We are on $(MyVar)"

# If you have a single stage, you can omit the stages keyword and directly specify the jobs keyword
stages:
- stage: CI
  displayName: CI stage
  # remove implicite depency on the previous stage, the stages will run in parallel
  dependsOn: []
  jobs:
  - job: ci_job
    pool: MyPool
    # If you have a single stage and a single job, you can omit the stages and jobs keywords and directly specify the steps keyword
    steps:
    - powershell: Write-Host "We are on CI"
      enabled: false  # disable the task

- stage: PROD
  displayName: PROD stage
  dependsOn: []
  jobs:
  # use deployment job with environment and strategy to use the approval mechanism
  - deployment: prod_job
    environment: PROD
    strategy:
      runOnce:
        deploy:
          steps: ${{ parameters.myStepList }}

Trigger

If you specify no push trigger, pushes to any branch trigger a build.
Yaml.svg
trigger:
  batch: true # batch changes if true; start a new build for every push if false (default)
  branches:
    include: # branch names which will trigger a build
    - master
    - release/*
    exclude: temp # branch names which will not
  paths:
    include: MyProject/* # file paths which must match to trigger a build
    exclude: temp # file paths which will not trigger a build

trigger: none # will disable CI builds entirely

Stage

Yaml.svg
stages:
- stage: stage_name  # (A-Z, a-z, 0-9, and underscore)
  # friendly name to display in the UI
  displayName: 'Stage name'

  variables: # ...
  condition: # ...

  jobs: [ job | templateReference]

Job

Yaml.svg
jobs:
- job: job_name  # (A-Z, a-z, 0-9, and underscore)
  # friendly name to display in the UI
  displayName: 'Job name'

  variables: # ...

  pool: PoolName

  # wait the dependencies to run and succeed before starting current job
  dependsOn: job1
  dependsOn:
  - job1
  - job2

  condition: # ...

  # what to clean up before the job runs
  workspace:
    clean: outputs | resources | all

  steps: [ script | bash | pwsh | powershell | checkout | task | templateReference ]

Deployment job

Yaml.svg
jobs:
- deployment: job_name  # [A-Za-z0-9_]+
  # same properties as normal job
  # ...
  environment: CI             # target environment name
               CI.MyResource  # and optionally a resource-name to record the deployment history; format: <environment-name>.<resource-name>
  pool: PoolName
  strategy:
    runOnce:
      preDeploy: # Used to run steps that initialize resources before application deployment starts.
        steps:
        # ...
      deploy: # Used to run steps that deploy your application.
        steps:
        # ...
      routeTraffic: # Used to run steps that serve the traffic to the updated version.
        steps:
        # ...
      postRouteTraffic: # Used to run the steps after the traffic is routed.
                        # Typically, these tasks monitor the health of the updated version for defined interval.      
        steps:
        # ...
      on:
        failure: # Used to run steps for rollback actions.
          steps:
          # ...
        success: # Used to run steps for clean-up. 
          steps:
          # ...

Task

Yaml.svg
steps:

# step
- task: TaskToCall@Version  # VSBuild@1
  name: StepName  # A-Z, a-z, 0-9, and underscore
  displayName: 'My Step'
  timeoutInMinutes: 120
  inputs:
    key: 'value'
  enabled: boolean

# runs a script in Bash
- bash: |
    which bash
    echo Hello $(whoami)
  # friendly name displayed in the UI
  displayName: Multiline Bash script

# runs a script in PowerShell Core
- pwsh: Write-Host Hello $(name)

# runs a script in Windows PowerShell
- powershell: Write-Host Hello $(name)

# checkout git repo
- checkout: self | none | repo name # self represents the repo where the initial Pipelines YAML file was found
  clean: boolean                    # if true, run `execute git clean -ffdx && git reset --hard HEAD` before fetching. Default: false
  fetchDepth: number                # the depth of commits to ask Git to fetch

PowerShell task

Yaml.svg
- powershell: 'cmd'
# 4 blank characters before the command
- powershell: |
    cmd1
    cmd2

# pwsh runs PowerShell Core, which must be installed on the agent or container.
- pwsh: 'cmd'

- task: PowerShell@2
  displayName: 'PowerShell Script'
  inputs:
    targetType: filePath
    filePath: 'MyScript.ps1'
    arguments: '-Arg1 "Arg1"'

- task: PowerShell@2
  displayName: 'PowerShell Script'
  inputs:
    targetType: inline
    script: |
      cmd1
      cmd2

Publish pipeline artifact

Yaml.svg
steps:
- publish: MyProject\bin\Release  # path to the folder to publish
  artifact: MyProject             # artifact name

- task: PublishPipelineArtifact@1
  inputs:
    targetPath: MyProject\bin\Release  # path to the folder to publish
    artifactName: MyProject
    parallel: true                     # Select whether to copy files in parallel using multiple threads. Default: false
    parallelCount: 8 	               # number of threads used to publish a package. Default: 8

Download pipeline artifact

Yaml.svg
steps:

# to stop artifacts from being downloaded automatically
- download: none

- download: current  # options: current (default), specific
  artifact: ArtifactName

# equivalent to
- task: DownloadPipelineArtifact@2
  inputs:
    artifact: ArtifactName

# artifact from another pipeline
- task: DownloadPipelineArtifact@2
  inputs:
    buildType: 'specific' # current (default), specific
    project: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' # Required when buildType = specific
    pipeline: '0000' # Required when source = specific
    buildVersionToDownload: 'specific'
    buildId: '000000' # Required when source = specific && RunVersion = specific
    artifact: ArtifactName
    downloadPath: $(System.DefaultWorkingDirectory)  # default: $(Pipeline.Workspace). relative path is concat to $(System.DefaultWorkingDirectory)
  • The Download Pipeline Artifact task can download both build artifacts (published with the Publish Build Artifacts task) and pipeline artifacts.
  • By default, files are downloaded to $(Pipeline.Workspace)/{artifact name} (E:\Agent\Release_INT_2\_work\1\ArtefactName)
  • Downloading artifacts
  • Get project GUID: https://dev.azure.com/{organization}/_apis/projects
  • Get the pipeline definition ID: https://dev.azure.com/{organization}/{project}/_apis/build/definitions?name={pipeline name}
  • Get the build ID: https://dev.azure.com/{organization}/{project}/_apis/build/builds?definitions={pipeline definition ID}&buildNumber=x.x.xxxx.xxxxx
  • Get Artifact name: https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}/artifacts
  • Publish and download artifacts in Azure Pipelines

Download Build Artifacts

Yaml.svg
- task: DownloadBuildArtifacts@0
  inputs:
    buildType: 'specific' # current (default), specific
    project: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' # Required when buildType = specific
    pipeline: '0000' # Required when source = specific
    buildVersionToDownload: 'specific'
    buildId: '000000' # Required when source = specific && RunVersion = specific
    artifactName: ArtifactName
    downloadPath: '$(System.ArtifactsDirectory)'
  • By default, files are downloaded to $(System.ArtifactsDirectory)/{artifact}

Windows Machine File Copy

Yaml.svg
- task: WindowsMachineFileCopy@2
  displayName: 'Copy Files'
  inputs:
    sourcePath: '$(sourcePath)'
    machineNames: '$(machineNames='
    adminUserName: '$(adminUserName)'
    adminPassword: '$(adminPassword)'
    targetPath: '$(targetPath)'
    additionalArguments: '/r:1 /w:10' # if a file copy fails, wait 10 seconds and retry 1 time
    cleanTargetBeforeCopy: true

PowerShell on Target Machines

Yaml.svg
- task: PowerShellOnTargetMachines@3
  inputs:
    machines: '$(machines)'
    userName: '$(userName)'
    userPassword: '$(userPassword)'
    scriptType: 'filePath' # inline by default
    scriptPath: '$(scriptPath)'
    scriptArguments: '-Arg1 $(Arg1Value)'
    communicationProtocol: http

- task: PowerShellOnTargetMachines@3
  inputs:
    machines: '$(machines)'
    userName: '$(userName)'
    userPassword: '$(userPassword)'
    scriptType: 'inline' # inline by default
    inlineScript: |
      echo 'Hello !'
    communicationProtocol: http

NuGet

Yaml.svg
- task: NuGetCommand@2
  inputs:
    command: restore
    restoreSolution: MySolution.sln  # relative path to a solution or project file

Visual Studio Build task

Yaml.svg
- task: VSBuild@1
  inputs:
    solution: MyProject\MyProject.csproj  # relative path to a solution or project file
    platform: AnyCPU                      # options: Win32, x86, x64 or any cpu. For a project use AnyCPU
    configuration: release                # options: debug or release
    clean: false                          # Set to false to make this an incremental build, it might reduce the build time. Make sure not to clean the repo too.
    msbuildArchitecture: x64              # options: x86 (default), x64

FileTransform

Replace tokens with variable values in XML or JSON configuration files.

Yaml.svg
- job:
  variables:
    Name1: Value1  # replace ... by Value1 <setting name="Name1"><value>...</value></setting>

  - task: FileTransform@1
    inputs:
      folderPath: $(Pipeline.Workspace)\MyArtefact
      fileType: xml
      targetFiles: MyProject.exe.config

MagicChunk

Modify a XML, JSON or YAML file.

Yaml.svg
- task: MagicChunks@2
  inputs:
  sourcePath: $(System.DefaultWorkingDirectory)\Folder\App.config
  transformations: |
    {
      "root/node[@attribute1='value']/@attribute2": "true"
    }

- task: MagicChunks@2
  inputs:
  sourcePath: $(System.DefaultWorkingDirectory)\Folder\App.config
  transformationType: file
  transformationsFile: $(System.DefaultWorkingDirectory)\Folder\Transformations.json
Transformations.json
{
  "root/node[@attribute1='value']/@attribute2": "true"
}

Pool

Yaml.svg
# if you use a private pool and don't need to specify demands
pool: PoolName

pool:
  name: PoolName
  demands:
  - myCustomCapability       # check for existence of capability
  - agent.os -equals Darwin  # check for specific string in capability

Variables

Yaml.svg
# scope: root, stage, job
variables:
  name1: 'value1' # name [\w\d._]+
  name2: '${{ variables.name1 }} - value2' # reuse the value defined for name1

variables:
- name: variable_name
  value: 'value'
# groups are used to make variables inside that group available across multiple pipelines
- group: group_name

steps:
- powershell: 'echo $(variable_name)' # pipeline variable
- powershell: 'echo $(System.StageName)' # system variable
- powershell: 'echo $(group_variable_name)' # variable in the group_name group variables
- powershell: 'echo $env:VARIABLE_NAME' # environment variable

- powershell: 'echo $(variable_name)' # macro syntax, processed during runtime before a task runs
- powershell: 'echo ${{ variables.variable_name }}' # template expression syntax, processed at compile time, before runtime starts
- powershell: 'echo $[variables.variable_name]' # Runtime expression syntax, processed during runtime, designed for use with conditions and expressions

# Set a job-scoped variable from a script
# the variable is available to downstream steps within the same job
- powershell: echo "##vso[task.setvariable variable=variable_name]variable value"
- powershell: echo $(variable_name)

# Set a stage-scoped variable from a script
jobs:
- job: A
  steps:
  - powershell: echo "##vso[task.setvariable variable=variable_name;isOutput=true]variable value"
    name: task_name
  - powershell: echo $(task_name.variable_name)
- job: B
  dependsOn: A
  variables:
    var_from_job_A: $[ dependencies.A.outputs['task_name.variable_name'] ] # expressions require single quotes

# Set an inter-stage-scoped variable from a script
stages:
- stage: A
  jobs:
  - job: A1
    steps:
     - powershell: echo "##vso[task.setvariable variable=variable_name;isOutput=true]variable value"
       name: task_name
- stage: B
  dependsOn: A
  variables:
    var_from_stage_A: $[ stageDependencies.A.A1.outputs['task_name.variable_name'] ]

Runtime parameters

Ces paramètres sont sélectionnable à chaque lancement de la pipeline.

Yaml.svg
parameters:
- name: my_parameter
  displayName: My parameter
  type: string
  default: Value1
  values:
  - Value1
  - Value2

Repositories

By default current pipeline repo is checked out.
Yaml.svg
resources:
  repositories: 
  - repository: RepoId  # A-Z, a-z, 0-9, and underscore
    name: ProjectName/RepoName  # repository name (format depends on `type`)
    type: git 
    ref: master

steps:
  - checkout: self    # checkout in $(Build.SourcesDirectory)\CurrentRepoId (C:\agent\_work\1\s\CurrentRepoId)
  - checkout: RepoId  # checkout in $(Build.SourcesDirectory)\RepoId (C:\agent\_work\1\s\RepoId)
  • Single repository: If you have a single checkout step in your job, (or no checkout step which is equivalent to checkout: self), your source code is checked out into $(Build.SourcesDirectory).
    Ex: (C:\agent\_work\1\s)
  • Multiple repositories: If you have multiple checkout steps in your job, your source code is checked out into directories named after the repositories as a subfolder of $(Build.SourcesDirectory).
    Ex: C:\agent\_work\1\s\MyRepo

Condition and dependency

It's as if you specified condition: succeeded()
Condition Description
succeeded() Only when all previous dependencies have succeeded. This is the default if there is not a condition set in the YAML.
succeededOrFailed() Even if a previous dependency has failed, unless the run was canceled.
always() Even if a previous dependency has failed, even if the run was canceled.
failed() Only when a previous dependency has failed.
Yaml.svg
# scope: stage, job, task
condition: eq(variables.Var1, 'Value1')
# parameter
condition: eq('${{ parameters.Param1 }}', 'Value1')
# and, or, eq, ne
condition: or(eq(variables.Var1, 'Value1'), ne(variables.Var2, 'Value2'))
# in
condition: in(variables['Build.Reason'], 'IndividualCI', 'BatchedCI')

# with dependencies: wait Job1 and Job2 to run
dependsOn:
- Job1
- Job2
# test Job1 and Job2 have succeeded and test Var1 = Value1
condition: and(succeeded(), eq(variables.Var1, 'Value1'))
# test Job2 has succeeded and test Var1 = Value1
condition: and(succeeded('Job2'), eq(variables.Var1, 'Value1'))
# test Job1 and Job2 have succeeded or skipped
condition: |
  and
  (
    in(dependencies.Job1.result, 'Succeeded', 'SucceededWithIssues', 'Skipped'),
    in(dependencies.Job2.result, 'Succeeded', 'SucceededWithIssues', 'Skipped')
  )

# stage dependency
- stage: Stage2
  # on Stage1
  condition: in(dependencies.Stage1.result, 'Succeeded', 'SucceededWithIssues', 'Skipped')
  condition: eq(dependencies.Stage1.result, '') # Stage1 has not been selected
  # on Stage1.Job1.Task1.Var1 output variable
  condition: and(succeeded(), eq(dependencies.Stage1.outputs['Job1.Task1.Var1'], 'true'))
  # job dependency
  jobs:
    - job: Job2
      # on Stage1.Job1
      condition: in(stageDependencies.Stage1.Job1.result, 'Succeeded', 'SucceededWithIssues', 'Skipped')
      # on Stage1.Job1.Task1.Var1 output variable
      condition: eq(stageDependencies.Stage1.Job1.outputs['Task1.Var1'], 'true')
      # on current stage Job3
      condition: in(dependencies.Job3.result, 'Succeeded', 'SucceededWithIssues', 'Skipped')
      # on current stage Job3.Task1.Var1 output variable
      condition: eq(dependencies.Job3.outputs['Task1.Var1'], 'true')
You can specify conditions under which a step, job, or stage will run.
By default, each stage in a pipeline depends on the one just before it in the YAML file.

if statement

Yaml.svg
# conditional insertion
variables:
  ${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
    var_name: value

steps:
- ${{ if eq(parameters.param_name, 'value') }}:
  - task:
- task:
  inputs:
    ${{ if eq(variables['Build.SourceBranchName'], 'master') }}:
      input_name: 'value'

Template

Insert a template

Reuse stages, jobs ans tasks.

stages-template.yml
stages:
- stage: Stage1
  jobs:
  - job: Job1
    steps:
    - script: echo 'stages template'
jobs-template.yml
jobs:
- job: Job1
  steps:
  - script: echo 'jobs template'
tasks-template.yml
steps:
- script: echo 'tasks template'
my-pipeline.yml
stages:
- template: stages-template.yml
- stage: Stage2
  jobs:
  - template: jobs-template.yml
- stage: Stage3
  jobs:
  - job: ExcecuteTasks
    steps:
    - template: tasks-template.yml

Extend from a template

Copy the template in the pipeline.

my-template.yml
resources:
  repositories: 
  - repository: MyRepo
    type: git 
    name: MyProject/MyRepo

steps:
- script: echo "My template"
my-pipepline.yml
extends:
  template: my-template.yml

Parameters

tasks-template.yml
parameters:
  parameterName: 'parameter value'

steps:
- script: echo ${{ parameters.parameterName}}
my-pipepline.yml
variables:
  var: 'value'

steps:
- template: tasks-template.yml
  parameters:
    parameterName: 'value'
    parameterName: ${{ variables.var }}  # $(var) doesn't work

Condition on a template

It is not possible to set a condition on a template.

Solution 1: pass a parameter to the template.

steps-template.yml
parameters:
  enabled: true

steps:
- script: echo 'Task 1'
  condition: ${{ parameters.enabled }}
my-pipepline.yml
steps:
- template: steps-template.yml
  parameters:
      enabled: false

Solution 2: use if.

my-pipepline.yml
steps:
- ${{ if eq($(MyVar), 'value') }}:
  - template: steps-template.yml

Errors

Environment xxx could not be found

  1. Pipelines → Environments → Select the env xxx → 3 dots button on top right → Security
  2. Add the user with User role