Gitlab

Links

• archlinux wiki

Pipeline

• Development guide for GitLab CI/CD templates
• .NET Gitlab CI template
• .NET Code coverage on GitLab
• Unit test report examples

Ensure you have installed and registered runners

Gitlab runs each job in a new container

Default stages are .pre, build, test, deploy, .post

Gitlab → top left menu → Projects → My project → left column: Repository → Files

.gitlab-ci.yml

include: template: Template1.gitlab-ci.yml # define the docker image to use image: mcr.microsoft.com/dotnet/sdk:7.0 variables: VAR1: value job1: variables: VAR2: "The $VAR1 is invalid" GIT_STRATEGY: none # skips all Git operations. Useful for deployment jobs. script: - local_var=value - 'echo "$local_var ${VAR2}"' - echo ${VAR2} rules: # the jib is executed only if one of the rules matches - if: $CI_PIPELINE_SOURCE == "merge_request_event" when: manual # manual job allow_failure: true # the pipeline continues running even if the manual job is not run - if: $CI_PIPELINE_SOURCE == "schedule" # if the first rule doesn’t match, then the second rule is evaluated when: on_success # default allow_failure: false # default artifacts: name: "$CI_JOB_NAME" # name of the current job name: "$CI_COMMIT_REF_SLUG" # name of the current branch or tag untracked: true # add all Git untracked files as artifacts paths: - binaries/ exclude: - binaries/**/*.o # Unlike artifacts:paths, exclude paths are not recursive # jobs download all artifacts from the completed jobs in previous stages by default dependencies: [] # prevent a job from downloading any artifacts cache: key: $CI_COMMIT_REF_SLUG # Share caches between jobs in the same branch key: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG" # per-job and per-branch caching key: "$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG" # per-stage and per-branch caching key: one-key-to-rule-them-all # share a cache across all branches and all jobs, use the same key for everything key: $CI_JOB_NAME # share a cache between branches, but have a unique cache for each job paths: - '$NUGET_PACKAGES_DIRECTORY' policy: pull-push # pull-push for changes to the default branch, pull for changes to other branches.

.NET pipeline

• DOTNET CI/CD with Gitlab

.gitlab-ci.yml

# merge request pipeline workflow: rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' image: mcr.microsoft.com/dotnet/sdk:7.0 cache: key: "$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG" # Per-stage and per-branch caching. paths: - obj/project.assets.json - obj/*.csproj.nuget.* - .nuget - 'tools' policy: pull-push build: stage: build script: - 'dotnet build' test: stage: test script: # run unit tests with Coverlet data collector and log the results # JunitXML.TestLogger nuget package has to be added to the unit tests projects - 'dotnet test --collect="XPlat Code Coverage" --logger:"junit;LogFilePath=TestResults.xml"' # install reportgenerator - '[[ -x tools/reportgenerator ]] || dotnet tool install dotnet-reportgenerator-globaltool --tool-path tools' # merge the code coverage reports into 1 Cobertura report file and generate a text summary report file - 'tools/reportgenerator -reports:"*/TestResults/*/coverage.cobertura.xml" -targetdir:CoverageReport -reporttypes:"Cobertura;TextSummary"' # print the text summary report file - "sed -n '/Summary/,/^$/p' CoverageReport/Summary.txt" # read the line coverage from the printed text summary report file coverage: '/Line coverage: [0-9.]+%/' artifacts: reports: junit: - "*/TestResults.xml" coverage_report: coverage_format: cobertura path: "CoverageReport/Cobertura.xml" quality: stage: test script: - 'dotnet build' - '[[ -x tools/roslynator ]] || dotnet tool install roslynator.dotnet.cli --tool-path tools' - 'tools/roslynator analyze -o roslynator.xml || true' - '[[ -x tools/cq ]] || dotnet tool install CodeQualityToGitlab --tool-path tools' - 'tools/cq roslynator roslynator.xml gl-code-quality-report.json "$CI_BUILDS_DIR/$CI_JOB_ID/$CI_PROJECT_NAME"' artifacts: paths: - roslynator.xml - gl-code-quality-report.json expose_as: 'code_quality_reports' reports: codequality: gl-code-quality-report.json deploy: stage: deploy environment: production script: - 'dotnet publish --configuration Release' - 'mv bin/Release/net7.0/publish ./dotnetcore' artifacts: paths: - dotnetcore

Predefined variables

Name	Description
CI_COMMIT_REF_NAME	The branch or tag name for which project is built
CI_COMMIT_REF_SLUG	CI_COMMIT_REF_NAME in lowercase, shortened to 63 bytes, and with everything except 0-9 and a-z replaced with -

Merge request pipeline

Branch pipelines are the default pipelines, they run when you push a new commit to a branch.

.gitlab-ci.yml

# the entire pipeline will only run on a merge request workflow: rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' # job1 will only run on a merge request job1: script: - echo "This job runs in merge request pipelines" rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

Environments and deployments

.gitlab-ci.yml

deploy_staging: stage: deploy script: - echo "Deploy to staging server" environment: name: staging url: https://staging.example.com

Code Quality

Pipeline details view is not available in Gitlab Free.

Log

Log files are in /var/log/gitlab

Change sidekiq log level

/usr/share/webapps/gitlab/config/initializers/sidekiq.rb

Sidekiq.configure_server do |config| # force WARN level config.logger.level = Logger::WARN # or set the same log level as Rails config.logger.level = Rails.logger.level

Rails Console Session

sudo -u gitlab $(cat environment | xargs) bundle-2.7 exec rails console -e production

Add new user

1. from admin account → menu → admin
2. Overview → Users
3. click on the button New user (top right)

Resend confirmation email: https://gitlab.domain.net/users/confirmation/new

Installation

1. install and configure PostgreSQL

sudo pacman gitlab

/etc/webapps/gitlab/gitlab.yml

host: gitlab.domain.net port: 443 https: true

Secret string

Fill the following files with a secret:

hexdump -v -n 64 -e '1/1 "%02x"' /dev/urandom | sudo dd of=/etc/webapps/gitlab/secret sudo chmod 640 /etc/webapps/gitlab/secret hexdump -v -n 64 -e '1/1 "%02x"' /dev/urandom | sudo dd of=/etc/webapps/gitlab-shell/secret sudo chmod 640 /etc/webapps/gitlab-shell/secret

Set secrets in the following configuration file:

/etc/webapps/gitlab/secrets.yml

production: secret_key_base: [secret] db_key_base: [secret] otp_key_base: [secret] openid_connect_signing_key: [secret] encrypted_settings_key_base: [secret]

Redis

# Add user gitlab to group redis cat /etc/group | grep redis sudo gpasswd -a gitlab redis

Listen on socket

/etc/redis/redis.conf

unixsocket /run/redis/redis.sock unixsocketperm 770 # don't listen on TCP #port 6379 port 0

/etc/webapps/gitlab/resque.yml

development: url: unix:/run/redis/redis.sock test: url: unix:/run/redis/redis.sock production: url: unix:/run/redis/redis.sock

PostgreSQL database

# switch to the PostgreSQL user sudo -iu postgres # create a user named gitlab with superuser rights createuser --interactive # login into the databases server psql # change the password of the gitlab user \password gitlab # create the database create database gitlabdb OWNER gitlab;

/var/lib/postgres/data/pg_hba.conf

# TYPE DATABASE USER METHOD local gitlabdb gitlab scram-sha-256

# login as gitlab psql -U gitlab -d gitlabdb

/etc/webapps/gitlab/database.yml

production: main: adapter: postgresql encoding: unicode database: gitlabdb username: gitlab password: [password] # host: localhost # port: 5432 socket: /run/postgresql/.s.PGSQL.5432

# start redis and gitlab-gitaly sc-start redis sc-start gitlab-gitaly cd /usr/share/webapps/gitlab sudo -u gitlab $(cat environment | xargs) bundle-2.7 exec rake gitlab:setup

Check

# Adjust modifier bits sudo chmod -R ug+rwX,o-rwx /var/lib/gitlab/repositories/ sudo chmod -R ug-s /var/lib/gitlab/repositories sudo find /var/lib/gitlab/repositories/ -type d -print0 | xargs -0 sudo chmod g+s sc-start postgresql sc-start redis sc-start gitlab.target sudo -u gitlab $(cat environment | xargs) bundle-2.7 exec rake gitlab:env:info sudo -u gitlab $(cat environment | xargs) bundle-2.7 exec rake gitlab:check

Nginx

/etc/nginx/sites-available/gitlab.conf

upstream gitlab-workhorse { server unix:/run/gitlab/gitlab-workhorse.socket fail_timeout=0; } server { listen 80; # IPv4 HTTP #listen 443 ssl http2; # uncomment to enable IPv4 HTTPS + HTTP/2 #listen [::]:80; # uncomment to enable IPv6 HTTP #listen [::]:443 ssl http2; # uncomment to enable IPv6 HTTPS + HTTP/2 server_name gitlab.localhost; access_log /var/log/gitlab/nginx_access.log; error_log /var/log/gitlab/nginx_error.log; #ssl_certificate ssl/example.com.crt; #ssl_certificate_key ssl/example.com.key; location ~ ^/(assets)/ { root /usr/share/webapps/gitlab/public; gzip_static on; # to serve pre-gzipped version expires max; add_header Cache-Control public; } location / { # unlimited upload size in nginx (so the setting in GitLab applies) client_max_body_size 0; # proxy timeout should match the timeout value set in /etc/webapps/gitlab/puma.rb proxy_read_timeout 60; proxy_connect_timeout 60; proxy_redirect off; proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; #proxy_set_header X-Forwarded-Ssl on; proxy_pass http://gitlab-workhorse; } error_page 404 /404.html; error_page 422 /422.html; error_page 500 /500.html; error_page 502 /502.html; error_page 503 /503.html; location ~ ^/(404|422|500|502|503)\.html$ { root /usr/share/webapps/gitlab/public; internal; } }

Fast lookup of authorized SSH keys

/etc/ssh/sshd_config

# Gitlab fast SSH key lookup AuthorizedKeysCommand /var/lib/gitlab/gitlab-shell/bin/gitlab-shell-authorized-keys-check gitlab %u %k AuthorizedKeysCommandUser gitlab

# generate private and public keys: ~/.ssh/id_ed25519 ~/.ssh/id_ed25519.pub ssh-keygen -t ed25519 -C "$(whoami)@$(hostname)" # start ssh daemon sc-start sshd

Add the public key to Gitlab:

1. click on the user icone top right → edit profile
2. SSH Keys on left menu
3. paste the public key (~/.ssh/id_ed25519.pub)
4. click on the Add key button

SMTP

/etc/webapps/gitlab/smtp_settings.rb

if Rails.env.production? Rails.application.config.action_mailer.delivery_method = :smtp secrets = Gitlab::Email::SmtpConfig.secrets ActionMailer::Base.delivery_method = :smtp ActionMailer::Base.smtp_settings = { address: "smtp.domain.net", port: 587, user_name: secrets.username, password: secrets.password, domain: "domain.net", authentication: :login, enable_starttls_auto: true, openssl_verify_mode: 'peer' # See ActionMailer documentation for other possible options } end

# add the username and password to the smtp:secret sudo -u gitlab $(cat environment | xargs) bundle-2.7 exec rake gitlab:smtp:secret:edit EDITOR=nano # open a Rails Console Session sudo -u gitlab $(cat environment | xargs) bundle-2.7 exec rails console -e production # check the configuration ActionMailer::Base.delivery_method # => :smtp ActionMailer::Base.smtp_settings # => {:address=>"smtp.domain.net", :port=587, ... # send a test email Notify.test_email('user@domain.net', 'Hello World', 'This is a test message').deliver_now

Runner

# install sudo pacman -S gitlab-runner # register sudo gitlab-runner register # GitLab instance URL: https://www.mygitlab.net/ # Registration token: Gitlab (admin) → top left menu → Admin → left column: CI/CD → Runners → right: Register an instance runner button → copy the registration token # Executor: docker # Default Docker image: mcr.microsoft.com/dotnet/sdk:6.0 # Configuration (with the authentication token) is saved in "/etc/gitlab-runner/config.toml" # bring runner online manually sudo gitlab-runner run # by running the service sc-start gitlab-runner

• Advanced configuration options

/etc/gitlab-runner/config.toml

log_level = "debug" # change the loglevel [[runners]] url = "https://www.mygitlab.net/" clone_url = "https://www.mygitlab.net/" # override the url defined in /etc/webapps/gitlab/gitlab.yml environment = ["ENV=value", "LC_ALL=en_US.UTF-8"] # set environment variables [runners.docker] image = "mcr.microsoft.com/dotnet/sdk:7.0" # default docker image, used if no docker image is defined in the pipeline extra_hosts = ["www.mygitlab.net:host-gateway"] # map www.mygitlab.net with the host, so the runner can access the website www.mygitlab.net running on the host

Errors

Number of Sidekiq processes (cluster/worker) ... 0/1

Sidekiq: ... Running? ... yes
Number of Sidekiq processes (cluster/worker) ... 0/1
  Try fixing it:
  sudo systemctl restart gitlab-sidekiq.service
  Please fix the error above and rerun the checks.

All migrations up? ... Exception: No such file or directory - bundle

Database config exists? ... yes
All migrations up? ... Exception: No such file or directory - bundle

Cable config exists? ... no

Cable config exists? ... no
  Try fixing it:
  Copy config/cable.yml.example to config/cable.yml
  Update config/cable.yml to match your setup
  For more information see:
  doc/install/installation.md in section "GitLab"
  Please fix the error above and rerun the checks.