Aller au contenu

Gitlab

De Banane Atomic

Links

Continuous Integration (CI)

Pipeline

Ensure you have installed and registered runners
Gitlab runs each job in a new container
Default stages are .pre, build, test, deploy, .post

Gitlab → top left menu → Projects → My project → left column: Repository → Files

.gitlab-ci.yml 
include:
  template: Template1.gitlab-ci.yml

# define the docker image to use
image: mcr.microsoft.com/dotnet/sdk:7.0

variables:
  VAR1: value

job1:
  variables:
    VAR2: "The $VAR1 is invalid"
    GIT_STRATEGY: none  # skips all Git operations. Useful for deployment jobs.
  script:
    - local_var=value
    - 'echo "$local_var ${VAR2}"'
    - echo ${VAR2}
  rules:  # the jib is executed only if one of the rules matches
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      when: manual  # manual job
      allow_failure: true  # the pipeline continues running even if the manual job is not run
    - if: $CI_PIPELINE_SOURCE == "schedule"  # if the first rule doesn’t match, then the second rule is evaluated
      when: on_success  # default
      allow_failure: false  # default
  artifacts:
    name: "$CI_JOB_NAME"  # name of the current job
    name: "$CI_COMMIT_REF_SLUG"  # name of the current branch or tag
    untracked: true  # add all Git untracked files as artifacts
    paths:
      - binaries/
    exclude:
      - binaries/**/*.o  # Unlike artifacts:paths, exclude paths are not recursive
  # jobs download all artifacts from the completed jobs in previous stages by default
  dependencies: []  # prevent a job from downloading any artifacts

cache:
  key: $CI_COMMIT_REF_SLUG  # Share caches between jobs in the same branch
  key: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"  # per-job and per-branch caching
  key: "$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG"  # per-stage and per-branch caching
  key: one-key-to-rule-them-all  # share a cache across all branches and all jobs, use the same key for everything
  key: $CI_JOB_NAME  # share a cache between branches, but have a unique cache for each job
  paths:
    - '$NUGET_PACKAGES_DIRECTORY'

  policy: pull-push  # pull-push for changes to the default branch, pull for changes to other branches.

.NET pipeline

.gitlab-ci.yml 
# merge request pipeline
workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

image: mcr.microsoft.com/dotnet/sdk:7.0

cache:
  key: "$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG"  # Per-stage and per-branch caching.
  paths:
    - obj/project.assets.json
    - obj/*.csproj.nuget.*
    - .nuget
    - 'tools'
  policy: pull-push

build:
  stage: build
  script:
    - 'dotnet build'

test:
  stage: test
  script:
    # run unit tests with Coverlet data collector and log the results
    # JunitXML.TestLogger nuget package has to be added to the unit tests projects
    - 'dotnet test --collect="XPlat Code Coverage" --logger:"junit;LogFilePath=TestResults.xml"'
    # install reportgenerator
    - '[[ -x tools/reportgenerator ]] || dotnet tool install dotnet-reportgenerator-globaltool --tool-path tools'
    # merge the code coverage reports into 1 Cobertura report file and generate a text summary report file
    - 'tools/reportgenerator -reports:"*/TestResults/*/coverage.cobertura.xml" -targetdir:CoverageReport -reporttypes:"Cobertura;TextSummary"'
    # print the text summary report file
    - "sed -n '/Summary/,/^$/p' CoverageReport/Summary.txt"
  # read the line coverage from the printed text summary report file
  coverage: '/Line coverage: [0-9.]+%/'
  artifacts:
    reports:
      junit:
        - "*/TestResults.xml"
      coverage_report:
        coverage_format: cobertura
        path: "CoverageReport/Cobertura.xml"

quality:
  stage: test
  script:
    - 'dotnet build'
    - '[[ -x tools/roslynator ]] || dotnet tool install roslynator.dotnet.cli --tool-path tools'
    - 'tools/roslynator analyze -o roslynator.xml || true'
    - '[[ -x tools/cq ]] || dotnet tool install CodeQualityToGitlab --tool-path tools'
    - 'tools/cq roslynator roslynator.xml gl-code-quality-report.json "$CI_BUILDS_DIR/$CI_JOB_ID/$CI_PROJECT_NAME"'
  artifacts:
    paths:
      - roslynator.xml
      - gl-code-quality-report.json
    expose_as: 'code_quality_reports'
    reports:
      codequality: gl-code-quality-report.json

deploy:
  stage: deploy
  environment: production
  script:
    - 'dotnet publish --configuration Release'
    - 'mv bin/Release/net7.0/publish ./dotnetcore'
  artifacts:
    paths:
      - dotnetcore

Predefined variables

Name Description
CI_COMMIT_REF_NAME The branch or tag name for which project is built
CI_COMMIT_REF_SLUG CI_COMMIT_REF_NAME in lowercase, shortened to 63 bytes, and with everything except 0-9 and a-z replaced with -

Merge request pipeline

Branch pipelines are the default pipelines, they run when you push a new commit to a branch.
.gitlab-ci.yml 
# the entire pipeline will only run on a merge request
workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

# job1 will only run on a merge request
job1:
  script:
    - echo "This job runs in merge request pipelines"
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

Environments and deployments

.gitlab-ci.yml 
deploy_staging:
  stage: deploy
  script:
    - echo "Deploy to staging server"
  environment:
    name: staging
    url: https://staging.example.com

Code Quality

Pipeline details view is not available in Gitlab Free.

Log

Log files are in /var/log/gitlab

Change sidekiq log level

/usr/share/webapps/gitlab/config/initializers/sidekiq.rb 
Sidekiq.configure_server do |config|
  # force WARN level
  config.logger.level = Logger::WARN
  # or set the same log level as Rails
  config.logger.level = Rails.logger.level

Rails Console Session

sudo -u gitlab $(cat environment | xargs) bundle-2.7 exec rails console -e production

Add new user

  1. from admin account → menu → admin
  2. Overview → Users
  3. click on the button New user (top right)

Resend confirmation email: https://gitlab.domain.net/users/confirmation/new

Installation

  1. install and configure PostgreSQL
sudo pacman gitlab
/etc/webapps/gitlab/gitlab.yml 
host: gitlab.domain.net
port: 443
https: true

Secret string

Fill the following files with a secret: 

hexdump -v -n 64 -e '1/1 "%02x"' /dev/urandom | sudo dd of=/etc/webapps/gitlab/secret
sudo chmod 640 /etc/webapps/gitlab/secret

hexdump -v -n 64 -e '1/1 "%02x"' /dev/urandom | sudo dd of=/etc/webapps/gitlab-shell/secret
sudo chmod 640 /etc/webapps/gitlab-shell/secret

Set secrets in the following configuration file:

/etc/webapps/gitlab/secrets.yml 
production:
  secret_key_base: [secret]
  db_key_base: [secret]
  otp_key_base: [secret]
  openid_connect_signing_key: [secret]
  encrypted_settings_key_base: [secret]

Redis

# Add user gitlab to group redis
cat /etc/group | grep redis
sudo gpasswd -a gitlab redis

Listen on socket

/etc/redis/redis.conf 
unixsocket /run/redis/redis.sock
unixsocketperm 770

# don't listen on TCP
#port 6379
port 0
/etc/webapps/gitlab/resque.yml 
development:
  url: unix:/run/redis/redis.sock
test:
  url: unix:/run/redis/redis.sock
production:
  url: unix:/run/redis/redis.sock

PostgreSQL database

# switch to the PostgreSQL user
sudo -iu postgres

# create a user named gitlab with superuser rights
createuser --interactive

# login into the databases server
psql

# change the password of the gitlab user
\password gitlab

# create the database
create database gitlabdb OWNER gitlab;
/var/lib/postgres/data/pg_hba.conf 
# TYPE  DATABASE        USER            METHOD
local   gitlabdb        gitlab          scram-sha-256
# login as gitlab
psql -U gitlab -d gitlabdb
/etc/webapps/gitlab/database.yml 
production:
  main:
    adapter: postgresql
    encoding: unicode
    database: gitlabdb
    username: gitlab
    password: [password]
    # host: localhost
    # port: 5432
    socket: /run/postgresql/.s.PGSQL.5432
# start redis and gitlab-gitaly
sc-start redis
sc-start gitlab-gitaly

cd /usr/share/webapps/gitlab
sudo -u gitlab $(cat environment | xargs) bundle-2.7 exec rake gitlab:setup

Check

# Adjust modifier bits
sudo chmod -R ug+rwX,o-rwx /var/lib/gitlab/repositories/
sudo chmod -R ug-s /var/lib/gitlab/repositories
sudo find /var/lib/gitlab/repositories/ -type d -print0 | xargs -0 sudo chmod g+s

sc-start postgresql
sc-start redis
sc-start gitlab.target

sudo -u gitlab $(cat environment | xargs) bundle-2.7 exec rake gitlab:env:info
sudo -u gitlab $(cat environment | xargs) bundle-2.7 exec rake gitlab:check

Nginx

/etc/nginx/sites-available/gitlab.conf 
upstream gitlab-workhorse {
  server unix:/run/gitlab/gitlab-workhorse.socket fail_timeout=0;
}

server {
  listen 80;                  # IPv4 HTTP
  #listen 443 ssl http2;      # uncomment to enable IPv4 HTTPS + HTTP/2
  #listen [::]:80;            # uncomment to enable IPv6 HTTP
  #listen [::]:443 ssl http2; # uncomment to enable IPv6 HTTPS + HTTP/2
  server_name gitlab.localhost;

  access_log  /var/log/gitlab/nginx_access.log;
  error_log   /var/log/gitlab/nginx_error.log;

  #ssl_certificate ssl/example.com.crt;
  #ssl_certificate_key ssl/example.com.key;

  location ~ ^/(assets)/ {
    root /usr/share/webapps/gitlab/public;
    gzip_static on; # to serve pre-gzipped version
    expires max;
    add_header Cache-Control public;
  }

  location / {
      # unlimited upload size in nginx (so the setting in GitLab applies)
      client_max_body_size 0;

      # proxy timeout should match the timeout value set in /etc/webapps/gitlab/puma.rb
      proxy_read_timeout 60;
      proxy_connect_timeout 60;
      proxy_redirect off;

      proxy_set_header Host $http_host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;

      #proxy_set_header X-Forwarded-Ssl on;

      proxy_pass http://gitlab-workhorse;
  }

  error_page 404 /404.html;
  error_page 422 /422.html;
  error_page 500 /500.html;
  error_page 502 /502.html;
  error_page 503 /503.html;
  location ~ ^/(404|422|500|502|503)\.html$ {
    root /usr/share/webapps/gitlab/public;
    internal;
  }
}

Fast lookup of authorized SSH keys

/etc/ssh/sshd_config 
# Gitlab fast SSH key lookup
AuthorizedKeysCommand /var/lib/gitlab/gitlab-shell/bin/gitlab-shell-authorized-keys-check gitlab %u %k
AuthorizedKeysCommandUser gitlab
# generate private and public keys: ~/.ssh/id_ed25519 ~/.ssh/id_ed25519.pub
ssh-keygen -t ed25519 -C "$(whoami)@$(hostname)"

# start ssh daemon
sc-start sshd

Add the public key to Gitlab:

  1. click on the user icone top right → edit profile
  2. SSH Keys on left menu
  3. paste the public key (~/.ssh/id_ed25519.pub)
  4. click on the Add key button

SMTP

/etc/webapps/gitlab/smtp_settings.rb 
if Rails.env.production?
  Rails.application.config.action_mailer.delivery_method = :smtp
  secrets = Gitlab::Email::SmtpConfig.secrets

  ActionMailer::Base.delivery_method = :smtp
  ActionMailer::Base.smtp_settings = {
    address: "smtp.domain.net",
    port: 587,
    user_name: secrets.username,
    password: secrets.password,
    domain: "domain.net",
    authentication: :login,
    enable_starttls_auto: true,
    openssl_verify_mode: 'peer' # See ActionMailer documentation for other possible options
  }
end
# add the username and password to the smtp:secret
sudo -u gitlab $(cat environment | xargs) bundle-2.7 exec rake gitlab:smtp:secret:edit EDITOR=nano

# open a Rails Console Session
sudo -u gitlab $(cat environment | xargs) bundle-2.7 exec rails console -e production

# check the configuration
ActionMailer::Base.delivery_method
# => :smtp
ActionMailer::Base.smtp_settings
# => {:address=>"smtp.domain.net", :port=587, ...

# send a test email
Notify.test_email('user@domain.net', 'Hello World', 'This is a test message').deliver_now

Runner

# install
sudo pacman -S gitlab-runner

# register
sudo gitlab-runner register
# GitLab instance URL: https://www.mygitlab.net/
# Registration token: Gitlab (admin) → top left menu → Admin → left column: CI/CD → Runners → right: Register an instance runner button → copy the registration token
# Executor: docker
# Default Docker image: mcr.microsoft.com/dotnet/sdk:6.0

# Configuration (with the authentication token) is saved in "/etc/gitlab-runner/config.toml"

# bring runner online manually
sudo gitlab-runner run
# by running the service
sc-start gitlab-runner
/etc/gitlab-runner/config.toml 
log_level = "debug"  # change the loglevel

[[runners]]
  url = "https://www.mygitlab.net/"
  clone_url = "https://www.mygitlab.net/"  # override the url defined in /etc/webapps/gitlab/gitlab.yml
  environment = ["ENV=value", "LC_ALL=en_US.UTF-8"]  # set environment variables
  [runners.docker]
    image = "mcr.microsoft.com/dotnet/sdk:7.0"  # default docker image, used if no docker image is defined in the pipeline
    extra_hosts = ["www.mygitlab.net:host-gateway"]  # map www.mygitlab.net with the host, so the runner can access the website www.mygitlab.net running on the host

Errors

Number of Sidekiq processes (cluster/worker) ... 0/1

Sidekiq: ... Running? ... yes
Number of Sidekiq processes (cluster/worker) ... 0/1
  Try fixing it:
  sudo systemctl restart gitlab-sidekiq.service
  Please fix the error above and rerun the checks.

All migrations up? ... Exception: No such file or directory - bundle

Database config exists? ... yes
All migrations up? ... Exception: No such file or directory - bundle

Cable config exists? ... no

Cable config exists? ... no
  Try fixing it:
  Copy config/cable.yml.example to config/cable.yml
  Update config/cable.yml to match your setup
  For more information see:
  doc/install/installation.md in section "GitLab"
  Please fix the error above and rerun the checks.
Récupérée de "http://wiki.bananeatomic.fr/index.php?title=Gitlab&oldid=33098"