Version du 22 novembre 2023 à 00:14

Présentation

Matomo est un programme d'analyse statistique pour site web.

Mise à jour

Piwik's One Click update

chown -R http:http /usr/share/webapps/piwik
# mise à jour
chown -R root:root /usr/share/webapps/piwik
chown -R http:http /usr/share/webapps/piwik/tmp
chown http:http /usr/share/webapps/piwik/config/config.ini.php
chown http:http /usr/share/webapps/piwik/piwik.js

# si besoin, maj de la bdd
php /usr/share/webapps/piwik/console core:update

Désactiver le tracking

config/config.ini.php

; disable Piwik Tracking
[Tracker]
record_statistics = 0

Faire une sauvegarde du fichier de configuration

« config/config.ini.php »

MAJ des fichiers

FTP

Télécharger la nouvelle version de PIWIK et l'envoyer sur le FTP en remplaçant les anciens fichiers.

Veiller à ce que le transfert FTP se fasse en mode BINAIRE.

SSH

wget http://builds.piwik.org/latest.zip # download latest version
cd /srv/http # se rendre là où se trouve le dossier piwik
7z x latest.zip # extraire l'archive
rm latest.zip How\ to\ install\ Piwik.html # supprimer l'archive et le fichier d'aide

MAJ de la base de données

Aller sur la page d'accueil de votre PIWIK, un lien vous permettant de mettre à jour la BdD sera disponible.

# en ligne de commande
php /usr/share/webapps/piwik/console core:update

Configuration

Support Do Not Track preference

Settings → Privacy → Support Do Not Track preference

Exclure des IPs

Settings → Websites → Global list of Excluded IPs

Version

Le fichier « piwik/core/Version.php » contient la version de piwik.

Import server log instead of using JS tracker

sudo -u www-data \
python3 /var/www/matomo/misc/log-analytics/import_logs.py \
--url=https://matomo.localhost \
--idsite 1 \
--enable-http-errors --enable-http-redirects --enable-bots \
/var/log/nginx/website1-access.log

# process the log data
sudo -u www-data /var/www/matomo/console core:archive --force-all-websites --url='https://matomo.localhost'

# archivage et recalcule de tous les rapports historiques 
/var/www/matomo/console core:archive \
--force-all-websites \
--force-all-periods=315576000 \
--force-date-last-n=1000 \
--url='https://matomo.localhost'

# utiliser l'utilisateur « http »
su -c "/usr/bin/php /usr/share/webapps/piwik/console core:archive --url=http://piwik" http

# « php-gd » doit être installé

ReadMe

Options import_logs.py
url	url de piwik
idsite	dans le cas où hostname n’apparaît pas dans le log, il faut spécifier l'id du site
log-format-name	spécifie le format du log: common, common_vhost, ncsa_extended, common_complete, ... s'il n'est pas spécifié tous sont essayés
recorders	nombre de cpu à utiliser
show-progress	affiche le nombre de lignes traitées en temps réel
--debug --debug	liste les lignes invalides
dry-run	aucune donnée n'est ajoutée à piwik
skip=N	passe les N première ligne du fichier de log
enable-http-errors	traite les erreurs HTTP (status code 4xx et 5xx)
enable-http-redirects	traite les redirections HTTP (status code 3xx sauf 304)
exclude-path=/url	ignore l'url

ERROR CoreConsole Got invalid response from API request

/etc/php/php.ini

extension=curl.so

Format du log

Regex piwik _NCSA_EXTENDED_LOG_FORMAT	Log Nginx combined	Exemple de log Nginx combined
ip	$remote_addr	x.x.x.x
userid	$remote_user	-
date	$time_local	1/Apr/2016:00:00:00
timezone	$time_local	+0200
path	$request	/page.php
status	$status	200
length	$body_bytes_sent	10000
referrer	$http_referer	-
user_agent	$http_user_agent	Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Regex piwik _NCSA_EXTENDED_LOG_FORMAT
(?P<ip>\S+)\s+\S+\s+(?P<userid>\S+)\s+\[(?P<date>.*?)\s+(?P<timezone>.*?)\]\s+"\S+\s+(?P<path>.*?)\s+\S+"\s+(?P<status>\S+)\s+(?P<length>\S+)\s+"(?P<referrer>.*?)"\s+"(?P<user_agent>.*?)"

Log Nginx combined
$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"

Exemple de log Nginx combined
x.x.x.x - - [1/Apr/2016:00:00:00 +0200] "GET /page.php HTTP/1.1" 200 10000 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

logrotate

/etc/nginx/nginx.conf

# Créer un log journalier dans nginx
access_log  /var/log/nginx/mediawiki_access_daily.piwik;

/etc/logrotate.d/nginx

/var/log/nginx/mediawiki_access_daily.piwik {
    # archive le log tous les jours à 00h00
    daily
    # conserve 7 archives
    rotate 7
    # ajoute la date aux noms des archives
    dateext
    # format par défaut de la date: -%Y%m%d
    dateformat -%Y-%m-%d
    # ajoute la date avant l'extension
    extension .piwik
    missingok
    create 640 http log
    su http log
    compress
    postrotate
        test ! -r /run/nginx.pid || kill -USR1 `cat /run/nginx.pid`
    endscript
}

crontab

# tous les jours à 1:00
0 1 * * *	import_piwik_log_and_archive.sh

import_piwik_log_and_archive.sh

#!/bin/bash

set -e
set -o pipefail
set -u

python2 /usr/share/webapps/piwik/misc/log-analytics/import_logs.py \
  --url='http://piwik' \
  --idsite 1 \
  $(date +/var/log/nginx/mediawiki_access_daily-\%Y-\%m-\%d.piwik.gz) \
  --log-format-name ncsa_extended \
  --recorders=4

/usr/share/webapps/piwik/console core:archive --url='http://piwik'

Script perso

crontab

* * * * *	/path/to/piwik-import-log.sh 2>&1 | mail -s "Piwik Import Log" -r cron@mon-domain.fr xxx@gmail.com

piwik-import-log.sh

#!/bin/bash

set -e
set -o pipefail
set -u


log_file='/var/log/nginx/bananeatomic_access.log'


nb_of_lines_to_skip=0
if [[ -r .piwik-skip ]] ; then
    nb_of_lines_to_skip=$(cat .piwik-skip)
fi


nb_of_lines_in_the_log_file=$(wc -l < $log_file)
# si il n'y a pas eu de nouvelles entrées dans le log on ne fait rien
if [[ $nb_of_lines_to_skip -eq $nb_of_lines_in_the_log_file ]] ; then
    exit 0
# si il le log a été archivé
elif [[ $nb_of_lines_to_skip -gt $nb_of_lines_in_the_log_file ]] ; then
    nb_of_lines_to_skip=0
fi
echo $nb_of_lines_in_the_log_file > .piwik-skip


python2 /srv/http/piwik/misc/log-analytics/import_logs.py \
--url=http://piwik.mon-domain.fr \
"$log_file" \
--recorders=4 \
--exclude-path="/load.php" \
--exclude-path="/api.php" \
--skip=$nb_of_lines_to_skip

# déclencher manuellement l'archivage / la création des rapports
su http -m -c "/usr/bin/php /srv/http/piwik/console core:archive --url=http://piwik.mon-domain.fr"
# « http » n'ayant pas le droit de se connecter à un shell, il faut utiliser l'option « -m »

GeoLocalisation

Enable the GeoIp2 plugin
Download GeoIP database to /var/www/matomo/misc

/var/www/matomo/plugins/GeoIp2/config/config.php

'path.geoip2' => DI\string('{path.root}/misc/'),

It seems the database name dbip-city-lite-2023-11.mmdb is not recognized, rename it to DBIP-City-Lite.mmdb

GeoLocalisation OLD

Installation GeoIP

pacman -S php-geoip geoip-database-extra
# les paquets geoip et geoip-database seront installés comme dépendances
# geoip-database contient GeoIP.dat
# geoip-database-extra contient GeoIPCity.dat

/etc/php/conf.d/geoip.ini

extension=geoip.so

; redéfinir le dossier où se trouve les fichiers dat
geoip.custom_directory=/usr/share/GeoIP/

	Tester: Piwik → Administration → Geolocalisation

Installation PECL

	PECL est plus rapide que la version GeoIP PHP

pacman -S autoconf
pecl install geoip

Appliquez la géolocalisation aux anciens log visiteurs

# Mettre à jour la base de données :
/usr/bin/php /srv/http/piwik/console usercountry:attribute 2015-01-01,2016-01-01

# Reconstruire les rapports
/usr/bin/php /srv/http/piwik/console core:archive --force-all-websites --force-all-periods=315576000 --force-date-last-n=1000 --url=http://piwik

Delete data

# supprimer toutes les tables commençant par archive_
select concat('drop table ', group_concat(table_name), ';')
from information_schema.tables where table_schema = 'matomo' and table_name like 'archive_n%' or 'archive_b%';
# DROP TABLE archive_*;

delete from log_visit where idsite = 1;
delete from log_link_visit_action where idsite = 1; 
delete from log_conversion where idsite = 1; 
delete from log_conversion_item where idsite = 1;

Installation

wget https://builds.matomo.org/matomo.zip
unzip matomo.zip
rm -f matomo.zip How\ to\ install\ Matomo.html

sudo mv matomo /var/www
sudo chown -R root:root /var/www/matomo

sudo chown www-data:www-data /var/www/matomo/tmp
sudo chown www-data:www-data /var/www/matomo/matomo.js  # the js tracker has to be writable
sudo chown -R www-data:www-data /var/www/matomo/config  # only for the installation

# after the installation, reset access rights to the config folder
sudo chown -R root:root /var/www/matomo/config
# but allow to modifiy the config.ini.php file
sudo chown www-data:www-data /var/www/matomo/config/config.ini.php

NGINX

/etc/nginx/sites-available/matomo.conf

server {
    server_name matomo.localserver;
    listen 80;
    # Redirect all HTTP requests to HTTPS with a 301 Moved Permanently response.
    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    server_name matomo.localserver;
    listen      443 ssl http2;
    root        /var/www/matomo;
    index       index.php;

    ssl_certificate     /etc/letsencrypt/live/matomo.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/matomo.example.com/privkey.pem;

    add_header Referrer-Policy origin always; # make sure outgoing links don't show the URL to the Matomo instance
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    allow 192.168.0.0/24;
    deny all;

    access_log  /var/log/nginx/matomo-access.log;
    error_log   /var/log/nginx/matomo-error.log;

    ## only allow accessing the following php files
    location ~ ^/(index|matomo|piwik|js/index|plugins/HeatmapSessionRecording/configs)\.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_param HTTP_PROXY ""; # prohibit httpoxy: https://httpoxy.org/
        fastcgi_pass unix:/var/run/php/php-fpm.sock;
    }

    ## deny access to all other .php files
    location ~* ^.+\.php$ {
        deny all;
        return 403;
    }

    ## serve all other files normally
    location / {
        try_files $uri $uri/ =404;
    }

    ## disable all access to the following directories
    location ~ ^/(config|tmp|core|lang) {
        deny all;
        return 403; # replace with 404 to not show these directories exist
    }

    location ~ /\.ht {
        deny  all;
        return 403;
    }

    location ~ js/container_.*_preview\.js$ {
        expires off;
        add_header Cache-Control 'private, no-cache, no-store';
    }

    location ~ \.(gif|ico|jpg|png|svg|js|css|htm|html|mp3|mp4|wav|ogg|avi|ttf|eot|woff|woff2)$ {
        allow all;
        ## Cache images,CSS,JS and webfonts for an hour
        ## Increasing the duration may improve the load-time, but may cause old files to show after an Matomo upgrade
        expires 1h;
        add_header Pragma public;
        add_header Cache-Control "public";
    }

    location ~ ^/(libs|vendor|plugins|misc|node_modules) {
        deny all;
        return 403;
    }

    ## properly display textfiles in root directory
    location ~/(.*\.md|LEGALNOTICE|LICENSE) {
        default_type text/plain;
    }
}

Nginx et uwsgi

/etc/nginx/nginx.conf

server {
    listen 80;
    server_name piwik;
    root        /usr/share/webapps/piwik;
    index       index.php;
    
    location ~ \.php$ {
        include uwsgi_params;
        uwsgi_modifier1 14;
        uwsgi_pass unix:/run/uwsgi/piwik.sock;
    }
}

/etc/uwsgi/piwik.ini

[uwsgi]

; log
logger = file:/var/log/uwsgi/%n.log

; maximum number of worker processes
processes = 40
; start with only 4 and spawn the others on demand, maintaining a minimal pool of 4 processes.
cheaper = 1

; the user and group id of the process once it’s started
uid = http
gid = http

master = true
socket = /run/uwsgi/%n.sock
chdir = /usr/share/webapps/%n

; clear environment on exit
vacuum = true

; php
plugins = php
php-docroot = /usr/share/webapps/%n
php-index = index.php
php-allowed-ext = .php
; For some mysterious reason, the opcode cache is disabled in the embed SAPI.
; You can bypass the problem by telling the PHP engine that is running under the apache SAPI
php-sapi-name = apache

; extensions
php-set = extension=pdo_mysql.so
php-set = extension=iconv.so
php-set = extension=geoip.so

; php config
php-set = open_basedir=/usr/share/webapps/piwik/:/tmp/:/usr/share/GeoIP/

php-set = iconv.input_encoding=ISO-8859-1
php-set = iconv.internal_encoding=ISO-8859-1
php-set = iconv.output_encoding=ISO-8859-1

; To give Piwik enough memory to process your web analytics reports, increase the memory limit to 512M
php-set = memory_limit=512M

; define the GeoIP directory
php-set = geoip.custom_directory=/usr/share/GeoIP/

Database Setup

create database matomo default character set utf8mb4 collate utf8mb4_unicode_ci;
create user matomo@localhost identified by 'password';
grant all on matomo.* to matomo@localhost;
flush privileges;

/etc/mysql/mariadb.conf.d/50-server.cnf

max_allowed_packet = 64MB

PHP configuration

/etc/php/8.2/fpm/pool.d/matomo.conf

listen = /run/php/php8.2-fpm-matomo.sock

pm = ondemand
pm.max_children = 2

; Enable LOAD DATA INFILE
php_value[mysqli.allow_local_infile] = On
php_value[mysqli.local_infile_directory] = /var/www/matomo/tmp/assets/

/etc/nginx/sites-available/matomo.conf

fastcgi_pass unix:/var/run/php/php8.2-fpm-matomo.sock;

Plugins

be sure to have the write rights on /var/www/matomo/plugins
Marketplace on the left menu → Browse

Useful plugins and themes
Name	Description
Dark Theme	a dark theme for Matomo web site
Security Info	provides security information about your PHP environment
Log Viewer	display log messages logged by Matomo
Bot Tracker	detection of bots & spiders and count their visits without tracking them in the visitor-log

Erreurs

Can't get stat of '/usr/share/webapps/piwik/tmp/assets/option-xx.csv' (Errcode: 13 "Permission denied")

Administration → Diagnostic → System Check → Database abilities: LOAD DATA INFILE

# le dossier assets n'est pas accessible par mysql
ll /usr/share/webapps/piwik/tmp/assets
# drwxr-x---

chmod o+rx /usr/share/webapps/piwik/tmp/assets
ll /usr/share/webapps/piwik/tmp/assets
# drwxr-xr-x

The mysql driver is not currently installed

Décommentez la ligne suivante

/etc/php/php.ini

extension=pdo_mysql.so

# redemarrez le serveur
sudo systemctl restart httpd

SQLSTATE[HY000] [2002] Connection refused

Mauvais hôte, identifiant ou mot de passe.
Essayez localhost au lieu de 127.0.0.1

You need to configure and rebuild PHP with "iconv" support enabled, --with-iconv

Décommentez les lignes suivantes

/etc/php/php.ini

extension=iconv.so

[iconv]
iconv.input_encoding = ISO-8859-1
iconv.internal_encoding = ISO-8859-1
iconv.output_encoding = ISO-8859-1

# redemarrez le serveur
sudo systemctl restart httpd

@@ Ligne 245 : / Ligne 245 : @@
 # « http » n'ayant pas le droit de se connecter à un shell, il faut utiliser l'option « -m »
 </filebox>
-= Confidentialité =
-Créer un lien vers <nowiki>« http://monsite/piwik/index.php?module=CoreAdminHome&action=optOut&language=fr »</nowiki> pour créer un cookie signifiant à Piwiki de ne pas collecter les données.<br>
-[http://piwik.org/docs/privacy Configure Privacy Settings in Piwik]
 = [https://matomo.org/faq/how-to/faq_163/ GeoLocalisation] =

« Matomo » : différence entre les versions

Version du 22 novembre 2023 à 00:14

Présentation

Piwik's One Click update

Désactiver le tracking

Faire une sauvegarde du fichier de configuration

MAJ des fichiers

FTP

SSH

MAJ de la base de données

Configuration

Support Do Not Track preference

Exclure des IPs

Version

ERROR CoreConsole Got invalid response from API request

Format du log

logrotate

Script perso

GeoLocalisation OLD

Nginx et uwsgi

Database Setup

PHP configuration

Plugins

Erreurs

Can't get stat of '/usr/share/webapps/piwik/tmp/assets/option-xx.csv' (Errcode: 13 "Permission denied")

The mysql driver is not currently installed

SQLSTATE[HY000] [2002] Connection refused

You need to configure and rebuild PHP with "iconv" support enabled, --with-iconv

Menu de navigation

Rechercher

« Matomo » : différence entre les versions