Présentation
Matomo est un programme d'analyse statistique pour site web.
|
# change the folder rights to allow the update
chown -R www-data:www-data /var/www/matomo
# mise à jour
chown -R root:root /usr/share/webapps/piwik
chown -R http:http /usr/share/webapps/piwik/tmp
chown http:http /usr/share/webapps/piwik/config/config.ini.php
chown http:http /usr/share/webapps/piwik/piwik.js
# si besoin, maj de la bdd
php /usr/share/webapps/piwik/console core:update
|
Backup the config/config.ini.php file.
|
|
# download the latest version
wget https://builds.matomo.org/matomo.zip
# extract it to "matomo/" directory
unzip -o matomo.zip
# delete the archive
rm matomo.zip
# check for any obsolete files, Matomo 4.13.2+ only
./console diagnostics:unexpected-files
# delete any obsolete files, Matomo 4.13.2+ only
./console diagnostics:unexpected-files --delete
|
Once Matomo is successfully upgraded, visit the Admin > Diagnostics > System check report and review the report and follow any recommended actions.
Désactiver le tracking
| config/config.ini.php
|
; disable Piwik Tracking
[Tracker]
record_statistics = 0
|
Configuration
Support Do Not Track preference
Settings → Privacy → Support Do Not Track preference
Exclure des IPs
Settings → Websites → Global list of Excluded IPs
|
|
sudo crontab -u www-data -e
|
| crontab
|
5 * * * * www-data /usr/bin/php /var/www/matomo/console core:archive --url=https://matomo.domain.net > /var/log/matomo-archive.log
|
Administration → System → General settings → Archiving settings → Archive reports when viewed from the browser = No
 |
Python 3.x is required. |
|
|
sudo -u www-data \
python3 /var/www/matomo/misc/log-analytics/import_logs.py \
--url=https://matomo.localhost \
--idsite 1 \
--enable-http-errors --enable-http-redirects --enable-bots \
/var/log/nginx/website1-access.log
# process the log data
sudo -u www-data /var/www/matomo/console core:archive --force-all-websites --url='https://matomo.localhost'
# --force-all-periods=315576000
# --force-date-last-n=1000
|
Matomo Server Log Analytics on GitHub
Options for the import_logs.py script
| url |
url de piwik
|
| idsite |
dans le cas où hostname n’apparaît pas dans le log, il faut spécifier l'id du site
|
| log-format-name |
spécifie le format du log: common, common_vhost, ncsa_extended, common_complete, ...
s'il n'est pas spécifié tous sont essayés
|
| recorders |
nombre de cpu à utiliser
|
| show-progress |
affiche le nombre de lignes traitées en temps réel
|
| --debug --debug |
liste les lignes invalides
|
| dry-run |
aucune donnée n'est ajoutée à piwik
|
| skip=N |
passe les N première ligne du fichier de log
|
| enable-http-errors |
traite les erreurs HTTP (status code 4xx et 5xx)
|
| enable-http-redirects |
traite les redirections HTTP (status code 3xx sauf 304)
|
| exclude-path=/url |
ignore l'url
|
ERROR CoreConsole Got invalid response from API request
| /etc/php/php.ini
|
extension=curl.so
|
Format du log
| Regex piwik _NCSA_EXTENDED_LOG_FORMAT
|
Log Nginx combined
|
Exemple de log Nginx combined
|
| ip |
$remote_addr |
x.x.x.x
|
| userid |
$remote_user |
-
|
| date |
$time_local |
1/Apr/2016:00:00:00
|
| timezone |
$time_local |
+0200
|
| path |
$request |
/page.php
|
| status |
$status |
200
|
| length |
$body_bytes_sent |
10000
|
| referrer |
$http_referer |
-
|
| user_agent |
$http_user_agent |
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
|
Regex piwik _NCSA_EXTENDED_LOG_FORMAT
(?P<ip>\S+)\s+\S+\s+(?P<userid>\S+)\s+\[(?P<date>.*?)\s+(?P<timezone>.*?)\]\s+"\S+\s+(?P<path>.*?)\s+\S+"\s+(?P<status>\S+)\s+(?P<length>\S+)\s+"(?P<referrer>.*?)"\s+"(?P<user_agent>.*?)"
Log Nginx combined
$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"
Exemple de log Nginx combined
x.x.x.x - - [1/Apr/2016:00:00:00 +0200] "GET /page.php HTTP/1.1" 200 10000 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
logrotate
| /etc/nginx/nginx.conf
|
# Créer un log journalier dans nginx
access_log /var/log/nginx/mediawiki_access_daily.piwik;
|
| /etc/logrotate.d/nginx
|
/var/log/nginx/mediawiki_access_daily.piwik {
# archive le log tous les jours à 00h00
daily
# conserve 7 archives
rotate 7
# ajoute la date aux noms des archives
dateext
# format par défaut de la date: -%Y%m%d
dateformat -%Y-%m-%d
# ajoute la date avant l'extension
extension .piwik
missingok
create 640 http log
su http log
compress
postrotate
test ! -r /run/nginx.pid || kill -USR1 `cat /run/nginx.pid`
endscript
}
|
| crontab
|
# tous les jours à 1:00
0 1 * * * import_piwik_log_and_archive.sh
|
| import_piwik_log_and_archive.sh
|
#!/bin/bash
set -e
set -o pipefail
set -u
python2 /usr/share/webapps/piwik/misc/log-analytics/import_logs.py \
--url='http://piwik' \
--idsite 1 \
$(date +/var/log/nginx/mediawiki_access_daily-\%Y-\%m-\%d.piwik.gz) \
--log-format-name ncsa_extended \
--recorders=4
/usr/share/webapps/piwik/console core:archive --url='http://piwik'
|
Script perso
| crontab
|
* * * * * /path/to/piwik-import-log.sh 2>&1 | mail -s "Piwik Import Log" -r cron@mon-domain.fr xxx@gmail.com
|
| piwik-import-log.sh
|
#!/bin/bash
set -e
set -o pipefail
set -u
log_file='/var/log/nginx/bananeatomic_access.log'
nb_of_lines_to_skip=0
if [[ -r .piwik-skip ]] ; then
nb_of_lines_to_skip=$(cat .piwik-skip)
fi
nb_of_lines_in_the_log_file=$(wc -l < $log_file)
# si il n'y a pas eu de nouvelles entrées dans le log on ne fait rien
if [[ $nb_of_lines_to_skip -eq $nb_of_lines_in_the_log_file ]] ; then
exit 0
# si il le log a été archivé
elif [[ $nb_of_lines_to_skip -gt $nb_of_lines_in_the_log_file ]] ; then
nb_of_lines_to_skip=0
fi
echo $nb_of_lines_in_the_log_file > .piwik-skip
python2 /srv/http/piwik/misc/log-analytics/import_logs.py \
--url=http://piwik.mon-domain.fr \
"$log_file" \
--recorders=4 \
--exclude-path="/load.php" \
--exclude-path="/api.php" \
--skip=$nb_of_lines_to_skip
# déclencher manuellement l'archivage / la création des rapports
su http -m -c "/usr/bin/php /srv/http/piwik/console core:archive --url=http://piwik.mon-domain.fr"
# « http » n'ayant pas le droit de se connecter à un shell, il faut utiliser l'option « -m »
|
GeoLocalisation
DBIP / GeoIP 2 (HTTP Server Module)
- install the GeoIP module for Nginx
 |
php-geoip is no more available after php version 7.4 |
- Enable the GeoIp2 plugin
- Download GeoIP database to /var/www/matomo/misc (DBIP ip-to-city-lite)
| /var/www/matomo/plugins/GeoIp2/config/config.php
|
'path.geoip2' => DI\string('{path.root}/misc/'),
|
|
It seems the database name dbip-city-lite-2023-11.mmdb is not recognized, rename it to DBIP-City-Lite.mmdb |
GeoLocalisation OLD
|
|
pacman -S php-geoip geoip-database-extra
# les paquets geoip et geoip-database seront installés comme dépendances
# geoip-database contient GeoIP.dat
# geoip-database-extra contient GeoIPCity.dat
|
| /etc/php/conf.d/geoip.ini
|
extension=geoip.so
; redéfinir le dossier où se trouve les fichiers dat
geoip.custom_directory=/usr/share/GeoIP/
|
|
Tester: Piwik → Administration → Geolocalisation |
|
PECL est plus rapide que la version GeoIP PHP |
|
|
pacman -S autoconf
pecl install geoip
|
|
|
# Mettre à jour la base de données :
/usr/bin/php /srv/http/piwik/console usercountry:attribute 2015-01-01,2016-01-01
# Reconstruire les rapports
/usr/bin/php /srv/http/piwik/console core:archive --force-all-websites --force-all-periods=315576000 --force-date-last-n=1000 --url=http://piwik
|
|
# drop all tables starting with archive_ except archive_invalidations
select concat('drop table ', group_concat(table_name), ';')
from information_schema.tables where table_schema = 'matomo' and table_name like 'archive_n%' or 'archive_b%';
delete from log_visit where idsite = 1;
delete from log_link_visit_action where idsite = 1;
delete from log_conversion where idsite = 1;
delete from log_conversion_item where idsite = 1;
|
|
|
wget https://builds.matomo.org/matomo.zip
unzip matomo.zip
rm -f matomo.zip How\ to\ install\ Matomo.html
sudo mv matomo /var/www
sudo chown -R root:root /var/www/matomo
sudo chown www-data:www-data /var/www/matomo/tmp
sudo chown www-data:www-data /var/www/matomo/matomo.js # the js tracker has to be writable
sudo chown -R www-data:www-data /var/www/matomo/config # only for the installation
# after the installation, reset access rights to the config folder
sudo chown -R root:root /var/www/matomo/config
# but allow to modifiy the config.ini.php file
sudo chown www-data:www-data /var/www/matomo/config/config.ini.php
|
| /etc/nginx/sites-available/matomo.conf
|
server {
server_name matomo.localserver;
listen 80;
# Redirect all HTTP requests to HTTPS with a 301 Moved Permanently response.
location / {
return 301 https://$host$request_uri;
}
}
server {
server_name matomo.localserver;
listen 443 ssl http2;
root /var/www/matomo;
index index.php;
ssl_certificate /etc/letsencrypt/live/matomo.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/matomo.example.com/privkey.pem;
add_header Referrer-Policy origin always; # make sure outgoing links don't show the URL to the Matomo instance
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
allow 192.168.0.0/24;
deny all;
access_log /var/log/nginx/matomo-access.log;
error_log /var/log/nginx/matomo-error.log;
## only allow accessing the following php files
location ~ ^/(index|matomo|piwik|js/index|plugins/HeatmapSessionRecording/configs)\.php$ {
include snippets/fastcgi-php.conf;
fastcgi_param HTTP_PROXY ""; # prohibit httpoxy: https://httpoxy.org/
fastcgi_pass unix:/var/run/php/php-fpm.sock;
}
## deny access to all other .php files
location ~* ^.+\.php$ {
deny all;
return 403;
}
## serve all other files normally
location / {
try_files $uri $uri/ =404;
}
## disable all access to the following directories
location ~ ^/(config|tmp|core|lang) {
deny all;
return 403; # replace with 404 to not show these directories exist
}
location ~ /\.ht {
deny all;
return 403;
}
location ~ js/container_.*_preview\.js$ {
expires off;
add_header Cache-Control 'private, no-cache, no-store';
}
location ~ \.(gif|ico|jpg|png|svg|js|css|htm|html|mp3|mp4|wav|ogg|avi|ttf|eot|woff|woff2)$ {
allow all;
## Cache images,CSS,JS and webfonts for an hour
## Increasing the duration may improve the load-time, but may cause old files to show after an Matomo upgrade
expires 1h;
add_header Pragma public;
add_header Cache-Control "public";
}
location ~ ^/(libs|vendor|plugins|misc|node_modules) {
deny all;
return 403;
}
## properly display textfiles in root directory
location ~/(.*\.md|LEGALNOTICE|LICENSE) {
default_type text/plain;
}
}
|
Nginx et uwsgi
| /etc/nginx/nginx.conf
|
server {
listen 80;
server_name piwik;
root /usr/share/webapps/piwik;
index index.php;
location ~ \.php$ {
include uwsgi_params;
uwsgi_modifier1 14;
uwsgi_pass unix:/run/uwsgi/piwik.sock;
}
}
|
| /etc/uwsgi/piwik.ini
|
[uwsgi]
; log
logger = file:/var/log/uwsgi/%n.log
; maximum number of worker processes
processes = 40
; start with only 4 and spawn the others on demand, maintaining a minimal pool of 4 processes.
cheaper = 1
; the user and group id of the process once it’s started
uid = http
gid = http
master = true
socket = /run/uwsgi/%n.sock
chdir = /usr/share/webapps/%n
; clear environment on exit
vacuum = true
; php
plugins = php
php-docroot = /usr/share/webapps/%n
php-index = index.php
php-allowed-ext = .php
; For some mysterious reason, the opcode cache is disabled in the embed SAPI.
; You can bypass the problem by telling the PHP engine that is running under the apache SAPI
php-sapi-name = apache
; extensions
php-set = extension=pdo_mysql.so
php-set = extension=iconv.so
php-set = extension=geoip.so
; php config
php-set = open_basedir=/usr/share/webapps/piwik/:/tmp/:/usr/share/GeoIP/
php-set = iconv.input_encoding=ISO-8859-1
php-set = iconv.internal_encoding=ISO-8859-1
php-set = iconv.output_encoding=ISO-8859-1
; To give Piwik enough memory to process your web analytics reports, increase the memory limit to 512M
php-set = memory_limit=512M
; define the GeoIP directory
php-set = geoip.custom_directory=/usr/share/GeoIP/
|
Database Setup
|
|
create database matomo default character set utf8mb4 collate utf8mb4_unicode_ci;
create user matomo@localhost identified by 'password';
grant all on matomo.* to matomo@localhost;
flush privileges;
|
| /etc/mysql/mariadb.conf.d/50-server.cnf
|
max_allowed_packet = 64MB
|
PHP configuration
| /etc/php/8.2/fpm/pool.d/matomo.conf
|
listen = /run/php/php8.2-fpm-matomo.sock
pm = ondemand
pm.max_children = 2
; Enable LOAD DATA INFILE
php_value[mysqli.allow_local_infile] = On
php_value[mysqli.local_infile_directory] = /var/www/matomo/tmp/assets/
|
| /etc/nginx/sites-available/matomo.conf
|
fastcgi_pass unix:/var/run/php/php8.2-fpm-matomo.sock;
|
Plugins
- be sure to have the write rights on /var/www/matomo/plugins
- Marketplace on the left menu → Browse
Useful plugins and themes
| Name
|
Description
|
| Security Info |
provides security information about your PHP environment and offers suggestions based on PhpSecInfo
|
| Tracking Spam Prevention |
prevent spammers and bots from making your data inaccurate
|
| Provider |
reports the Internet Service Provider of the visitors
|
| Log Viewer |
display log messages logged by Matomo
|
| Bot Tracker |
detection of bots & spiders and count their visits without tracking them in the visitor-log
|
Erreurs
Can't get stat of '/usr/share/webapps/piwik/tmp/assets/option-xx.csv' (Errcode: 13 "Permission denied")
Administration → Diagnostic → System Check → Database abilities: LOAD DATA INFILE
|
|
# le dossier assets n'est pas accessible par mysql
ll /usr/share/webapps/piwik/tmp/assets
# drwxr-x---
chmod o+rx /usr/share/webapps/piwik/tmp/assets
ll /usr/share/webapps/piwik/tmp/assets
# drwxr-xr-x
|
The mysql driver is not currently installed
Décommentez la ligne suivante
| /etc/php/php.ini
|
extension=pdo_mysql.so
|
|
|
# redemarrez le serveur
sudo systemctl restart httpd
|
SQLSTATE[HY000] [2002] Connection refused
Mauvais hôte, identifiant ou mot de passe.
Essayez localhost au lieu de 127.0.0.1
You need to configure and rebuild PHP with "iconv" support enabled, --with-iconv
Décommentez les lignes suivantes
| /etc/php/php.ini
|
extension=iconv.so
[iconv]
iconv.input_encoding = ISO-8859-1
iconv.internal_encoding = ISO-8859-1
iconv.output_encoding = ISO-8859-1
|
|
|
# redemarrez le serveur
sudo systemctl restart httpd
|
