Hello Ludovic, Yes, I've installed the mariadb-backup tool on all the nodes.
I've played a bit around since last time, and I now have less errors, but still some. The databases are still not syncing... I thought it has something to do with the fact that I was trying to join the two nodes with the third one powered down, but event after bringing back the third node to life it doesn't work any better. It looks like Galera is trying to reach both nodes and it doesn't copy anything because a file is missing on the 2nd node, but while doing a mysqlcheck on the first node databases they seem to be okay (weird). Could it be something like wrong hostnames in the cluster.conf file that is causing theses issues ? Also, when I reload the configuration on the second and third nodes at step 3.5.2 of the clustering guide, I have a lot of errors that begin by "could not write namespace...", maybe is this related ? Here are the mysql logs for the main and secondary nodes. Many thanks in advance for your answers, Best regards, Gabriel MORIN Node_1 : Jul 25 13:46:46 node_1 mysqld[25863]: 2023-07-25 15:46:46 0 [ERROR] WSREP: Command did not run: wsrep_sst_mariabackup --role 'donor' --address '<ip_address>:4444/xtrabackup_sst//1' --local-port 3306 --socket '/var/lib/mysql/mysql.sock' --progress 0 --datadir '/var/lib/mysql/' --defaults-file '/usr/local/pf/var/conf/mariadb.conf' --gtid '13780657-2aed-11ee-9612-5362c158f4cb:848' --gtid-domain-id -1062706128 --binlog 'mariadb-bin' --mysqld-args --defaults-file=/usr/local/pf/var/conf/mariadb.conf --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --wsrep_on=ON --wsrep_provider=/usr/lib/galera/libgalera_smm.so --wsrep_on=ON --wsrep-new-cluster --pid-file=node_1.pid --socket=/var/lib/mysql/mysql.sock --port=3306 --wsrep_start_position=13780657-2aed-11ee-9612-5362c158f4cb:276,-1062706128-1-275 Jul 25 13:47:07 node_1 mysqld[25863]: 2023-07-25 15:47:07 0 [ERROR] WSREP: Failed to read from: wsrep_sst_mariabackup --role 'donor' --address '<ip_address>:4444/xtrabackup_sst//1' --local-port 3306 --socket '/var/lib/mysql/mysql.sock' --progress 0 --datadir '/var/lib/mysql/' --defaults-file '/usr/local/pf/var/conf/mariadb.conf' --gtid '13780657-2aed-11ee-9612-5362c158f4cb:853' --gtid-domain-id -1062706128 --binlog 'mariadb-bin' --mysqld-args --defaults-file=/usr/local/pf/var/conf/mariadb.conf --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --wsrep_on=ON --wsrep_provider=/usr/lib/galera/libgalera_smm.so --wsrep_on=ON --wsrep-new-cluster --pid-file=node_1.pid --socket=/var/lib/mysql/mysql.sock --port=3306 --wsrep_start_position=13780657-2aed-11ee-9612-5362c158f4cb:276,-1062706128-1-275 Jul 25 13:47:07 node_1 mysqld[25863]: 2023-07-25 15:47:07 0 [ERROR] WSREP: Process completed with error: wsrep_sst_mariabackup --role 'donor' --address '<ip_address>:4444/xtrabackup_sst//1' --local-port 3306 --socket '/var/lib/mysql/mysql.sock' --progress 0 --datadir '/var/lib/mysql/' --defaults-file '/usr/local/pf/var/conf/mariadb.conf' --gtid '13780657-2aed-11ee-9612-5362c158f4cb:853' --gtid-domain-id -1062706128 --binlog 'mariadb-bin' --mysqld-args --defaults-file=/usr/local/pf/var/conf/mariadb.conf --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --wsrep_on=ON --wsrep_provider=/usr/lib/galera/libgalera_smm.so --wsrep_on=ON --wsrep-new-cluster --pid-file=node_1.pid --socket=/var/lib/mysql/mysql.sock --port=3306 --wsrep_start_position=13780657-2aed-11ee-9612-5362c158f4cb:276,-1062706128-1-275: 22 (Invalid argument) Node_2: Jul 25 13:48:33 node_2 mysqld[95215]: 2023-07-25 15:48:33 0 [ERROR] WSREP: Process completed with error: wsrep_sst_mariabackup --role 'joiner' --address '<ip_address>' --datadir '/var/lib/mysql/' --defaults-file '/usr/local/pf/var/conf/mariadb.conf' --parent 95214 --progress 0 --binlog 'mariadb-bin' --mysqld-args --defaults-file=/usr/local/pf/var/conf/mariadb.conf --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --wsrep_on=ON --wsrep_provider=/usr/lib/galera/libgalera_smm.so --wsrep_on=ON --pid-file=node_2.pid --socket=/var/lib/mysql/mysql.sock --port=3306 --wsrep_start_position=00000000-0000-0000-0000-000000000000:-1,0-0-0: 2 (No such file or directory) Jul 25 13:48:33 node_2 mysqld[95215]: 2023-07-25 15:48:33 0 [ERROR] WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script. Jul 25 13:48:33 node_2 mysqld[95215]: 2023-07-25 15:48:33 2 [ERROR] WSREP: Application received wrong state: Jul 25 13:48:33 node_2 mysqld[95215]: 2023-07-25 15:48:33 2 [ERROR] WSREP: Application state transfer failed. This is unrecoverable condition, restart required. Jul 25 13:48:34 node_2 mysqld[95215]: 230725 15:48:34 [ERROR] mysqld got signal 11 ; Jul 25 13:48:34 node_2 mysqld[96185]: 2023-07-25 15:48:34 0 [ERROR] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some plugins may be not loaded Jul 25 13:48:34 node_2 mysqld[96185]: 2023-07-25 15:48:34 0 [ERROR] Can't open and lock privilege tables: Table 'mysql.servers' doesn't exist Jul 25 13:48:54 node_2 mysqld[96564]: 2023-07-25 15:48:54 0 [ERROR] WSREP: ./gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():1205: Will never receive state. Need to abort. Jul 25 13:48:54 node_2 mysqld[96564]: 2023-07-25 15:48:54 0 [ERROR] WSREP: Process completed with error: wsrep_sst_mariabackup --role 'joiner' --address '<ip_address>' --datadir '/var/lib/mysql/' --defaults-file '/usr/local/pf/var/conf/mariadb.conf' --parent 96563 --progress 0 --binlog 'mariadb-bin' --mysqld-args --defaults-file=/usr/local/pf/var/conf/mariadb.conf --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --wsrep_on=ON --wsrep_provider=/usr/lib/galera/libgalera_smm.so --wsrep_on=ON --pid-file=node_2.pid --socket=/var/lib/mysql/mysql.sock --port=3306 --wsrep_start_position=00000000-0000-0000-0000-000000000000:-1,0-0-0: 2 (No such file or directory) ________________________________ De : Zammit, Ludovic <luza...@akamai.com> Envoyé : jeudi 20 juillet 2023 15:00 À : packetfence-users@lists.sourceforge.net <packetfence-users@lists.sourceforge.net> Cc : Gabriel Morin <gabriel.mo...@sermeta.com> Objet : Re: [PacketFence-users] Issue while syncing the MariaDB databases at cluster creation ATTENTION: Cet e-mail provient d'une personne externe à Sermeta. Ne cliquez pas sur les liens ou n'ouvrez pas les pièces jointes si vous ne connaissez pas l'expéditeur et que vous n'êtes pas sûr du contenu. Confidentialité : Ce message est exclusivement destiné à la personne titulaire de l'adresse électronique à laquelle il a été envoyé. Il contient des informations de nature confidentielle et couvertes par le secret professionnel. Si vous n'en n'êtes pas le destinataire, vous êtes avisé que toute divulgation, copie ou distribution de cette communication ou des éléments qu'elle contient, est strictement interdite. Si vous recevez ce message par erreur, veuillez nous le faire savoir par retour de courrier électronique, à l'adresse mentionnée ci-dessus, puis le détruire. Confidentiality: The information contained in this E-mail is only intended for the use of the individual or above named holder. It contains privileged information covered by the professional confidentiality. If you are not the intended recipient, you are hereby notified that any disclosure, forwarding or copy of this message or its content is strictly forbidden. Hence, if you have received this message in error, please notify us immediately by returning the original message to us at the above mentioned E-mail address, and then delete it.
_______________________________________________ PacketFence-users mailing list PacketFence-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/packetfence-users