Hello,
I've been following the cluster setup guide to setup a 3-nodes PacketFence
cluster, and everything has been running smoothly until the database syncing
step.
Starting the packetfence-mariadb service on the main node with the
--force-new-cluster is working perfectly, but starting it on the other nodes
doesn't work.
This is how I proceed : only one of the other two nodes is powered on. I sync
the files, and then I try to sync the database. Sadly, I see that the second
node joins the first one, and then they immediately desynchronize eachother and
it keeps looping.
Furthermore, the logs are full of messages indicating that some files are not
present nor accessible while trying to start the service on both the second and
third nodes.
I tried clustering using both the zen OVA file and the PacketFence iso
availiable on the website, and it keeps getting the same errors.
Here are the mysql logs for the main and secondary nodes :
Many thanks in advance for your answers,
Best regards,
Gabriel MORIN
1st node :
Jul 17 12:04:06 node_1 mysqld[80239]: 2023-07-17 12:04:06 2 [Note] WSREP:
================================================
Jul 17 12:04:06 node_1 mysqld[80239]: View:
Jul 17 12:04:06 node_1 mysqld[80239]: id:
e7b51203-2186-11ee-9758-a70beafa5e98:54094
Jul 17 12:04:06 node_1 mysqld[80239]: status: primary
Jul 17 12:04:06 node_1 mysqld[80239]: protocol_version: 4
Jul 17 12:04:06 node_1 mysqld[80239]: capabilities: MULTI-MASTER,
CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ,
INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
Jul 17 12:04:06 node_1 mysqld[80239]: final: no
Jul 17 12:04:06 node_1 mysqld[80239]: own_index: 0
Jul 17 12:04:06 node_1 mysqld[80239]: members(2):
Jul 17 12:04:06 node_1 mysqld[80239]: 0:
33705641-2473-11ee-bbe6-f7033c9996f6, node_1
Jul 17 12:04:06 node_1 mysqld[80239]: 1:
434a4440-2489-11ee-906a-7e678365dfea, node_2
Jul 17 12:04:06 node_1 mysqld[80239]:
=================================================
Jul 17 12:04:06 node_1 mysqld[80239]: 2023-07-17 12:04:06 2 [Note] WSREP:
wsrep_notify_cmd is not defined, skipping notification.
Jul 17 12:04:06 node_1 mysqld[80239]: 2023-07-17 12:04:06 2 [Note] WSREP:
Lowest cert index boundary for CC from group: 54086
Jul 17 12:04:06 node_1 mysqld[80239]: 2023-07-17 12:04:06 2 [Note] WSREP: Min
available from gcache for CC from group: 27378
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP:
Deferred close timer started for socket with remote endpoint:
tcp://192.168.100.48:34664
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP:
forgetting 434a4440-906a (tcp://192.168.100.48:4567)
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP: Node
33705641-bbe6 state prim
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP:
view(view_id(PRIM,33705641-bbe6,173) memb {
Jul 17 12:04:07 node_1 mysqld[80239]: 33705641-bbe6,0
Jul 17 12:04:07 node_1 mysqld[80239]: } joined {
Jul 17 12:04:07 node_1 mysqld[80239]: } left {
Jul 17 12:04:07 node_1 mysqld[80239]: } partitioned {
Jul 17 12:04:07 node_1 mysqld[80239]: 434a4440-906a,0
Jul 17 12:04:07 node_1 mysqld[80239]: })
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP: save
pc into disk
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP:
forgetting 434a4440-906a (tcp://192.168.100.48:4567)
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP:
Deferred close timer handle_wait Operation aborted. for 0x55a7b6bb0c00
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP:
Deferred close timer destruct
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP: New
COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP:
STATE_EXCHANGE: sent state UUID: 448a2f9b-2489-11ee-9585-3ffcdf8d99ab
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP: STATE
EXCHANGE: sent state msg: 448a2f9b-2489-11ee-9585-3ffcdf8d99ab
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP: STATE
EXCHANGE: got state msg: 448a2f9b-2489-11ee-9585-3ffcdf8d99ab from 0 (node_1)
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP:
Quorum results:
Jul 17 12:04:07 node_1 mysqld[80239]: version = 6,
Jul 17 12:04:07 node_1 mysqld[80239]: component = PRIMARY,
Jul 17 12:04:07 node_1 mysqld[80239]: conf_id = 172,
Jul 17 12:04:07 node_1 mysqld[80239]: members = 1/1 (joined/total),
Jul 17 12:04:07 node_1 mysqld[80239]: act_id = 54095,
Jul 17 12:04:07 node_1 mysqld[80239]: last_appl. = 54085,
Jul 17 12:04:07 node_1 mysqld[80239]: protocols = 2/10/4
(gcs/repl/appl),
Jul 17 12:04:07 node_1 mysqld[80239]: vote policy= 0,
Jul 17 12:04:07 node_1 mysqld[80239]: group UUID =
e7b51203-2186-11ee-9758-a70beafa5e98
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP:
Flow-control interval: [16, 16]
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 2 [Note] WSREP:
####### processing CC 54096, local, ordered
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 2 [Note] WSREP:
####### My UUID: 33705641-2473-11ee-bbe6-f7033c9996f6
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 2 [Note] WSREP:
Skipping cert index reset
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 2 [Note] WSREP: REPL
Protocols: 10 (5)
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 2 [Note] WSREP:
####### Adjusting cert position: 54095 -> 54096
Jul 17 12:04:07 node_1 mysqld[80239]: 2023-07-17 12:04:07 0 [Note] WSREP:
Service thread queue flushed.
==> Then, it redoes the sequence but only finds the node_1 server, and it loops
again, and it finds both nodes, indefinitely....
2nd node :
Part. 1 :
Jul 17 07:48:05 node_2 mysqld[3720374]: 2023-07-17 9:48:05 0 [Note] WSREP:
Start replication
Jul 17 07:48:05 node_2 mysqld[3720374]: 2023-07-17 9:48:05 0 [Note] WSREP:
Connecting with bootstrap option: 0
Jul 17 07:48:05 node_2 mysqld[3720374]: 2023-07-17 9:48:05 0 [Note] WSREP:
Setting GCS initial position to 00000000-0000-0000-0000-000000000000:-1
Jul 17 07:48:05 node_2 mysqld[3720374]: 2023-07-17 9:48:05 0 [Note] WSREP:
protonet asio version 0
Jul 17 07:48:05 node_2 mysqld[3720374]: 2023-07-17 9:48:05 0 [Note] WSREP:
Using CRC-32C for message checksums.
Jul 17 07:48:05 node_2 mysqld[3720374]: 2023-07-17 9:48:05 0 [Note] WSREP:
backend: asio
Jul 17 07:48:05 node_2 mysqld[3720374]: 2023-07-17 9:48:05 0 [Note] WSREP:
gcomm thread scheduling priority set to other:0
Jul 17 07:48:05 node_2 mysqld[3720374]: 2023-07-17 9:48:05 0 [Warning] WSREP:
access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory)
Jul 17 07:48:05 node_2 mysqld[3720374]: 2023-07-17 9:48:05 0 [Note] WSREP:
restore pc from disk failed
Jul 17 07:48:05 node_2 mysqld[3720374]: 2023-07-17 9:48:05 0 [Note] WSREP:
GMCast version 0
Jul 17 07:48:05 node_2 mysqld[3720374]: 2023-07-17 9:48:05 0 [Note] WSREP:
(43c1434f-a41c, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
Jul 17 07:48:05 node_2 mysqld[3720374]: 2023-07-17 9:48:05 0 [Note] WSREP:
(43c1434f-a41c, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
part. 2 :
3719997]: 2023-07-17 9:47:59 0 [Note] InnoDB: 128 rollback segments are active.
Jul 17 07:47:59 node_2 mysqld[3719997]: 2023-07-17 9:47:59 0 [Note] InnoDB:
Creating shared tablespace for temporary tables
Jul 17 07:47:59 node_2 mysqld[3719997]: 2023-07-17 9:47:59 0 [Note] InnoDB:
Setting file '/var/lib/mysql/ibtmp1' size to 12 MB. Physically writing the file
full; Please wait ...
Jul 17 07:47:59 node_2 mysqld[3719997]: 2023-07-17 9:47:59 0 [Note] InnoDB:
File '/var/lib/mysql/ibtmp1' size is now 12 MB.
Jul 17 07:47:59 node_2 mysqld[3719997]: 2023-07-17 9:47:59 0 [Note] InnoDB:
10.5.19 started; log sequence number 38160; transaction id 8
Jul 17 07:47:59 node_2 mysqld[3719997]: 2023-07-17 9:47:59 0 [Warning] InnoDB:
Skipping buffer pool dump/restore during wsrep recovery.
Jul 17 07:47:59 node_2 mysqld[3719997]: 2023-07-17 9:47:59 0 [Note] Plugin
'FEEDBACK' is disabled.
Jul 17 07:47:59 node_2 mysqld[3719997]: 2023-07-17 9:47:59 0 [ERROR] Could not
open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some plugins may
be not loaded
Jul 17 07:47:59 node_2 mysqld[3719997]: 2023-07-17 9:47:59 0 [Warning]
'thread-concurrency' was removed. It does nothing now and exists only for
compatibility with old my.cnf files.
Jul 17 07:47:59 node_2 mysqld[3719997]: 2023-07-17 9:47:59 1 [Warning] Failed
to load slave replication state from table mysql.gtid_slave_pos: 1017: Can't
find file: './mysql/' (errno: 2 "No such file or directory")
Jul 17 07:47:59 node_2 mysqld[3719997]: 2023-07-17 9:47:59 0 [ERROR] Can't
open and lock privilege tables: Table 'mysql.servers' doesn't exist
Jul 17 07:47:59 node_2 mysqld[3719997]: 2023-07-17 9:47:59 0 [Note] Server
socket created on IP: '192.168.100.48'.
Jul 17 07:47:59 node_2 mysqld[3719997]: 2023-07-17 9:47:59 0 [Note] WSREP:
Recovered position: 00000000-0000-0000-0000-000000000000:-1,0-0-0
Jul 17 07:47:59 node_2 mysqld_safe[3720013]: mysqld from pid file
/var/lib/mysql/node_2.pid ended
Jul 17 07:48:04 node_2 pf-mariadb[3720078]: Starting MySQL with command:
mysqld_safe --defaults-file=/usr/local/pf/var/conf/mariadb.conf
Jul 17 07:48:05 node_2 pf-mariadb[3720078]: 230717 09:48:05 mysqld_safe Logging
to syslog.
Jul 17 07:48:05 node_2 pf-mariadb[3720078]: 230717 09:48:05 mysqld_safe
Starting mariadbd daemon with databases from /var/lib/mysql
Jul 17 07:48:05 node_2 mysqld_safe[3720336]: Starting mariadbd daemon with
databases from /var/lib/mysql
Jul 17 07:48:05 node_2 mysqld_safe[3720345]: WSREP: Running position recovery
with --disable-log-error --pid-file='/var/lib/mysql/node_2-recover.pid'
Confidentialité : Ce message est exclusivement destiné à la personne titulaire
de l'adresse électronique à laquelle il a été envoyé. Il contient des
informations de nature confidentielle et couvertes par le secret professionnel.
Si vous n'en n'êtes pas le destinataire, vous êtes avisé que toute divulgation,
copie ou distribution de cette communication ou des éléments qu'elle contient,
est strictement interdite. Si vous recevez ce message par erreur, veuillez nous
le faire savoir par retour de courrier électronique, à l'adresse mentionnée
ci-dessus, puis le détruire.
Confidentiality: The information contained in this E-mail is only intended for
the use of the individual or above named holder. It contains privileged
information covered by the professional confidentiality. If you are not the
intended recipient, you are hereby notified that any disclosure, forwarding or
copy of this message or its content is strictly forbidden. Hence, if you have
received this message in error, please notify us immediately by returning the
original message to us at the above mentioned E-mail address, and then delete
it.
_______________________________________________
PacketFence-users mailing list
PacketFence-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/packetfence-users
