The first thing to look is the logs, specifically, the
/var/log/cassandra/system.log file on each node.
5 seconds time drift is enough to cause Cassandra to fail. You should
ensure the time difference between Cassandra nodes is very low by ensure
time sync is working correctly, otherwise cross node timeout may happen,
and a node with time relatively slightly behind may think everything is
fine, but a node with time relatively slightly ahead will think the
other nodes are down.
On 08/08/2023 03:54, vishnu vanced wrote:
Hi All,
I am very new to Cassandra. I have a 5 nodes cluster setup in Centos
servers for our internal team testing, couple of days ago our network
team has asked us to stop 3 of the nodes let's say C1,C2,C3 for OS
patching activity. After the activity I started the nodes again but
now interestingly in C1 node it was showing as C2 node was down and in
C2 node it was showing C1 as down. But in remaining all three nodes
everything is UN.i have tried disabling gossip and enabling it.
Restarting all the nodes nothing changed. So I stopped this cluster
and tried to build freshly. But C1 and C2 only join the cluster if
other node is not present. So I first added C1 to the cluster and C2
only joins when I mention it as seed node. Now in C1 nodetool status
C2 is showing as '?N' and vice-versa. But in other nodes showing all
as 'UN'. I have checked connectivity between all the servers and
everything is fine. NTP in the three stopped servers differs by 5
secs, could that be the problem? But C3 node is not showing any issues.
Due to this while creating schemas and getting errors like schema
version mismatch repairs are failing. Can anyone give any solution as
to how this can be fixed? Thanks!
P.S are there any telegram/whatsapp groups for Cassandra?
Regards
Vishnu
