Cassandra 3.0.9 is a very old version, many bugs have been discovered
and fixed since then. You really should consider upgrading to the latest
version (3.0.26 / 3.11 / 4.0).
Write failures (such as timeout) can generate hints even when all nodes
are up. You should check the logs and metrics to make sure writes were
not failing.
Tombstone won't directly cause nodes creating hints, but the side effect
of too many tombstones, such as long GC pauses, may cause it.
I'm not sure where does the "Unknown column during serialization" come
from, but if this is on the write path, it may cause hints generation.
You may need to dig into the source code to confirm the source of this
message.
On 16/02/2022 19:57, Inquistive allen wrote:
Hello Team,
I'm using Cassandra 3.0.9.
Recently I have observed that, all the nodes in my cluster are
generated hints.
All the nodes are up but I do see hints getting generated continuously.
In the logs
I see tombstone warnings, but frequency is less
I also see frequent message
Unknown column during serialization.
In the nodetool tpstats o/p i do see hintsdispatcher has actiive
thread as 1 and pending as 42.
The cluster is spread over multiple datacenters as well.
Kindly suggest what may be inferred from above.
Thanks
