Jai Bheemsen Rao Dhanwada created CASSANDRA-15850:
-----------------------------------------------------
Summary: Delay between Gossip settle and CQL port opening during
the startup
Key: CASSANDRA-15850
URL: https://issues.apache.org/jira/browse/CASSANDRA-15850
Project: Cassandra
Issue Type: Bug
Reporter: Jai Bheemsen Rao Dhanwada
Hello,
When I am bootstrapping/restarting a Cassandra Node, there is a delay between
gossip settle and CQL port opening. Can someone please explain me where this
delay is configured and can this be changed? I don't see any information in the
logs
In my case if you see there is a ~3 minutes delay and this increases if I
increase the #of tables and #of nodes and DC.
{code:java}
INFO [main] 2020-05-31 23:51:07,554 Gossiper.java:1692 - Waiting for gossip to
settle...
INFO [main] 2020-05-31 23:51:15,555 Gossiper.java:1723 - No gossip backlog;
proceeding
INFO [main] 2020-05-31 23:54:06,867 NativeTransportService.java:70 - Netty
using native Epoll event loop
INFO [main] 2020-05-31 23:54:06,913 Server.java:155 - Using Netty Version:
[netty-buffer=netty-buffer-4.0.44.Final.452812a,
netty-codec=netty-codec-4.0.44.Final.452812a,
netty-codec-haproxy=netty-codec-haproxy-4.0.44.Final.452812a,
netty-codec-http=netty-codec-http-4.0.44.Final.452812a,
netty-codec-socks=netty-codec-socks-4.0.44.Final.452812a,
netty-common=netty-common-4.0.44.Final.452812a,
netty-handler=netty-handler-4.0.44.Final.452812a,
netty-tcnative=netty-tcnative-1.1.33.Fork26.142ecbb,
netty-transport=netty-transport-4.0.44.Final.452812a,
netty-transport-native-epoll=netty-transport-native-epoll-4.0.44.Final.452812a,
netty-transport-rxtx=netty-transport-rxtx-4.0.44.Final.452812a,
netty-transport-sctp=netty-transport-sctp-4.0.44.Final.452812a,
netty-transport-udt=netty-transport-udt-4.0.44.Final.452812a]
INFO [main] 2020-05-31 23:54:06,913 Server.java:156 - Starting listening for
CQL clients on /x.x.x.x:9042 (encrypted)...
{code}
Also during this 3-10 minutes delay, I see
{noformat}
nodetool compactionstats
{noformat}
command is hung and never respond, until the CQL port is up and running.
Can someone please help me understand the delay here?
Cassandra Version: 3.11.3
The issue can be easily reproducible with around 300 Tables and 100 nodes in a
cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
