[
https://issues.apache.org/jira/browse/CASSANDRA-18845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766917#comment-17766917
]
Stefan Miklosovic commented on CASSANDRA-18845:
-----------------------------------------------
I noticed now this in your comment above:
_This is going to be very difficult todo. dtests setup clusters on loopback
addresses and waitToSettle code path has a guard against it if using a loopback
address. Also, the problems mostly become apparent with large clusters._
This is really true (1) Gossip.waitToSettle is called only in case it is not on
loopback. Since our dtests are all on loopback (right?) I do not think that
code was ever invoked during dtests so its revert was not necessary.
_If I redo the patch and remove the changes to ECHO and show those tests do not
have regression would this allow the ticket to move forward?_
I think that is reasonable, wdyt, [~brandon.williams]?
I think that what was unfortunate was that we mixed flood of echos solution /
change with waiting for at least one node to be up. I think that the waiting
for at least 1 node can go in and we will focus on the echos separately.
(1)
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L400-L401
(2)
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/dht/BootStrapper.java#L213-L214
(3)
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/dht/BootStrapper.java#L235-L236
(4)
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/MigrationCoordinator.java#L696-L697
> Waiting for gossip to settle on live endpoints
> ----------------------------------------------
>
> Key: CASSANDRA-18845
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18845
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Cameron Zemek
> Priority: Normal
> Attachments: delay.log, example.log,
> image-2023-09-14-11-16-23-020.png, test1.log, test2.log, test3.log
>
>
> This is a follow up to CASSANDRA-18543
> Although that ticket added ability to set cassandra.gossip_settle_min_wait_ms
> this is tedious and error prone. On a node just observed a 79 second gap
> between waiting for gossip and the first echo response to indicate a node is
> UP.
> The problem being that do not want to start Native Transport until gossip
> settles otherwise queries can fail consistency such as LOCAL_QUORUM as it
> thinks the replicas are still in DOWN state.
> Instead of having to set gossip_settle_min_wait_ms I am proposing that
> (outside single node cluster) wait for UP message from another node before
> considering gossip as settled. Eg.
> {code:java}
> if (currentSize == epSize && currentLive == liveSize && liveSize
> > 1)
> {
> logger.debug("Gossip looks settled.");
> numOkay++;
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
