[
https://issues.apache.org/jira/browse/CASSANDRA-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cameron Zemek updated CASSANDRA-18546:
--------------------------------------
Description:
In going through trying to understand the Gossiper code I come across:
{code:java}
/**
* The gossip digest is built based on randomization
* rather than just looping through the collection of live endpoints.
*
* @param gDigests list of Gossip Digests.
*/
private void makeRandomGossipDigest(List<GossipDigest> gDigests) {code}
But I couldn't see what purpose randomization had. In fact in 3.11 it will call:
{code:java}
doSort(gDigestList); {code}
On the receiving end, negating the purpose of the randomization.
In discussion with [~stefan.miklosovic] he found this ticket CASSANDRA-14174
So it seems to me this randomization may have been to allow for limited sizes
of SYN messages. But this feature doesn't exist and as such by randomizing it
is:
* creating more garbage
* using more CPU (sure its mostly trival; see next point)
* more time spent on unnecessary functionality on the *single threaded* gossip
stage.
* complicating the code and making it more difficult to understand
In fact there is a bug in the implementation:
{code:java}
int generation = 0;
int maxVersion = 0; // local epstate will be part of
endpointStateMap
List<InetAddress> endpoints = new
ArrayList<InetAddress>(endpointStateMap.keySet());
Collections.shuffle(endpoints, random);
for (InetAddress endpoint : endpoints)
{
epState = endpointStateMap.get(endpoint);
if (epState != null)
{
generation = epState.getHeartBeatState().getGeneration();
maxVersion = getMaxEndpointStateVersion(epState);
}
gDigests.add(new GossipDigest(endpoint, generation, maxVersion));
} {code}
If epState is null and we already had a non-null epState, then the next digest
will use the generation and maxVersion of the previous iterated epState.
Here is change to remove this randomization and fix the above bug,
[https://github.com/apache/cassandra/pull/2357/files#diff-99267a2170b04fd7dd24d6c6bf2ba1fc26d6dc896cd74f8c5bd56c476e2540e4]
was:
In going through trying to understand the Gossiper code I come across:
{code:java}
/**
* The gossip digest is built based on randomization
* rather than just looping through the collection of live endpoints.
*
* @param gDigests list of Gossip Digests.
*/
private void makeRandomGossipDigest(List<GossipDigest> gDigests) {code}
But I couldn't see what purpose randomization had. In fact in 3.11 it will call:
{code:java}
doSort(gDigestList); {code}
On the receiving end, negating the purpose of the randomization.
In discussion with [~stefan.miklosovic] he found this ticket CASSANDRA-14174
So it seems to me this randomization may have been to allow for limited sizes
of SYN messages. But this feature doesn't exist and as such by randomizing it
is:
* creating more garbage
* using more CPU (sure its mostly trival; see next point)
* more time spent on unnecessary functionality on the *single threaded* gossip
stage.
* complicating the code and making it more difficult to understand
In fact there is a bug in the implementation:
{code:java}
int generation = 0;
int maxVersion = 0; // local epstate will be part of
endpointStateMap
List<InetAddress> endpoints = new
ArrayList<InetAddress>(endpointStateMap.keySet());
Collections.shuffle(endpoints, random);
for (InetAddress endpoint : endpoints)
{
epState = endpointStateMap.get(endpoint);
if (epState != null)
{
generation = epState.getHeartBeatState().getGeneration();
maxVersion = getMaxEndpointStateVersion(epState);
}
gDigests.add(new GossipDigest(endpoint, generation, maxVersion));
} {code}
If epState is null and we already had a non-null epState, then the next digest
will use the generation and maxVersion of the previous iterated epState.
Here is change to remove this randomization and fix the above bug,
https://github.com/apache/cassandra/pull/2357/files#diff-99267a2170b04fd7dd24d6c6bf2ba1fc26d6dc896cd74f8c5bd56c476e2540e4
> Remove Gossiper#makeRandomGossipDigest
> --------------------------------------
>
> Key: CASSANDRA-18546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18546
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Cameron Zemek
> Priority: Normal
>
> In going through trying to understand the Gossiper code I come across:
> {code:java}
> /**
> * The gossip digest is built based on randomization
> * rather than just looping through the collection of live endpoints.
> *
> * @param gDigests list of Gossip Digests.
> */
> private void makeRandomGossipDigest(List<GossipDigest> gDigests) {code}
> But I couldn't see what purpose randomization had. In fact in 3.11 it will
> call:
> {code:java}
> doSort(gDigestList); {code}
> On the receiving end, negating the purpose of the randomization.
>
> In discussion with [~stefan.miklosovic] he found this ticket CASSANDRA-14174
> So it seems to me this randomization may have been to allow for limited sizes
> of SYN messages. But this feature doesn't exist and as such by randomizing it
> is:
> * creating more garbage
> * using more CPU (sure its mostly trival; see next point)
> * more time spent on unnecessary functionality on the *single threaded*
> gossip stage.
> * complicating the code and making it more difficult to understand
> In fact there is a bug in the implementation:
> {code:java}
> int generation = 0;
> int maxVersion = 0; // local epstate will be part of
> endpointStateMap
> List<InetAddress> endpoints = new
> ArrayList<InetAddress>(endpointStateMap.keySet());
> Collections.shuffle(endpoints, random);
> for (InetAddress endpoint : endpoints)
> {
> epState = endpointStateMap.get(endpoint);
> if (epState != null)
> {
> generation = epState.getHeartBeatState().getGeneration();
> maxVersion = getMaxEndpointStateVersion(epState);
> }
> gDigests.add(new GossipDigest(endpoint, generation, maxVersion));
> } {code}
> If epState is null and we already had a non-null epState, then the next
> digest will use the generation and maxVersion of the previous iterated
> epState.
>
> Here is change to remove this randomization and fix the above bug,
> [https://github.com/apache/cassandra/pull/2357/files#diff-99267a2170b04fd7dd24d6c6bf2ba1fc26d6dc896cd74f8c5bd56c476e2540e4]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
