JosephGao created CASSANDRA-10402:
-------------------------------------
Summary: reduce the memory usage of Metadata.tokenMap.tokenToHost
Key: CASSANDRA-10402
URL: https://issues.apache.org/jira/browse/CASSANDRA-10402
Project: Cassandra
Issue Type: Improvement
Components: Drivers (now out of tree)
Reporter: JosephGao
Fix For: 2.1.x
My application uses 2000+ keyspaces, and will dynamically create keyspaces and
tables. And then in java client, the Metadata.tokenMap.tokenToHost would use
about 1g memory. so this will cause a lot of full gc.
As I see, the key of the tokenToHost is keyspace, and the value is a
tokenId_to_replicateNodes map.
When I try to solve this problem, I find something not sure: all keyspaces
have same 'tokenId_to_replicateNodes' map.
My replication strategy of all keyspaces is : simpleStrategy and
replicationFactor is 3
So would it be possible if keyspaces use same strategy, the value of
tokenToHost map use a same map. So it would extremely reduce the memory usage
ps： the following is effected code:
for (KeyspaceMetadata keyspace : keyspaces)
{
ReplicationStrategy strategy = keyspace.replicationStrategy();
Map<Token, Set<Host>> ksTokens = (strategy == null)
? makeNonReplicatedMap(tokenToPrimary)
: strategy.computeTokenToReplicaMap(tokenToPrimary, ring);
tokenToHosts.put(keyspace.getName(), ksTokens);
}
tokenToPrimary is all same, ring is all same, and if strategy is all same ,
strategy.computeTokenToReplicaMap would return 'same' map but different object(
cause every calling returns a new HashMap)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
