In practice there are not many good reasons to use that many keyspaces and tables. If the use case is multi tenancy then you’re almost always better off just using a combination of version tables and tenantId to give you flexibility as well as separation of client data. If you have that many data types then it maybe worth considering a blog or text value for the data and then a data type column so you can serialize and deserialize on the client.
> On Sep 22, 2015, at 3:09 AM, horschi <hors...@gmail.com> wrote: > > Hi Joseph, > > I think 2000 keyspaces might be just too much. Fewer keyspaces (and CFs) will > probably work much better. > > kind regards, > Christian > > > On Tue, Sep 22, 2015 at 9:29 AM, joseph gao <gaojf.bok...@gmail.com > <mailto:gaojf.bok...@gmail.com>> wrote: > Hi, anybody could help me? > > 2015-09-21 0:47 GMT+08:00 joseph gao <gaojf.bok...@gmail.com > <mailto:gaojf.bok...@gmail.com>>: > ps : that's the code in java drive , in MetaData.TokenMap.build: > for (KeyspaceMetadata keyspace : keyspaces) > { > ReplicationStrategy strategy = keyspace.replicationStrategy(); > Map<Token, Set<Host>> ksTokens = (strategy == null) > ? makeNonReplicatedMap(tokenToPrimary) > : strategy.computeTokenToReplicaMap(tokenToPrimary, ring); > > tokenToHosts.put(keyspace.getName(), ksTokens); > tokenToPrimary is all same, ring is all same, and if strategy is all same , > strategy.computeTokenToReplicaMap would return 'same' map but different > object( cause every calling returns a new HashMap > > 2015-09-21 0:22 GMT+08:00 joseph gao <gaojf.bok...@gmail.com > <mailto:gaojf.bok...@gmail.com>>: > cassandra: 2.1.7 > java driver: datastax java driver 2.1.6 > > Here is the problem: > My application uses 2000+ keyspaces, and will dynamically create keyspaces > and tables. And then in java client, the Metadata.tokenMap.tokenToHost would > use about 1g memory. so this will cause a lot of full gc. > As I see, the key of the tokenToHost is keyspace, and the value is a > tokenId_to_replicateNodes map. > > When I try to solve this problem, I find something not sure: all keyspaces > have same 'tokenId_to_replicateNodes' map. > My replication strategy of all keyspaces is : simpleStrategy and > replicationFactor is 3 > > So would it be possible if keyspaces use same strategy, the value of > tokenToHost map use a same map. So it would extremely reduce the memory usage > > thanks a lot > > -- > ------ > Joseph Gao > PhoneNum:15210513582 > QQ: 409343351 > > > > -- > ------ > Joseph Gao > PhoneNum:15210513582 > QQ: 409343351 > > > > -- > ------ > Joseph Gao > PhoneNum:15210513582 > QQ: 409343351 >
