[
https://issues.apache.org/jira/browse/CASSANDRA-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12994124#comment-12994124
]
Eddie commented on CASSANDRA-2158:
----------------------------------
I think I have a usecase where larger memtables would help a lot. I have a
combination of fat columns that can update frequently, and I have lots of
memory (currently 96 gb). I know I could also handle this by putting more
boxes in the cluster, but I think I can get a lot more out of the boxes I have.
I am experimenting with breaking up my cf into multiple ones to get the same
effect as the bigger sstable. So far it seems to perform well, but feels hacky.
Even if you decide to have a hard limit on the memtable size, it should
probably fail loudly instead of generating hundreds of sstables. With my
understanding of the current defaults, any default install of cassandra with
more than 32 gb of memory will default to this state and will be hard for new
users to understand (32/2 -> 16 gig heap | 16/8 -> 2gb default CF memtable
throughput). I would much prefer the option than the hard limit though :).
> memtable_throughput_in_mb can not support sizes over 2.2 gigs because of an
> integer overflow.
> ---------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-2158
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2158
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Eddie
>
> If memtable_throughput_in_mb is set past 2.2 gigs, no errors are thrown.
> However, as soon as data starts being written it is almost immediately being
> flushed. Several hundred SSTables are created in minutes. I am almost
> positive that the problem is that when memtable_throughput_in_mb is being
> converted into bytes the result is stored in an integer, which is overflowing.
> From memtable.java:
> private final int THRESHOLD;
> private final int THRESHOLD_COUNT;
> ...
> this.THRESHOLD = cfs.getMemtableThroughputInMB() * 1024 * 1024;
> this.THRESHOLD_COUNT = (int) (cfs.getMemtableOperationsInMillions() * 1024 *
> 1024);
> NOTE:
> I also think currentThroughput also needs to be changed from an int to a
> long. I'm not sure if it is as simple as this or if this also is used in
> other places.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
