[
https://issues.apache.org/jira/browse/CASSANDRA-6107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13787971#comment-13787971
]
Sylvain Lebresne commented on CASSANDRA-6107:
---------------------------------------------
CASSANDRA-5981 is indeed about limiting the size at the protocol level. However
it's a global frame limitation. In particular this is the hard limit for
queries with their values and for that reason the current hard-coded limit is
relatively high (256MB). And we can bikeshed on the exact default to user and
CASSANDRA-5981 will probably allow the user to play with that limit, but in any
case, it will definitively have to be higher than the 1MB. The other detail is
that the limit done by CASSANDRA-5981 is on the sent bytes, not the in-memory
size of the query, but that probably don't matter much.
Anyway, provided that a prepared statement doesn't include values, it wouldn't
be absurd to have a specific, lower limit on their size. Though my own
preference would be to just leave it to a global limit on the
preparedStatements cache map (but it could make sense to reject statements that
blow up the entire limit on their own, so as to make sure to respect it). Too
many hard-coded limitations make me nervous.
> CQL3 Batch statement memory leak
> --------------------------------
>
> Key: CASSANDRA-6107
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6107
> Project: Cassandra
> Issue Type: Bug
> Components: API, Core
> Environment: - CASS version: 1.2.8 or 2.0.1, same issue seen in both
> - Running on OSX MacbookPro
> - Sun JVM 1.7
> - Single local cassandra node
> - both CMS and G1 GC used
> - we are using the cass-JDBC driver to submit our batches
> Reporter: Constance Eustace
> Assignee: Lyuben Todorov
> Priority: Minor
> Fix For: 1.2.11
>
> Attachments: 6107.patch, 6107_v2.patch, 6107_v3.patch, 6107-v4.txt,
> Screen Shot 2013-10-03 at 17.59.37.png
>
>
> We are doing large volume insert/update tests on a CASS via CQL3.
> Using 4GB heap, after roughly 750,000 updates create/update 75,000 row keys,
> we run out of heap, and it never dissipates, and we begin getting this
> infamous error which many people seem to be encountering:
> WARN [ScheduledTasks:1] 2013-09-26 16:17:10,752 GCInspector.java (line 142)
> Heap is 0.9383457210434385 full. You may need to reduce memtable and/or
> cache sizes. Cassandra will now flush up to the two largest memtables to
> free up memory. Adjust flush_largest_memtables_at threshold in
> cassandra.yaml if you don't want Cassandra to do this automatically
> INFO [ScheduledTasks:1] 2013-09-26 16:17:10,753 StorageService.java (line
> 3614) Unable to reduce heap usage since there are no dirty column families
> 8 and 12 GB heaps appear to delay the problem by roughly proportionate
> amounts of 75,000 - 100,000 rowkeys per 4GB. Each run of 50,000 row key
> creations sees the heap grow and never shrink again.
> We have attempted to no effect:
> - removing all secondary indexes to see if that alleviates overuse of bloom
> filters
> - adjusted parameters for compaction throughput
> - adjusted memtable flush thresholds and other parameters
> By examining heapdumps, it seems apparent that the problem is perpetual
> retention of CQL3 BATCH statements. We have even tried dropping the keyspaces
> after the updates and the CQL3 statement are still visible in the heapdump,
> and after many many many CMS GC runs. G1 also showed this issue.
> The 750,000 statements are broken into batches of roughly 200 statements.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
