[
https://issues.apache.org/jira/browse/CASSANDRA-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12848031#action_12848031
]
Chris Goffinet commented on CASSANDRA-896:
------------------------------------------
ArrayBlockingQueue helps keep memory under control. I set the capacity to 4096
(stagingQueueSize). Test is still running but within first 15 minutes its
stable. I will measure different sizes, as this is going to limit writes
(blocking queue). I think we will have to make this a tunable so users can
trade off memory for higher write throughput.
> GC issues with LinkedBlockingQueue, very large heap sizes
> ---------------------------------------------------------
>
> Key: CASSANDRA-896
> URL: https://issues.apache.org/jira/browse/CASSANDRA-896
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.6
> Reporter: Chris Goffinet
> Assignee: Chris Goffinet
> Fix For: 0.6
>
> Attachments: Screen shot 2010-03-14 at 10.19.09 PM.png, Screen shot
> 2010-03-14 at 10.20.31 PM.png
>
>
> We were doing a large import over thrift today (250M rows) with 50 columns
> each. We noticed when this process started, that the heap usage slowly began
> to rise. Eventually hitting 10GB. After investigating heap dumps we noticed
> this is linked to GC problems in the JVM for LinkedBlockingQueue. Sending a
> hint through jconsole to perform GC didn't help either.
> See this article:
> http://tech.puredanger.com/2009/02/11/linkedblockingqueue-garbagecollection/
> JDK 1.6.18 doesn't have this fix yet either. We could possibly add the
> change from:
> http://svn.apache.org/repos/asf/harmony/standard/classlib/trunk/modules/concurrent/src/main/java/java/util/concurrent/LinkedBlockingQueue.java
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
