[
https://issues.apache.org/jira/browse/CASSANDRA-3537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214675#comment-13214675
]
Sylvain Lebresne commented on CASSANDRA-3537:
---------------------------------------------
No I meant that we would wrap the initial Runnable into another Runnable that
catches every exception of the first one and log them. In other words, as far
as STPE is concerned, our runnable would never ever fail (and thus would never
be suppressed).
> ExpiringMap timer is not exception-proof
> ----------------------------------------
>
> Key: CASSANDRA-3537
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3537
> Project: Cassandra
> Issue Type: Bug
> Environment: 当前堆大小:
> 5,815,955 Kb
> 堆大小的最大值:
> 6,045,696 Kb
> 分配的内存:
> 6,045,696 Kb
> 暂挂结束操作:
> 0 个对象
> 垃圾收集器:
> Name = 'ParNew', Collections = 3,294, Total time spent = 2 minutes
> 垃圾收集器:
> Name = 'ConcurrentMarkSweep', Collections = 5,909, Total time spent = 2 hours
> 17 minutes
>
> 操作系统:
> Linux 2.6.32.12-0.7-default
> 体系结构:
> amd64
> 处理器的数目:
> 16
> 分配的虚拟内存:
> 42,748,416 Kb
> 物理内存总量:
> 24,568,836 Kb
> 可用物理内存:
> 7,136,380 Kb
> 交换空间总量:
> 2,104,472 Kb
> 可用交换空间:
> 1,970,800 Kb
>
> VM 参数:
> -ea -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms6G -Xmx6G
> -Xmn2400M -XX:+HeapDumpOnOutOfMemoryError -Xss128k -XX:+UseParNewGC
> -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8
> -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75
> -XX:+UseCMSInitiatingOccupancyOnly -Djava.net.preferIPv4Stack=true
> -Dcom.sun.management.jmxremote.port=9000
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dpasswd.properties=/opt/obs/cassandra/conf/passwd.properties
> -Dpasswd.mode=MD5 -Dlog4j.configuration=log4j-server.properties
> -Dlog4j.defaultInitOverride=true
> 类路径:
> /opt/obs/cassandra/conf:/opt/obs/cassandra/build/classes/main:/opt/obs/cassandra/build/classes/thrift:/opt/obs/cassandra/lib/antlr-3.2.jar:/opt/obs/cassandra/lib/apache-cassandra-1.0.0.jar:/opt/obs/cassandra/lib/apache-cassandra-clientutil-1.0.0.jar:/opt/obs/cassandra/lib/apache-cassandra-thrift-1.0.0.jar:/opt/obs/cassandra/lib/avro-1.4.0-fixes.jar:/opt/obs/cassandra/lib/avro-1.4.0-sources-fixes.jar:/opt/obs/cassandra/lib/cassandra_simple_authentication.jar:/opt/obs/cassandra/lib/commons-cli-1.1.jar:/opt/obs/cassandra/lib/commons-codec-1.2.jar:/opt/obs/cassandra/lib/commons-lang-2.4.jar:/opt/obs/cassandra/lib/compress-lzf-0.8.4.jar:/opt/obs/cassandra/lib/concurrentlinkedhashmap-lru-1.2.jar:/opt/obs/cassandra/lib/guava-r08.jar:/opt/obs/cassandra/lib/high-scale-lib-1.1.2.jar:/opt/obs/cassandra/lib/jackson-core-asl-1.4.0.jar:/opt/obs/cassandra/lib/jackson-mapper-asl-1.4.0.jar:/opt/obs/cassandra/lib/jamm-0.2.5.jar:/opt/obs/cassandra/lib/jline-0.9.94.jar:/opt/obs/cassandra/lib/json-simple-1.1.jar:/opt/obs/cassandra/lib/libthrift-0.6.jar:/opt/obs/cassandra/lib/log4j-1.2.16.jar:/opt/obs/cassandra/lib/servlet-api-2.5-20081211.jar:/opt/obs/cassandra/lib/slf4j-api-1.6.1.jar:/opt/obs/cassandra/lib/slf4j-log4j12-1.6.1.jar:/opt/obs/cassandra/lib/snakeyaml-1.6.jar:/opt/obs/cassandra/lib/snappy-java-1.0.3.jar
> Reporter: MaHaiyang
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.0.9
>
> Attachments: 3537-v2.txt, 3537.txt
>
>
> I have 4 cassandra nodes ,and I put about 30G data to db for every nodes .
> It's just 4 days before I start the cluster ,but now every 4 nodes have the
> same problem ,JVM heap is full ,and GC take no effect ,There must be some
> memory leak . Jmap the memory as follow:
> Object Histogram:
> num #instances #bytes Class description
> --------------------------------------------------------------------------
> 1: 15793606 758093088 java.nio.HeapByteBuffer
> 2: 2153811 320138208 java.lang.Object[]
> 3: 6163192 197222144 org.apache.cassandra.db.Column
> 4: 2543836 175890256 int[]
> 5: 2168816 155397192 long[]
> 6: 2078123 116374888
> org.cliffc.high_scale_lib.ConcurrentAutoTable$CAT
> 7: 1847111 73884440 java.math.BigInteger
> 8: 1234243 59243664 java.util.Hashtable
> 9: 1770829 58233000 char[]
> 10: 1770627 56660064 java.lang.String
> 11: 1665886 39981264 org.apache.cassandra.db.DecoratedKey
> 12: 692706 38791536
> org.cliffc.high_scale_lib.NonBlockingHashMap$CHM
> 13: 1234274 37172088 java.util.Hashtable$Entry[]
> 14: 1133541 36273312 java.net.Inet4Address
> 15: 738528 35449344
> org.apache.cassandra.service.ReadCallback
> 16: 2078118 33249888 org.cliffc.high_scale_lib.Counter
> 17: 1373886 32973264 org.apache.cassandra.db.ReadResponse
> 18: 1234023 29616552 org.apache.cassandra.net.Message
> 19: 1234019 29616456 org.apache.cassandra.net.Header
> 20: 1846185 29538960 org.apache.cassandra.dht.BigIntegerToken
> 21: 891378 28524096
> org.apache.cassandra.utils.ExpiringMap$CacheableObject
> 22: 692706 27708240
> org.cliffc.high_scale_lib.NonBlockingHashMap
> 23: 1148252 27558048 java.util.Collections$SynchronizedSet
> 24: 541977 26014896
> org.apache.cassandra.db.SliceFromReadCommand
> 25: 998001 23952024
> java.util.concurrent.ConcurrentSkipListMap$Node
> 26: 928792 22291008 java.util.ArrayList
> 27: 692715 22166880
> java.util.concurrent.atomic.AtomicReferenceFieldUpdater$AtomicReferenceFieldUpdaterImpl
> 28: 891378 21393072 org.apache.cassandra.net.CallbackInfo
> 29: 1148247 18371952 java.util.Hashtable$KeySet
> 30: 731859 17564616 org.apache.cassandra.db.Row
> 31: 529991 16959712
> org.apache.cassandra.db.ArrayBackedSortedColumns
> 32: 691425 16594200
> org.apache.cassandra.db.AbstractColumnContainer$DeletionInfo
> 33: 648580 15565920 org.apache.cassandra.db.filter.QueryPath
> 34: 648338 15560112
> org.apache.cassandra.service.RowDigestResolver
> 35: 971376 15542016
> java.util.concurrent.atomic.AtomicInteger
> 36: 837418 13398688
> org.apache.cassandra.utils.SimpleCondition
> 37: 535614 12854736 org.apache.cassandra.db.ColumnFamily
> 38: 725634 11610144
> java.util.concurrent.atomic.AtomicReference
> 39: 195117 9365616 org.apache.cassandra.db.ThreadSafeSortedColumns
> 40: 281921 9021472
> java.util.concurrent.ConcurrentSkipListMap$HeadIndex
> 41: 277679 8885728
> java.util.concurrent.locks.ReentrantLock$NonfairSync
> 42: 314424 7546176 java.util.concurrent.ConcurrentSkipListMap$Index
> 43: 275186 6604464
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject
> 44: 270280 6486720 java.util.concurrent.LinkedBlockingQueue$Node
> 45: 219553 5269272
> org.apache.cassandra.io.sstable.IndexSummary$KeyPosition
> 46: 106436 5108928 java.util.TreeMap
> 47: 122185 4887400 org.apache.cassandra.db.ExpiringColumn
> 48: 189968 4559232 org.apache.cassandra.db.SuperColumn
> 49: 275659 4410544 java.util.concurrent.locks.ReentrantLock
> 50: 90213 4330224 java.util.concurrent.LinkedBlockingQueue
> 51: 107026 4281040 java.util.TreeMap$Entry
> 52: 30501 4222056 * ConstMethodKlass
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
