[
https://issues.apache.org/jira/browse/CASSANDRA-13008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15853296#comment-15853296
]
Jay Zhuang commented on CASSANDRA-13008:
----------------------------------------
[~Stefania] would you please review this patch?
> Add vm.max_map_count StartupCheck
> ---------------------------------
>
> Key: CASSANDRA-13008
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13008
> Project: Cassandra
> Issue Type: Improvement
> Components: Configuration
> Reporter: Jay Zhuang
> Assignee: Jay Zhuang
> Priority: Minor
> Fix For: 3.0.x
>
> Attachments: 13008-3.0.txt
>
>
> It's recommended to set {{vm.max_map_count}} to 1048575 (CASSANDRA-3563)
> When the max_map_count is low, it throws OOM exception, which is hard to link
> to the real issue of vm.max_map_count.
> The problem happened when we tried to remove one node, all the other nodes in
> cluster crashed. As each node was trying to load more local SSTable files for
> streaming.
> I would suggest to add a StartupCheck for max_map_count, at least it could
> give a warning message to help the debug.
> {code}
> ERROR [STREAM-IN-] JVMStabilityInspector.java:140 - JVM state determined to
> be unstable. Exiting forcefully due to:
> java.lang.OutOfMemoryError: Map failed
> at sun.nio.ch.FileChannelImpl.map0(Native Method) ~[na:1.8.0_112]
> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:937)
> ~[na:1.8.0_112]
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:152)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:280)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:216)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:173)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.util.MmappedRegions.<init>(MmappedRegions.java:70)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.util.MmappedRegions.<init>(MmappedRegions.java:58)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:96)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.util.CompressedSegmentedFile.<init>(CompressedSegmentedFile.java:47)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.complete(CompressedSegmentedFile.java:132)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:177)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.util.SegmentedFile$Builder.buildData(SegmentedFile.java:193)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.openFinal(BigTableWriter.java:276)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.access$600(BigTableWriter.java:50)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$TransactionalProxy.doPrepare(BigTableWriter.java:313)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.prepareToCommit(Transactional.java:173)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.finish(Transactional.java:184)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.finish(SSTableWriter.java:213)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.finish(SimpleSSTableMultiWriter.java:56)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.streaming.StreamReceiveTask.received(StreamReceiveTask.java:109)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.streaming.StreamSession.receive(StreamSession.java:599)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:482)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:296)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
