[
https://issues.apache.org/jira/browse/CASSANDRA-14840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16665726#comment-16665726
]
Jeff Jirsa commented on CASSANDRA-14840:
----------------------------------------
No, its an open issue. There are some proposed patches flying around that may
work for you (check the linked JIRA), but it's not committed or tested or
guaranteed to work.
Also, you dont have to put iptables on every host, you can put the rules only
on hte bootstrapping host, you just need to be able to block most of the
cluster quickly (and undo it quickly).
> Bootstrap of new node fails with OOM in a large cluster
> -------------------------------------------------------
>
> Key: CASSANDRA-14840
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14840
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
> Reporter: Jai Bheemsen Rao Dhanwada
> Priority: Critical
>
> We are seeing new node addition fails with OOM during bootstrap in a cluster
> of more than 80 nodes and 3000 CF without any data in those CFs.
>
> Steps to reproduce:
> # Launch a 3 node cluster
> # Create 3000 CF in the cluster
> # Start adding nodes to the cluster one by one
> # After adding 75-80 nodes, the new node bootstrap fails with OOM.
> {code:java}
> ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2018-10-24 03:26:47,870
> JVMStabilityInspector.java:78 - Exiting due to error while processing commit
> log during initialization.
> java.lang.OutOfMemoryError: Java heap space
> at java.util.regex.Pattern.matcher(Pattern.java:1093) ~[na:1.8.0_151]
> at java.util.Formatter.parse(Formatter.java:2547) ~[na:1.8.0_151]
> at java.util.Formatter.format(Formatter.java:2501) ~[na:1.8.0_151]
> at java.util.Formatter.format(Formatter.java:2455) ~[na:1.8.0_151]
> at java.lang.String.format(String.java:2940) ~[na:1.8.0_151]
> at
> org.apache.cassandra.db.commitlog.AbstractCommitLogService$1.run(AbstractCommitLogService.java:105)
> ~[apache-cassandra-2.1.16.jar:2.1.16]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]{code}
> Cassandra Version: 2.1.16
> OS: CentOS7
> num_tokens: 256 on each node.
>
> This behavior is blocking us from adding extra capacity when needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
