[
https://issues.apache.org/jira/browse/SOLR-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429132#comment-13429132
]
Markus Jelsma commented on SOLR-3685:
-------------------------------------
Each node has two cores and allow only one warming searcher at any time. The
problem is triggered on start up after graceful shutdown as well as a hard
power off. I've seen it happening not only when the whole cluster if restarted
(i don't think i've ever done that) but just one node of the 6 shard 2 replica
test cluster.
The attached log is of one node being restarted out of the whole cluster.
Could the off-heap RAM be part of data being sent over the wire?
We've worked around the problem for now by getting more RAM.
> solrcloud crashes on startup due to excessive memory consumption
> ----------------------------------------------------------------
>
> Key: SOLR-3685
> URL: https://issues.apache.org/jira/browse/SOLR-3685
> Project: Solr
> Issue Type: Bug
> Components: replication (java), SolrCloud
> Affects Versions: 4.0-ALPHA
> Environment: Debian GNU/Linux Squeeze 64bit
> Solr 5.0-SNAPSHOT 1365667M - markus - 2012-07-25 19:09:43
> Reporter: Markus Jelsma
> Priority: Critical
> Fix For: 4.1
>
> Attachments: info.log
>
>
> There's a serious problem with restarting nodes, not cleaning old or unused
> index directories and sudden replication and Java being killed by the OS due
> to excessive memory allocation. Since SOLR-1781 was fixed index directories
> get cleaned up when a node is being restarted cleanly, however, old or unused
> index directories still pile up if Solr crashes or is being killed by the OS,
> happening here.
> We have a six-node 64-bit Linux test cluster with each node having two
> shards. There's 512MB RAM available and no swap. Each index is roughly 27MB
> so about 50MB per node, this fits easily and works fine. However, if a node
> is being restarted, Solr will consistently crash because it immediately eats
> up all RAM. If swap is enabled Solr will eat an additional few 100MB's right
> after start up.
> This cannot be solved by restarting Solr, it will just crash again and leave
> index directories in place until the disk is full. The only way i can restart
> a node safely is to delete the index directories and have it replicate from
> another node. If i then restart the node it will crash almost consistently.
> I'll attach a log of one of the nodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
