Re: SolrCloud 6.6.2 suddenly crash due to slow queries and Log4j issue

2020-10-19 Thread Dominique Bejean
Shawn, According to the log4j description ( https://bz.apache.org/bugzilla/show_bug.cgi?id=57714), the issue is related to lock during appenders collection process. In addition to CONSOLE and file appenders in the default log4j.properties, my customer added 2 extra FileAppender dedicated to all

Re: SolrCloud 6.6.2 suddenly crash due to slow queries and Log4j issue

2020-10-19 Thread Dominique Bejean
Hi Shawn, Thank you for your response. You are confirming my diagnosis. This is in fact a 8 nodes cluster with one single collection with 4 shards and 1 replica (8 cores). 4 Gb heap and 90 Gb Ram When no issue occurs nearly 50% of the heap is used. Num Docs in collection : 10.000.000 Num

Re: SolrCloud 6.6.2 suddenly crash due to slow queries and Log4j issue

2020-10-18 Thread Shawn Heisey
On 10/18/2020 3:22 AM, Dominique Bejean wrote: A few months ago, I reported an issue with Solr nodes crashing due to the old generation heap growing suddenly and generating OOM. This problem occurred again this week. I have threads dumps for each minute during the 3 minutes the problem occured.

SolrCloud 6.6.2 suddenly crash due to slow queries and Log4j issue

2020-10-18 Thread Dominique Bejean
Hi, A few months ago, I reported an issue with Solr nodes crashing due to the old generation heap growing suddenly and generating OOM. This problem occurred again this week. I have threads dumps for each minute during the 3 minutes the problem occured. I am using fastthread.io in order to analyse