Re: Solr core corrupted for version 7.4.0, please help!

2018-08-25 Thread Erick Erickson
Well, how it's marked really has little to do with whether it gets addressed or not. The nature of Open Source software is that it's, well, open. If this is a big enough pain point for you, you can pull the code and fix it. Whether you then submit a patch for review/commit is up to you. There are

Re: Solr core corrupted for version 7.4.0, please help!

2018-08-24 Thread mmb1234
Thank you for https://issues.apache.org/jira/browse/SOLR-12691. I see it's marked as minor. Can we bump up the priority please ? The example of 2 cores ingest + transientCacheSize==1 was provided for reproduction reference only, and is not running in not production. Production setup on AWS use

Re: Solr core corrupted for version 7.4.0, please help!

2018-08-22 Thread Shawn Heisey
On 8/22/2018 7:09 PM, mmb1234 wrote: The original post clarified the current state. "we have about 75 cores with "transientCacheSize" set to 32". If transientCacheSize is increased to match current cores, we'll differ the issue. It's going to hit 100's cores per solr instance shortly. Majority o

Re: Solr core corrupted for version 7.4.0, please help!

2018-08-22 Thread mmb1234
> Having 100+ cores on a Solr node and a transient cache size of 1 The original post clarified the current state. "we have about 75 cores with "transientCacheSize" set to 32". If transientCacheSize is increased to match current cores, we'll differ the issue. It's going to hit 100's cores per sol

Re: Solr core corrupted for version 7.4.0, please help!

2018-08-22 Thread Erick Erickson
Then until you can try to fix the code, you need to increase your transient cache size. Having 100+ cores on a Solr node and a transient cache size of 1 to "so jvm heap and os filecache is aligned to a manageable number of open cores we can serve" implies that you are trying to compensate for an

Re: Solr core corrupted for version 7.4.0, please help!

2018-08-22 Thread mmb1234
> The problem here is that you may have M requests queued up for the _same_ core, each with a new update request. With transientCacheSize ==1, as soon as the update request for Core B is received, Core B encounters data corruption not Core A. Both Core A and Core B are receiving update requets.

Re: Solr core corrupted for version 7.4.0, please help!

2018-08-22 Thread Erick Erickson
It should not be required that your transient core size is greater than or equal to the number of simultaneous updates. Theoretically, it works like this: - A request comes in and a reference-counted core is opened to serve it. That may require loading the core. - If another request comes in that

Re: Solr core corrupted for version 7.4.0, please help!

2018-08-22 Thread Shawn Heisey
On 8/16/2018 7:14 PM, Michael Hu (CMBU) wrote: Environment: * solr 7.4.1 * all cores are vanilla cores with "loadOnStartUp" set to false, and "transient" set to true * we have about 75 cores with "transientCacheSize" set to 32 Issue: we have core corruption from time to time (2-

Re: Solr core corrupted for version 7.4.0, please help!

2018-08-22 Thread Mikhail Khludnev
t; From: Michael Hu (CMBU) > Sent: Friday, August 17, 2018 1:22 PM > To: solr-user@lucene.apache.org > Cc: Mohsin Beg > Subject: Re: Solr core corrupted for version 7.4.0, please help! > > > > Can someone advise me how to solve this issue? > > Thank you! > > --

Re: Solr core corrupted for version 7.4.0, please help!

2018-08-22 Thread Michael Hu (CMBU)
Can someone advise me how to solve this issue, please? Thank you so much! --Michael From: Michael Hu (CMBU) Sent: Friday, August 17, 2018 1:22 PM To: solr-user@lucene.apache.org Cc: Mohsin Beg Subject: Re: Solr core corrupted for version 7.4.0, please help

Re: Solr core corrupted for version 7.4.0, please help!

2018-08-17 Thread Michael Hu (CMBU)
Can someone advise me how to solve this issue? Thank you! --Michael From: Michael Hu (CMBU) Sent: Thursday, August 16, 2018 6:14 PM To: solr-user@lucene.apache.org Cc: Mohsin Beg Subject: Solr core corrupted for version 7.4.0, please help! Environment

Solr core corrupted for version 7.4.0, please help!

2018-08-16 Thread Michael Hu (CMBU)
Environment: * solr 7.4.1 * all cores are vanilla cores with "loadOnStartUp" set to false, and "transient" set to true * we have about 75 cores with "transientCacheSize" set to 32 Issue: we have core corruption from time to time (2-3 core corruption a day) How to reproduce: *