Re: daily SolrCloud collection wipes

2019-11-18 Thread Andrzej Białecki
This default autoscaling config helps to keep some aspects of SolrCloud clean - specifically: * Inactive shard plan: it periodically checks whether there are old shards in INACTIVE state that can be removed. Shards in this state are left-over parent shards remaining after a *successful*

Re: daily SolrCloud collection wipes

2019-11-15 Thread Erick Erickson
Werner: Who is sending the delete-by-query? What I’m really wondering is if it’s done by something internal to Solr (in which case I’d like to track it down) or something outside Solr in which case we don’t need to be concerned... Thanks, Erick > On Nov 15, 2019, at 1:38 AM, Werner Detter

Re: daily SolrCloud collection wipes

2019-11-14 Thread Werner Detter
Hi, > Currently logging is disabled due to performance on the live setup. But > tonight, bevor > this happens, we'll enable logging an we'll hopefully see something to track > down the > source for the documents deletion in the collection. tcpdump and the logs revealed the source of the

Re: daily SolrCloud collection wipes

2019-11-14 Thread Shawn Heisey
On 11/14/2019 9:17 AM, Werner Detter wrote: first, thanks for your response. By "reset" I mean: collection still exists but documents have been dropped (from actually round 50k to 0). It happened twice within the same timeframe early in the morning the last two days so I was wondering if

Re: daily SolrCloud collection wipes

2019-11-14 Thread Werner Detter
Hi Shawn, > There is only one thing I know of that can delete data from an index > without an external trigger.  It is the document expiration feature. > > https://lucidworks.com/post/document-expiration/ > > Without some kind of action or intentional config, Solr will never > delete anything

Re: daily SolrCloud collection wipes

2019-11-14 Thread Shawn Heisey
On 11/14/2019 12:28 AM, Werner Detter wrote: I've got a SolrCloud instance with two collections running (Solr 7.7.2) on Debian Stretch VMs. Every morning round about 03:3* am the collection gets reset by $something and I have no clue what causes this and how to prevent it as there areeven no log

Re: daily SolrCloud collection wipes

2019-11-14 Thread Erick Erickson
There’s nothing in Solr that will wipe collections by itself outside of Time Routed Aliases, and there’s no indication that you’re using them. Even if you were, those should only delete collections past their expiration date, they’re pretty well tested to behave correctly. This really looks

daily SolrCloud collection wipes

2019-11-14 Thread Werner Detter
Hi, I've got a SolrCloud instance with two collections running (Solr 7.7.2) on Debian Stretch VMs. Every morning round about 03:30 the collection gets reset by $something and I have no clue what causes this and how to prevent it as there are even no log entries in SolrCloud (even with

daily SolrCloud collection wipes

2019-11-13 Thread Werner Detter
Hi, I've got a SolrCloud instance with two collections running (Solr 7.7.2) on Debian Stretch VMs. Every morning round about 03:3* am the collection gets reset by $something and I have no clue what causes this and how to prevent it as there areeven no log entries in SolrCloud (even with