Hey Shawn, Thanks for the reply - funnily enough that is exactly what i'm trialing now. I've significantly lowered the autoWarm (as well as the size) and still have a 0.95+ cache hit rate through searcher loads.
I'm going to continue to tweak these values down so long as i keep the hit rate above 90, which should reduce some memory pressure at least. Thanks Karl ________________________________ From: Shawn Heisey <apa...@elyograg.org> Sent: 29 January 2020 21:01 To: solr-user@lucene.apache.org <solr-user@lucene.apache.org> Subject: Re: Solr Searcher 100% Latency Spike On 1/29/2020 12:44 PM, Karl Stoney wrote: > Looking for a bit of support here. When we soft commit (every 10 > minutes), we get a latency spike that means response times for solr are > loosely double, as you can see in this screenshot: Attachments almost never make it to the list. We cannot see any of your screenshots. > They do correlate to filterCache warmup, which seem to take between 10s > and 30s: > > We don't have any other caches enabled, due to the high level of > cardinality of the queries. > > The spikes are specifically on /select > > > We have the following autowarm configuration for the filterCache: > > <filterCache class="solr.FastLRUCache" > size="8192" > initialSize="8192" > cleanupThread="true" > autowarmCount="900"/> Autowarm, especially on filterCache, can be an extremely lengthy process. What Solr must do in order to warm the cache here is execute up to 900 queries, sequentially, on the new index. That can take a lot of time and use a lot of resources like CPU and I/O. In order to reduce the impact of cache warming, I had to reduce my own autowarmCount on the filterCache to 4. Thanks, Shawn This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office: 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 9439967). This email and any files transmitted with it are confidential and may be legally privileged, and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This email message has been swept for the presence of computer viruses.
