On 7/26/16 5:46 AM, Shawn Heisey wrote:
On 7/22/2016 10:15 AM, Rallavagu wrote:<filterCache class="solr.FastLRUCache" size="5000" initialSize="5000" autowarmCount="500"/><queryResultCache class="solr.LRUCache" size="20000" initialSize="20000" autowarmCount="500"/>As Erick indicated, these settings are incompatible with Near Real Time updates. With those settings, every time you commit and create a new searcher, Solr will execute up to 1000 queries (potentially 500 for each of the caches above) before that new searcher will begin returning new results. I do not know how fast your filter queries execute when they aren't cached... but even if they only take 100 milliseconds each, that's could take up to a minute for filterCache warming. If each one takes two seconds and there are 500 entries in the cache, then autowarming the filterCache would take nearly 17 minutes. You would also need to wait for the warming queries on queryResultCache. The autowarmCount on my filterCache is 4, and warming that cache *still* sometimes takes ten or more seconds to complete. If you want true NRT, you need to set all your autowarmCount values to zero. The tradeoff with NRT is that your caches are ineffective immediately after a new searcher is created.
Will look into this and make changes as suggested.
Looking at the "top" screenshot ... you have plenty of memory to cache the entire index. Unless your queries are extreme, this is usually enough for good performance. One possible problem is that cache warming is taking far longer than your autoSoftCommit interval, and the server is constantly busy making thousands of warming queries. Reducing autowarmCount, possibly to zero, *might* fix that. I would expect higher CPU load than what your screenshot shows if this were happening, but it still might be the problem.
Great point. Thanks for the help.
Thanks, Shawn
