Don't go overboard warming here, you often hit diminishing returns very quickly. For instance, if the size is 512 you might set your autowarm count to 16 and get the most bang for your buck. Beyond some (usually small) number, the additional work you put in to warming is wasted. This is especially true if your autocommit (soft, or hard with openSearcher=true) is short.
So while you're correct in your sizing bit, practically it's rarely that complicated since the autowarm count is usually so much smaller than the size that there's no danger of swapping them out. YMMV of course. Best, Erick On Wed, Apr 16, 2014 at 10:33 AM, Matt Kuiper <matt.kui...@issinc.com> wrote: > Thanks Erick, this is helpful information! > > So it sounds like, at minimum the cache size (at least for filterCache and > queryResultCache) should be the sum of the autowarmCount for that cache and > the number of queries defined for the newSearcher listener. Otherwise some > items in the caches will be evicted right away. > > Matt > > -----Original Message----- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: Tuesday, April 15, 2014 5:21 PM > To: solr-user@lucene.apache.org > Subject: Re: cache warming questions > > bq: What does it mean that items will be regenerated or prepopulated from the > current searcher's cache... > > You're right, the values aren't cached. They can't be since the internal > Lucene document id is used to identify docs, and due to merging the internal > ID may bear no relation to the old internal ID for a particular document. > > I find it useful to think of Solr's caches as a map where the key is the > "query" and the value is some representation of the found documents. The > details of the value don't matter, so I'll skip them. > > What matters is the key. Consider the filter cache. You put something like > &fq=price:[0 TO 100] on a URL. Solr then uses the fq clause as the key to > the filterCache. > > Here's the sneaky bit. When you specify an autowarm count of N for the > filterCache, when a new searcher is opened the first N keys from the map are > re-executed in the new searcher's context and the results put into the new > searcher's filterCache. > > bq: ...how does auto warming and explicit warming work together? > > They're orthogonal. IOW, the autowarming for each cache is executed as well > as the newSearcher static warming queries. Use the static queries to do > things like fill the sort caches etc. > > Incidentally, this bears on why there's a "firstSearcher" and "newSearcher". > The newSearcher queries are run in addition to the cache autowarms. > firstSearcher static queries are only run when a Solr server is started the > first time, and there are no cache entries to autowarm. So the firstSearcher > queries might be quite a bit more complex than newSearcher queries. > > HTH, > Erick > > On Tue, Apr 15, 2014 at 1:55 PM, Matt Kuiper <matt.kui...@issinc.com> wrote: >> Hello, >> >> I have a few questions regarding how Solr caches are warmed. >> >> My understanding is that there are two ways to warm internal Solr caches >> (only one way for document cache and lucene FieldCache): >> >> Auto warming - occurs when there is a current searcher handling requests and >> new searcher is being prepared. "When a new searcher is opened, its caches >> may be prepopulated or "autowarmed" with cached object from caches in the >> old searcher. autowarmCount is the number of cached items that will be >> regenerated in the new searcher." >> http://wiki.apache.org/solr/SolrCaching#autowarmCount >> >> Explicit warming - where the static warming queries specified in >> Solrconfig.xml for newSearcher and firstSearcher listeners are executed when >> a new searcher is being prepared. >> >> What does it mean that items will be regenerated or prepopulated from the >> current searcher's cache to the new searcher's cache? I doubt it means >> copy, as the index has likely changed with a commit and possibly invalidated >> some contents of the cache. Are the queries, or filters, that define the >> contents of the current caches re-executed for the new searcher's caches? >> >> For the case where auto warming is configured, a current searcher is active, >> and static warming queries are defined how does auto warming and explicit >> warming work together? Or do they? Is only one type of warming activated to >> fill the caches? >> >> Thanks, >> Matt
