Erik, I have a followup question on this topic.
If we have used 10 unique FQs and when we configure filterCache=100 & autoWarm=5, then which 5 out of the 10 will be repopulated in the case of new searcher? I don't think there is a way to set the preference or there is? Thanks, Kranti K. Parisa http://www.linkedin.com/in/krantiparisa On Thu, Apr 17, 2014 at 5:25 PM, Matt Kuiper <matt.kui...@issinc.com> wrote: > Ok, that makes sense. > > Thanks again, > Matt > > Matt Kuiper - Software Engineer > Intelligent Software Solutions > p. 719.452.7721 | matt.kui...@issinc.com > www.issinc.com | LinkedIn: intelligent-software-solutions > > -----Original Message----- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: Thursday, April 17, 2014 9:26 AM > To: solr-user@lucene.apache.org > Subject: Re: cache warming questions > > Don't go overboard warming here, you often hit diminishing returns very > quickly. For instance, if the size is 512 you might set your autowarm count > to 16 and get the most bang for your buck. Beyond some (usually small) > number, the additional work you put in to warming is wasted. This is > especially true if your autocommit (soft, or hard with > openSearcher=true) is short. > > So while you're correct in your sizing bit, practically it's rarely that > complicated since the autowarm count is usually so much smaller than the > size that there's no danger of swapping them out. YMMV of course. > > Best, > Erick > > On Wed, Apr 16, 2014 at 10:33 AM, Matt Kuiper <matt.kui...@issinc.com> > wrote: > > Thanks Erick, this is helpful information! > > > > So it sounds like, at minimum the cache size (at least for filterCache > and queryResultCache) should be the sum of the autowarmCount for that cache > and the number of queries defined for the newSearcher listener. Otherwise > some items in the caches will be evicted right away. > > > > Matt > > > > -----Original Message----- > > From: Erick Erickson [mailto:erickerick...@gmail.com] > > Sent: Tuesday, April 15, 2014 5:21 PM > > To: solr-user@lucene.apache.org > > Subject: Re: cache warming questions > > > > bq: What does it mean that items will be regenerated or prepopulated > from the current searcher's cache... > > > > You're right, the values aren't cached. They can't be since the internal > Lucene document id is used to identify docs, and due to merging the > internal ID may bear no relation to the old internal ID for a particular > document. > > > > I find it useful to think of Solr's caches as a map where the key is > the "query" and the value is some representation of the found documents. > The details of the value don't matter, so I'll skip them. > > > > What matters is the key. Consider the filter cache. You put something > like &fq=price:[0 TO 100] on a URL. Solr then uses the fq clause as the > key to the filterCache. > > > > Here's the sneaky bit. When you specify an autowarm count of N for the > filterCache, when a new searcher is opened the first N keys from the map > are re-executed in the new searcher's context and the results put into the > new searcher's filterCache. > > > > bq: ...how does auto warming and explicit warming work together? > > > > They're orthogonal. IOW, the autowarming for each cache is executed as > well as the newSearcher static warming queries. Use the static queries to > do things like fill the sort caches etc. > > > > Incidentally, this bears on why there's a "firstSearcher" and > "newSearcher". The newSearcher queries are run in addition to the cache > autowarms. firstSearcher static queries are only run when a Solr server is > started the first time, and there are no cache entries to autowarm. So the > firstSearcher queries might be quite a bit more complex than newSearcher > queries. > > > > HTH, > > Erick > > > > On Tue, Apr 15, 2014 at 1:55 PM, Matt Kuiper <matt.kui...@issinc.com> > wrote: > >> Hello, > >> > >> I have a few questions regarding how Solr caches are warmed. > >> > >> My understanding is that there are two ways to warm internal Solr > caches (only one way for document cache and lucene FieldCache): > >> > >> Auto warming - occurs when there is a current searcher handling > requests and new searcher is being prepared. "When a new searcher is > opened, its caches may be prepopulated or "autowarmed" with cached object > from caches in the old searcher. autowarmCount is the number of cached > items that will be regenerated in the new searcher." > http://wiki.apache.org/solr/SolrCaching#autowarmCount > >> > >> Explicit warming - where the static warming queries specified in > Solrconfig.xml for newSearcher and firstSearcher listeners are executed > when a new searcher is being prepared. > >> > >> What does it mean that items will be regenerated or prepopulated from > the current searcher's cache to the new searcher's cache? I doubt it means > copy, as the index has likely changed with a commit and possibly > invalidated some contents of the cache. Are the queries, or filters, that > define the contents of the current caches re-executed for the new > searcher's caches? > >> > >> For the case where auto warming is configured, a current searcher is > active, and static warming queries are defined how does auto warming and > explicit warming work together? Or do they? Is only one type of warming > activated to fill the caches? > >> > >> Thanks, > >> Matt >
