bq: autowarmcount=1024... That's the point, this is quite a high number in my experience.
I've rarely seen numbers above 128 show much of any improvement. I've seen a large number of installations use much smaller autowarm numbers, as in the 16-32 range and be quite content. I _really_ recommend you try to use much smaller numbers then _measure_ whether the first few queries after a commit show unacceptable response times before trying to make things "better". This really feels like premature optimization. Of course you know your problem space better than I do, it's just that I've spent too much of my professional life fixing the wrong "problem"; I've become something of a "measure first" curmudgeon. FWIW, Erick On Sun, Jul 27, 2014 at 10:48 PM, YouPeng Yang <yypvsxf19870...@gmail.com> wrote: > Hi Erick > > We do the DIH job from the DB and committed frequently.It takes a long time > to autowarm the filterCaches after commit or soft commit happened when > setting the autowarmcount=1024,which I do think is small enough. > So It comes up an idea that whether it could directly pass the reference > of the caches over to the new caches so that the autowarm processing will > take much fewer time . > > > > 2014-07-28 2:30 GMT+08:00 Erick Erickson <erickerick...@gmail.com>: > > > Why do you think you _need_ to autowarm the entire cache? It > > is, after all, an LRU cache, the theory being that the most recent > > queries are most likely to be reused. > > > > Personally I'd run some tests on using small autowarm counts > > before getting at all mixed up in some complex scheme that > > may not be useful at all. Say an autowarm count of 16. Then > > measure using that, then say 32 then... Insure you have a real > > problem before worrying about a solution! ;) > > > > Best, > > Erick > > > > > > On Fri, Jul 25, 2014 at 6:45 AM, Shawn Heisey <s...@elyograg.org> wrote: > > > > > On 7/24/2014 8:45 PM, YouPeng Yang wrote: > > > > To Matt > > > > > > > > Thank you,your opinion is very valuable ,So I have checked the > source > > > > codes about how the cache warming up. It seems to just put items of > > the > > > > old caches into the new caches. > > > > I will pull Mark Miller into this discussion.He is the one of the > > > > developer of the Solr whom I had contacted with. > > > > > > > > To Mark Miller > > > > > > > > Would you please check out what we are discussing in the last two > > > > posts.I need your help. > > > > > > Matt is completely right. Any commit can drastically change the Lucene > > > document id numbers. It would be too expensive to determine which > > > numbers haven't changed. That means Solr must throw away all cache > > > information on commit. > > > > > > Two of Solr's caches support autowarming. Those caches use queries as > > > keys and results as values. Autowarming works by re-executing the top > N > > > queries (keys) in the old cache to obtain fresh Lucene document id > > > numbers (values). The cache code does take *keys* from the old cache > > > for the new cache, but not *values*. I'm very sure about this, as I > > > wrote the current (and not terribly good) LFUCache. > > > > > > Thanks, > > > Shawn > > > > > > > > >
