Drop those cache numbers. Way down. I warm up 30 million documents in about 2
minutes with the following configuration:
<documentCache
class="solr.FastLRUCache"
size="128"
initialSize="10"
cleanupThread="true" />
<queryResultCache
class="solr.FastLRUCache"
size="128"
initialSize="10"
autowarmCount="20"
cleanupThread="true" />
<fieldValueCache
class="solr.FastLRUCache"
size="128"
initialSize="10"
autowarmCount="20"
cleanupThread="true" />
<filterCache
class="solr.FastLRUCache"
size="128"
initialSize="10"
autowarmCount="20"
cleanupThread="true" />
Mind you, I also use Solr 1.4. Also, setup a decent warming query or two, as
so:
<lst> <str name="q">date:[NOW-2DAYS TO NOW]</str> <str name="start">0</str>
<str name="rows">1000000</str> <str name="sort">date desc</str></lst>
Don't warm facets that have a large amount of terms or you will kill your warm
up time.
Hope this helps!
On Feb 17, 2010, at 8:55 AM, Stefan Neumann wrote:
> Hi all,
>
> we are facing extremly increasing warmup times the last 15 days, which
> we are not able to explain, since the number of documents and their size
> is stable. Before the increase we can commit our changes in nearly 20
> minutes, now it is about 2 hours.
>
> We were able to identify the warmup of the caches (queryresultCache and
> filterCache) as the reason. We tried to decrease the number of warmup
> elements from 30000 to 10000 without any impact.
>
> What influences the runtime during the warmup? Is there any possibility
> to boost the warmup?
>
> I attach some more information and statistics.
>
> Thanks a lot for your help.
>
> Stefan
>
>
> Solr: 1.3
> Documents: 4.000.000
> -Xmx 12G
> index size/disc 4.7G
>
> config:
>
> <queryResultWindowSize>100</queryResultWindowSize>
> <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
>
> No queries configured for warming.
>
> CACHES:
> =======
>
> name: queryResultCache
> class: org.apache.solr.search.LRUCache
> version: 1.0
> description: LRU Cache(maxSize=200000,
> initialSize=30000,
> autowarmCount=10000,
> regenerator=org.apache.solr.search.solrindexsearche...@36eb7331)
> stats:
>
> lookups: 15958
> hits : 9589
> hitratio: 0.60
> inserts: 16211
> evictions: 0
> size: 16169
> warmupTime : 1960239
> cumulative_lookups: 436250
> cumulative_hits: 260678
> cumulative_hitratio: 0.59
> cumulative_inserts: 174066
> cumulative_evictions: 0
>
>
> name: filterCache
> class: org.apache.solr.search.LRUCache
> version: 1.0
> description: LRU Cache(maxSize=200000,
> initialSize=30000,
> autowarmCount=30000,
> regenerator=org.apache.solr.search.solrindexsearche...@9818f80)
> stats:
> lookups: 6313622
> hits: 6304004
> hitratio: 0.99
> inserts: 42266
> evictions: 0
> size: 40827
> warmupTime: 1268074
> cumulative_lookups: 118887830
> cumulative_hits: 118605224
> cumulative_hitratio: 0.99
> cumulative_inserts: 296134
> cumulative_evictions: 0
>
>
>
