Wouldn't collection swapping be a better strategy in that case? Load and optimise in a separate server, then swap it in. On 30 Dec 2015 10:08 am, "Walter Underwood" <wun...@wunderwood.org> wrote:
> The only time that a force merge might be useful is when you reindex all > content every night or every week, then do not make any changes until the > next reindex. But even then, it probably does not matter. > > Just let Solr do its thing. Solr is pretty smart. > > A long time ago (1996-2006), I worked on an enterprise search engine with > the same merging algorithm as Solr (Ultraseek Server). We always had > customers asking about force-merge/optimize. It never made a useful > difference. Even with twenty servers at irs.gov <http://irs.gov/>, it > didn’t make a difference. > > wunder > K6WRU > Walter Underwood > CM87wj > http://observer.wunderwood.org/ (my blog) > > > On Dec 29, 2015, at 6:59 PM, Zheng Lin Edwin Yeo <edwinye...@gmail.com> > wrote: > > > > Hi Walter, > > > > Thanks for your reply. > > > > Then how about optimization after indexing? > > Normally the index size is much larger after indexing, then after > > optimization, the index size reduces. Do we still need to do that? > > > > Regards, > > Edwin > > > > On 30 December 2015 at 10:45, Walter Underwood <wun...@wunderwood.org> > > wrote: > > > >> Do not “optimize". > >> > >> It is a forced merge, not an optimization. It was a mistake to ever name > >> it “optimize”. Solr automatically merges as needed. There are a few > >> situations where a force merge might make a small difference. Maybe 10% > or > >> 20%, no one had bothered to measure it. > >> > >> If your index is continually updated, clicking that is a complete waste > of > >> resources. Don’t do it. > >> > >> wunder > >> Walter Underwood > >> wun...@wunderwood.org > >> http://observer.wunderwood.org/ (my blog) > >> > >>> On Dec 29, 2015, at 6:35 PM, Zheng Lin Edwin Yeo <edwinye...@gmail.com > > > >> wrote: > >>> > >>> Hi, > >>> > >>> I am facing a situation, when I do an optimization by clicking on the > >>> "Optimized" button on the Solr Admin Overview UI, the memory usage of > the > >>> server increases gradually, until it reaches near the maximum memory > >>> available. There is 64GB of memory available in the server. > >>> > >>> Even after the optimized is completed, the memory usage stays near the > >> 100% > >>> range, and could not be reduced until I stop Solr. Why could this be > >>> happening? > >>> > >>> Also, I don't think the optimization is completed, as the admin page > says > >>> the index is not optimized again after I go back to the Overview page, > >> even > >>> though I did not do any updates to the index. > >>> > >>> I am using Solr 5.3.0, with 1 shard and 2 replica. My index size is > >> 183GB. > >>> > >>> Regards, > >>> Edwin > >> > >> > >
