Re: Changing merge policy config on production

2017-12-16 Thread alexpusch
Thanks Erick, good point on maxMergedSegmentMB, many of my segments really are max out. My index isn't 800G, but it's not far from it - it's about 250G per server. I have high confidence in Solr and my EC2 i3-2xl instances, so far I got pretty good results. -- Sent from:

Re: Changing merge policy config on production

2017-12-16 Thread Erick Erickson
So I'm guessing you have something on the order of an 800G index? The max segment size is roughly 5G (by default) and assuming all your segments are close to the max size I get 160 * 5G = 800G, but that may be off. I think you're barking up the wrong tree if these numbers are close to correct.

Re: Changing merge policy config on production

2017-12-16 Thread alexpusch
To be clear - I'm talking about query performance, not indexing performance. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Changing merge policy config on production

2017-12-16 Thread alexpusch
Thanks for the quick answer Erick, I'm hoping to improve performance by reducing the number of segments. Currently I have ~160 segments. Am I wrong thinking it might improve performance? -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Changing merge policy config on production

2017-12-15 Thread Erick Erickson
The merge rate will be limited by the number of merge threads. You'll merge more often though so the load will change. That said, I wouldn't be concerned unless you have a very high indexing rate. Why do you want to change anyway? Unless you've tried the new settings in a Dev environment, the

Changing merge policy config on production

2017-12-15 Thread alexpusch
Hi, Is it safe to change the mergePolicyFactory config on production servers? Specifically maxMergeAtOnce and segmentsPerTier. How will solr reconcile the current state of the segments with the new config? In case of setting segmentsPerTier to a lower number - will subsequent merges be particulary