On 1/8/2017 11:21 PM, Manan Sheth wrote: > Currently, We are in process of upgrading existing Solr indexes from Solr 4.x > to Solr 6.2.1. In order to upgrade existing indexes we are planning to use > IndexUpgrader class in sequential manner from Solr 4.x to Solr 5.x and Solr > 5.x to Solr 6.2.1. > > While preforming the upgrdation a strange behaviour is noticed where all the > previous segments are getting merged to one single large segment. We need to > preserve the original segments as single large segment is getting bulkier (~ > 2-4 TBs). > > Please let me know how to tune the process or write custom logic to overcome > it.
I've taken a very quick look at the IndexUpgrader code. It does the upgrade by calling forceMerge ... which is the Lucene term for what Solr still calls "optimize." What you are seeing is completely normal. This is how it is designed to work. Changing it *might* be possible, but it would involve development work in Lucene. It likely would not be quick and easy. Single-segment indexes are slightly faster than multi-segment indexes containing the same data, so I am failing to see why this is a problem. After the upgrade, the total amount of disk space for your index would either go down or stay the same, although it will temporarily double in size during the upgrade. FYI -- this kind of segment merging can happen as a consequence of normal indexing, so this is something your system must be prepared to handle even when you are not using the upgrader tool, on ANY version of Solr. Thanks, Shawn
