Thanks Shawn. It does seem like more than anything, the check for LATEST-1 is a safeguard against potential future incompatible change at this point. I tried removing the check in SegmentInfos.java ( https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.11.1/lucene/core/src/java/org/apache/lucene/index/SegmentInfos.java#L321) , compiled the code and ran a full sequence of index upgrades from 5.x -> 6.x -> 7.x ->8.x. The upgrade goes through fine. Also search/update operations work without any issues.
I would still follow-up on the lucene mailing list to know the nitty-gritties to have a complete understanding in case I am missing anything. In my opinion such kinds of restrictions limit the practical usability of an otherwise awesome project. Personally, I feel complete reindexing is not a practical solution for many systems in the wild and it would be great if there could be inbuilt support for such upgrades without having a hard break in between. Thanks, Rahul On Fri, Dec 31, 2021 at 12:51 AM Shawn Heisey <apa...@elyograg.org> wrote: > On 12/30/2021 9:46 PM, Rahul Goswami wrote: > > What is the reason for blocking the upgrade ? If someone has been able to > > successfully upgrade an index written in 6.x in the past to 8.x without > > doing a full reindex, I would love to get some ideas. > > I'm told that Lucene has never guaranteed that an index can be upgraded > through two major versions, even if going to the interim major version > first. The guarantee is one major version back, which means 8.x > versions can read indexes originally built by 7.0 and later. > > The Lucene IndexUpgrader utility just does a ForceMerge on the index. > (Solr calls it Optimize, Lucene calls it ForceMerge). The fact that the > original segments were written by a 6.x version is preserved by a 7.x > ForceMerge, and when 8.x sees that, it refuses to load the index, rather > than risk the index not working correctly. > > If you want nitty gritty low level details about what kind of problems > might occur from using an index originally built by 6.x in 8.x, you're > going to need to consult a Lucene expert on a Lucene mailing list. I do > not understand Lucene at a low enough level to provide those answers. > > It is always best to be able to do a full reindex from the original > source data at any time. Almost all Solr installs will require schema > changes as business needs shift, and as the administrator gains more > knowledge about Solr. Most schema changes require a reindex, and a > reindex is strongly recommended with ANY Solr/Lucene upgrade, even from > one minor version to the next minor version. > > Thanks, > Shawn >
