Chris, Thanks for taking a stab at this. If I understand correctly, the flow you are planning on is Solr X converts all X-1 segments to X on a hot index --> shut down server (this is very important in the context of the code presented here due to possible race condition with commit) --> run this script on a cold index ---> bring up the server ?
> actually run this on a test index originally created with Solr/Lucene 7, but running on Solr 8. There were no issues stopping Solr 8, running this to force the index to move to version 8, then restarting Solr. Everything works as expected. Curious to know how you converted all segments to Solr 8 before triggering this code here. I ask because if you didn't, it likely would still open the index fine, but there are minVersion/version checks in several flows in Lucene and you don't want to end up in an inconsistent state that way. It would be quite reasonable for Lucene to see indexCreatedVersionMajor to be X, but segments having traces of X-1 and declare the index corrupted in some flow. Also, not correctly converting all segments to version X *in every aspect* before the major version flip would violate the guarantee that even in case of a breaking change, your index is indeed lossless/uncorrupted after upgrade. I am also not sure how I feel about the use of reflection to force set the indexCreatedVersionMajor (a private final currently) . That _might_ be a blocker for Solr to maintain this code (?). It would be preferable if we could rely on APIs (effort to push for the same underway at https://github.com/apache/lucene/pull/14607 ). Personally, I would like to see this materialize this way. Meanwhile I do believe there is value in exploring a fallback to doing this entirely within Solr. Mike McCandless's suggestion at the talk to open a new IndexWriter in a parallel data directory and copy over the SegmentInfo list from the older SegmentInfos seems like a reasonable option in that direction. If you have the bandwidth, would you be willing to give that a try? Thanks, Rahul On Thu, Sep 18, 2025 at 5:55 PM Christopher Schultz <schu...@apache.org> wrote: > All, > > On 2025/09/14 16:49:16 Christopher Schultz wrote: > > I've written (with some help from ChatGPT) a short CLI utility to check > > the segments and segment info and optionally force a change to the major > > version. It should only work if the requested version is *higher* than > > the current version AND if all the segments are already compatible. You > > also have to use a command-line argument to enable WRITE mode. > > > > So, you can run it on an index and get info, and verify beforehand that > > it SHOULD work before asking it to actually write to the segment info > > and "forge" the new indexCreatedVersionMajor field. > > I have made a few modifications and actually run this on a test index > originally created with Solr/Lucene 7, but running on Solr 8. There were no > issues stopping Solr 8, running this to force the index to move to version > 8, then restarting Solr. Everything works as expected. > > My next step is to try running this newly-forged index on Solr 9. > > Here is the latest code: > > import org.apache.lucene.index.*; > import org.apache.lucene.store.FSDirectory; > import org.apache.lucene.util.Version; > > import java.io.IOException; > import java.lang.reflect.Field; > import java.nio.file.Path; > import java.nio.file.Paths; > > public class LuceneIndexUpgrader { > > public static void main(String[] args) { > if (args.length < 1) { > System.err.println("Usage: java LuceneIndexUpgrader > <path-to-index> [newMajorVersion] [--write]"); > System.exit(1); > } > > Path indexPath = null; > Integer newMajorVersion = null; > boolean shouldWrite = false; > > // Parse args > for (String arg : args) { > if (arg.equals("--write")) { > shouldWrite = true; > } else if (arg.matches("\\d+")) { > newMajorVersion = Integer.parseInt(arg); > } else if (indexPath == null) { > indexPath = Paths.get(arg); > } else { > System.err.println("Unrecognized argument: " + arg); > System.exit(1); > } > } > > if (indexPath == null) { > System.err.println("Error: Index path is required."); > System.exit(1); > } > > if (newMajorVersion != null && (newMajorVersion < 1 || > newMajorVersion > 99)) { > System.err.println("Invalid Lucene major version: " + > newMajorVersion); > System.exit(1); > } > > // Get Lucene library version > Version luceneVersion = Version.LATEST; > int currentLuceneMajor = luceneVersion.major; > > System.out.println("Running with Lucene library version: " + > luceneVersion); > > if (newMajorVersion != null && newMajorVersion > > currentLuceneMajor) { > System.err.printf( > "Aborting: Requested index version %d is higher than the > current Lucene library version %d%n", > newMajorVersion, currentLuceneMajor > ); > System.exit(1); > } > > try (FSDirectory directory = FSDirectory.open(indexPath)) { > SegmentInfos segmentInfos = > SegmentInfos.readLatestCommit(directory); > > int currentMajorVersion = > segmentInfos.getIndexCreatedVersionMajor(); > > // Print index-level version info > System.out.println("\nLucene Index Metadata:"); > > System.out.println("--------------------------------------------------"); > System.out.println("Index Created Version Major: " + > currentMajorVersion); > System.out.println("Min Segment Lucene Version: " + > segmentInfos.getMinSegmentLuceneVersion()); > > System.out.println("\nSegment Details:"); > boolean compatible = true; > > for (SegmentCommitInfo sci : segmentInfos) { > SegmentInfo si = sci.info; > Version version = si.getVersion(); > Version minVersion = si.getMinVersion(); > > > System.out.println("--------------------------------------------------"); > System.out.println("Segment name: " + si.name); > System.out.println(" Lucene version: " + version); > System.out.println(" Min Lucene ver: " + minVersion); > > if (newMajorVersion != null) { > int actualVersion = minVersion != null ? > minVersion.major : > (version != null ? version.major : -1); > > if (actualVersion < newMajorVersion) { > System.err.printf(" Incompatible: segment > minVersion %d < requested indexCreatedVersionMajor %d%n", > actualVersion, newMajorVersion); > compatible = false; > } > } > } > > if (newMajorVersion != null) { > System.out.printf("\nRequested new > indexCreatedVersionMajor: %d%n", newMajorVersion); > > // Check: new version not lower than current > if (newMajorVersion < currentMajorVersion) { > System.err.printf("Aborting: new major version %d is > less than current index version %d%n", > newMajorVersion, currentMajorVersion); > System.exit(1); > } > > // Check: new version same as current > if (newMajorVersion.equals(currentMajorVersion)) { > System.out.println("No change needed: index already > has the requested version."); > return; > } > > if (!compatible) { > System.err.println("Aborting: Not all segments are > compatible with the requested version."); > System.exit(1); > } > > if (shouldWrite) { > // Proceed with update > Field field = > SegmentInfos.class.getDeclaredField("indexCreatedVersionMajor"); > field.setAccessible(true); > field.setInt(segmentInfos, newMajorVersion); > segmentInfos.commit(directory); > > System.out.printf("Updated indexCreatedVersionMajor to > %d and saved to disk.%n", newMajorVersion); > } else { > System.out.println("Dry run: indexCreatedVersionMajor > would be updated, but --write was not set."); > } > } else { > System.out.println("\n(No version change requested.)"); > } > > } catch (IOException | NoSuchFieldException | > IllegalAccessException e) { > System.err.println("Error while processing Lucene index: " + > e.getMessage()); > e.printStackTrace(); > } > } > } >
