xiangfu0 commented on code in PR #18919:
URL: https://github.com/apache/pinot/pull/18919#discussion_r3563924741
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/text/TextIndexType.java:
##########
@@ -154,14 +155,24 @@ public TextIndexReader
createIndexReader(SegmentDirectory.Reader segmentReader,
File segmentDir = segmentReader.toSegmentDirectory().getPath().toFile();
TextIndexConfig indexConfig =
fieldIndexConfigs.getConfig(StandardIndexes.text());
if (indexConfig.isStoreInSegmentFile()) {
- PinotDataBuffer textIndexBuffer = null;
- try {
- textIndexBuffer =
segmentReader.getIndexFor(metadata.getColumnName(), StandardIndexes.text());
- } catch (IOException e) {
- throw new RuntimeException(e);
+ // Legacy-directory-first: if the on-disk text index is still the
Lucene directory (a V3
+ // segment TextIndexHandler has not migrated yet — its combined-format
conversion is
+ // v3-gated — or any V1/V2 segment backed by FilePerIndexDirectory),
read it directly and
+ // skip the consolidated probe entirely. FilePerIndexDirectory
resolves getIndexFor to that
+ // directory and fails to map it (not a regular file), which would
kill the segment load
+ // before any fallback could run. This keeps storeInSegmentFile=true
rolling-upgrade-safe
+ // for existing text segments.
+ File legacyTextIndex =
SegmentDirectoryPaths.findTextIndexIndexFile(segmentDir,
metadata.getColumnName());
Review Comment:
This calls `findTextIndexIndexFile()` before trying the consolidated
text-index entry, but that helper requires `segmentDir` to be a local
directory. For non-local `SegmentDirectory` implementations, segment load can
fail here before `getIndexFor(..., text())` has a chance to read the valid
`columns.psf` entry. The vector reader guards the same sidecar probe with
`segmentDir.isDirectory()`; this path should do the same and add a non-local
consolidated-entry regression test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
