xiangfu0 commented on code in PR #18919:
URL: https://github.com/apache/pinot/pull/18919#discussion_r3558089753
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/text/TextIndexType.java:
##########
@@ -154,14 +155,24 @@ public TextIndexReader
createIndexReader(SegmentDirectory.Reader segmentReader,
File segmentDir = segmentReader.toSegmentDirectory().getPath().toFile();
TextIndexConfig indexConfig =
fieldIndexConfigs.getConfig(StandardIndexes.text());
if (indexConfig.isStoreInSegmentFile()) {
- PinotDataBuffer textIndexBuffer = null;
- try {
- textIndexBuffer =
segmentReader.getIndexFor(metadata.getColumnName(), StandardIndexes.text());
- } catch (IOException e) {
- throw new RuntimeException(e);
+ // Legacy-directory-first: if the on-disk text index is still the
Lucene directory (a V3
+ // segment TextIndexHandler has not migrated yet — its combined-format
conversion is
+ // v3-gated — or any V1/V2 segment backed by FilePerIndexDirectory),
read it directly and
+ // skip the consolidated probe entirely. FilePerIndexDirectory
resolves getIndexFor to that
+ // directory and fails to map it (not a regular file), which would
kill the segment load
+ // before any fallback could run. This keeps storeInSegmentFile=true
rolling-upgrade-safe
+ // for existing text segments.
+ File legacyTextIndex =
SegmentDirectoryPaths.findTextIndexIndexFile(segmentDir,
metadata.getColumnName());
Review Comment:
This probe needs the same local-directory guard used by the vector reader.
`SegmentDirectoryPaths.findTextIndexIndexFile()` calls `findFormatFile()`,
which requires `segmentDir.isDirectory()`. For a remote/tiered
`SegmentDirectory`, `getPath()` can be a non-existent local path while
`getIndexFor()` can still return the consolidated `columns.psf` buffer, so
`storeInSegmentFile=true` can fail segment load before reading that buffer.
Please guard the probe with `segmentDir.isDirectory()` and add a non-local
consolidated-entry regression test like the vector reader has.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
