xiangfu0 opened a new pull request, #18919:
URL: https://github.com/apache/pinot/pull/18919
## Problem
With `storeInSegmentFile=true`, `TextIndexType.ReaderFactory` probed the
consolidated segment-file entry (`segmentReader.getIndexFor(column,
StandardIndexes.text())`) unconditionally.
On a V1/V2 segment backed by `FilePerIndexDirectory` whose column still has
the legacy Lucene text-index **directory** on disk, `getFileFor` resolves that
directory (`File.exists()` is true for directories) and `mapForReads` rejects
it with `IllegalArgumentException: File: ... must be a regular file` — killing
the whole segment load before any fallback can run. This makes flipping
`storeInSegmentFile=true` non-rolling-upgrade-safe for existing V1/V2 text
segments.
`TextIndexHandler` cannot rescue these segments either: its combined-format
conversion is v3-gated, so the reader factory must handle them.
This is the text-index twin of the vector index bug fixed in #18852.
## Fix
Legacy-directory-first gate, same pattern as #18852: if
`SegmentDirectoryPaths.findTextIndexIndexFile` finds a Lucene **directory** on
disk (an unmigrated V3 segment, or any V1/V2 segment), construct the
directory-based `LuceneTextIndexReader` directly and skip the consolidated
probe entirely. The probe only runs when no legacy directory exists.
## Tests
New `TextIndexTypeTest` mirrors the vector regression tests:
- `testReaderFactoryUsesLegacyTextDirectoryWithoutProbingConsolidatedEntry`
— builds a real legacy Lucene text index in the V1 root layout, stubs
`getIndexFor` to throw `IllegalArgumentException("... must be a regular
file")`, asserts the reader is constructed, serves queries, and the probe never
runs.
- `testReaderFactoryUsesLegacyTextDirectoryInV3LayoutWithoutProbing` — same
for the `v3/` subdirectory layout.
Both tests fail without the fix (verified by stashing the main change) and
pass with it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
