raghavyadav01 opened a new pull request, #16688:
URL: https://github.com/apache/pinot/pull/16688
This PR introduces a new configuration option to merge Lucene text index
files into consolidated column file. This enhancement reduces the number of
files generated during text index creation and enables the
LuceneTextIndexReader to operate directly on PinotBuffer.
**Key Changes**
- New Configuration Flag: Added useCombineFiles option in field
configuration
- File Consolidation: Lucene text index files are now merged into single
column files when enabled
- Buffer Integration: LuceneTextIndexReader now works seamlessly with
PinotBuffer
**Configuration**
To enable file combination, add the following to your field configuration:
{
"name": "your_text_column",
"encodingType": "DICTIONARY",
"indexTypes": ["TEXT"],
"properties": {
"useCombineFiles": "true"
}
}
**Testing**
- LuceneTextIndexCombinedTest - Tests file combination functionality
- LuceneTextIndexBufferReaderTest - Tests buffer-based reading
- TextSearchQueriesWithCombinedFilesTest - Integration tests with combined
files
**Backward Compatibility:**
- Changes are backward compatible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
