lszskye opened a new pull request, #490: URL: https://github.com/apache/paimon-cpp/pull/490
### Purpose Implement precise file-index evaluation for mixed scalar and full-text predicates, with optional vector or full-text TopK applied after filtering. - Add `EnablePreciseFileIndexEvaluate` for append-only reads and primary-key reads with deletion vectors. - Evaluate AND/OR predicate trees and use available selectivity estimates to order AND children. - Recheck scalar index candidates against the data. - Apply predicates, deletion vectors and row ranges before root TopK, then push the final row selection into the data reader. - Support multiple named score columns and keep scores aligned with selected rows. - Reuse the index container through an instance-based `FileIndexEvaluator`. Precise evaluation requires bitmap32-compatible file row positions and rejects bitmap64 deletion vectors and merge reads. Full-text and vector searches require suitable indexes; only scalar predicates support evaluation without an index. TopK remains per file. ### Tests read_context_test.cpp empty_file_index_reader_test.cpp file_index_reader_test.cpp complete_index_score_file_batch_reader_test.cpp file_index_evaluator_test.cpp file_index_read_executor_test.cpp internal_read_context_test.cpp raw_file_split_read_test.cpp table_read_test.cpp predicate_abi_inte_test.cpp read_inte_with_index_test.cpp write_and_read_inte_test.cpp ### API and Format - Add `ReadContextBuilder::EnablePreciseFileIndexEvaluate`, defaulting to false, and carry the setting through `ReadContext`. - Require precise evaluation and `file-index.read.enabled=true` for file-level vector/full-text searches and FullTextMatch predicates. - Require explicit `_<field>_INDEX_SCORE` projections in precise mode; reject `_INDEX_SCORE`. - Extend `FileIndexReader` with search capability checks and selectivity estimates. Full-text TopK returns scored results, and FullTextMatch supports optional scores. - Add `FileIndexer::CreateEmptyReader` to preserve search capabilities for empty indexes. - Handle FullTextMatch through `FileIndexReader` rather than the generic scalar `FunctionVisitor`. ### Documentation Update the read API comments and `docs/source/api/file_index.rst` with the precise evaluation option, search requirements and named score projection examples. ### Generative AI tooling Generated-by: OpenAI Codex (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
