lxy-9602 opened a new pull request, #546:
URL: https://github.com/apache/paimon-cpp/pull/546

   …-text search
   
   <!-- PR titles must follow Conventional Commits: <type>(<optional-scope>): 
<description> -->
   
   ### Purpose
   
   <!-- Linking this pull request to the issue -->
   Linked issue: #158 
   
   Enable precise index evaluation for real-time append and deduplicate reads, 
combining scalar predicates with vector and full-text search.
   
   Scalar predicates and pinned deletion visibility are applied before TopK, so 
filtered or deleted rows do not consume result slots. Queries read each segment 
within the intersection of its pinned bounds and the requested offset range, 
excluding committed prefixes and later writes.
   
   - Reuse predicate evaluation, selectivity ordering, search filtering, and 
score completion between disk and real-time readers.
   - Introduce shared index reader capabilities and optional segment-level 
real-time index sources.
   - Support offset selection and deletion filtering when materializing 
real-time results.
   - Validate the pinned store schema before reading.
   
   Real-time primary-key merge remains unsupported in precise mode. Lumina 
real-time search integration is deferred to a separate PR; search behavior is 
covered with mocks.
   
   <!-- What is the purpose of the change -->
   
   ### Tests
   
   Added and updated unit tests for predicate evaluation, search capabilities, 
score alignment, offset bounds, deletion visibility, and pinned-source lifetime.
   
   Added six integration tests covering:
   
   1. Exact scalar filtering before vector TopK, including unsupported and 
missing scalar indexes.
   2. FullTextSearch and mixed FullTextMatch/scalar/vector predicates with 
separate score columns.
   3. Reads across multiple disk files, sealed segments, and building segments.
   4. In-memory deduplication updates and deletes before TopK.
   5. Committed disk deletion vectors combined with real-time deletions across 
prepare, commit, and refresh.
   6. Pinned plans and readers retaining their offset bounds and deletion 
visibility across writes and refresh.
   
   <!--
   Describe tests added or updated and what they cover; explain if no test 
changes are needed.
   Omit execution commands, results, and reasons tests were not run locally.
   -->
   
   ### API and Format
   Public C++ API changes include:
   
   - `IndexReaderCapabilities`, shared by file and real-time index readers.
   - Real-time full-text/vector search methods and offset-based search result 
types.
   - Optional `RealtimeIndexedReadView` and `RealtimeIndexReadSource` 
interfaces.
   - Offset selection, offset deletions, and scalar-index pruning control in 
`RealtimeQueryContext`.
   
   The reader inheritance and query-context layout change. No storage format or 
serialized protocol changes.
   
   <!-- Does this change affect API in include dir or storage format or 
protocol -->
   
   ### Documentation
   Updated API comments for supported table modes, row-ID domains, pinned 
visibility, and search filtering requirements.
   
   <!-- Does this change introduce a new feature -->
   
   ### Generative AI tooling
   Generated-by: OpenAI Codex
   <!--
   If generative AI tooling has been used in the process of authoring this 
patch, please include the
   phrase: 'Generated-by: ' followed by the name of the tool and its version.
   If no, write 'No'.
   Please refer to the [ASF Generative Tooling 
Guidance](https://www.apache.org/legal/generative-tooling.html) for details.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to