lxy-9602 opened a new pull request, #269: URL: https://github.com/apache/paimon-cpp/pull/269
<!-- PR titles must follow Conventional Commits: <type>(<optional-scope>): <description> --> ### Purpose <!-- Linking this pull request to the issue --> Linked issue: #158 <!-- What is the purpose of the change --> This PR moves real-time offset handling from `RealtimeStore` plugins into the Paimon read and write layers. The main changes are: - Remove `offset_begin` from `RealtimeStore::CreateQueryReaders`. Store plugins now return candidate rows without applying visibility offsets. - Remove `enable_predicate_pushdown` from `RealtimeQueryContext`. Callers decide whether a predicate should be passed to the plugin. - Add `_REALTIME_OFFSET` to append-mode transport batches before they are handed to the real-time store. - Add `RealtimeOffsetBatchReader` to filter rows by the visible half-open offset range and remove `_REALTIME_OFFSET` from query output. - Reuse `RealtimeOffsetBatchReader` and `KeyValueDataFileRecordReader` for primary-key real-time reads. - Generalize `KeyValueDataFileRecordReader` to accept a `BatchReader`. File row positions remain available when the wrapped reader is a `FileBatchReader`. - Resolve primary-key transport fields by name, making their field order insignificant and removing unnecessary upfront schema and offset coverage validation. - Simplify the built-in Arrow real-time store query reader to return candidate batches with an all-valid bitmap. ### Tests Added `RealtimeOffsetBatchReaderTest`. <!-- List UT and IT cases to verify this change --> ### API and Format This PR changes the public real-time store plugin API: - `RealtimeStore::CreateQueryReaders` no longer accepts `offset_begin`. - `RealtimeQueryContext` no longer contains `enable_predicate_pushdown`. - Append-mode transport batches now contain a leading `_REALTIME_OFFSET` field. - Primary-key transport fields are resolved by name rather than a fixed order. <!-- Does this change affect API in include dir or storage format or protocol --> ### Documentation <!-- Does this change introduce a new feature --> ### Generative AI tooling Generated-by: OpenAI Codex (GPT-5) <!-- If generative AI tooling has been used in the process of authoring this patch, please include the phrase: 'Generated-by: ' followed by the name of the tool and its version. If no, write 'No'. Please refer to the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) for details. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
