lxy-9602 commented on code in PR #481:
URL: https://github.com/apache/paimon-cpp/pull/481#discussion_r4422248273
##########
src/paimon/common/reader/blob_fallback_batch_reader.cpp:
##########
@@ -101,24 +105,35 @@ BlobFallbackBatchReader::BlobFallbackBatchReader(
arrow_pool_(arrow_pool),
finished_reader_metrics_(std::make_shared<MetricsImpl>()) {}
+Result<BatchReader::ReadBatchWithBitmap>
BlobFallbackBatchReader::ReadSegmentBatch(
+ const Segment& segment, int32_t max_rows, bool read_payload) {
+ const std::optional<BlobBatchRequest> previous_request =
segment.blob_context->request;
+ segment.blob_context->request = BlobBatchRequest{max_rows, read_payload};
+ ScopeGuard reset_request([context = segment.blob_context,
previous_request]() {
+ context->request = previous_request;
+ });
+ return segment.reader->NextBatchWithBitmap();
Review Comment:
I explored both `Skip()` and bounded `NextBatch(max_rows)` APIs. They would
either require extending `FileBatchReader` and carefully implementing the
contract in wrappers such as `ApplyBitmapIndexBatchReader` and
`ApplyDeletionVectorBatchReader`, or require the fallback reader to access
`BlobFileBatchReader` directly, introducing a dependency on the format
implementation.
To keep this PR small ahead of the release, I’d prefer to retain the current
implementation and address broader API changes separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
