lxy-9602 commented on code in PR #481:
URL: https://github.com/apache/paimon-cpp/pull/481#discussion_r4421890619
##########
src/paimon/common/reader/blob_fallback_batch_reader.cpp:
##########
@@ -101,24 +105,35 @@ BlobFallbackBatchReader::BlobFallbackBatchReader(
arrow_pool_(arrow_pool),
finished_reader_metrics_(std::make_shared<MetricsImpl>()) {}
+Result<BatchReader::ReadBatchWithBitmap>
BlobFallbackBatchReader::ReadSegmentBatch(
+ const Segment& segment, int32_t max_rows, bool read_payload) {
+ const std::optional<BlobBatchRequest> previous_request =
segment.blob_context->request;
+ segment.blob_context->request = BlobBatchRequest{max_rows, read_payload};
+ ScopeGuard reset_request([context = segment.blob_context,
previous_request]() {
+ context->request = previous_request;
+ });
+ return segment.reader->NextBatchWithBitmap();
Review Comment:
Thanks for the review. This optimization targets large Blob values, where
avoiding payload I/O and decoding should generally outweigh per-call overhead.
Java also resolves fallback row by row using `next()` and `skip()`.
I agree descriptor mode deserves separate measurement. For this PR, I’d keep
read/skip semantics: merging skip runs into reads would access covered payloads
again and could expose stale-data errors. Batching improvements can follow
separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
