lucasfang commented on code in PR #513:
URL: https://github.com/apache/paimon-cpp/pull/513#discussion_r4545114982


##########
src/paimon/format/orc/orc_file_batch_reader.cpp:
##########
@@ -211,10 +222,20 @@ Status OrcFileBatchReader::SeekToRow(uint64_t row_number) 
{
 }
 
 Result<std::vector<std::pair<uint64_t, uint64_t>>> 
OrcFileBatchReader::PreBufferRange() {
-    return reader_->PreBufferRange(target_column_ids_);
+    using ByteRanges = std::vector<std::pair<uint64_t, uint64_t>>;
+    std::pair<uint64_t, uint64_t> row_range;
+    PAIMON_ASSIGN_OR_RAISE(ByteRanges ranges,
+                           reader_->PreBufferRange(target_column_ids_, 
&row_range));
+    pre_buffered_row_range_ = row_range;
+    return ranges;
 }
 
 Result<BatchReader::ReadBatch> OrcFileBatchReader::NextBatch() {
+    uint64_t next_row = reader_->GetNextRowToRead();
+    if (read_ahead_ && (!pre_buffered_row_range_ || next_row < 
pre_buffered_row_range_->first ||
+                        next_row >= pre_buffered_row_range_->second)) {
+        PAIMON_RETURN_NOT_OK(read_ahead_->PrepareRanges(this));

Review Comment:
   Thanks for the detailed reproduction. You're right about both the seek 
ordering and the test's inability to prove that decoding consumes the cached 
bytes.
   
   In 5a70ca51, I removed the ORC per-stripe warmup optimization and its 
dedicated tests from this PR to keep the scope focused on decoupling RAW warmup 
and read-ahead caching from batch prefetch. ORC now retains its existing 
all-stripe `PreBufferRange()` implementation, with no new range-preparation 
hooks in `NextBatch()` or `SeekToRow()`. This removes the late per-stripe 
preparation path identified here. The existing cost of discovering all stripe 
ranges is documented; any future per-stripe optimization should cover 
seek-triggered I/O and verify physical reads/cache hits as you suggested.
   
   The real-ORC decoupling tests remain, covering synchronous reads, parallel 
prefetch, adaptive fallback, and late materialization. The five affected test 
targets passed locally (5,228 passed, 2 conditional skips).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to