gripleaf commented on code in PR #323:
URL: https://github.com/apache/paimon-cpp/pull/323#discussion_r4002798219
##########
src/paimon/format/avro/avro_file_batch_reader.cpp:
##########
@@ -180,6 +235,24 @@ Status AvroFileBatchReader::SetReadSchema(::ArrowSchema*
read_schema,
reader_ = std::move(reader);
array_builder_ = std::move(array_builder);
decode_context_.ClearBuilderMetadata();
+ selection_iterator_.reset();
+ selection_end_.reset();
+ selection_bitmap_ = selection_bitmap;
+ if (!block_index_complete_) {
+ block_index_.clear();
+ block_index_disabled_ = false;
+ }
+ if (selection_bitmap_ && !selection_bitmap_->IsEmpty() &&
block_index_complete_ &&
+ !block_index_.empty()) {
+ selection_iterator_ = selection_bitmap_->Begin();
+ selection_end_ = selection_bitmap_->End();
+ const uint64_t selected_row =
static_cast<uint32_t>(**selection_iterator_);
+ auto block =
+ std::upper_bound(block_index_.begin(), block_index_.end(),
selected_row,
+ [](uint64_t row, const auto& entry) { return row
< entry.first; });
+ selected_block_ = std::distance(block_index_.begin(), block) - 1;
+ }
+ previous_row_ids_.clear();
Review Comment:
Thanks for the review. I’ve refactored the bitmap support introduced by this
PR:
- Bitmap selection works on the first read, without requiring a preceding
full scan. AdvanceToRow() skips records sequentially when no completed block
index is available; an existing index only accelerates positioning.
- SelectionCursor owns the bitmap and its iterators, replacing repeated
Contains() checks with sequential iteration and keeping their lifetimes
together.
- BlockIndex encapsulates block positions and explicit Building / Ready /
Disabled states. This separates index lifecycle management from the current
selection and projection.
- The batch loop stays directly in NextBatch(). Positioning, row
preparation, and decoding/skipping are handled by AdvanceToRow(),
PrepareNextRow(), and ReadCurrentRow(), respectively. The separate
ReadRowsIntoBuilder() wrapper has been removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
