This is an automated email from the ASF dual-hosted git repository. mrhhsg pushed a commit to branch adaptive_batch_size in repository https://gitbox.apache.org/repos/asf/doris.git
commit 90823c029115905f34cb07865972182731a49624 Author: Hu Shenggang <[email protected]> AuthorDate: Wed Apr 8 22:55:01 2026 +0800 [fix](be) Fix ASAN heap-buffer-overflow in segment iterator with adaptive batch size ### What problem does this PR solve? Issue Number: close #xxx Problem Summary: When the adaptive batch size predictor reduces `block_row_max` before the first `_lazy_init` call, columns are reserved for the reduced prediction (e.g., 256 rows). On subsequent batches, if the predictor increases `block_row_max` (e.g., to 4096), the segment iterator tries to read more rows than the column capacity, causing a heap-buffer-overflow in `ColumnDictI32::insert_many_dict_data`. The fix uses `_initial_block_row_max` (the original unreduced ceiling) for column reservation in `_lazy_init`, ensuring columns always have enough capacity for the maximum possible batch size. ### Release note None ### Check List (For Author) - Test: Unit Test (38 AdaptiveBlockSizePredictor + 29 BlockBudget/ByteBudget tests pass) - Behavior changed: No - Does this need documentation: No Co-authored-by: Copilot <[email protected]> --- be/src/storage/segment/segment_iterator.cpp | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/be/src/storage/segment/segment_iterator.cpp b/be/src/storage/segment/segment_iterator.cpp index aa895009203..92a941ab4c3 100644 --- a/be/src/storage/segment/segment_iterator.cpp +++ b/be/src/storage/segment/segment_iterator.cpp @@ -522,10 +522,14 @@ Status SegmentIterator::_lazy_init(Block* block) { _range_iter.reset(new BitmapRangeIterator(_row_bitmap)); } - // If the row bitmap size is smaller than block_row_max, there's no need to reserve that many column rows. - auto nrows_reserve_limit = std::min(_row_bitmap.cardinality(), uint64_t(_opts.block_row_max)); + // Reserve columns for _initial_block_row_max (the original max before any adaptive + // prediction) because the predictor may increase block_row_max on subsequent batches + // up to this ceiling. Using the current (possibly reduced) _opts.block_row_max would + // cause heap-buffer-overflow if a later prediction is larger. + auto nrows_reserve_limit = + std::min(_row_bitmap.cardinality(), uint64_t(_initial_block_row_max)); if (_lazy_materialization_read || _opts.record_rowids || _is_need_expr_eval) { - _block_rowids.resize(_opts.block_row_max); + _block_rowids.resize(_initial_block_row_max); } _current_return_columns.resize(_schema->columns().size()); --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
