RoanHeNaN opened a new pull request, #70600:
URL: https://github.com/apache/doris/pull/70600
### Background
A virtual slot (CSE common expression or ANN distance pushed into the OLAP
scan) is a ColumnNothing placeholder in the scanner's block, and
SegmentIterator materializes it in the block it is given. Storage Top-N
(VCollectIterator::_topn_next) needs key-ordered rows from each rowset, so
a rowset with overlapping segments is read through VMergeIterator, which
lets every segment fill its own block and copies the merged rows into the
caller's block.
### What problem does this PR solve?
Issue Number: close #70489
Related PR: #52701 (introduced virtual slots), #57243 (virtual slot init in
_topn_next)
PR Type: Bugfix
Problem Summary:
A storage Top-N query whose scan has a virtual slot fails with E-7412
(Bad cast from ColumnNothing to ColumnVector) when a rowset holds
overlapping segments. It affects DUP and MOW tables using CSE virtual
slots (`experimental_enable_virtual_slot_for_cse`) or ANN distance with
`ORDER BY key LIMIT n`. VMergeIteratorContext::copy_rows now replaces a
ColumnNothing destination with an empty column of the source type before
copying.
### Details
- **Frequency:** Every such query once a rowset has overlapping segments,
e.g. after one large load with auto compaction lagging behind.
- **Root cause:** copy_rows calls insert_range_from on the caller's
virtual slot, which is still the ColumnNothing placeholder. Its
insert_range_from only adds to the row count and drops the values, so
the following MutableBlock::add_rows in _topn_next copies ColumnNothing
into the real column of its Top-N accumulator and fails the cast.
- **Why existing tests missed it:** Existing virtual slot and ANN tests
load small data, so each rowset has one segment or non-overlapping
segments and is read through the union iterator, where SegmentIterator
writes into the caller's block directly.
- **Fix:** Before the first copy into a ColumnNothing destination, replace
it with the source column's clone_empty() and type. The source is the
column SegmentIterator materialized, so its type matches what
_topn_next allocates for its accumulator.
- **Other paths and branches:** VCollectIterator::Level1Iterator's
_merge_next(Block*) copies rows the same way, but DUP/MOW queries with
virtual slots do not reach it today (read_orderby_key is set together
with the Top-N limit). Not changed here. BlockView copies keep row
references to the source block and are not affected. The root cause,
ColumnNothing placeholders living outside SegmentIterator, is tracked in
#70599, which plans to remove this local handling. branch-4.0 and
branch-4.1 have the same copy_rows, _topn_next and placeholder code and
should take this fix; branch-4.0 uses the old be/src/vec/olap paths.
### Testing
- New regression test
query_p0/virtual_slot_ref/topn_virtual_slot_on_overlapping_segments
builds a single rowset with overlapping segments (small write buffer,
auto compaction disabled, shuffled keys) on DUP, MOW and ANN tables,
asserts the rowset is OVERLAPPING and the plan has a virtual column and
SORT LIMIT, and checks ASC/DESC Top-N results. It fails with E-7412
before the fix and passes after; the generated .out matches the results
with `experimental_enable_virtual_slot_for_cse=false` for all 6 queries.
- query_p0/virtual_slot_ref (7 suites) and ann_index_p0 small_segment,
ann_range_search, index_scan_opt pass.
- All run locally on a Release BE built from master; not run on
branch-4.0 or branch-4.1.
### Release note
Fix E-7412 errors for ORDER BY key LIMIT queries with pushed-down virtual
columns (common subexpressions or ANN distance) on tablets whose rowsets
contain overlapping segments.
### Check List (For Author)
- Behavior changed: No
- Does this need documentation: No
- Pre-submission status: Partially verified: no BE unit test; only run
locally, not in CI; backports not built.
### AI Disclosure
- Tool: Claude Code (Claude Opus 5.5)
- Generated: code, regression test, this description
- Human review: The author reviewed the root-cause analysis and chose the
fix approach.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]