RoanHeNaN opened a new pull request, #70600:
URL: https://github.com/apache/doris/pull/70600

   ### Background
   
   A virtual slot (CSE common expression or ANN distance pushed into the OLAP
   scan) is a ColumnNothing placeholder in the scanner's block, and
   SegmentIterator materializes it in the block it is given. Storage Top-N
   (VCollectIterator::_topn_next) needs key-ordered rows from each rowset, so
   a rowset with overlapping segments is read through VMergeIterator, which
   lets every segment fill its own block and copies the merged rows into the
   caller's block.
   
   ### What problem does this PR solve?
   
   Issue Number: close #70489
   Related PR: #52701 (introduced virtual slots), #57243 (virtual slot init in 
_topn_next)
   PR Type: Bugfix
   
   Problem Summary:
   
   A storage Top-N query whose scan has a virtual slot fails with E-7412
   (Bad cast from ColumnNothing to ColumnVector) when a rowset holds
   overlapping segments. It affects DUP and MOW tables using CSE virtual
   slots (`experimental_enable_virtual_slot_for_cse`) or ANN distance with
   `ORDER BY key LIMIT n`. VMergeIteratorContext::copy_rows now replaces a
   ColumnNothing destination with an empty column of the source type before
   copying.
   
   ### Details
   
   - **Frequency:** Every such query once a rowset has overlapping segments,
     e.g. after one large load with auto compaction lagging behind.
   - **Root cause:** copy_rows calls insert_range_from on the caller's
     virtual slot, which is still the ColumnNothing placeholder. Its
     insert_range_from only adds to the row count and drops the values, so
     the following MutableBlock::add_rows in _topn_next copies ColumnNothing
     into the real column of its Top-N accumulator and fails the cast.
   - **Why existing tests missed it:** Existing virtual slot and ANN tests
     load small data, so each rowset has one segment or non-overlapping
     segments and is read through the union iterator, where SegmentIterator
     writes into the caller's block directly.
   - **Fix:** Before the first copy into a ColumnNothing destination, replace
     it with the source column's clone_empty() and type. The source is the
     column SegmentIterator materialized, so its type matches what
     _topn_next allocates for its accumulator.
   - **Other paths and branches:** VCollectIterator::Level1Iterator's
     _merge_next(Block*) copies rows the same way, but DUP/MOW queries with
     virtual slots do not reach it today (read_orderby_key is set together
     with the Top-N limit). Not changed here. BlockView copies keep row
     references to the source block and are not affected. The root cause,
     ColumnNothing placeholders living outside SegmentIterator, is tracked in
     #70599, which plans to remove this local handling. branch-4.0 and
     branch-4.1 have the same copy_rows, _topn_next and placeholder code and
     should take this fix; branch-4.0 uses the old be/src/vec/olap paths.
   
   ### Testing
   
   - New regression test
     query_p0/virtual_slot_ref/topn_virtual_slot_on_overlapping_segments
     builds a single rowset with overlapping segments (small write buffer,
     auto compaction disabled, shuffled keys) on DUP, MOW and ANN tables,
     asserts the rowset is OVERLAPPING and the plan has a virtual column and
     SORT LIMIT, and checks ASC/DESC Top-N results. It fails with E-7412
     before the fix and passes after; the generated .out matches the results
     with `experimental_enable_virtual_slot_for_cse=false` for all 6 queries.
   - query_p0/virtual_slot_ref (7 suites) and ann_index_p0 small_segment,
     ann_range_search, index_scan_opt pass.
   - All run locally on a Release BE built from master; not run on
     branch-4.0 or branch-4.1.
   
   ### Release note
   
   Fix E-7412 errors for ORDER BY key LIMIT queries with pushed-down virtual
   columns (common subexpressions or ANN distance) on tablets whose rowsets
   contain overlapping segments.
   
   ### Check List (For Author)
   
   - Behavior changed: No
   - Does this need documentation: No
   - Pre-submission status: Partially verified: no BE unit test; only run 
locally, not in CI; backports not built.
   
   ### AI Disclosure
   
   - Tool: Claude Code (Claude Opus 5.5)
   - Generated: code, regression test, this description
   - Human review: The author reviewed the root-cause analysis and chose the 
fix approach.
   
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to