Doris-Breakwater commented on issue #70687: URL: https://github.com/apache/doris/issues/70687#issuecomment-6945325040
Breakwater-GitHub-Analysis-Slot: slot_ba74eaf8ca14 **Initial assessment:** This warrants BE storage-iterator/CSE triage because the supplied trace shows a fatal column-type assertion in Storage Top-N. Source inspection at the exact reported commit, `4a3fce2a8fc1000b90f14793241aeeb574402e7e`, identifies a concrete placeholder-preservation path consistent with the crash. This is a static analysis; I have not reproduced the query or validated a fix. The issue currently has no labels or assignee. **What the code establishes** 1. A newly allocated scanner block initializes virtual slots as `ColumnNothing(0)`: [`ScannerContext::get_free_block`](https://github.com/apache/doris/blob/4a3fce2a8fc1000b90f14793241aeeb574402e7e/be/src/exec/scan/scanner_context.cpp#L299-L310) constructs the block from slot descriptors, and [`SlotDescriptor::get_empty_mutable_column`](https://github.com/apache/doris/blob/4a3fce2a8fc1000b90f14793241aeeb574402e7e/be/src/runtime/descriptors.cpp#L192-L197) supplies the placeholder. Clearing the block preserves that runtime column class. 2. Storage key Top-N requires ordered rowset output. [`BetaRowsetReader::is_merge_iterator`](https://github.com/apache/doris/blob/4a3fce2a8fc1000b90f14793241aeeb574402e7e/be/src/storage/rowset/beta_rowset_reader.h#L61-L64) selects the merge path when ordered output is required, the reader covers more than one segment, and the rowset is marked as having overlapping segments. Overlap across separate rowsets alone does not establish this condition. 3. SegmentIterator normally materializes virtual values before returning its output; see [`_materialization_of_virtual_column`](https://github.com/apache/doris/blob/4a3fce2a8fc1000b90f14793241aeeb574402e7e/be/src/storage/segment/segment_iterator.cpp#L3920-L3958). However, [`VMergeIteratorContext::copy_rows(Block*)`](https://github.com/apache/doris/blob/4a3fce2a8fc1000b90f14793241aeeb574402e7e/be/src/storage/iterator/vgeneric_iterators.cpp#L130-L156) inserts those source values into the caller's existing destination columns without replacing placeholder destinations. `ColumnNothing` inherits [`IColumnDummy::insert_range_from`](https://github.com/apache/doris/blob/4a3fce2a8fc1000b90f14793241aeeb574402e7e/be/src/core/column/column_dummy.h#L81-L85), which only increments the size and ignores the source values. Thus, even materialized segment output can become a nonempty `ColumnNothing` in merge output. 4. [`VCollectIterator::_topn_next`](https://github.com/apache/doris/blob/4a3fce2a8fc1000b90f14793241aeeb574402e7e/be/src/storage/iterator/vcollect_iterator.cpp#L294-L305) already creates concrete virtual columns for its candidate accumulator. After reading the rowset, it calls `mutable_block.add_rows` at line 396. A placeholder in the source then reaches [`ColumnVector::insert_range_from`](https://github.com/apache/doris/blob/4a3fce2a8fc1000b90f14793241aeeb574402e7e/be/src/core/column/column_vector.cpp#L360-L373), whose checked cast matches the supplied crash stack. This suggests a more specific failure mechanism than a segment simply returning an unmaterialized expression: **the merge destination can preserve the placeholder and discard already materialized values, with the fatal cast occurring later in Top-N**. The posted stack identifies Top-N's copy as the abort site; it does not itself show a cast inside VMergeIterator or identify the source runtime type. **Information needed to confirm the reported execution** - Minimal `CREATE TABLE`, failing SQL, and data/load steps. Include the key model, sort/cluster keys, nullability, `ORDER BY`/`LIMIT`, and the repeated expression that becomes a virtual slot. - `EXPLAIN VERBOSE` and relevant session settings, especially `experimental_enable_virtual_slot_for_cse` and `enable_segment_limit_pushdown`; confirm both FE and BE build commits and any local patches. - The BE fatal log immediately preceding the stack, especially `Bad cast from type:... to ...`. The supplied stack identifies the destination class but omits the source class. - The actual affected tablet/rowset and segment metadata showing multiple overlapping segments in the same reader. The stack header's `tablet id: 0` is insufficient to identify that rowset. Explain how loads/flushes and compaction produce or preserve the layout, plus reproduction frequency. **Recommended next steps** - In an isolated reproduction, capture the virtual slot ordinal, declared datatype, runtime column class, and size at segment output, immediately before/after merge `copy_rows`, and before Top-N `add_rows`. A concrete source becoming a `Nothing` merge destination would directly confirm this path. - For a session-scoped isolation/workaround trial, use `SET experimental_enable_virtual_slot_for_cse = false;` and re-plan the query, removing any query hint that overrides it. At this commit, [`the CSE pushdown rule is explicitly gated by that setting`](https://github.com/apache/doris/blob/4a3fce2a8fc1000b90f14793241aeeb574402e7e/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushDownVirtualColumnsIntoOlapScan.java#L160-L186). Verify the virtual slots disappear and compare results. This bypasses the identified CSE entry condition; it is not a tested fix for this report. - A PR should establish concrete, correctly typed merge destinations before copying materialized virtual values, including initial allocation and block reuse. Normalizing only the Top-N accumulator is already present and does not address this path. Replacing a nonempty placeholder with default values afterward would lose expression results. - Add a merge-iterator regression with materialized child virtual columns and a placeholder caller destination, checking actual values and runtime types. Add SQL coverage that reliably produces overlapping segments in one rowset, exercises Storage Top-N with CSE enabled, and checks results against CSE disabled. Cover ascending/descending reads, nullable/constant expressions, block reuse, and final batches returned with EOF. A no-crash assertion alone would miss the value loss. The source path is a strong root-cause candidate. Confirming that it caused this particular abort still requires the missing runtime evidence and reproduction; no affected release range or existing fixed version is established here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
