RoanHeNaN opened a new issue, #70599: URL: https://github.com/apache/doris/issues/70599
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description A virtual slot (a common subexpression pushed into the OLAP scan by `experimental_enable_virtual_slot_for_cse`, an ANN distance, `score()`, or a MATCH projection) starts its life as a `ColumnNothing` placeholder. The placeholder is created far away from `SegmentIterator`, the only component that materializes virtual slots, and it travels through the scanner, the tablet reader, the collect/merge iterators and the point-query path. Each of these components has to know that a column may be a placeholder. When one of them does not, the result is either an error such as E-7412 (#70489) or, in the point-query case below, a silently wrong result. This issue describes the current lifecycle, the problems it causes, and a refactor that confines `ColumnNothing` to `SegmentIterator`. Code references are at ee73acc6131e6c6c752eb59e252404590be6ad77. #### 1. Where the placeholder is created | Creator | Users | Virtual slot column | |---|---|---| | `SlotDescriptor::get_empty_mutable_column()` ([descriptors.cpp:192](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/runtime/descriptors.cpp#L192)) | `Block(slots)`; scanner free blocks ([scanner_context.cpp:309](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/exec/scan/scanner_context.cpp#L309)); `Scanner::get_block` ([scanner.cpp:181](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/exec/scan/scanner.cpp#L181)); point-query block pool ([point_query_executor.cpp:158](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/service/point_query_executor.cpp#L158)) | `ColumnNothing` when `get_virtual_column_expr() != nullptr` | | `ReadSchema::create_read_block()` ([schema.cpp:285](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/storage/schema.cpp#L285)) | merge-iterator context blocks, `Level0Iterator` blocks | `ColumnNothing` for `__DORIS_VIRTUAL_COL__*` | FE sets a virtual column expression only on the OLAP scan output tuple (`PhysicalPlanTranslator.computePhysicalOlapScan`). Callers of `get_empty_mutable_column()` that use other tuples (load, meta/schema scans, TVFs, projection outputs, materialization) never see a placeholder. #### 2. How each path handles it today `ColumnNothing` is harmless as long as it is either filled by `SegmentIterator` or read through `VirtualSlotRef`, which computes the expression on the spot when it sees a placeholder ([virtual_slot_ref.cpp:123](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/exprs/virtual_slot_ref.cpp#L123)). It breaks when code copies it like a normal column: `ColumnNothing::insert_range_from` / `insert_default` only bump a row count and drop the values. | Path | What happens to the virtual slot | Status | |---|---|---| | Plain scan (union iterator) | The caller's block is passed down unchanged to `SegmentIterator::next_batch`, which resets the slot to `ColumnNothing` ([`_init_virtual_columns`](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/storage/segment/segment_iterator.cpp#L3911)), then replaces it in [`_output_columns_to_block`](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/storage/segment/segment_iterator.cpp#L2682) / [`_materialization_of_virtual_column`](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/storage/segment/segment_iterator.cpp#L3920) | OK | | Storage Top-N, rowset with overlapping segments | `VMergeIterator` lets each segment fill its own context block, then [`copy_rows`](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/storage/iterator/vgeneric_iterators.cpp#L133) copies into the caller's block whose slot is still `ColumnNothing`; values are dropped and `_topn_next`'s `add_rows` fails | #70489, fixed by patching `copy_rows` | | Storage Top-N accumulator | `_topn_next` swaps real columns into its `clone_block` by hand ([vcollect_iterator.cpp:297](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/storage/iterator/vcollect_iterator.cpp#L297)) | Workaround | | Cross-rowset merge, `Level1Iterator::_merge_next(Block*)` ([vcollect_iterator.cpp:883](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/storage/iterator/vcollect_iterator.cpp#L883)) | Same `insert_range_from` pattern as `copy_rows`; not reached by DUP/MOW queries with virtual slots today | Latent | | `SegmentIterator` at EOF | `next_batch` resets the caller's slot to `ColumnNothing` on entry, and both EOF returns in `_next_batch_internal` reach [`_process_eof`](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/storage/segment/segment_iterator.cpp#L3434) before any materialization (the slot in `_current_columns` is also `ColumnNothing`, [:2675](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/storage/segment/segment_iterator.cpp#L2675)), so the caller's block keeps a placeholder after EOF | Leaks out | | Scanner padding / projection | `_merge_padding_block` only merges blocks with rows ([scanner.h:135](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/exec/scan/scanner.h#L135)), which are already materialized | OK | | Scanner output | `_make_sure_virtual_col_is_materialized` ([scanner_scheduler.cpp:366](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/exec/scan/scanner_scheduler.cpp#L366)) throws if a non-empty block still has `ColumnNothing` | Safety net | | Block self-checks | `Block` / `ColumnWithTypeAndName` skip `ColumnNothing` ([block.cpp:334](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/core/block/block.cpp#L334), [column_with_type_and_name.cpp:170](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/core/block/column_with_type_and_name.cpp#L170)) | Type of virtual slots is never checked | | Short-circuit point query | See section 4 | Depends on `ColumnNothing` for correctness | #### 3. Problems 1. **The placeholder lives too long.** It is created by generic block builders, survives `clear_column_data()`, and is put back by `SegmentIterator` at the start of every `next_batch` and left behind at EOF. Whether a caller's block holds a real column or a placeholder depends on what happened to that block before. For example, in `_topn_next` the same read buffer is reused across rowsets: after a non-overlapping rowset is read to EOF the slot is `ColumnNothing` again, so the next overlapping rowset meets a placeholder regardless of how the block was created. 2. **Every component that copies rows must special-case it.** `_topn_next`, `VMergeIteratorContext::copy_rows` (after #70489) and, latently, `Level1Iterator::_merge_next` each handle it locally. A new copy path that forgets it fails with E-7412 or drops values. 3. **`ColumnNothing` doubles as an "unmaterialized" flag outside `SegmentIterator`.** `VirtualSlotRef` recomputes the expression whenever it sees a placeholder. Code outside the storage layer, like the point query, relies on this implicitly. 4. **Virtual-slot types are never validated.** Because self-checks skip `ColumnNothing`, a mismatch between the slot type and the expression type (for example nullability, see #55694) only shows up deep inside the scan. #### 4. Short-circuit point query relies on the placeholder When CSE pushes a virtual column into the scan, the short-circuit point query reuses the scan tuple, so its blocks contain the virtual slot. It never runs `SegmentIterator`: 1. `Reusable::init` builds the block pool with `Block::create_unique(tuple_desc()->slots(), 2)` ([point_query_executor.cpp:158](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/service/point_query_executor.cpp#L158)); the virtual slot is `ColumnNothing`. `extract_slot_ref` only collects `SLOT_REF` nodes, so the virtual slot is not treated as a missing column. 2. The row is decoded from the row store with `JsonbSerializeUtil::jsonb_to_columns` ([point_query_executor.cpp:600](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/service/point_query_executor.cpp#L600)). The virtual column is not in the row store, so it is filled with `insert_default()` ([serialize.cpp:148](https://github.com/apache/doris/blob/ee73acc6131e6c6c752eb59e252404590be6ad77/be/src/util/jsonb/serialize.cpp#L148)). On `ColumnNothing` this only adds a row. 3. The output exprs read the slot through `VirtualSlotRef`, which sees `ColumnNothing` and computes the expression from the decoded columns. If the placeholder is simply removed from `get_empty_mutable_column()`, step 2 writes a real default value and step 3 returns it. This was verified locally: ```sql CREATE TABLE pq (id INT NOT NULL, v INT NOT NULL) UNIQUE KEY(id) DISTRIBUTED BY HASH(id) BUCKETS 1 PROPERTIES ("replication_num" = "1", "enable_unique_key_merge_on_write" = "true", "store_row_column" = "true"); INSERT INTO pq VALUES (1, 10), (5, 700), (9, 3); SET enable_sql_cache = false; SET experimental_enable_virtual_slot_for_cse = true; -- explain shows SHORT-CIRCUIT and a __DORIS_VIRTUAL_COL__ slot; -- each run increments the BE tablet_fetch_data RPC counter. SELECT abs(v - 500) * 3 + 1 AS a, (abs(v - 500) * 3 + 1) * 2 AS b FROM pq WHERE id = 5; ``` | BE | id = 1 | id = 5 | id = 9 | |---|---|---|---| | CSE off | `1471 2942` | `601 1202` | `1492 2984` | | CSE on, current code | `1471 2942` | `601 1202` | `1492 2984` | | CSE on, `get_empty_mutable_column()` always returns a real column | **`0 0`** | **`0 0`** | **`0 0`** | No error is raised. With the same experimental BE all suites under `regression-test/suites/query_p0/virtual_slot_ref` still pass, so existing tests do not cover this path. ### Solution Make `ColumnNothing` an implementation detail of `SegmentIterator`: every block outside it holds real, correctly typed columns for virtual slots. 1. **Creators return real columns.** `SlotDescriptor::get_empty_mutable_column()` and `ReadSchema::create_read_block()` create `type()->create_column()` for virtual slots. Use one type source for the virtual slot and assert at scan/expr preparation that the slot type equals `virtual_column_expr->data_type()`, so mismatches fail early instead of deep in the scan. 2. **`SegmentIterator` owns the placeholder for exactly one `next_batch` call.** It decides which ordinals are virtual from its read schema / `virtual_column_exprs`, may still put `ColumnNothing` into the block on entry (in-iterator predicate evaluation through `VirtualSlotRef` relies on it), and guarantees that every exit leaves a real column of the slot type: normal output (already true), zero selected rows (already true), **EOF in `_process_eof`**, and **error returns** (for example with a scope guard), because scanner blocks are recycled. 3. **Point query materializes virtual slots explicitly.** Skip virtual slots in `jsonb_to_columns` (no default fill), and after decoding evaluate each virtual column expression over the decoded block and write the result back before running the output exprs. Alternatively keep a local placeholder that never leaves the executor, but explicit materialization keeps the contract the same everywhere. 4. **Remove the local workarounds.** Drop the virtual-slot initialization of `clone_block` in `_topn_next` and the `ColumnNothing` handling added to `VMergeIteratorContext::copy_rows` for #70489, replacing them with `DCHECK`s that no `ColumnNothing` reaches a copy. `Level1Iterator::_merge_next` then needs no change. 5. **Let self-checks cover virtual slots.** Once nothing outside `SegmentIterator` holds a placeholder, `Block` / `ColumnWithTypeAndName` checks validate virtual-slot columns like any other; `_make_sure_virtual_col_is_materialized` can stay as an assertion. Tests to add before the refactor (all pass on the current code, so they guard behavior): - Short-circuit point query with a CSE virtual column on a row-store MOW table (the case above). This is the one that silently breaks if step 3 is missed. - Storage Top-N over a tablet with a non-overlapping rowset that is read to EOF followed by a rowset with overlapping segments (exercises the EOF leak in problem 1). - Storage Top-N with lazy materialization (`rowid_fetcher`) and a virtual column. By reading FE code (`LazyMaterializeTopN` only defers slots with an original column) virtual slots should never be fetched lazily, but this has not been verified by a query. - A BE unit test that makes `SegmentIterator::next_batch` return an error and EOF and asserts the block holds no `ColumnNothing`. Related: #70489 (bug caused by the long-lived placeholder), #70381 (virtual slots recomputed per reference, also a consequence of `VirtualSlotRef` treating `ColumnNothing` as "compute now"), #55694 (virtual slot nullability mismatch). ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
