linrrzqqq opened a new pull request, #70777:
URL: https://github.com/apache/doris/pull/70777

   ### Background
   
   A `ColumnNullable` stores its values and its NULL flags in separate columns. 
An expression such as `NULLIF(k, 1)` can share the values column with `k`. For 
a row where `k = 1`, the expression reads as NULL while the query can still 
return the original `1`.
   
   ### What problem does this PR solve?
   
   Issue Number: None
   Related PR: None
   PR Type: Bugfix
   
   Consider a join on `NULLIF(l.k, 1) <=> NULLIF(r.k, 2)` that also selects 
`l.k, r.k`. The row with keys `1` and `2` should match and return `(1, 2)`. 
Preparing the probe key could overwrite the shared `1` with `0`, changing the 
projected value. Load conversion had the same ownership problem one level up: 
deduplicating a Map in the output could remove entries from the input Block 
because both Blocks referenced the same Map object.
   
   ### Details
   
   - **Frequency:** The write is visible when another key, output slot or Block 
still references the same column.
   - **Root cause:** Key preparation wrote through borrowed pointers. Filtering 
checked only the nested column's reference count, which can be one even when 
two Blocks share its Nullable parent. Load validation also replaced child 
pointers inside shared parent columns.
   - **Why existing tests missed it:** A filter can select the correct rows 
even after overwriting the source bytes under NULL. A load can produce the 
correct deduplicated Map even after modifying the input Map. Checking only the 
result misses both writes.
   - **Fix:**
     - For a single non-string probe key whose NULLs must match, obtain a 
writable key with `IColumn::mutate`, set its NULL positions to the default 
value, and keep it alive until the join finishes probing that input Block.
     - When packing fixed-width keys, write zero bytes for NULL directly into 
the packed key. If two fields share the same input, packing `(NULL, 7)` now 
preserves the second field's `7`.
     - Before combining a nullable filter's value bytes with its NULL flags, 
check `ColumnNullable::is_exclusive()`. If the column is shared, copy the value 
bytes and apply the mask to the copy.
     - Pass the destination `ColumnPtr` by reference during load validation so 
a converted child can be installed in the correct output field. Deduplicate Map 
keys on a cloned outer Map, rebuild an Array or copy a Struct when a child 
changes, and obtain writable decimal data before writing a rounded value.
   - **Other paths and branches:** The fixed-width key change also applies to 
grouping on multiple columns that share input data.
   
   ### Testing
   
   The BE and FE build and 67 ASAN unit tests passed. The tests check that 
packing keys preserves the source values, filtering preserves the source byte 
under NULL, and deduplicating a Map with entries `(7, 10), (7, 20)` keeps `(7, 
20)` in the output and both entries in the input.
   
   Both P0 suites passed locally. `test_hash_join_probe_side_zero_copy` checks 
the projected `(1, 2)` row in the join above. `test_string_len_complex` checks 
string length handling in nested columns during load.
   
   ### Release note
   
   Null-safe joins preserve the original values of projected join columns, and 
converting nested data for a load preserves values shared with other output 
fields.
   
   ### Check List (For Author)
   
   - Behavior changed: Yes (see Details).
   - Does this need documentation: No.
   - Pre-submission status: Build and tests passed. Clang-tidy reports function 
size and cognitive complexity in 
`OlapTableBlockConvertor::_internal_validate_column`.
   
   ### AI Disclosure
   
   - Tool: OpenAI Codex, GPT-6.
   - Generated: Implementation, tests and commit description.
   - Human review: Pending.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to