eldenmoon opened a new pull request, #64092:
URL: https://github.com/apache/doris/pull/64092
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Queries that evaluate complex VARIANT expressions after
local exchange can share input blocks across downstream pipeline tasks. Variant
casts and Variant serialization finalized the source column in-place, and
nested-loop join lazy materialization also reused mutable probe-side columns
while expanding rows. This could make local-shuffle query results unstable and
could trigger lazy nested-loop join nullable-column size mismatches. This
change finalizes private Variant copies for casts and serialization,
deep-clones finalized Variant columns, and copies the lazy probe block before
nested-loop join materialization appends rows.
### Release note
None
### Check List (For Author)
- Test:
- `PATH=/tmp:$PATH build-support/clang-format.sh
be/src/core/column/column_variant.cpp be/src/core/column/column_variant.h
be/src/core/data_type/data_type_variant.cpp
be/src/exec/operator/nested_loop_join_probe_operator.cpp
be/src/exec/operator/nested_loop_join_probe_operator.h
be/src/exprs/function/cast/cast_to_variant.h
be/test/core/column/column_variant_test.cpp
be/test/exprs/function/cast/function_variant_cast_test.cpp`
- `git diff --check upstream/master..HEAD`
- `./run-be-ut.sh --run
--filter=FunctionVariantCast.CastFromVariantDoesNotFinalizeSourceColumn:ColumnVariantTest.clone_finalized_deep_copies_columns:ColumnVariantTest.serialize_does_not_finalize_source_column:NLJAppendProbeDataWithNullTest.*`
- `./build.sh --be`
- Manual test: local 4BE repro query for nullable nested-loop join lazy
materialization, 8 workers x 80 loops, 0 failures and stable output hash
- Manual test: local 4BE RQG-shaped Variant/local-shuffle query, local
shuffle off baseline vs 50 local shuffle on runs, 0 mismatches
- Behavior changed: No
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
