eldenmoon opened a new pull request, #66674: URL: https://github.com/apache/doris/pull/66674
## What problem does this PR solve? Variant V2 execution rebuilt shredded columns through encoded rows and repeatedly copied typed children when layouts changed across batches or segments. This made selection, union, assembly, and INSERT INTO SELECT pay avoidable row-wise conversion and copy costs. Related PR: #66204 ## What is changed? - Preserve a bounded block-local shredded layout across range/indices selection and multi-segment merge. - Transfer physical typed children when ownership is unique, otherwise bulk-gather with `insert_indices_from`; only conflicting rows enter the encoded residual lane. - Reuse exact typed leaves and merge presence/null maps, including alternating layouts and frozen-layout filter/permute cases. - Fuse consecutive constant Variant path segments and avoid materializing exact typed leaves. - Let the native Variant writer consume shredded columns directly, covering full and partial materialization in INSERT INTO SELECT. - Use the trusted internal shredded builder on validated critical paths and retain explicit validation in tests. - Add copy/layout metrics and high-cardinality layout bounds. ## Test - 103/103 focused ASAN BE unit tests passed on the final commit. - `variant_native_shredded_writer_v2`: passed with `enable_variant_v2=true`, `use_fuzzy_session_variable=true`, and `fuzzy_test_type=p0`. - Full `variant_p0`: 166/167 suites passed under the same V2/fuzzy settings. The only failure was the external OSS case `outfile_csv_variant_doc_value` with `InvalidAccessKeyId`. - `build-support/check-format.sh`: passed with clang-format 16. - Full ASAN BE build: passed. ## Notes - `run-clang-tidy.sh` was attempted but blocked by the local toolchain/pre-existing diagnostics (`stddef.h` not found and an unmatched `NOLINTEND` in `be/src/core/types.h`). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
