Mryange opened a new pull request, #70606:
URL: https://github.com/apache/doris/pull/70606

   ### Background
   
   Doris columns use copy-on-write ownership, and composite columns can retain 
immutable shared subcolumns. Filtering shared columns already creates an output 
containing the selected rows.
   
   ### What problem does this PR solve?
   
   Issue Number: None
   Related PR: None
   PR Type: Performance
   
   Problem Summary:
   
   Several function and writer paths eagerly copy data that is only read or 
immediately filtered. Share the null map for null-safe equality, share 
unchanged subcolumns during Iceberg ORC normalization, and filter shared Hive 
input columns without first cloning the whole input.
   
   ### Details
   
   - **Why it was slow:** Hive partition filtering copied every input row per 
partition before discarding unselected rows. Null-safe equality copied an 
unchanged null map, and Iceberg ORC normalization cloned unchanged null maps 
and offsets and detached unmodified children.
   - **Optimization:** Reuse existing immutable ColumnPtr ownership, shared 
composite constructors, and the existing COW-aware block filter.
   - **Results:** Source inspection confirms removal of the eager copy 
operations. No performance benchmark was run; no throughput or latency 
improvement is claimed.
   - **Applicability and cost:** Applies to the two all-null operand branches 
of null-safe equality, Hive partition filtering, and Iceberg ORC binary 
normalization. Later writes can still trigger normal COW detachment.
   
   ### Testing
   
   - ASAN BE build: `sh build.sh --be -j48` passed on a Linux development 
machine.
   - `build-support/clang-format.sh`, `build-support/check-format.sh` 
(clang-format 16), and `git diff --check` passed.
   - `build-support/run-clang-tidy.sh` passed for all three changed files, with 
no warnings on changed lines. Analysis used the build flags, exact source 
entries derived from Unity compilation where needed, and the matching toolchain 
resource directory.
   - No new BE unit tests were added. Unit tests, regression tests, and 
performance benchmarks were not run.
   
   ### Release note
   
   Reduce intermediate column copying in null-safe equality, Hive partition 
filtering, and Iceberg ORC normalization.
   
   ### Check List (For Author)
   
   - Behavior changed: No
   - Does this need documentation: No
   - Pre-submission status: Partially verified: ASAN BE build, formatting, and 
changed-line static analysis passed; runtime tests and performance benchmarks 
were not run.
   
   ### AI Disclosure
   
   - Tool: Codex (GPT-6)
   - Generated: Code changes and PR description.
   - Human review: The author discussed and approved the implementation 
approach; final code review is pending.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to