BiteTheDDDDt opened a new pull request, #65300:
URL: https://github.com/apache/doris/pull/65300
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
Hash join probe-side output may still copy projected probe columns even when
the output rows map one-to-one to the original probe block. In that
ALL_MATCH_ONE case, `_probe_indexs` covers the whole probe block in order, so
copying probe-side columns through `insert_range_from()` is unnecessary.
This PR adds a zero-copy fast path for eligible probe-side output columns:
- Detect batches where `_probe_indexs` is `0..probe_block.rows()-1` and
output row count equals the probe block row count.
- Keep the mutable output block shape by mocking eligible probe-side columns
during output construction.
- After materializing the output `Block`, replace eligible probe-side output
columns with the original probe block columns.
- Limit the fast path to join types where the probe-side output type is
unchanged: inner join, left outer join, left semi join, left anti join, ASOF
left inner join, and ASOF left outer join.
- Keep conservative guards for cases that can change row filtering or column
materialization: other join conjuncts, mark join conjuncts, local conjuncts,
variant finalization, right/full joins, and null-aware joins.
Local TPC-DS SF100 q47 profile data on a single-BE release build:
Query settings:
```sql
set enable_profile=true;
set profile_level=2;
set disable_join_reorder=false;
```
End-to-end elapsed time is noisy and did not show a stable query-level
improvement because build-side output and projection still dominate and
pipeline overlap hides part of the local CPU saving.
| Run | Query/Profile id | q47 elapsed | Output check |
| --- | --- | ---: | --- |
| Baseline run 1 | N/A | 7.31s | baseline output |
| Baseline run 2 | `a3e69b5226774770-a08c4bf2539526ce` | 6.42s | baseline
output |
| After this PR | `dfeb8b9ab13546a5-820c7ea81e3cbe23` | 6.93s | identical to
baseline |
Focused hash join counters show the intended reduction in probe-side output
time:
| Operator | Counter | Baseline | After this PR | Change |
| --- | --- | ---: | ---: | ---: |
| HASH_JOIN_OPERATOR id=9 | `ProbeWhenProbeSideOutputTime` | 481.862ms |
52.714ms | -89.1% |
| HASH_JOIN_OPERATOR id=9 | `ProbeWhenBuildSideOutputTime` | 1sec779ms |
1sec858ms | noise / unchanged |
| HASH_JOIN_OPERATOR id=9 | `ProbeWhenSearchHashTableTime` | 229.461ms |
232.211ms | noise / unchanged |
| HASH_JOIN_OPERATOR id=9 | `ProjectionTime` | 667.907ms | 820.823ms | noise
|
| HASH_JOIN_OPERATOR id=9 | `ExecTime` | 3sec435ms | 3sec228ms | -6.0% |
| HASH_JOIN_OPERATOR id=7 | `ProbeWhenProbeSideOutputTime` | 275.201ms |
41.642ms | -84.9% |
| HASH_JOIN_OPERATOR id=7 | `ProbeWhenBuildSideOutputTime` | 94.786ms |
110.221ms | noise / unchanged |
| HASH_JOIN_OPERATOR id=7 | `ExecTime` | 717.242ms | 546.437ms | -23.8% |
| HASH_JOIN_OPERATOR id=8 | `ProbeWhenProbeSideOutputTime` | 327.954ms |
394.574ms | not eligible / noise |
The q47 result file from the final run was byte-identical to the baseline
result file.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [x] Manual test (add detailed scripts or steps below)
- `build-support/clang-format.sh
be/src/exec/operator/join/process_hash_table_probe.h
be/src/exec/operator/join/process_hash_table_probe_impl.h`
- `git diff --check`
- `./build.sh --be`
- TPC-DS SF100 q47 on a local single-BE release cluster with the
settings above; compared final output with the baseline output.
- Tried `build-support/run-clang-tidy.sh --base upstream/master
--build-dir be/build_Release`; it failed on existing large-function/complexity
diagnostics in `process_hash_table_probe_impl.h` and static assertions from
included `jni-util.h`, not on the new changed lines.
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
