HappenLee opened a new pull request, #69685:
URL: https://github.com/apache/doris/pull/69685
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
`RuntimeFilterExpr::execute_filter` scans the result mask three times for
every non-constant filter: count live input rows, AND the new filter into the
mask, then count live output rows.
Fuse these operations in a shared SSE helper used by both ordinary and
nullable filters. It processes four 16-byte vectors per iteration, accumulates
zero counts in byte lanes (at most four per lane), and reduces them with
`_mm_sad_epu8`. The existing `sse2neon` dependency maps the same implementation
to NEON on AArch64. A scalar tail/fallback handles remaining rows.
The helper preserves the existing bitwise filtering and nullable semantics,
counts nonzero bytes without assuming the inputs are normalized to 0/1, and
retains the runtime-filter Profile/selectivity updates.
#### Local microbenchmark
Intel Xeon Platinum 8457C, Clang 21.1.8, `-O3 -DNDEBUG -msse4.2 -mavx2`,
pinned to one CPU. Five repetitions with random interleaving; median CPU time
per batch. The incoming result mask is all ones and the new filter has 50%
passing rows.
| Rows per batch | Before | SSE fusion | CPU time reduction |
|---:|---:|---:|---:|
| 4,096 | 239.57 ns | 188.42 ns | 21.35% |
| 16,384 | 1,014.96 ns | 759.39 ns | 25.18% |
Each iteration restores fresh masks outside the timed region so repeated AND
operations do not gradually turn the inputs into zeros. Independent blocks use
a cache-resident working set; the benchmark measures only mask merging and
counting, not RF evaluation or SQL latency. AArch64 cross-compilation confirms
NEON code generation; ARM performance has not been measured.
### Release note
Reduce CPU overhead when merging runtime-filter masks and counting live rows.
### Check List (For Author)
- Test
- [ ] Regression test
- [ ] Unit Test
- [x] Manual test
- Added BE unit tests for ordinary/nullable filters, empty batches,
unaligned buffers, SIMD boundaries/tails, and arbitrary byte values; the test
file compiles independently.
- Independent ASAN/UBSan oracle checks passed 11,776 cases for the
SSE path and 11,776 cases for the forced scalar fallback, comparing both counts
and the entire output buffer.
- Ran the SSE microbenchmark matrix for
64/256/1,024/4,093/4,096/16,384 rows and 0/50/100% filter pass rates.
- Cross-compiled the isolated helper through the actual `sse2neon.h`
for AArch64 and inspected NEON code generation. This is not a full ARM BE build.
- Formatting, header/build hygiene, `git diff --check`, and
clang-tidy on the added helper/test lines passed.
- Attempted `./run-be-ut.sh -j 48 --run
--filter='BitsTest.FilterAndCount*'`; existing local `ColumnVector`/`PODArray`
constructor compilation errors blocked the full build, so the standard BE unit
tests did not execute. Full-TU clang-tidy is also incomplete because of local
compilation/environment errors.
- Behavior changed:
- [x] No.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
