HappenLee opened a new pull request, #69818:
URL: https://github.com/apache/doris/pull/69818
### What problem does this PR solve?
Issue Number: None
Problem Summary:
`SegmentIterator` expands partially selected SIMD masks one set bit at a
time, creating a serial dependency for every selected row. Expand x86 masks
with more than two selected rows using a constexpr eight-row lookup table and
fixed-width stores. Keep the existing bit iteration for sparse masks and other
architectures. Empty/full masks, scalar tails, row order and downstream column
materialization retain their existing behavior.
The output selector already has one slot per input row. At a complete SIMD
block, the number of emitted indices is at most the input position, so the
fixed-width stores fit within that allocation; only the returned prefix is
consumed. The lookup table has constant initialization and introduces no
dynamic allocation or mutable shared state.
Add unit coverage and a standalone benchmark that measures the original loop
and the actual production helper. On an Intel Xeon Platinum 8457C, with
4096-row batches, random 50% selection and Clang 20.1.8 `-O3 -march=native`:
| Benchmark | Before (ns/input row) | After (ns/input row) | Speedup |
| --- | ---: | ---: | ---: |
| Row-index generation | 0.693280 | 0.119248 | 5.81x |
| Predicate comparison + selection + 8-byte copy | 1.170464 | 0.643518 |
1.82x |
Performance tradeoff: Clang 16.0.6 with `-march=x86-64-v3` regresses from
0.104319 to 0.119946 ns/input row at random 1% selection (about 15% slower).
The Clang 20 native 1% case is approximately unchanged. These are standalone
measurements; complete SQL query speedup has not been measured.
### Release note
None
### Check List (For Author)
- Test
- [ ] Regression test
- [ ] Unit Test
- [x] Manual test (details below)
Validation completed:
- Standalone benchmark with native and AVX2 compiler settings.
- ASan + UBSan verification of all 256 byte masks, 65536 mixed masks, scalar
tails, row order, output bounds, near-uint16 row limits and payload copies.
- clang-format 16.0.6, repository build-hygiene checks and focused
clang-tidy for the helper, test source and standalone benchmark.
- Prepared this PR as one commit on the latest master; the helper, test and
benchmark source blobs are identical to the verified versions.
Unit tests added: empty/full/single-bit masks, exhaustive byte and mixed
masks, large row bases and consecutive appends. They have **not executed
through the standard BE test runner**. Both `./run-be-ut.sh -j 48 --run
--filter=SelectionTest.*:BitsTest.*` and `BUILD_TYPE=ASAN ./build.sh --be -j
48` were attempted in the development workspace, but compilation failed in
unrelated existing local `ColumnVector` edits at `column_vector.h:95` before
the tests could execute. Full BE build and full `SegmentIterator` static
analysis remain incomplete. No SQL or ARM execution.
Standalone reproduction (x86, from the repository root):
```bash
c++ -O3 -DNDEBUG -std=c++20 -march=native -Ibe/src \
be/benchmark/selection_compaction_bench.cpp -o
/tmp/selection_compaction_bench
/tmp/selection_compaction_bench --verify-only
/tmp/selection_compaction_bench --pattern random --rate 50 --mode select \
--repeats 9 --min-ms 60
/tmp/selection_compaction_bench --pattern random --rate 50 --input values \
--mode copy8 --repeats 9 --min-ms 80
```
- Behavior changed:
- [x] No.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
