HappenLee opened a new pull request, #67584:
URL: https://github.com/apache/doris/pull/67584
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
`sequence(2147483646, 2147483647, 2)` and its `array_range` equivalent
should return `[2147483646]`. The current Int32 generator overflows when
incrementing past the last element and can keep appending instead of
terminating. The wider preflight size calculation does not protect the Int32
generation loop.
Use Int64 distance/cursor arithmetic, check the exact element count before
allocation, resize the nested output once per multi-element row, and fill it
with a counted loop. Initialize the element null map once per block. Empty and
singleton fast paths avoid division and general-fill overhead for short ranges.
The Int32 return type, exclusive upper bound, invalid-input NULLs and
array-size limit remain unchanged.
Four BE unit tests cover both function names, all constant/vector argument
combinations, overflow boundaries, default arguments, NULL/invalid inputs,
mixed growing/empty/NULL rows, and the exact size limit and limit+1.
### Performance
Function-level Google Benchmark calls through the real function factory,
including output allocation. Both binaries use the same benchmark and master
baseline `eea19b3f3cfef9e1bbbd559f9ea42954d8891e0f`; only the generator
implementation differs.
Intel Xeon Platinum 8457C, Clang 20.1.8, Release `-O3`, AVX2, pinned to CPU
24. Each case uses non-constant input columns, `start = row % 17`, and the
length/step below. Run order: before, after, after, before; 7 repetitions per
run, minimum 0.3 s measurement and 0.2 s warmup. Values are the median CPU time
across 14 samples per case.
| Input rows | Elements per array | Step | Before (µs/batch) | After
(µs/batch) | Speedup |
|---:|---:|---:|---:|---:|---:|
| 4096 | 0 | 1 | 9.34 | 9.27 | 1.01× |
| 4096 | 1 | 1 | 15.64 | 11.37 | 1.38× |
| 4096 | 16 | 1 | 119.95 | 32.01 | 3.75× |
| 4096 | 256 | 1 | 2737.57 | 931.27 | 2.94× |
| 4096 | 1024 | 1 | 20430.45 | 3763.61 | 5.43× |
| 4096 | 256 | 7 | 2615.14 | 942.04 | 2.78× |
| 1 | 1000000 | 1 | 1973.84 | 214.92 | 9.18× |
This is a shared host with CPU scaling enabled, so the empty-array result
should be treated as unchanged. These are BE function timings, not end-to-end
SQL speedups. Raw samples show higher variation for the large multi-row
allocation cases; the reported values are medians.
Reproduce with the registered `BM_ArrayRange` benchmark:
```bash
BUILD_TYPE=Release ./build.sh --benchmark -j 48
taskset -c 24 be/build_Release/bin/benchmark_test \
--benchmark_filter=BM_ArrayRange \
--benchmark_min_time=0.3s --benchmark_min_warmup_time=0.2 \
--benchmark_repetitions=7 --benchmark_out=results.json \
--benchmark_out_format=json
```
### Release note
Fix integer overflow in `sequence`/`array_range` near INT32_MAX and
accelerate integer range generation with batched output filling.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test: `./run-be-ut.sh --run
--filter='FunctionArrayRangeTest.*' -j 48` — all 4 ASAN tests passed and the
process exited normally.
- [x] Manual test: actual-function Release benchmarks above.
- Behavior changed:
- [ ] No.
- [x] Yes: valid ranges whose final increment exceeds INT32_MAX now
terminate with the expected array.
- Does this need documentation?
- [x] No.
- [ ] Yes.
Validation notes:
- clang-format 16 and build-hygiene checks passed.
- The Release benchmark compiled, linked, installed and ran successfully.
The build script's subsequent generic packaging step fails because
benchmark-only builds do not populate `be/output/bin/*`.
- The repository clang-tidy script was attempted but did not pass because of
the existing unmatched `NOLINTEND` in `be/src/core/types.h` and signed
comparisons in unchanged array-range code. No unrelated fixes are included.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
