JingsongLi opened a new pull request, #8457: URL: https://github.com/apache/paimon/pull/8457
## Summary Optimize `RoaringNavigableMap64#toRangeList` so large contiguous bitmaps do not have to expand every row id through the boxed iterator path. The implementation uses an adaptive path: dense-looking bitmaps use `select` with exponential/binary range discovery, while sparse bitmaps fall back to a primitive `LongIterator` scan. ## Changes - Add adaptive range extraction for `RoaringNavigableMap64#toRangeList`. - Avoid merging `Long.MAX_VALUE` and `Long.MIN_VALUE` across unsigned iterator wrap-around. - Add tests for large contiguous ranges, high-bitmap-boundary ranges, and wrap-around behavior. - Add a micro benchmark for dense and sparse `toRangeList` cases. - Fix the micro-benchmark harness to use the benchmark module's unshaded `commons-lang3` dependency. ## Testing - [x] `mvn -pl paimon-common -Pfast-build -Dtest=RoaringNavigableMap64Test test` - [x] `mvn -pl paimon-benchmark/paimon-micro-benchmarks -am -Pfast-build,runTest -DfailIfNoTests=false -Dtest=RoaringNavigableMap64Benchmark#testDenseRangeToRangeList test` - [x] `mvn -pl paimon-benchmark/paimon-micro-benchmarks -am -Pfast-build,runTest -DfailIfNoTests=false -Dtest=RoaringNavigableMap64Benchmark#testSparseRangeToRangeList test` - [x] `mvn -pl paimon-common,paimon-benchmark/paimon-micro-benchmarks -am -DskipTests test-compile` ## Benchmark On Apple M4 Pro / JDK 8: - Dense 10M contiguous row ids: iterator baseline best ~37 ms, optimized `toRangeList` best ~0.096 ms, about 386x faster. - Sparse 100K singleton row ids: remains in the same millisecond range as the iterator baseline, avoiding the select-path regression for sparse data. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
