nodece opened a new pull request, #26183: URL: https://github.com/apache/pulsar/pull/26183
### Motivation `SegmentedLongArray` previously used Netty `ByteBuf` as its backing store. Every `readLong`/`writeLong` operation went through `ByteBuf.getLong()`/`setLong()`, adding extra indirection on the hot path of `TripleLongPriorityQueue` heap operations(`siftUp`/`siftDown`). PR [#26010](https://github.com/apache/pulsar/pull/26010) identified `SegmentedLongArray` as the remaining bottleneck after optimizing the heap algorithms, and suggested replacing the backing storage with primitive arrays. This PR replaces the `ByteBuf` backing store with segmented heap-backed `long[][]` arrays while preserving the segmented architecture required to support capacities beyond `Integer.MAX_VALUE` elements (PR [#16490](https://github.com/apache/pulsar/pull/16490)). ### Modifications - **`SegmentedLongArray`** - Replace the `ByteBuf` backing store with segmented heap-backed `long[][]` arrays. - Use bit-shift/bit-mask for segment lookup instead of division/modulo. - Grow the outer segment array using an amortized expansion strategy to avoid copying on every new segment allocation. ### Benchmark Results JMH benchmarks from `TripleLongPriorityQueueBenchmark`(2 forks, 3 warmup iterations, 5 measurement iterations, JDK 21.0.5). #### Before (ByteBuf) | Benchmark | Size | Score (us/op) | Error | |-----------|-----:|--------------:|------:| | recoveryBulkAddThenPop | 500,000 | 454,423 | ±142,390 | | recoveryBulkAddThenPop | 2,000,000 | 2,406,656 | ±680,108 | | interleavedAddPop | 500,000 | 335,764 | ±461,320 | | interleavedAddPop | 2,000,000 | 1,560,565 | ±403,289 | | steadyState | 500,000 | 261,449 | ±11,931 | | steadyState | 2,000,000 | 1,148,007 | ±699,420 | #### After (heap-backed `long[][]`) | Benchmark | Size | Score (us/op) | Error | |-----------|-----:|--------------:|------:| | recoveryBulkAddThenPop | 500,000 | 337,634 | ±6,460 | | recoveryBulkAddThenPop | 2,000,000 | 1,805,059 | ±99,786 | | interleavedAddPop | 500,000 | 231,696 | ±1,425 | | interleavedAddPop | 2,000,000 | 1,102,539 | ±89,740 | | steadyState | 500,000 | 261,390 | ±7,856 | | steadyState | 2,000,000 | 1,054,573 | ±58,502 | #### Comparison (Size = 2,000,000) | Benchmark | Before (ms) | After (ms) | Improvement | |-----------|------------:|-----------:|------------:| | recoveryBulkAddThenPop | 2,407 | 1,805 | **+25%** | | interleavedAddPop | 1,561 | 1,103 | **+29%** | | steadyState | 1,148 | 1,055 | **+8%** | > **Note:** The baseline measurements showed relatively large variance in some > scenarios, while the new implementation produced noticeably more stable > results. The reported improvements should therefore be interpreted together > with the benchmark error margins. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
