Caideyipi opened a new pull request, #18358:
URL: https://github.com/apache/iotdb/pull/18358
## Description
### Problem
#18249 made aligned MemTable bitmaps lazy, and #18330 reconciled their
actual memory usage. On Windows, profiling the remaining write-path regression
showed a hot allocation chain:
```text
BitMapLongImpl.getByteArray
-> AlignedTVList.containsMarkedBit
-> AlignedTVList.containsNonNullValue / markNullBitmapRange
-> AlignedTVList.arrayCopy / putAlignedValues
```
For a dynamically selected, long-backed bitmap, `getByteArray()`
materializes a new byte array. The common aligned-tablet path checks a range
starting at offset 0 twice per column, so a 64-column batch creates 128
temporary arrays (about 4 KiB with the measured object size).
### Fix
- Use `BitMap.isAllUnmarked(length)` for ranges starting at offset 0, so the
backing bitmap is inspected directly without materializing a byte-array copy.
- Keep the existing byte-array range scanner unchanged for non-zero offsets.
- Add correctness coverage for long-backed and array-backed bitmaps,
including empty, prefix, partial, byte-boundary, and multi-long ranges.
- Add a default-disabled manual performance UT that reports current-thread
CPU time and allocated bytes for both the legacy and optimized implementations.
A bit-by-bit non-zero-offset fast path was also evaluated, but its
worst-case CPU cost was about 3x the legacy scanner. It is intentionally not
included.
### Performance
Measured on Windows with Java 17. The range-check benchmark used 64
long-backed bitmaps, 102,400,000 checks per round, and the median of 7 rounds.
| Range | Legacy CPU | Optimized CPU | CPU delta | Legacy allocation |
Optimized allocation |
| --- | ---: | ---: | ---: | ---: | ---: |
| Prefix (`start=0, length=64`) | 15.869 ns/check | 1.221 ns/check | -14.648
ns/check | 32 B/check | 0 B/check |
| Partial (`start=1, length=63`) | 15.411 ns/check | 14.954 ns/check |
-0.458 ns/check | 32 B/check | 32 B/check |
The optimized prefix removes the allocation and is faster. The
non-zero-offset fallback has no measurable CPU or allocation regression.
The existing aligned bitmap accounting benchmark was also run with 64
columns, 64 rows, 10,000 batches per round, and 7 rounds:
| Scenario | Allocation before | Allocation after | Delta |
| --- | ---: | ---: | ---: |
| Dense | 19,792 B/batch | 19,792 B/batch | 0 B/batch |
| Null-heavy | 30,600 B/batch | 26,480 B/batch | -4,120 B/batch |
Null-heavy write CPU remained 12.500 us/batch before and after. The dense
path does not use source bitmaps and is unchanged.
Enable the new benchmark with:
```shell
mvn -pl iotdb-core/datanode test
-Dtest=AlignedBitmapRangeCheckPerformanceTest
-Diotdb.aligned.bitmap.range-check.perf.enabled=true
```
### Verification
- `mvn -o -nsu spotless:apply -pl iotdb-core/datanode`
- `mvn -o -nsu -pl iotdb-core/datanode test-compile -DskipTests`
- Checkstyle: 0 violations
- `AlignedTVListTest`: 16 passed
- `AlignedBitmapRangeCheckPerformanceTest`: 1 passed when enabled
- Default-off performance UT behavior: 1 skipped as expected
- `AlignedBitmapMemoryAccountingPerformanceTest`: 1 passed
- `git diff --check`: passed
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added comments explaining the intent of the fast path.
- [x] added or updated unit tests to cover the new code paths.
<hr>
##### Key changed/added classes
- `AlignedTVList`: allocation-free prefix bitmap range check.
- `AlignedTVListTest`: correctness coverage for prefix and partial ranges
across bitmap implementations.
- `AlignedBitmapRangeCheckPerformanceTest`: opt-in CPU and allocation
comparison against the legacy implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
