zhangyue19921010 opened a new pull request, #11131:
URL: https://github.com/apache/paimon/pull/11131
### Purpose
Fixes #11130.
Bitmap global-index range lookups currently decode every dictionary block in
each selected file. Use the existing logical-key binary search to scan only
blocks covering the bounds, retaining the exact predicate checks within those
blocks.
Local reader measurements for 50,000 integer keys, four rows per key,
default 16 KiB dictionary blocks and no compression:
| Lookup | Bytes read, before → after | Warm median, before → after |
| --- | --- | --- |
| BETWEEN -5 AND 4 (40 rows) | 450,188 → 16,991 | 2.56 → 0.64 ms |
| All keys (200,000 rows) | 1,799,918 → 1,799,918 | 61.6 → 64.9 ms |
JDK 11, 8-vCPU Xeon Platinum 8260 VM, local filesystem, 512 MiB heap; median
of three JVM medians, each with three warmups and seven measurements,
alternating base/head order. All-row per-JVM medians overlap (58.6–65.6 ms
before, 58.7–67.6 ms after). LZ4 and a 1,000-key control were also checked.
This is an index-reader microbenchmark, not a whole-query speedup. Heap
allocation was not measured.
### Tests
The dictionary-read regression fails on unchanged production code for
none/LZ4/ZSTD/LZO (106–107 seeks, limit 12), and passes with the change.
Extended numeric cases cover signed key order, inclusive/exclusive bounds and
bounds absent from the dictionary, full ranges, reversed bounds and NULL rows.
Additional local differential probes passed 6,816
string/codec/block-size/file-layout comparisons and 360 Data Evolution table
comparisons with partial index coverage and compound predicates, on both base
and patch.
143 related tests passed on JDK 11, with Checkstyle, Spotless, Enforcer and
RAT, without `fast-build`:
```sh
mvn -pl paimon-core -am -DwildcardSuites=none -DfailIfNoTests=false \
-Dtest=LazyFilteredBitmapIndexReaderTest,MultiValueBitmapIndexReaderTest,BitmapGlobalIndexTableTest,SortedGlobalIndexWriterTest,SortedGlobalIndexScannerTest,GlobalIndexQueryTest,GlobalIndexEvaluatorTest,SortedFileMetaSelectorTest
verify
```
AI-generated with OpenAI Codex; independently reviewed by an AI agent. No
human review is claimed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]