zhangyue19921010 opened a new pull request, #11131:
URL: https://github.com/apache/paimon/pull/11131

   ### Purpose
   
   Fixes #11130.
   
   Bitmap global-index range lookups currently decode every dictionary block in 
each selected file. Use the existing logical-key binary search to scan only 
blocks covering the bounds, retaining the exact predicate checks within those 
blocks.
   
   Local reader measurements for 50,000 integer keys, four rows per key, 
default 16 KiB dictionary blocks and no compression:
   
   | Lookup | Bytes read, before → after | Warm median, before → after |
   | --- | --- | --- |
   | BETWEEN -5 AND 4 (40 rows) | 450,188 → 16,991 | 2.56 → 0.64 ms |
   | All keys (200,000 rows) | 1,799,918 → 1,799,918 | 61.6 → 64.9 ms |
   
   JDK 11, 8-vCPU Xeon Platinum 8260 VM, local filesystem, 512 MiB heap; median 
of three JVM medians, each with three warmups and seven measurements, 
alternating base/head order. All-row per-JVM medians overlap (58.6–65.6 ms 
before, 58.7–67.6 ms after). LZ4 and a 1,000-key control were also checked. 
This is an index-reader microbenchmark, not a whole-query speedup. Heap 
allocation was not measured.
   
   ### Tests
   
   The dictionary-read regression fails on unchanged production code for 
none/LZ4/ZSTD/LZO (106–107 seeks, limit 12), and passes with the change. 
Extended numeric cases cover signed key order, inclusive/exclusive bounds and 
bounds absent from the dictionary, full ranges, reversed bounds and NULL rows.
   
   Additional local differential probes passed 6,816 
string/codec/block-size/file-layout comparisons and 360 Data Evolution table 
comparisons with partial index coverage and compound predicates, on both base 
and patch.
   
   143 related tests passed on JDK 11, with Checkstyle, Spotless, Enforcer and 
RAT, without `fast-build`:
   
   ```sh
   mvn -pl paimon-core -am -DwildcardSuites=none -DfailIfNoTests=false \
     
-Dtest=LazyFilteredBitmapIndexReaderTest,MultiValueBitmapIndexReaderTest,BitmapGlobalIndexTableTest,SortedGlobalIndexWriterTest,SortedGlobalIndexScannerTest,GlobalIndexQueryTest,GlobalIndexEvaluatorTest,SortedFileMetaSelectorTest
 verify
   ```
   
   AI-generated with OpenAI Codex; independently reviewed by an AI agent. No 
human review is claimed.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to