JingsongLi opened a new pull request, #11238:
URL: https://github.com/apache/paimon/pull/11238

   ### Purpose
   
   Add a packed R-tree global index for bounding-box queries on a single 
`GEOMETRY` column in row-tracked Data Evolution append tables. Java callers can 
build an `rtree` index and use `PredicateBuilder.bboxIntersects` to find 
envelopes that overlap or touch a query rectangle. Empty geometries remain 
available to non-null queries.
   
   The writer packs Hilbert-ordered envelopes into checksummed V1 pages. The 
reader prunes files and subtrees, coalesces adjacent page reads and searches up 
to four files concurrently by default. Queries and the byte-bounded cache share 
the backing array of each contiguous read. The format documentation and golden 
fixtures cover file layout, row IDs, nulls and empty geometries.
   
   An indexed bbox can participate in a compound scalar filter for ranked 
queries. The evaluator tracks whether all branches contributed so a second, 
unsupported predicate on the same column still gets evaluated against the data.
   
   ### Tests
   
   - Ran 430 focused Maven tests across common, format and core, including V1 
validation, WKB parsing, mixed predicates, DE scans, vector/full-text 
filtering, concurrency and cache eviction.
   - Verified that the Java writer reproduced the frozen V1 fixture byte for 
byte, and that Java and Rust could read each other's fixture.
   - Ran the Java benchmark against JTS HPRtree and STRtree on uniform points, 
clustered boxes and elongated boxes. Ran the Java and Rust file readers on the 
same 1,048,576 uniform points. Each worker compared complete query result 
bitmaps with a brute-force scan.
   - Built the documentation site and ran the root Apache RAT check.
   
   During development, at sixteen concurrent files with 20 ms added per 
physical read, sharing page buffers reduced the median sampled Java query peak 
from 435.1 to 364.6 MiB. Large-window warm queries took 105.47 and 105.66 ms 
respectively. This workload used 16 files, an 8 MiB cache, one query client and 
three independent Corretto 8 JVMs per configuration.
   
   The Java/JTS benchmark can be reproduced from the repository root with JDK 
8, Maven and Python 3.
   
   ```sh
   python3 paimon-benchmark/paimon-micro-benchmarks/run-spatial-benchmark.py \
     --engines paimon-file,jts-hpr,jts-str --rows 1048576 \
     --distributions uniform-points --forks 3 --queries 12 --rounds 1
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to