huan233usc opened a new pull request, #17173:
URL: https://github.com/apache/iceberg/pull/17173
> [!IMPORTANT]
> Draft / proof-of-concept, stacked on #17161 (Compute geometry bounding box
metrics).
> The first commit here is #17161; review that one there. This PR is only
the second
> commit — a demonstration that the bounds #17161 writes are usable for data
skipping.
> Not intended to merge as-is.
## Why
#17161 writes a per-file geometry bounding box into
`lower_bounds`/`upper_bounds`, but
nothing consumes it yet: `InclusiveMetricsEvaluator` has no geo branch,
there is no
spatial predicate in the `Expression` API, and
`GeospatialPredicateEvaluators` (the
bbox-intersection evaluator that already exists) is not wired into scan
planning. So it
is fair to ask whether those bounds are actually useful. This PoC answers
that: yes —
the stored bbox is sufficient to prune files.
## What it shows
`TestGeospatialDataSkipping` writes two geometry files with disjoint point
clusters,
reads each file's bbox back from `DataFile.lowerBounds()/upperBounds()` (via
`Conversions.fromByteBuffer(GEOMETRY, …)` → `GeospatialBound`), builds a
`BoundingBox`,
and intersects a query window against each using the existing
`GeospatialPredicateEvaluators.create(type).intersects(query, fileBbox)`:
- a window over the first cluster keeps file A and **prunes** file B;
- a window over the second cluster keeps file B and prunes file A.
This is the full write→store→read→intersect chain, end to end, on real
Parquet files.
## What is still missing for production pruning (not in this PoC)
1. A spatial predicate in the `Expression` API to carry the query bbox
(there is no
`ST_Intersects`/`Operation` for it today; geo types also have no
`Comparator`, so
ordinary comparison predicates can't bind to a geo column).
2. Wiring the bbox intersection into `InclusiveMetricsEvaluator` /
`ManifestReader` so scan planning skips files automatically.
Opening this as a draft to make the value of #17161 concrete and to anchor
discussion on
the pruning-side design.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
