amogh-jahagirdar opened a new pull request, #18604:
URL: https://github.com/apache/iceberg/pull/18604
This adds `StatsUtil.sourceBoundsFor`, which returns inclusive bounds on the
source column values allowed by a partition value for order-preserving
transforms (identity, truncate, year, month, day, and hour).
A follow-up PR will use this in the v3 to v4 adapter
(`TrackedFileAdapters`), so that adapted data files report bounds for partition
source columns derived from the partition tuple when the file doesn't already
have stats for those columns. This is one of the primitives needed to
eventually drop the v4 partition tuple, since filtering can rely on source
column stats instead. This should also end up producing bounds that we can
*later derive the partition tuple from* (tracked file to data file adaptation)
for equality delete matching.
Test plan:
- Identity:
- Bounds are the partition value itself
- Bounds use the source column type when the partition value predates
promotion (int to long, float to double)
- NaN partition values have no bounds
- Truncate numbers:
- Upper bound is the partition value plus width - 1 for int, long, and
decimal (unscaled), including negative values
- Upper bound stops at the type's max instead of overflowing
- Decimal upper bound stops at the column's largest value (decimal(9,2)
caps at 9999999.99)
- Truncate strings and binary:
- Upper bound for a full-width prefix is the next higher prefix
- Values shorter than the width have both bounds equal to that value
- Width is counted in code points, so a string with a [surrogate
pair](https://www.unicode.org/glossary/#surrogate_pair) (a character above
U+FFFF that Java stores as two `char`s) that is shorter than the width in code
points has both bounds equal to that value
- Prefixes with no higher prefix (max code point, all 0xFF bytes) have
no upper bound
- Binary bounds do not share the partition value's buffer, since
partition tuples can be reused
- Year, month, day, and hour:
- Bounds cover the whole unit for date, timestamp, and timestamp_ns
sources
- Lower bound starts one unit earlier for stored values <= 0 to cover
older writers (#1981), except for day on dates
- date and timestamp_ns upper bounds stop at the max value when the end
of the unit does not fit
- Invalid arguments:
- Null partition values are rejected
- Partition fields that do not preserve order (bucket, void) are rejected
---
**AI Disclosure**
- Model: Claude Opus 5.5
- Platform/Tool: OpenCode
- Human Oversight:
- Prompt Summary:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
