plusplusjiajia opened a new pull request, #8732:
URL: https://github.com/apache/paimon/pull/8732
### Purpose
`IcebergDataFileMeta.create` publishes wrong stats when a `SimpleStats`
slot is
unknown (null), silently corrupting Iceberg's metrics-based pruning:
- an unknown null count was published as `0`, so `IS NULL` pruning skips
files
that actually contain nulls;
- a required field's unknown min/max was read without a null check (the
getter of
a non-nullable type has none) and published as garbage bounds like `[0,
0]`;
- bounds-ineligible types (nested, variant, vector, blob) were rejected
only after
reading the stats slots, which can crash on a required field.
Unknown counts and bounds are now omitted (a missing entry means unknown
per the
Iceberg spec), the getter is built from the nullable copy of the type, and
the
eligibility check runs before any slot is read. No format or API change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
