mattfaltyn opened a new pull request, #2597: URL: https://github.com/apache/iceberg-go/pull/2597
Fixes #2596. ## Summary - Read null count and min/max presence from the Parquet footer. Arrow's typed statistics can represent an absent null count as zero and a single bound as a min/max pair. - Omit incomplete Iceberg file aggregates independently. Keep valid bounds across provably all-null row groups and continue recording complete column sizes and value counts. - Use the same footer presence checks for row-group pruning and shredded Variant bounds. When suppressing partial bounds removes the basis for partition inference, require an explicit partition value. ## Validation The regression imports a two-row-group Parquet file with an omitted null count, omitted min/max, or a one-sided bound. It checks persisted metrics and filtered scans, and verifies that a copy-on-write delete preserves a null row. Complete footer and all-null row-group cases cover valid pruning. Additional cases cover shredded Variant bounds and explicit partition values. When footer statistics are incomplete, observed bounds can still reject a contradictory supplied partition value. They are omitted from the Iceberg manifest and cannot be used to infer a partition. Existing manifest entries with incorrect metrics are outside this change. - `make test` passed (41 package suites). - `make lint` passed with zero findings. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
