Anurag-S1ngh opened a new pull request, #4458: URL: https://github.com/apache/iceberg-python/pull/4458
<!-- Closes #2699 --> # Rationale for this change Nested fields were downgraded to counts-only statistics in writer stats planning, so struct columns could not be pruned on bounds. That matters for schemas such as GeoParquet, which keep bounding boxes in struct fields. Add `write.metadata.metrics.max-inferred-column-defaults` (default 100) and apply `write.metadata.metrics.default` to at most that many primitive fields, mirroring Java's MetricsConfig. With the limit in place, nested fields collect the same bounds as top-level fields. Behavior details: - If `write.metadata.metrics.default` is set explicitly, it still applies to every column. - `write.metadata.metrics.column.*` overrides still apply to columns beyond the limit. - A negative limit falls back to 100. ## Are these changes tested? Yes. New tests cover the limit, overrides beyond the limit, the explicit default bypassing the limit, and the negative fallback. Existing stats tests were updated to expect nested bounds. `tests/io/test_pyarrow_stats.py`, `tests/table`, and the local write tests pass. ## Are there any user-facing changes? Yes. Nested columns now collect bounds by default, and tables with more than 100 primitive columns stop collecting inferred metrics beyond the first 100 unless they configure a default mode or per-column overrides. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
