JingsongLi opened a new pull request, #11303: URL: https://github.com/apache/paimon/pull/11303
### Purpose The pure Python writer only populated manifest value statistics for `metadata.stats-mode=full`. As a result, an explicit `truncate(16)` on a table shared with Java produced empty value statistics on Python-written files. - Match Java's `truncate(16)` default and honor `none`, `counts`, `full`, column overrides, and primary-key level and changelog modes. - Reuse the Parquet writer's in-memory footer for scalar min/max bounds when available. Keep the Arrow fallback when a row group lacks reliable bounds, and preserve the existing handling of NaN and signed zero. - Apply the policy to ordinary data files, changelogs, and Data Evolution partial-column writes. Related to #10173. ### Tests - On current master, 61 focused Python tests passed using PyArrow 22.0.0, with 11 skipped. This covers manifest bounds, Parquet page indexes, changelog retry behavior, and REST writes. - The broader local regression run passed 1,568 tests. It excluded an existing pandas 3 dtype assertion reproduced on master and Ray cases requiring the optional dependency. - Local PyArrow 22 A/B benchmark on a 200,000-row, 64-column ZSTD append reduced statistics CPU time from 20.3 ms to 2.9 ms and median write-plus-commit time from 471.5 ms to 457.0 ms over six measured runs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
