Zouxxyy opened a new pull request, #1479: URL: https://github.com/apache/paimon-rust/pull/1479
### Purpose Follow-up to #1189 and #1190 for file-level writer statistics. A Parquet file containing 5 and NaN can have footer bounds [5, 5]. Copying them to the manifest entry can cause another engine to skip the data file on an equality filter for NaN. Using partial comparisons for primary-key column statistics can also collapse the two signs of zero. ### Brief change log * Compute bounds for full/truncate FLOAT and DOUBLE columns from incoming Arrow batches, accumulating across writes and row groups. Other supported value columns continue to use the footer. Preserve counts/none modes and dense field mapping. * Share Java-order comparisons with the existing manifest partition predicate/collector and with the key column stats collector. Use typed Arrow arrays to select batch extrema. * Preserve physical sort order, merge identity and complete key endpoints. The committed-file test checks the scalar key/value bounds separately from the physical first and last composite keys. ### Tests Ran the column-statistics and manifest-partition-statistics unit tests, the Parquet unit suite, and the pk_value_stats_test and float_partition_filter_test integrations (119 tests total). Coverage includes positive and negative zero alone and together, NaN at either end, negative NaN, infinities, multiple batches and row groups, nulls and dense counts-only columns. ### API and Format The manifest schema and the Parquet file format are unchanged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
