Zouxxyy opened a new pull request, #11317: URL: https://github.com/apache/paimon/pull/11317
### Purpose Keep writer statistics sound for floating-point values across engines. Positive and negative zero are distinct under Java's ordering and all NaNs compare above positive infinity. * Compute Python partition and file bounds in Java order. Use the same ordering for sorting composite primary keys and grouping equal keys, while retaining the original column values. Determine the first and last key across the full file. * When Java copies an encoded manifest, project only its partition column to rebuild floating or incomplete statistics. Continue to copy the original Avro blocks. A complete-looking legacy range such as [5, 5] can hide a NaN partition, and an unknown null count must not become zero during copying. * When importing Parquet from disk, retain known floating null counts and omit floating bounds. External footers may omit NaNs. The normal Java writer continues to use its own in-memory metadata. Related to the writer side of #10492. ### Tests * Added regressions that copy an actual legacy Python manifest and import a PyArrow file containing 5, NaN and null. Exercised distinct signed zeros, positive zero alone, NaNs, nullable/composite partitions, key identity, and file/key/value statistics. * Ran seven Java suites (178 tests), seven Python suites (244 tests), Spotless, RAT and the Python file lint. * Recomputed bounds from the contents of 45 Python-written manifests. All stored partition bounds and null counts matched. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
