ninglinLiu opened a new pull request, #3529: URL: https://github.com/apache/iceberg-rust/pull/3529
## Which issue does this PR close? - Closes #3409. #3476 fixed the first half of the issue (valid floats such as `123.456` being rejected on read). This PR covers the second half: non-finite partition values silently becoming `null`. ## What changes are included in this PR? JSON has no representation for `NaN` or `±Infinity`, and `serde_json` writes them as `null`. So `serialize_data_file_to_json` turned a non-finite float or double partition value into `null`, and `deserialize_data_file_from_json` then read it back as a null partition value, with no error anywhere. That is how the incorrect metadata in the issue got committed. `serialize_data_file_to_json` now checks the partition tuple before serializing and returns a `DataInvalid` error naming the partition field when a float or double value is not finite, instead of writing `null`. Finite values are unchanged. The Avro manifest path is not touched, since Avro stores these values exactly. ## Are these changes tested? Yes. `test_data_file_to_json_rejects_non_finite_partition_values` round-trips a data file with finite float and double partition values, and checks that `NaN`, `+Infinity` (float) and `NaN`, `-Infinity` (double) are rejected with an error naming the field. The test fails on `main` and passes with the change. `cargo test -p iceberg --lib spec::manifest` and `cargo clippy -p iceberg --all-targets -- -D warnings` pass. This PR was prepared with AI assistance and reviewed/tested by the author. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
