iSparshP opened a new pull request, #24856: URL: https://github.com/apache/datafusion/pull/24856
## Which issue does this PR close? Closes #24786. ## Rationale for this change Reading a Parquet file with `datafusion.execution.parquet.coerce_int96` set is only supposed to change the time unit that INT96 timestamp columns decode at. In practice it also empties the metadata of every struct, list and map field in the file. The cause is in `Int96Coercer` (`datafusion/datasource-parquet/src/schema_coercion.rs`): leaf fields are rebuilt via `field_with_new_type`, which clones the field and swaps only its data type, so their metadata is retained. Container fields, however, were reconstructed with `Field::new_struct` / `Field::new_list` / `Field::new`, all of which start from empty metadata. Name, type and nullability were carried across, but `current_field.metadata()` was never read, so it was silently discarded. ## What changes are included in this PR? - Rebuild struct, list and map container fields through the existing `field_with_new_type` helper (clone the original field, swap only the data type) so name, nullability **and** metadata are all preserved, consistent with how leaf fields are already handled. - Add a regression test (`coerce_int96_preserves_container_field_metadata`) that attaches metadata to struct/list/map container fields and asserts it survives coercion. ## Are these changes tested? Yes. The new test fails before this change (container metadata is dropped) and passes after. Existing `schema_coercion` tests, plus `cargo fmt` and `cargo clippy` for the crate, are clean. ## Are there any user-facing changes? No API changes. INT96 coercion now preserves nested-field metadata that was previously dropped. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
