timmyb32r opened a new pull request, #12255: URL: https://github.com/apache/arrow-rs/pull/12255
# Which issue does this PR close? None — standalone bug fix. # Rationale for this change Tensor extension metadata cannot round-trip when optional parameters are absent. The serializer writes absent parameters as JSON `null`, but the map deserializer expects arrays. Consequently, attaching a tensor extension with `with_extension_type` succeeds, while reading it back with `try_extension_type` fails with `invalid type: null, expected a sequence`. This affects three of four optional-parameter combinations for `FixedShapeTensor` and seven of eight for `VariableShapeTensor`. # What changes are included in this PR? Deserialize optional metadata values as `Option`, treating explicit `null` and missing fields as absent. Track key presence separately from its value so duplicate keys are still rejected, including when the first value is `null`. The serialized representation and public API remain unchanged. # Are these changes tested? Expanded the existing field round-trip tests to cover all four optional-parameter combinations for fixed-shape tensors and all eight for variable-shape tensors. Added coverage for missing fields, explicit `null`, and duplicate keys containing null values. The regression tests reproduce the failures before the fix. Validation passed: - `cargo test -p arrow-schema --all-features`: 199 unit tests and 45 doctests. - `cargo +nightly miri test -p arrow-schema --features canonical_extension_types tensor`: 23 tests. - `cargo test --workspace`. - `cargo clippy --workspace --all-targets --all-features -- -D warnings`. - Formatting and spelling checks. # Are there any significant user-facing changes? Tensor extensions with absent optional parameters can now be read back from their own serialized metadata. Previously serialized metadata containing null optional values is also accepted. There are no public API changes. # AI assistance Codex helped investigate the issue, implement the fix and regression tests, run validation, and draft this PR description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
