timmyb32r opened a new pull request, #12255:
URL: https://github.com/apache/arrow-rs/pull/12255

   # Which issue does this PR close?
   
   None — standalone bug fix.
   
   # Rationale for this change
   
   Tensor extension metadata cannot round-trip when optional parameters are 
absent.
   
   The serializer writes absent parameters as JSON `null`, but the map 
deserializer expects arrays. Consequently, attaching a tensor extension with 
`with_extension_type` succeeds, while reading it back with `try_extension_type` 
fails with `invalid type: null, expected a sequence`.
   
   This affects three of four optional-parameter combinations for 
`FixedShapeTensor` and seven of eight for `VariableShapeTensor`.
   
   # What changes are included in this PR?
   
   Deserialize optional metadata values as `Option`, treating explicit `null` 
and missing fields as absent.
   
   Track key presence separately from its value so duplicate keys are still 
rejected, including when the first value is `null`.
   
   The serialized representation and public API remain unchanged.
   
   # Are these changes tested?
   
   Expanded the existing field round-trip tests to cover all four 
optional-parameter combinations for fixed-shape tensors and all eight for 
variable-shape tensors.
   
   Added coverage for missing fields, explicit `null`, and duplicate keys 
containing null values. The regression tests reproduce the failures before the 
fix.
   
   Validation passed:
   
   - `cargo test -p arrow-schema --all-features`: 199 unit tests and 45 
doctests.
   - `cargo +nightly miri test -p arrow-schema --features 
canonical_extension_types tensor`: 23 tests.
   - `cargo test --workspace`.
   - `cargo clippy --workspace --all-targets --all-features -- -D warnings`.
   - Formatting and spelling checks.
   
   # Are there any significant user-facing changes?
   
   Tensor extensions with absent optional parameters can now be read back from 
their own serialized metadata. Previously serialized metadata containing null 
optional values is also accepted.
   
   There are no public API changes.
   
   # AI assistance
   
   Codex helped investigate the issue, implement the fix and regression tests, 
run validation, and draft this PR description.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to