sdf-jkl opened a new pull request, #12245:
URL: https://github.com/apache/arrow-rs/pull/12245

   # Which issue does this PR close?
   
   - Closes #11938.
   
   # Rationale for this change
   
   `VariantType` accepted any `Struct` as storage, so `try_new`, `validate` and 
`Field::try_canonical_extension_type` treated structs like `Struct("unrelated": 
Int32)` as Variant. Extension wrappers such as DataFusion's `DFVariant` 
(apache/datafusion#26822) should be able to delegate this check to the Arrow 
type instead of building a `VariantArray`.
   
   # What changes are included in this PR?
   
   `VariantType::supports_data_type` now requires a struct with a binary 
`metadata` field (or a dictionary or run-end encoded array of binary) and at 
least one of a binary `value` field and a `typed_value` field.
   
   These are the top-level checks `VariantArray::try_new` already applies, so 
everything `VariantArray` reads still passes. That includes shredded storage 
without `value`, and the Parquet reader, which attaches `VariantType` to every 
VARIANT column. Nullability, `typed_value` types and nested shredded groups 
stay with `VariantArray`. The error uses the `<Name> data type mismatch, 
expected ..., found ...` format of the other canonical extension types.
   
   # Are these changes tested?
   
   Yes. New unit tests in `variant.rs` cover accepted layouts (each binary 
type, dictionary and run-end encoded `metadata`, shredded, and `typed_value` 
without `value`) and rejected ones. The existing `parquet-variant-compute` and 
`parquet` Variant tests, including `variant_integration`, pass unchanged.
   
   # Are there any significant user-facing changes?
   
   Storage types that are not Variant storage now fail `VariantType::try_new` 
and `validate`, so `Field::with_extension_type(VariantType)` panics on them and 
`try_with_extension_type` returns an error. Code that tagged placeholder 
structs such as `Struct(Fields::empty())` needs a real storage type. The error 
message for non-struct storage changed.
   
   # AI usage disclosure
   
   Implementation and tests prepared with Claude Code.
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to