sdf-jkl opened a new pull request, #12245:
URL: https://github.com/apache/arrow-rs/pull/12245
# Which issue does this PR close?
- Closes #11938.
# Rationale for this change
`VariantType` accepted any `Struct` as storage, so `try_new`, `validate` and
`Field::try_canonical_extension_type` treated structs like `Struct("unrelated":
Int32)` as Variant. Extension wrappers such as DataFusion's `DFVariant`
(apache/datafusion#26822) should be able to delegate this check to the Arrow
type instead of building a `VariantArray`.
# What changes are included in this PR?
`VariantType::supports_data_type` now requires a struct with a binary
`metadata` field (or a dictionary or run-end encoded array of binary) and at
least one of a binary `value` field and a `typed_value` field.
These are the top-level checks `VariantArray::try_new` already applies, so
everything `VariantArray` reads still passes. That includes shredded storage
without `value`, and the Parquet reader, which attaches `VariantType` to every
VARIANT column. Nullability, `typed_value` types and nested shredded groups
stay with `VariantArray`. The error uses the `<Name> data type mismatch,
expected ..., found ...` format of the other canonical extension types.
# Are these changes tested?
Yes. New unit tests in `variant.rs` cover accepted layouts (each binary
type, dictionary and run-end encoded `metadata`, shredded, and `typed_value`
without `value`) and rejected ones. The existing `parquet-variant-compute` and
`parquet` Variant tests, including `variant_integration`, pass unchanged.
# Are there any significant user-facing changes?
Storage types that are not Variant storage now fail `VariantType::try_new`
and `validate`, so `Field::with_extension_type(VariantType)` panics on them and
`try_with_extension_type` returns an error. Code that tagged placeholder
structs such as `Struct(Fields::empty())` needs a real storage type. The error
message for non-struct storage changed.
# AI usage disclosure
Implementation and tests prepared with Claude Code.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]