alamb commented on PR #8481:
URL: https://github.com/apache/arrow-rs/pull/8481#issuecomment-3353625324
> > It seems to me like the `unshred_variant` in this PR subsumes the code
in `cast_to_variant` -- we could probably switch `cast_to_variant` to create a
"dummy" variant array and call into `unshred_variant` 🤔
>
> There's definitely some overlap and I may have missed some opportunities
for builder code reuse, but the two functions are not doing the same thing:
I didn't mean to imply this PR should be changed
>
> * `cast_to_variant` exists to convert fully strongly-typed data to binary
variant
>
> * Among other things, this entails building a new `metadata` column from
scratch, because no metadata dictionaries exist yet.
That is true, but I think most of the arrow types that are converted are
primitive types (and thus have no metadata to convert -- the output array is
just the same metadata (0 byte) repeated over and over
> * NOTE: The function can't even detect if a caller passes variant data
-- shredded or otherwise -- because it takes a `&dyn Array` that carries no
extension type info. So it has to assume that any `StructArray` it sees is an
actual struct array whose rows should be converted to variant objects, and
whose fields might just happen to be called `metadata` and `value`.
🤔 that is a good point
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
