sdf-jkl commented on issue #8153: URL: https://github.com/apache/arrow-rs/issues/8153#issuecomment-4684507748
https://github.com/apache/arrow-rs/pull/9681#discussion_r3397271791 > > 2. Hmm... It's [[Variant] Support Shredded Objects in `variant_get`: access as `Some(DataType::Struct)` (nested shredding) #8153](https://github.com/apache/arrow-rs/issues/8153). > > > I could do 2 but I'm not sure about `variant_get` custom shredding. > > I checked Spark/duckdb/dremio `variant_get` implementations and it seems like no one does output shredding as a part of `variant_get` which is basically: > > ```rust > variant_get(arr, path, Some(shredding_schema)) ≡ shred_variant(variant_get(arr, path), schema) > ``` > > There's a https://github.com/datafusion-contrib/datafusion-variant `Variant` integration and it strives for parity with spark/databricks. > > Custom shredding will, however, skip unshred-reshred round trip that'd come from doing: > > ```rust > shred_variant(variant_get(arr, path), schema) > ``` > > That's really a #8153 discussion -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
