sdf-jkl commented on code in PR #9681:
URL: https://github.com/apache/arrow-rs/pull/9681#discussion_r3397271791
##########
parquet-variant-compute/src/variant_get.rs:
##########
@@ -204,7 +204,23 @@ fn shredded_get_path(
// Helper that shreds a VariantArray to a specific type.
let shred_basic_variant =
|target: VariantArray, path: VariantPath<'_>, as_field:
Option<&Field>| {
Review Comment:
1. It does already do that.
2. Hmm... It's #8153.
I could do 2 but I'm not sure about `variant_get` custom shredding.
I checked Spark/duckdb/dremio `variant_get` implementations and it seems
like no one does output shredding as a part of `variant_get` which is basically:
```rust
variant_get(arr, path, Some(shredding_schema)) ≡
shred_variant(variant_get(arr, path), schema)
```
There's a https://github.com/datafusion-contrib/datafusion-variant `Variant`
integration and it strives for parity with spark/databricks.
Custom shredding will, however, skip unshred-reshred round trip that'd come
from doing:
```rust
shred_variant(variant_get(arr, path), schema)
```
That's really a #8153 discussion
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
