sdf-jkl commented on PR #10157: URL: https://github.com/apache/arrow-rs/pull/10157#issuecomment-4899845803
> We implement Variant::as_xxx with Value-based lossless semantics and document this clearly, so that the user can know the semantics. and seems the value-based lossless semantics is enough for shred, we can board this if we need in the future @klion26 We can use the [Equivalence Class](https://github.com/apache/parquet-format/blob/master/VariantEncoding.md#encoding-types) from the spec to base the lossless cast for shredding on. ``` ┌───────────────────────┬───────────────────────────────────────────────────┐ │ Family (Equivalence │ Members │ │ Class) │ │ ├───────────────────────┼───────────────────────────────────────────────────┤ │ Exact Numeric │ int8, int16, int32, int64, decimal4, decimal8, │ │ │ decimal16 │ ├───────────────────────┼───────────────────────────────────────────────────┤ │ Timestamp │ timestamp[µs], timestamp[ns] │ │ (tz-adjusted) │ │ ├───────────────────────┼───────────────────────────────────────────────────┤ │ TimestampNTZ │ timestamp_ntz[µs], timestamp_ntz[ns] │ ├───────────────────────┼───────────────────────────────────────────────────┤ │ Boolean │ true (id 1), false (id 2) — two type-IDs, one │ │ │ logical type (trivial) │ └───────────────────────┴───────────────────────────────────────────────────┘ The rest - identity (no cross cast between types) ``` > Good fit for SQL uses of variant_get, yes. For field extraction in imperative code? Not so sure. I guess one could manually drill down the jsonpath in such cases... @scovich users could do smth like - `let age: Option<i64> = v.get_path(&path).and_then(|leaf| leaf.as_int64()); ` once we have the lossless cast available. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
