sdf-jkl commented on PR #10417: URL: https://github.com/apache/arrow-rs/pull/10417#issuecomment-5061148234
@HuaHuaY you are correct! I was reading more into it and saw where I'm wrong. > The arrow [primitive type mapping](https://arrow.apache.org/docs/format/CanonicalExtensions.html#primitive-type-mappings) you've linked is following the table from the spec here - https://github.com/apache/parquet-format/blob/master/VariantEncoding.md#encoding-types - It's about unshredded Variant. This point I made was incorrect and the arrow spec indeed says: "A field named `typed_value` which can be a [Primitive Type Mappings](https://arrow.apache.org/docs/format/CanonicalExtensions.html#variant-primitive-type-mapping) or a `List`, `LargeList`, `ListView` or `Struct`" So the PR and the issue make sense given the spec, **but** I wonder if the spec itself should support `Uint` for typed value. `Arrow Uint -> Parquet INT -> Arrow Int` Given any UInt will be written as a signed INT Parquet type and never read back as UInt, it never round trips. The only places a UInt typed_value can come from are in-memory construction or IPC. I'll raise a question on the Arrow mailing list. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
