findepi commented on issue #6736: URL: https://github.com/apache/arrow-rs/issues/6736#issuecomment-2518474187
There needs to be a way to represent a series of variant values having "no type in common" (variant integer, variant boolean, variant varchar, etc all mixed up). For that some blob-like representation with internal structure seems natural. Then there should be a way to carry-on the shredded columns without having to put them back into that blob, so yes, one type, different child layouts. It feels to me that the runtime representation will end up being similar to what is defined in Parquet (https://github.com/apache/parquet-format/blob/master/VariantShredding.md)... so maybe it should be the same representation to provide for an efficient read path. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
