gszadovszky commented on issue #42069: URL: https://github.com/apache/arrow/issues/42069#issuecomment-2841345548
> (...) could arrow just avoid dealing with shredding altogether and leave that up to the query engine? What does it mean exactly? Arrow want to avoid representing the shredded values altogether or represent as is (according to the variant specification)? For the first approach, a variant vector would be similar to a struct vector with varbinary fields for `metadata` and `value`. For the latter approach we would need an additional arbitrary typed field of `typed_value`. I think handling `typed_value` makes handling the values quite complicated. For example two different Parquet files might use different schema for shredding so two variant vectors would have different schema as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
