raghuvanshraj commented on issue #2546:
URL: https://github.com/apache/iceberg-rust/issues/2546#issuecomment-6852015215

   I'd like to take this. Now that #2880 reads unshredded variants, I'll make 
the Parquet writer accept a variant column (today `parquet_writer.rs` returns 
`FeatureUnsupported`) and write it as the `metadata`/`value` group with the 
`VARIANT` annotation, by enabling `variant_experimental` on the parquet 
dependency and keeping the `arrow.parquet.variant` tag on the field. One thing 
to flag: parquet 60.1 writes the annotation without `specification_version`, 
and Spark 4.1 rejects such files. arrow-rs main fixes that 
(apache/arrow-rs#11961) but it is not released yet, so the PR will need the 
next arrow-rs release or an assertion that fails until then.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to