Le 22/08/2024 à 17:08, Curt Hagenlocher a écrit :
(I also happen to want a canonical Arrow representation for variant data, as this type occurs in many databases but doesn't have a great representation today in ADBC results. That's why I filed [Format] Consider adding an official variant type to Arrow · Issue #42069 · apache/arrow (github.com) <https://github.com/apache/arrow/issues/42069>. Of course, there's no specific reason why a canonical Arrow representation for variants must align with Spark and/or Iceberg.)
Well, one reason is interoperability, especially as converting between different semi-structured encodings like this would probably be expensive.
Regards Antoine.
