Samrose-Ahmed commented on PR #46831: URL: https://github.com/apache/spark/pull/46831#issuecomment-2151413685
I suggest moving the implementation of Variant to a separate repo outside of the Spark project. I see the usage of "Open Variant" instead of "Spark Variant" in [recent announcements](https://www.databricks.com/blog/introducing-open-variant-data-type-delta-lake-and-apache-spark). Other projects and table formats like Delta Lake and Apache Iceberg have adopted or are [considering adopting this format](https://docs.google.com/document/d/1QjhpG_SVNPZh3anFcpicMQx90ebwjL7rmzFYfUP89Iw/edit), respectively. Can you clarifily if this is not a Spark specific internal Variant, because it so it would be useful in terms of specification and implementation, and interoperability for this to not be embedded inside of the Apache Spark project. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
