[ https://issues.apache.org/jira/browse/SPARK-24741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542207#comment-16542207 ]
Xiao Li commented on SPARK-24741: --------------------------------- [~mn-mikke] So far, we do not have a plan to improve the interoperability with Confluent schema registry. Adding the dependency to schema registry might be not a good idea. I am not familiar with ABRiS. If you have some good idea to improve the data source of AVRO. Welcome your contributions. > Have a built-in AVRO data source implementation > ----------------------------------------------- > > Key: SPARK-24741 > URL: https://issues.apache.org/jira/browse/SPARK-24741 > Project: Spark > Issue Type: New Feature > Components: SQL, Structured Streaming > Affects Versions: 2.4.0 > Reporter: Xiao Li > Assignee: Gengliang Wang > Priority: Major > > Apache Avro (https://avro.apache.org) is a data serialization format. It is > widely used in the Spark and Hadoop ecosystem, especially for Kafka-based > data pipelines. Using Spark-Avro package > (https://github.com/databricks/spark-avro), Spark SQL can read and write the > avro data. Making spark-Avro built-in can provide a better experience for > first-time users of Spark SQL and structured streaming. We expect the > built-in Avro data source can further improve the adoption of structured > streaming. We should consider inlining > https://github.com/databricks/spark-avro. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org