cloud-fan commented on pull request #31771: URL: https://github.com/apache/spark/pull/31771#issuecomment-792677194
Why do we add such an API? It just looks like a variant of the `avroSchemaUrl` option and does not worth a new API. You mentioned https://docs.databricks.com/spark/latest/structured-streaming/avro-dataframe.html#example-with-schema-registry , but it's a totally different thing. The schema register integration needs to not only get the avro schema from the schema registry server, but also need to recognize the special avro binary data that has the schema register schema ID encoded. This needs to add the schema register client dependency, that's why the API is not in the upstream Spark, as it changes dependencies. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
