gengliangwang edited a comment on pull request #31771: URL: https://github.com/apache/spark/pull/31771#issuecomment-792726114
@blcksrx I don't think the implementation of this PR can work with Kafka + Avro + schema registry as https://docs.databricks.com/spark/latest/structured-streaming/avro-dataframe.html#example-with-schema-registry describes. I was hoping you realize the problem after you create unit tests for the use case. For what has done in this PR, it can be done by using the option `avroSchemaUrl` anyway. If your purpose is to get the latest Avro schema from the schema registry and use it on a stream of Avro format without schema-registry schema ID, there is no need to add a new API. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
