HeartSaVioR commented on pull request #31771: URL: https://github.com/apache/spark/pull/31771#issuecomment-794498033
I guess we haven't decided the important things. Let's stop making progress and decide before continue. 1. Would we want to pull the dependency from Confluent Schema Registry? There's no issue on license, but I also see we are trying to be "vendor-neutral" via avoiding vendor related things directly. 2. If the answer for 1 is yes (let's not saying 2 before deciding 1), given SR is only working with Kafka topic, are we sure doing it via `from_avro`/`to_avro` is the right way? I'd rather say Spark SQL Kafka should be smart to support schema based on SR instead, but I agree it requires a bunch of hurdles, so it's OK if we just like to go through `from_avro` / `to_avro`. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
