viirya commented on pull request #31771: URL: https://github.com/apache/spark/pull/31771#issuecomment-795041758
Thanks for ping me, @dongjoon-hyun. I quickly go through previous comments and hope I don't miss anything. If anything I miss or misunderstand, please let me know. I think I can understand the comments from @cloud-fan @gengliangwang @HeartSaVioR. I agree that what this PR does currently is far from the Avro + Kafka + SR integration. Seems to me this only wants to fetch the latest schema from SR and so it does nothing with schema evolution here. I also agree that `avroSchemaUrl` looks like an option to achieve the goal. Although I also worry about it might be a bit confusing for users who wrongly think it as SR integration support. It sounds like @HeartSaVioR's option 4? I'm fine if others think it is okay to do. I guess @cloud-fan's idea about an advanced Kafka data source is another story. Actually I am not sure if the author expects a full integration of SR? Or just want something like option 4. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
