HeartSaVioR edited a comment on pull request #34089: URL: https://github.com/apache/spark/pull/34089#issuecomment-1038707810
I think it wouldn't happen to pull Kafka community to ask for the details on migration if I didn't concern it, but it was addressed in any way, so fair enough. I figured out KIP-679 is zero mentioned in "Notable changes" in Kafka 3.0 which feels me less comfortable to simply rely on Notable changes, but it would be never figured out if we didn't hear this from Kafka community and I didn't look into the details, so I'll consider this as like "I wasn't aware of". It is OK for me to not mention this, as Kafka community didn't mention this as one of major changes. I'll consider myself as wrong for determining importance of this. As a general comment, since Kafka 3.0 brings breaking changes I'd document the version changes into SS migration guide (we didn't), linking Notable changes in Kafka doc, and explicitly mention "please contact Kafka community for details of changes". At least it helps us to route end users' concerns to the Kafka community, keep us be transparent with changes of Kafka and leave our stance as just a one of usages of Kafka client. I don't see anyone actively participating on both communities, so for me it seems to be the only valid strategy we can take as of now. Another general comment is, I'd make sure we guarantee downgrading to Kafka 2.8.1 (in runtime or even with different set of artifacts) and give end users freedom to choose Kafka 2 vs 3. We simply consider like there is no demand for end users to stick with Kafka 2.8.1. If we were trying to provide separate Kafka data source artifacts for Kafka 3, I wouldn't concern at all. It's up to all others to take my general comment or ignore it. I wouldn't mind at all. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
