gaborgsomogyi commented on pull request #29729: URL: https://github.com/apache/spark/pull/29729#issuecomment-712837431
@HeartSaVioR > Personally I wouldn't mind if it's not compatible with 0.10/0.11 as 1.0 is released in Oct 2017 (already 3 years passed), but probably someone would mind if it's not supported in 1.x. Since we know the facts (`0.11.0.0+` is supported) I would say it's a fair price to eliminate a cluster stop issue (only 1.x support would be different). Naming the connector still `0-10...` is maybe not super elegant but I think renaming it would cause quite some trouble (this was the case last time when `kafka-clients` dependency increased to 2.x or so). > Btw, while doing exhaustive test is great, we might also want to ask this to the Kafka community, as we could claim again to provide a method on metadata update changing only client side if they confirm KIP-396 cannot be the resolution of issue on replacing poll(0). We can resurrect this part but I've heard from many sources that the offset fetching must happen from `AdminClient`. This is strategic thing from their perspective (and all the rest is hack). I'm maybe pessimistic but I don't see that metadata update functionality would appear in `Consumer` any time. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
