Github user markgrover commented on the pull request:
https://github.com/apache/spark/pull/11143#issuecomment-190845440
Kafka 0.9 still has the old high and low level consumers (and the old
producers too).
Correct but they are not compatible when using the 0.9 client with 0.8
brokers. So, your Kafka client app may need no code changes if you are using
the old consumer API but you'd still have to upgrade your broker first before
you upgrade your client.
Anyways, since @koeninger and @tdas don't feel comfortable about my
proposal upgrading to Kafka 0.9 (and dropping support for 0.8), I am working on
updating this PR to support 0.8 and 0.9. It's not just a simple maven profile,
because we use some non-private APIs (in AdminUtils, ZKClient, etc.) which have
changed between 0.8 and 0.9, so a) I have to either do a shim layer, or b)
change the code so it doesn't depend on those APIs. Will keep you all posted
here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
