Github user koeninger commented on the pull request:
https://github.com/apache/spark/pull/11863#issuecomment-211466703
As far as the name, I don't really care. If you name it beta, one day it
will be GA. If you name it newapi, one day it will be the old api.
Stuff's going to change either way. Plenty of people won't read the docs,
but having beta in the name should make it realllly obvious what the state
of the code is.
As far as python support, I'm not really a python user, so probably not the
best person to ask as far as making a nice api for python. My bet is that
adequately mapping all of the possible things that can be done to set up
the underlying consumer is going to be quite hard.
As far as examples, I'd be fine with just copying over the stuff from
https://github.com/koeninger/kafka-exactly-once/tree/kafka-0.9
As far as KafkaUtils, we can add it back in to avoid rocking the boat, but
honestly I don't think we get much benefit from it. In the old consumer
it ended up being a dumping ground for random stuff from 3 or 4 different
classes.
On Fri, Apr 15, 2016 at 7:25 PM, Mark Grover <[email protected]>
wrote:
> Thanks for this, @koeninger <https://github.com/koeninger>! This is
> looking really good. I have made some comments inline.
>
> I noticed that there wasn't a new example, a new KafkaUtils, python
> support or documentation. So, we should take care of those too.
>
> Also, I'd personally prefer a different name than kafka-beta, because the
> new consumer API will become GA some day and I would rather not have us
> rename the project then and break everyone. I would personally prefer
> calling it some thing like kafka-newapi (or something else, it's not that
> important). We can doc that this subproject is beta quality but as time
> goes and the subproject and kafka's new consumer API matures, we don't
have
> to rename, just update the docs.
>
> And, I would definitely like to help out with the work you have started
> here. Whether it's implementing some of the comments (if you agree with
> them), or the left over things that I listed out earlier. Let me know what
> you think.
>
> Thanks again for working on this!
>
> â
> You are receiving this because you were mentioned.
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/11863#issuecomment-210696664>
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
