Re: [DISCUSS] Add missing API to old high level consumer

2015-05-15 Thread Gwen Shapira
I thought we wanted security on 0.8.3 too... the SSL + Authz patches seem close to ready, no? On Fri, May 15, 2015 at 3:56 AM, Joe Stein joe.st...@stealth.ly wrote: Hey Becket, yeah good point. Officially there is no 0.8.3 https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan

Re: [DISCUSS] Add missing API to old high level consumer

2015-05-15 Thread Joe Stein
I peeked at the Java producer SSL changes, haven't tried it yet though. I can see about getting a Go version to help testing compatibility done in the next few weeks. I still don't understand the Auth pieces, I haven't been able to make the KIP lately I need to try to attend like every other or

Re: [DISCUSS] Add missing API to old high level consumer

2015-05-15 Thread Gwen Shapira
Makes sense, thanks! When we added Authz to Sqoop, we tested with mock authorizers that would either allow everything or deny everything and would log every call. Perhaps this will make sense for Kafka too. Parth has the patch up, so we can see how he tested it. Gwen On Fri, May 15, 2015 at

Re: [DISCUSS] Add missing API to old high level consumer

2015-05-15 Thread Jun Rao
Hi, Jiangjie, It seems that api is already in kafka.javaapi.consumer.ZookeeperConsumerConnector. We just need to add it to kafka.javaapi.consumer.ConsumerConnector. This is fine and I don't think we need a KIP. Since this is only added in trunk, it will be part of the 0.8.3 release. I agree with

Re: [DISCUSS] Add missing API to old high level consumer

2015-05-15 Thread Jiangjie Qin
Sounds good. Thanks, Jun. Jiangjie (Becket) Qin On 5/15/15, 5:21 PM, Jun Rao j...@confluent.io wrote: Hi, Jiangjie, It seems that api is already in kafka.javaapi.consumer.ZookeeperConsumerConnector. We just need to add it to kafka.javaapi.consumer.ConsumerConnector. This is fine and I don't

Re: [DISCUSS] Add missing API to old high level consumer

2015-05-14 Thread Jiangjie Qin
Hey Joe, Actually this API was asked for before, and we have several use cases in LinkedIn as well. I thought we have added that in KAFKA-1650 but obviously I forgot to do that. My understanding is that we won¹t really deprecate high level consumer until we move to 0.9.0. So we can have this API

Re: [DISCUSS] Add missing API to old high level consumer

2015-05-14 Thread Joe Stein
Hey Becket, yeah good point. Officially there is no 0.8.3 https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan release planned. I agree we should have the new consumer beta and a patch for the old one. If we do that in 0.8.3 that makes good sense, yup. We should also include

Re: [DISCUSS] Add missing API to old high level consumer

2015-05-13 Thread Joe Stein
My gut reaction is that this isn't that important for folks otherwise they would have complained already. If it is a blocker for folks upgrading to 0.8.2.1 then we should do a 0.8.2.2 release with this fix in it. For 0.9.0. we are pushing for folks to start using the new consumer and that is the