Makes sense, thanks! When we added Authz to Sqoop, we tested with "mock" authorizers that would either allow everything or deny everything and would log every call. Perhaps this will make sense for Kafka too. Parth has the patch up, so we can see how he tested it.
Gwen On Fri, May 15, 2015 at 10:02 AM, Joe Stein <joe.st...@stealth.ly> wrote: > I peeked at the Java producer SSL changes, haven't tried it yet though. I > can see about getting a Go version to help testing compatibility done in > the next few weeks. > > I still don't understand the Auth pieces, I haven't been able to make the > KIP lately I need to try to attend like every other or something. > > I will re-read Auth this weekend. I guess my question still is how do I > test it. With SSL it is configure and it works or it doesn't. Auth is not > so straightforward and more opinionated. > > ~ Joe Stein > - - - - - - - - - - - - - - - - - > > http://www.stealth.ly > - - - - - - - - - - - - - - - - - > > On Fri, May 15, 2015 at 2:27 AM, Gwen Shapira <gshap...@cloudera.com> > wrote: > > > I thought we wanted security on 0.8.3 too... the SSL + Authz patches seem > > close to ready, no? > > > > On Fri, May 15, 2015 at 3:56 AM, Joe Stein <joe.st...@stealth.ly> wrote: > > > > > Hey Becket, yeah good point. Officially there is no 0.8.3 > > > https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan > > > release > > > planned. > > > > > > I agree we should have the new consumer beta and a patch for the old > one. > > > If we do that in 0.8.3 that makes good sense, yup. We should also > include > > > https://issues.apache.org/jira/browse/KAFKA-1694 server side admin in > > > 0.8.3 > > > too. We are testing that next week in a few languages to get it through > > > testing first before committing. > > > > > > Maybe we branch 0.8.3 in the near future so that can go through getting > > > released while the security changes for 0.9 get on trunk. > > > > > > ~ Joe Stein > > > > > > On Thu, May 14, 2015 at 8:00 PM, Jiangjie Qin > <j...@linkedin.com.invalid > > > > > > wrote: > > > > > > > Hey Joe, > > > > > > > > Actually this API was asked for before, and we have several use cases > > in > > > > LinkedIn as well. I thought we have added that in KAFKA-1650 but > > > obviously > > > > I forgot to do that. > > > > > > > > My understanding is that we won¹t really deprecate high level > consumer > > > > until we move to 0.9.0. So we can have this API either in 0.8.3 or > > > > 0.8.2.2. Do you mean we only add them to those releases but not put > it > > > > into trunk? Any specific concern on that? > > > > > > > > Considering this API has already been provided in new consumer. > Adding > > > > this method probably won¹t cause any API compatibility issue even if > > > > people move to new consumer later. > > > > Given it is both backward and forward compatible and is a one line > > > change, > > > > I think it is probably OK to have it added. > > > > > > > > Thanks, > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > > > > > On 5/13/15, 3:18 PM, "Joe Stein" <joe.st...@stealth.ly> wrote: > > > > > > > > >My gut reaction is that this isn't that important for folks > otherwise > > > they > > > > >would have complained already. If it is a blocker for folks > upgrading > > to > > > > >0.8.2.1 then we should do a 0.8.2.2 release with this fix in it. For > > > > >0.9.0. > > > > >we are pushing for folks to start using the new consumer and that is > > the > > > > >upgrade path we should continue on, imho. If we are going to phase > out > > > the > > > > >scala clients then we need to strive to not be making changes to > them > > on > > > > >trunk. > > > > > > > > > >~ Joe Stein > > > > >- - - - - - - - - - - - - - - - - > > > > > > > > > > http://www.stealth.ly > > > > >- - - - - - - - - - - - - - - - - > > > > > > > > > >On Wed, May 13, 2015 at 6:01 PM, Jiangjie Qin > > <j...@linkedin.com.invalid > > > > > > > > >wrote: > > > > > > > > > >> Add the DISCUSS prefix to the email title : ) > > > > >> > > > > >> From: Jiangjie Qin <j...@linkedin.com<mailto:j...@linkedin.com>> > > > > >> Date: Tuesday, May 12, 2015 at 4:51 PM > > > > >> To: "dev@kafka.apache.org<mailto:dev@kafka.apache.org>" < > > > > >> dev@kafka.apache.org<mailto:dev@kafka.apache.org>> > > > > >> Subject: Add missing API to old high level consumer > > > > >> > > > > >> Hi, > > > > >> > > > > >> I just noticed that in KAFKA-1650 (which is before we use KIP) we > > > added > > > > >>an > > > > >> offset commit method in high level consumer that commits offsets > > > using a > > > > >> user provided offset map. > > > > >> > > > > >> public void commitOffsets(Map<TopicPartition, OffsetAndMetadata> > > > > >> offsetsToCommit, boolean retryOnFailure); > > > > >> > > > > >> This method was added to all the Scala classes but I forgot to add > > it > > > to > > > > >> Java API of ConsumerConnector. (Already regretting now. . .) > > > > >> This method is very useful in several cases and has been asked for > > > from > > > > >> time to time. For example, people have several threads consuming > > > > >>messages > > > > >> and processing them. Without this method, one thread will > > unexpectedly > > > > >> commit offsets for another thread, thus might lose some messages > if > > > > >> something goes wrong. > > > > >> > > > > >> I created KAFKA-2186 and hope we can add this missing method into > > the > > > > >>Java > > > > >> API of old high level consumer (literarily one line change). > > > > >> Although this method should have been there since KAFKA-1650, > > adding > > > > >>this > > > > >> method to Java API now is a public API change, just want to see if > > > > >>people > > > > >> think we need a KIP for this. > > > > >> > > > > >> Thanks. > > > > >> > > > > >> Jiangjie (Becket) Qin > > > > >> > > > > > > > > > > > > > >
