But its just a compile-time dependency, right? Since the third-party-authorizer-implementation will be installed on a broker where the common classes will exist anyway.
On Wed, Apr 20, 2016 at 3:13 PM, Ashish Singh <asi...@cloudera.com> wrote: > Jay, > > Thanks for the info. I think having common in clients jar makes sense, as > their is no direct usage of it. i.e., without depending on or using > clients. Authorizer is a bit different, as third party implementations do > not really need anything from clients or server, all they need is > Authorizer interface and related classes. If we move authorizer into > common, then third party implementations will have to depend on clients. > Though third party implementations depending on clients is not a big > problem, right now they depend on core, I think it is cleaner to have > dependency on minimal modules. Would you agree? > > On Wed, Apr 20, 2016 at 2:27 PM, Jay Kreps <j...@confluent.io> wrote: > >> I think it's great that we're moving the interface to java and fixing some >> of the naming foibles. >> >> This isn't explicit in the KIP which just refers to the java package name >> (I think), but it looks like you are proposing adding a new authorizer jar >> for this new package and adding it as a dependency for the client jar. This >> is a bit inconsistent with how we decided to package stuff when we started >> with the new clients so it'd be good to work that out. >> >> To date the categorization has been: >> 1. Anything which is just in the clients is in org.apache.clients under >> clients/ >> 2. Anything which is in the server is kafka.* which is under core/ >> 3. Anything which is needed in both places (as it sounds like some enums >> for authorization are?) is in common which is under clients/ >> >> org.apache.clients and org.apache.common are both pure java and dependency >> free other than the compression libraries and slf4j and are packaged into >> the kafka-clients.java, the server has it's own jar which has richer >> dependencies and depends on the client jar. >> >> There are other ways this could have been done--e.g. common could have been >> its own library or even split into multiple sub-libraries--but the decision >> at that time was just to keep it simple and hard to mess up. Based on the >> experience with the scala clients our plan was to be ultra-hostile to any >> added client dependencies. >> >> So I think if we're continuing this model we would put the shared >> authorizer code somewhere under >> clients/src/main/java/org/apache/kafka/common as with the other shared >> authorizer. If we're moving away from this model we should probably rethink >> things and be consistent with this, at the very least splitting up common >> and clients. >> >> -Jay >> >> On Wed, Apr 20, 2016 at 1:04 PM, Ashish Singh <asi...@cloudera.com> wrote: >> >> > Jun/ Jay/ Gwen/ Harsha/ Ismael, >> > >> > As you guys have provided feedback on this earlier, could you review the >> > KIP again? I have updated the PR < >> https://github.com/apache/kafka/pull/861> >> > as >> > well. >> > >> > On Wed, Apr 20, 2016 at 1:01 PM, Ashish Singh <asi...@cloudera.com> >> wrote: >> > >> > > Hi Grant, >> > > >> > > On Tue, Apr 19, 2016 at 8:13 AM, Grant Henke <ghe...@cloudera.com> >> > wrote: >> > > >> > >> Hi Ashish, >> > >> >> > >> Thanks for the updates. I have a few questions below: >> > >> >> > >> > Move following interfaces to new package, >> org.apche.kafka.authorizer. >> > >> > >> > >> > 1. Authorizer >> > >> > 2. Acl >> > >> > 3. Operation >> > >> > 4. PermissionType >> > >> > 5. Resource >> > >> > 6. ResourceType >> > >> > 7. KafkaPrincipal >> > >> > 8. Session >> > >> > >> > >> > >> > >> This means the client would be required to depend on the authorizer >> > >> package >> > >> as a part of KIP-4. Another option is to have the client objects in >> > >> common. >> > >> Have we ruled out leaving the interface in the core module? >> > >> >> > > With this entities that use Authorizer will depend only on Authorizer >> > > package. Third party implementations can have only the authorizer pkg >> as >> > > dependency. core and client modules will also have to depend on the >> > > authorizer with this approach. Do you see any issue with it? >> > > >> > >> >> > >> Authorizer interface will be updated to remove getter naming >> convention. >> > >> >> > >> >> > >> Now that this is Java do we still want to change to the Scala naming >> > >> convention? >> > >> >> > > Even in clients module I do not see getter naming convention being >> > > followed, it is better to be consistent I guess. >> > > >> > >> >> > >> >> > >> Since we are completely rewriting the interface, can we add some (at >> > least >> > >> one to start with) standard exceptions that each method is recommended >> > to >> > >> use/throw? This will help the server in KIP-4 provide meaningful error >> > >> codes. KAFKA-3507 <https://issues.apache.org/jira/browse/KAFKA-3507> >> is >> > >> tracking it right now. >> > >> >> > > That should be good to have. Will include that. Thanks. >> > > >> > >> >> > >> Thanks, >> > >> Grant >> > >> >> > >> >> > >> >> > >> On Tue, Apr 19, 2016 at 9:48 AM, Ashish Singh <asi...@cloudera.com> >> > >> wrote: >> > >> >> > >> > I have updated KIP-50 >> > >> > < >> > >> > >> > >> >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-50+-+Move+Authorizer+to+a+separate+package >> > >> > > >> > >> > and PR <https://github.com/apache/kafka/pull/861> as per recent >> > >> > discussions. Please take a look. >> > >> > >> > >> > @Harsha / Don, it would be nice if you guys can review the KIP and >> PR >> > as >> > >> > well. >> > >> > >> > >> > >> > >> > On Mon, Apr 11, 2016 at 7:36 PM, Ashish Singh <asi...@cloudera.com> >> > >> wrote: >> > >> > >> > >> > > Yes, Jun. I would like to try get option 2 in, if possible in >> 0.10. >> > I >> > >> am >> > >> > > not asking for delaying 0.10 for it, but some reviews and early >> > >> feedback >> > >> > > would be great. At this point this is what I have in mind. >> > >> > > >> > >> > > 1. Move authorizer and related entities to its own package. Note >> > that >> > >> I >> > >> > am >> > >> > > proposing to drop scala interface completely. Ranger team is fine >> > >> with it >> > >> > > and I will update Sentry. >> > >> > > 2. The only new public method that will be added to authorizer >> > >> interface >> > >> > > is description(). >> > >> > > 3. Update SimpleAclAuthorizer to use the new interface and >> classes. >> > >> > > >> > >> > > On Mon, Apr 11, 2016 at 6:38 PM, Jun Rao <j...@confluent.io> >> wrote: >> > >> > > >> > >> > >> Ashish, >> > >> > >> >> > >> > >> So, you want to take a shot at option 2 for 0.10.0? That's fine >> > with >> > >> me >> > >> > >> too. I am just not sure if we have enough time to think through >> the >> > >> > >> changes. >> > >> > >> >> > >> > >> Thanks, >> > >> > >> >> > >> > >> Jun >> > >> > >> >> > >> > >> On Mon, Apr 11, 2016 at 6:05 PM, Ashish Singh < >> asi...@cloudera.com >> > > >> > >> > >> wrote: >> > >> > >> >> > >> > >> > Hello Jun, >> > >> > >> > >> > >> > >> > The 3rd option will require Apache Sentry to go GA with current >> > >> > >> authorizer >> > >> > >> > interface, and at this point it seems that the interface won't >> > last >> > >> > >> long. >> > >> > >> > Within a few months, Sentry will have to make a breaking >> change. >> > I >> > >> do >> > >> > >> > understand that Kafka should not have to delay its release due >> to >> > >> one >> > >> > of >> > >> > >> > the authorizer implementations. However, can we assist Sentry >> > >> users to >> > >> > >> > avoid that breaking upgrade? I think it is worth a shot. If the >> > >> > changes >> > >> > >> are >> > >> > >> > not done by 0.10 code freeze, then sure lets punt it to next >> > >> release. >> > >> > >> Does >> > >> > >> > this seem reasonable to you? >> > >> > >> > >> > >> > >> > On Sun, Apr 10, 2016 at 11:42 AM, Jun Rao <j...@confluent.io> >> > >> wrote: >> > >> > >> > >> > >> > >> > > Ashish, >> > >> > >> > > >> > >> > >> > > A 3rd option is to in 0.10.0, just sanity check the principal >> > >> type >> > >> > in >> > >> > >> the >> > >> > >> > > implementation of addAcls/removeAcls of Authorizer, but don't >> > >> change >> > >> > >> the >> > >> > >> > > Authorizer api to add the getDescription() method. This fixes >> > the >> > >> > >> > immediate >> > >> > >> > > issue that an acl rule with the wrong principal type is >> > silently >> > >> > >> ignored. >> > >> > >> > > Knowing valid user types is nice, but not critical (we can >> > >> include >> > >> > the >> > >> > >> > > supported user type in the UnsupportedPrincipalTypeException >> > >> thrown >> > >> > >> from >> > >> > >> > > addAcls/removeAcls). This will give us more time to clean up >> > the >> > >> > >> > Authorizer >> > >> > >> > > api post 0.10.0. >> > >> > >> > > >> > >> > >> > > Thanks >> > >> > >> > > >> > >> > >> > > Jun >> > >> > >> > > >> > >> > >> > > On Fri, Apr 8, 2016 at 9:04 AM, Ashish Singh < >> > >> asi...@cloudera.com> >> > >> > >> > wrote: >> > >> > >> > > >> > >> > >> > > > Thanks for the input Don. One of the possible paths for >> > Option >> > >> 2 >> > >> > is >> > >> > >> to >> > >> > >> > > > completely drop Scala interface, would that be Ok with you >> > >> folks? >> > >> > >> > > > >> > >> > >> > > > On Thursday, April 7, 2016, Don Bosco Durai < >> > bo...@apache.org> >> > >> > >> wrote: >> > >> > >> > > > >> > >> > >> > > > > Ranger team would prefer option #2. Right now, we have to >> > >> access >> > >> > >> some >> > >> > >> > > of >> > >> > >> > > > > the nested constants using constructs like >> Group$.MODULE$, >> > >> which >> > >> > >> is >> > >> > >> > not >> > >> > >> > > > > intuitive in Java. >> > >> > >> > > > > >> > >> > >> > > > > Thanks >> > >> > >> > > > > >> > >> > >> > > > > Bosco >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > On 4/7/16, 4:30 PM, "Ashish Singh" <asi...@cloudera.com >> > >> > >> > > <javascript:;>> >> > >> > >> > > > > wrote: >> > >> > >> > > > > >> > >> > >> > > > > >Harsha/ Don, >> > >> > >> > > > > > >> > >> > >> > > > > >Are you guys OK with option 2? I am not aware of all the >> > >> > existing >> > >> > >> > > > > >authorizer implementations, however ranger has one for >> > sure. >> > >> > >> Getting >> > >> > >> > > > > direct >> > >> > >> > > > > >feedback from you guys will be really valuable. >> > >> > >> > > > > > >> > >> > >> > > > > >On Thu, Apr 7, 2016 at 3:52 PM, Ismael Juma < >> > >> ism...@juma.me.uk >> > >> > >> > > > > <javascript:;>> wrote: >> > >> > >> > > > > > >> > >> > >> > > > > >> Hi Don, >> > >> > >> > > > > >> >> > >> > >> > > > > >> This is true in Java 7, but Java 8 introduces default >> > >> methods >> > >> > >> and >> > >> > >> > > this >> > >> > >> > > > > >> workaround is no longer required. During the >> Interceptor >> > >> KIP >> > >> > >> > > > > discussion, it >> > >> > >> > > > > >> was decided that it was fine to stick to interfaces >> > given >> > >> > that >> > >> > >> we >> > >> > >> > > are >> > >> > >> > > > > >> likely to move to Java 8 in the nearish future >> (probably >> > >> no >> > >> > >> later >> > >> > >> > > than >> > >> > >> > > > > the >> > >> > >> > > > > >> Java 9 release). >> > >> > >> > > > > >> >> > >> > >> > > > > >> Ismael >> > >> > >> > > > > >> >> > >> > >> > > > > >> On Thu, Apr 7, 2016 at 11:36 PM, Don Bosco Durai < >> > >> > >> > bo...@apache.org >> > >> > >> > > > > <javascript:;>> wrote: >> > >> > >> > > > > >> >> > >> > >> > > > > >> > Hi Ashish >> > >> > >> > > > > >> > >> > >> > >> > > > > >> > If we are going by option #2, then I can suggest we >> > >> give an >> > >> > >> > > abstract >> > >> > >> > > > > >> > implementation of the Interface and recommend anyone >> > >> > >> > implementing >> > >> > >> > > > > their >> > >> > >> > > > > >> own >> > >> > >> > > > > >> > plugin to extend from the abstract class, rather >> than >> > >> > >> implement >> > >> > >> > > the >> > >> > >> > > > > >> > interface? >> > >> > >> > > > > >> > >> > >> > >> > > > > >> > The advantage is, in the future if we add add any >> new >> > >> > >> methods in >> > >> > >> > > the >> > >> > >> > > > > >> > Interface (e.g. Similar to getDescription()), then >> we >> > >> can >> > >> > >> give a >> > >> > >> > > > dummy >> > >> > >> > > > > >> > implementation of the new method and this won’t >> break >> > >> the >> > >> > >> > > > compilation >> > >> > >> > > > > of >> > >> > >> > > > > >> > any external implementation. Else over the time it >> > will >> > >> be >> > >> > >> > > > challenging >> > >> > >> > > > > >> for >> > >> > >> > > > > >> > anyone customizing the implementation to keep track >> of >> > >> > >> changes >> > >> > >> > to >> > >> > >> > > > the >> > >> > >> > > > > >> > Interface. >> > >> > >> > > > > >> > >> > >> > >> > > > > >> > Thanks >> > >> > >> > > > > >> > >> > >> > >> > > > > >> > Bosco >> > >> > >> > > > > >> > >> > >> > >> > > > > >> > >> > >> > >> > > > > >> > >> > >> > >> > > > > >> > >> > >> > >> > > > > >> > On 4/7/16, 11:21 AM, "Ashish Singh" < >> > >> asi...@cloudera.com >> > >> > >> > > > > <javascript:;>> wrote: >> > >> > >> > > > > >> > >> > >> > >> > > > > >> > >Hello Harsha, >> > >> > >> > > > > >> > > >> > >> > >> > > > > >> > >On Thu, Apr 7, 2016 at 11:03 AM, Harsha < >> > >> m...@harsha.io >> > >> > >> > > > > <javascript:;>> wrote: >> > >> > >> > > > > >> > > >> > >> > >> > > > > >> > >"My only ask is to have this in 0.10. As Jay >> pointed >> > >> out, >> > >> > >> right >> > >> > >> > > now >> > >> > >> > > > > >> > >> there >> > >> > >> > > > > >> > >> are not many implementations out there, we might >> > >> want to >> > >> > >> fix >> > >> > >> > it >> > >> > >> > > > > ASAP." >> > >> > >> > > > > >> > >> >> > >> > >> > > > > >> > >> Probably there aren't many implementations but >> > there >> > >> are >> > >> > >> lot >> > >> > >> > of >> > >> > >> > > > > users >> > >> > >> > > > > >> > >> using these implementations in production >> clusters. >> > >> > >> > > > > >> > >> Isn't this going to break the rolling upgrade? >> > >> > >> > > > > >> > > >> > >> > >> > > > > >> > >It will and it is a concern, in my previous mail I >> > have >> > >> > >> > mentioned >> > >> > >> > > > > this >> > >> > >> > > > > >> as >> > >> > >> > > > > >> > >an issue if we choose to go this route. However, if >> > we >> > >> > >> actually >> > >> > >> > > > > decide >> > >> > >> > > > > >> to >> > >> > >> > > > > >> > >do this, I would say it is better to do it sooner >> > than >> > >> > >> later, >> > >> > >> > as >> > >> > >> > > > > fewer >> > >> > >> > > > > >> > >implementations will be affected. Below is excerpt >> > >> from my >> > >> > >> > > previous >> > >> > >> > > > > >> mail. >> > >> > >> > > > > >> > > >> > >> > >> > > > > >> > >Increase scope of KIP-50 to move authorizer and >> > related >> > >> > >> classes >> > >> > >> > > to >> > >> > >> > > > a >> > >> > >> > > > > >> > >separate package. The new package will have java >> > >> > interface. >> > >> > >> > This >> > >> > >> > > > will >> > >> > >> > > > > >> > allow >> > >> > >> > > > > >> > >implementations to not depend on kafka core and >> just >> > on >> > >> > >> > > authorizer >> > >> > >> > > > > >> > package, >> > >> > >> > > > > >> > >make authorization interface follow kafka’s coding >> > >> > standards >> > >> > >> > and >> > >> > >> > > > will >> > >> > >> > > > > >> > allow >> > >> > >> > > > > >> > >java implementations to be cleaner. We can either >> > >> > completely >> > >> > >> > drop >> > >> > >> > > > > scala >> > >> > >> > > > > >> > >interface, which might be a pain for existing >> > >> > >> implementations, >> > >> > >> > or >> > >> > >> > > > we >> > >> > >> > > > > can >> > >> > >> > > > > >> > >have scala interface wrap java interface. Later >> > allows >> > >> a >> > >> > >> > cleaner >> > >> > >> > > > > >> > >deprecation path for existing scala authorizer >> > >> interface, >> > >> > >> > however >> > >> > >> > > > it >> > >> > >> > > > > may >> > >> > >> > > > > >> > or >> > >> > >> > > > > >> > >may not be feasible as Kafka server will have to >> > >> somehow >> > >> > >> decide >> > >> > >> > > > which >> > >> > >> > > > > >> > >interface it should be looking for while loading >> > >> > authorizer >> > >> > >> > > > > >> > implementation, >> > >> > >> > > > > >> > >this can probably be solved with a config or some >> > >> > >> reflection. >> > >> > >> > If >> > >> > >> > > we >> > >> > >> > > > > >> choose >> > >> > >> > > > > >> > >to go this route, I can dig deeper. >> > >> > >> > > > > >> > > >> > >> > >> > > > > >> > >If we go with option 2 and commit on getting this >> in >> > >> ASAP, >> > >> > >> > > > > preferably in >> > >> > >> > > > > >> > >0.10, there will be fewer implementations that will >> > be >> > >> > >> > affected. >> > >> > >> > > > > >> > > >> > >> > >> > > > > >> > >and also moving to Java , >> > >> > >> > > > > >> > >> a authorizer implementation going to run inside a >> > >> > >> KafkaBroker >> > >> > >> > > > and I >> > >> > >> > > > > >> > >> don't see why this is necessary to move to >> clients >> > >> > >> package. >> > >> > >> > > > > >> > >> Are we planning on introducing common module to >> > have >> > >> it >> > >> > >> > > > > independent of >> > >> > >> > > > > >> > >> broker and client code? >> > >> > >> > > > > >> > >> >> > >> > >> > > > > >> > >Yes, I think that would take away the requirement >> of >> > >> > >> depending >> > >> > >> > on >> > >> > >> > > > > Kafka >> > >> > >> > > > > >> > >core from authorizer implementations. Do you >> suggest >> > >> > >> otherwise? >> > >> > >> > > > > >> > > >> > >> > >> > > > > >> > > >> > >> > >> > > > > >> > >> -Harsha >> > >> > >> > > > > >> > >> >> > >> > >> > > > > >> > >> On Thu, Apr 7, 2016, at 10:52 AM, Ashish Singh >> > wrote: >> > >> > >> > > > > >> > >> > We might want to take a call here. Following >> are >> > >> the >> > >> > >> > options. >> > >> > >> > > > > >> > >> > >> > >> > >> > > > > >> > >> > 1. Let KIP-50 be the way it is, i.e., just >> add >> > >> > >> > > > > getDescription() >> > >> > >> > > > > >> to >> > >> > >> > > > > >> > >> > existing scala authorizer interface. It will >> > >> break >> > >> > >> > binary >> > >> > >> > > > > >> > >> > compatibility >> > >> > >> > > > > >> > >> > (only when using CLI and/or AdminCommand >> from >> > >= >> > >> > 0.10 >> > >> > >> > > > against >> > >> > >> > > > > >> > >> > authorizer >> > >> > >> > > > > >> > >> > implementations based on 0.9.). If we go >> this >> > >> > route, >> > >> > >> it >> > >> > >> > > is a >> > >> > >> > > > > >> > simpler >> > >> > >> > > > > >> > >> > change >> > >> > >> > > > > >> > >> > and existing implementations won’t have to >> > >> change >> > >> > >> > anything >> > >> > >> > > > on >> > >> > >> > > > > >> their >> > >> > >> > > > > >> > >> > end. >> > >> > >> > > > > >> > >> > 2. Increase scope of KIP-50 to move >> authorizer >> > >> and >> > >> > >> > related >> > >> > >> > > > > >> classes >> > >> > >> > > > > >> > to >> > >> > >> > > > > >> > >> > a >> > >> > >> > > > > >> > >> > separate package. The new package will have >> > java >> > >> > >> > > interface. >> > >> > >> > > > > This >> > >> > >> > > > > >> > will >> > >> > >> > > > > >> > >> > allow >> > >> > >> > > > > >> > >> > implementations to not depend on kafka core >> > and >> > >> > just >> > >> > >> on >> > >> > >> > > > > >> authorizer >> > >> > >> > > > > >> > >> > package, >> > >> > >> > > > > >> > >> > make authorization interface follow kafka’s >> > >> coding >> > >> > >> > > standards >> > >> > >> > > > > and >> > >> > >> > > > > >> > will >> > >> > >> > > > > >> > >> > allow >> > >> > >> > > > > >> > >> > java implementations to be cleaner. We can >> > >> either >> > >> > >> > > completely >> > >> > >> > > > > drop >> > >> > >> > > > > >> > >> > scala >> > >> > >> > > > > >> > >> > interface, which might be a pain for >> existing >> > >> > >> > > > > implementations, or >> > >> > >> > > > > >> > we >> > >> > >> > > > > >> > >> > can >> > >> > >> > > > > >> > >> > have scala interface wrap java interface. >> > Later >> > >> > >> allows a >> > >> > >> > > > > cleaner >> > >> > >> > > > > >> > >> > deprecation path for existing scala >> authorizer >> > >> > >> > interface, >> > >> > >> > > > > however >> > >> > >> > > > > >> > it >> > >> > >> > > > > >> > >> > may or >> > >> > >> > > > > >> > >> > may not be feasible as Kafka server will >> have >> > to >> > >> > >> somehow >> > >> > >> > > > > decide >> > >> > >> > > > > >> > which >> > >> > >> > > > > >> > >> > interface it should be looking for while >> > loading >> > >> > >> > > authorizer >> > >> > >> > > > > >> > >> > implementation, >> > >> > >> > > > > >> > >> > this can probably be solved with a config or >> > >> some >> > >> > >> > > > reflection. >> > >> > >> > > > > If >> > >> > >> > > > > >> we >> > >> > >> > > > > >> > >> > choose >> > >> > >> > > > > >> > >> > to go this route, I can dig deeper. >> > >> > >> > > > > >> > >> > >> > >> > >> > > > > >> > >> > If we decide to go with option 1, I think it >> > would >> > >> be >> > >> > >> fair >> > >> > >> > to >> > >> > >> > > > say >> > >> > >> > > > > >> that >> > >> > >> > > > > >> > >> > scala authorizer interface will be around for >> > some >> > >> > >> time, as >> > >> > >> > > > there >> > >> > >> > > > > >> > will be >> > >> > >> > > > > >> > >> > more implementations relying on it. If we go >> with >> > >> > >> option 2 >> > >> > >> > > and >> > >> > >> > > > > >> commit >> > >> > >> > > > > >> > on >> > >> > >> > > > > >> > >> > getting this in ASAP, preferably in 0.10, there >> > >> will >> > >> > be >> > >> > >> > fewer >> > >> > >> > > > > >> > >> > implementations that will be affected. >> > >> > >> > > > > >> > >> > >> > >> > >> > > > > >> > >> > *Another thing to notice is that scala >> authorizer >> > >> > >> interface >> > >> > >> > > is >> > >> > >> > > > > not >> > >> > >> > > > > >> > >> > annotated as unstable.* >> > >> > >> > > > > >> > >> > >> > >> > >> > > > > >> > >> > >> > >> > >> > > > > >> > >> > On Wed, Apr 6, 2016 at 9:41 AM, Ashish Singh < >> > >> > >> > > > > asi...@cloudera.com <javascript:;>> >> > >> > >> > > > > >> > >> wrote: >> > >> > >> > > > > >> > >> > >> > >> > >> > > > > >> > >> > > I see value in minimizing breaking changes >> and >> > I >> > >> do >> > >> > >> not >> > >> > >> > > > oppose >> > >> > >> > > > > the >> > >> > >> > > > > >> > >> idea of >> > >> > >> > > > > >> > >> > > increasing scope of KIP-50 to move auth >> > >> interface to >> > >> > >> > java. >> > >> > >> > > > > >> > >> > > >> > >> > >> > > > > >> > >> > > As authorizer implementations do not really >> > need >> > >> to >> > >> > >> > depend >> > >> > >> > > on >> > >> > >> > > > > >> Kafka >> > >> > >> > > > > >> > >> core, >> > >> > >> > > > > >> > >> > > I would suggest that we keep authorizer >> > interface >> > >> > and >> > >> > >> its >> > >> > >> > > > > >> components >> > >> > >> > > > > >> > >> in a >> > >> > >> > > > > >> > >> > > separate package. I share the concern that >> > right >> > >> now >> > >> > >> > using >> > >> > >> > > > > >> Resource, >> > >> > >> > > > > >> > >> > > Operation, etc, in java implementations is >> > >> messy. I >> > >> > >> had >> > >> > >> > to >> > >> > >> > > > deal >> > >> > >> > > > > >> with >> > >> > >> > > > > >> > >> lot of >> > >> > >> > > > > >> > >> > > it while writing Apache Sentry plugin. >> > >> > >> > > > > >> > >> > > >> > >> > >> > > > > >> > >> > > My only ask is to have this in 0.10. As Jay >> > >> pointed >> > >> > >> out, >> > >> > >> > > > right >> > >> > >> > > > > now >> > >> > >> > > > > >> > >> there >> > >> > >> > > > > >> > >> > > are not many implementations out there, we >> > might >> > >> > want >> > >> > >> to >> > >> > >> > > fix >> > >> > >> > > > it >> > >> > >> > > > > >> > ASAP. >> > >> > >> > > > > >> > >> I can >> > >> > >> > > > > >> > >> > > only speak of Sentry integration and I think >> > 0.10 >> > >> > >> will be >> > >> > >> > > the >> > >> > >> > > > > best >> > >> > >> > > > > >> > for >> > >> > >> > > > > >> > >> such >> > >> > >> > > > > >> > >> > > a change, as I should be able to adopt the >> > >> changes >> > >> > in >> > >> > >> > > Sentry >> > >> > >> > > > > >> > >> integration >> > >> > >> > > > > >> > >> > > before a lot of users start using it. >> > >> > >> > > > > >> > >> > > >> > >> > >> > > > > >> > >> > > On Wed, Apr 6, 2016 at 9:25 AM, Ismael Juma < >> > >> > >> > > > ism...@juma.me.uk >> > >> > >> > > > > <javascript:;>> >> > >> > >> > > > > >> > wrote: >> > >> > >> > > > > >> > >> > > >> > >> > >> > > > > >> > >> > >> It is small, but breaks binary >> compatibility. >> > >> > >> > > > > >> > >> > >> >> > >> > >> > > > > >> > >> > >> Ismael >> > >> > >> > > > > >> > >> > >> >> > >> > >> > > > > >> > >> > >> On Wed, Apr 6, 2016 at 5:20 PM, Grant Henke >> < >> > >> > >> > > > > ghe...@cloudera.com <javascript:;> >> > >> > >> > > > > >> > >> > >> > >> > > > > >> > >> wrote: >> > >> > >> > > > > >> > >> > >> >> > >> > >> > > > > >> > >> > >> > KIP-50 as defined is very small. I don't >> see >> > >> any >> > >> > >> harm >> > >> > >> > in >> > >> > >> > > > > >> putting >> > >> > >> > > > > >> > it >> > >> > >> > > > > >> > >> in >> > >> > >> > > > > >> > >> > >> as >> > >> > >> > > > > >> > >> > >> > is and then tackling the follow up work. >> > >> > >> > > > > >> > >> > >> > >> > >> > >> > > > > >> > >> > >> > >> > >> > >> > > > > >> > >> > >> > On Wed, Apr 6, 2016 at 11:16 AM, Ismael >> > Juma < >> > >> > >> > > > > >> ism...@juma.me.uk <javascript:;>> >> > >> > >> > > > > >> > >> wrote: >> > >> > >> > > > > >> > >> > >> > >> > >> > >> > > > > >> > >> > >> > > Thanks Grant. I wonder if KIP-50 should >> > >> just be >> > >> > >> done >> > >> > >> > > as >> > >> > >> > > > > part >> > >> > >> > > > > >> of >> > >> > >> > > > > >> > >> this >> > >> > >> > > > > >> > >> > >> > work. >> > >> > >> > > > > >> > >> > >> > > >> > >> > >> > > > > >> > >> > >> > > Ismael >> > >> > >> > > > > >> > >> > >> > > >> > >> > >> > > > > >> > >> > >> > > On Wed, Apr 6, 2016 at 5:12 PM, Grant >> > Henke >> > >> < >> > >> > >> > > > > >> > ghe...@cloudera.com <javascript:;>> >> > >> > >> > > > > >> > >> > >> wrote: >> > >> > >> > > > > >> > >> > >> > > >> > >> > >> > > > > >> > >> > >> > > > My work with KIP-4 found that many of >> > the >> > >> > Scala >> > >> > >> > > > classes >> > >> > >> > > > > >> used >> > >> > >> > > > > >> > in >> > >> > >> > > > > >> > >> the >> > >> > >> > > > > >> > >> > >> > > > Authorizer interface are needed in the >> > >> > Clients >> > >> > >> > > package >> > >> > >> > > > > when >> > >> > >> > > > > >> > >> adding >> > >> > >> > > > > >> > >> > >> the >> > >> > >> > > > > >> > >> > >> > > > various ACL requests and responses. I >> > also >> > >> > >> found >> > >> > >> > > that >> > >> > >> > > > we >> > >> > >> > > > > >> > don't >> > >> > >> > > > > >> > >> have >> > >> > >> > > > > >> > >> > >> > > > standard Exceptions defined for the >> > >> > authorizer >> > >> > >> > > > > interface. >> > >> > >> > > > > >> > This >> > >> > >> > > > > >> > >> means >> > >> > >> > > > > >> > >> > >> > that >> > >> > >> > > > > >> > >> > >> > > > when I add the Authorizer calls to the >> > >> broker >> > >> > >> and >> > >> > >> > > wire >> > >> > >> > > > > >> > >> protocols all >> > >> > >> > > > > >> > >> > >> > > > exceptions will be reported as an >> > "Unknown >> > >> > >> Error" >> > >> > >> > > back >> > >> > >> > > > > to >> > >> > >> > > > > >> the >> > >> > >> > > > > >> > >> user >> > >> > >> > > > > >> > >> > >> via >> > >> > >> > > > > >> > >> > >> > > the >> > >> > >> > > > > >> > >> > >> > > > wire protocol. >> > >> > >> > > > > >> > >> > >> > > > >> > >> > >> > > > > >> > >> > >> > > > I have written more about it on the >> > KIP-4 >> > >> > wiki >> > >> > >> and >> > >> > >> > > > > created >> > >> > >> > > > > >> > >> jiras to >> > >> > >> > > > > >> > >> > >> > track >> > >> > >> > > > > >> > >> > >> > > > those issues (See below). I think we >> > >> should >> > >> > >> wrap >> > >> > >> > up >> > >> > >> > > > this >> > >> > >> > > > > >> KIP >> > >> > >> > > > > >> > as >> > >> > >> > > > > >> > >> is >> > >> > >> > > > > >> > >> > >> and >> > >> > >> > > > > >> > >> > >> > > > tackle the Java/Exception changes as a >> > >> part >> > >> > of >> > >> > >> > those >> > >> > >> > > > > >> > jiras/kips. >> > >> > >> > > > > >> > >> > >> > > > >> > >> > >> > > > > >> > >> > >> > > > - KIP-4 "Follow Up Changes" >> > >> > >> > > > > >> > >> > >> > > > < >> > >> > >> > > > > >> > >> > >> > > > >> > >> > >> > > > > >> > >> > >> > > >> > >> > >> > > > > >> > >> > >> > >> > >> > >> > > > > >> > >> > >> >> > >> > >> > > > > >> > >> >> > >> > >> > > > > >> > >> > >> > >> > > > > >> >> > >> > >> > > > > >> > >> > >> > > > >> > >> > >> > > >> > >> > >> > >> > >> > >> >> > >> > >> > >> >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations#KIP-4-Commandlineandcentralizedadministrativeoperations-FollowUpChangesfollow-up-changes >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > - KAFKA-3509 < >> > >> > >> > > > > >> > >> https://issues.apache.org/jira/browse/KAFKA-3509 >> >: >> > >> > >> > > > > >> > >> > >> > > > Provide >> > >> > >> > > > > >> > >> > >> > > > an Authorizer interface using the >> > Java >> > >> > >> client >> > >> > >> > > > > enumerator >> > >> > >> > > > > >> > >> classes >> > >> > >> > > > > >> > >> > >> > > > - KAFKA-3507 < >> > >> > >> > > > > >> > >> https://issues.apache.org/jira/browse/KAFKA-3507 >> >: >> > >> > >> > > > > >> > >> > >> > > Define >> > >> > >> > > > > >> > >> > >> > > > standard exceptions for the >> > Authorizer >> > >> > >> > interface >> > >> > >> > > > > >> > >> > >> > > > >> > >> > >> > > > > >> > >> > >> > > > Thank you, >> > >> > >> > > > > >> > >> > >> > > > Grant >> > >> > >> > > > > >> > >> > >> > > > >> > >> > >> > > > > >> > >> > >> > > > On Wed, Apr 6, 2016 at 10:58 AM, Jay >> > >> Kreps < >> > >> > >> > > > > >> j...@confluent.io <javascript:;> >> > >> > >> > > > > >> > > >> > >> > >> > > > > >> > >> > >> wrote: >> > >> > >> > > > > >> > >> > >> > > > >> > >> > >> > > > > >> > >> > >> > > > > Hey Ismael, >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > Yeah I think this is a minor >> > cleanliness >> > >> > >> thing. >> > >> > >> > > > Since >> > >> > >> > > > > >> this >> > >> > >> > > > > >> > is >> > >> > >> > > > > >> > >> kind >> > >> > >> > > > > >> > >> > >> > of a >> > >> > >> > > > > >> > >> > >> > > > > power user interface I don't feel >> > >> strongly >> > >> > >> > either >> > >> > >> > > > way. >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > My motivation with Scala is just >> that >> > >> we've >> > >> > >> > tried >> > >> > >> > > to >> > >> > >> > > > > move >> > >> > >> > > > > >> > to >> > >> > >> > > > > >> > >> > >> having >> > >> > >> > > > > >> > >> > >> > the >> > >> > >> > > > > >> > >> > >> > > > > public interfaces be Java, and as a >> > >> group >> > >> > we >> > >> > >> > > > > definitely >> > >> > >> > > > > >> > >> struggled >> > >> > >> > > > > >> > >> > >> a >> > >> > >> > > > > >> > >> > >> > lot >> > >> > >> > > > > >> > >> > >> > > > > with understanding and maintaining >> > Scala >> > >> > >> > > > > compatibility in >> > >> > >> > > > > >> > the >> > >> > >> > > > > >> > >> > >> older >> > >> > >> > > > > >> > >> > >> > > > > clients. >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > -Jay >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > On Tue, Apr 5, 2016 at 11:46 PM, >> > Ismael >> > >> > Juma >> > >> > >> < >> > >> > >> > > > > >> > >> ism...@juma.me.uk <javascript:;>> >> > >> > >> > > > > >> > >> > >> > > wrote: >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > > Hi Jay, >> > >> > >> > > > > >> > >> > >> > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > On Wed, Apr 6, 2016 at 3:48 AM, >> Jay >> > >> > Kreps < >> > >> > >> > > > > >> > j...@confluent.io <javascript:;> >> > >> > >> > > > > >> > >> > >> > >> > >> > > > > >> > >> > >> > wrote: >> > >> > >> > > > > >> > >> > >> > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > > Given that we're breaking >> > >> compatibility >> > >> > >> > anyway >> > >> > >> > > > > should >> > >> > >> > > > > >> > we: >> > >> > >> > > > > >> > >> > >> > > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > We are not breaking source >> > >> compatibility >> > >> > >> since >> > >> > >> > > the >> > >> > >> > > > > new >> > >> > >> > > > > >> > >> method >> > >> > >> > > > > >> > >> > >> has a >> > >> > >> > > > > >> > >> > >> > > > > default >> > >> > >> > > > > >> > >> > >> > > > > > implementation. I take it that you >> > >> mean >> > >> > >> binary >> > >> > >> > > > > >> > >> compatibility? >> > >> > >> > > > > >> > >> > >> > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > > 1. Remove the get prefix on this >> > >> method >> > >> > >> and >> > >> > >> > > the >> > >> > >> > > > > >> > existing >> > >> > >> > > > > >> > >> one >> > >> > >> > > > > >> > >> > >> > which >> > >> > >> > > > > >> > >> > >> > > > > > violate >> > >> > >> > > > > >> > >> > >> > > > > > > our own code style guidelines >> > (Oops! >> > >> > >> Kind of >> > >> > >> > > sad >> > >> > >> > > > > we >> > >> > >> > > > > >> > went >> > >> > >> > > > > >> > >> > >> through >> > >> > >> > > > > >> > >> > >> > > the >> > >> > >> > > > > >> > >> > >> > > > > > whole >> > >> > >> > > > > >> > >> > >> > > > > > > KIP process and no one even >> > flagged >> > >> > this) >> > >> > >> > > > > >> > >> > >> > > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > I did flag this during the >> > discussion >> > >> and >> > >> > >> > Ashish >> > >> > >> > > > > said >> > >> > >> > > > > >> he >> > >> > >> > > > > >> > >> would >> > >> > >> > > > > >> > >> > >> > change >> > >> > >> > > > > >> > >> > >> > > > it >> > >> > >> > > > > >> > >> > >> > > > > if >> > >> > >> > > > > >> > >> > >> > > > > > other people felt that it should >> be >> > >> > >> changed. >> > >> > >> > > > > >> > >> > >> > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > > 2. Move the interface out of >> scala >> > >> to >> > >> > be >> > >> > >> a >> > >> > >> > > > normal >> > >> > >> > > > > >> Java >> > >> > >> > > > > >> > >> > >> interface >> > >> > >> > > > > >> > >> > >> > > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > > This breaks source compatibility >> > but >> > >> > >> > probably >> > >> > >> > > > > what we >> > >> > >> > > > > >> > >> should >> > >> > >> > > > > >> > >> > >> have >> > >> > >> > > > > >> > >> > >> > > > done >> > >> > >> > > > > >> > >> > >> > > > > > > originally I suspect. Probably >> > there >> > >> > are >> > >> > >> few >> > >> > >> > > > > enough >> > >> > >> > > > > >> > >> > >> > implementations >> > >> > >> > > > > >> > >> > >> > > > of >> > >> > >> > > > > >> > >> > >> > > > > > this >> > >> > >> > > > > >> > >> > >> > > > > > > that it is better to just rip >> the >> > >> > bandaid >> > >> > >> > off. >> > >> > >> > > > > >> > >> > >> > > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > Can you please explain the >> > >> motivation? It >> > >> > >> did >> > >> > >> > > come >> > >> > >> > > > > up >> > >> > >> > > > > >> in >> > >> > >> > > > > >> > >> > >> previous >> > >> > >> > > > > >> > >> > >> > > > > > discussions that some things like >> > >> > Operation >> > >> > >> > and >> > >> > >> > > > > >> > ResourceType >> > >> > >> > > > > >> > >> > >> should >> > >> > >> > > > > >> > >> > >> > > be >> > >> > >> > > > > >> > >> > >> > > > in >> > >> > >> > > > > >> > >> > >> > > > > > the clients library, but not >> > >> Authorizer >> > >> > >> > itself. >> > >> > >> > > > Are >> > >> > >> > > > > we >> > >> > >> > > > > >> > >> saying >> > >> > >> > > > > >> > >> > >> that >> > >> > >> > > > > >> > >> > >> > > any >> > >> > >> > > > > >> > >> > >> > > > > > pluggable interface should be in >> > Java >> > >> so >> > >> > >> that >> > >> > >> > > > users >> > >> > >> > > > > can >> > >> > >> > > > > >> > >> > >> implement >> > >> > >> > > > > >> > >> > >> > it >> > >> > >> > > > > >> > >> > >> > > > > > without including the Scala >> library? >> > >> > >> > > > > >> > >> > >> > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > Grant, you originally suggested >> that >> > >> some >> > >> > >> > things >> > >> > >> > > > > would >> > >> > >> > > > > >> > have >> > >> > >> > > > > >> > >> to >> > >> > >> > > > > >> > >> > >> be >> > >> > >> > > > > >> > >> > >> > in >> > >> > >> > > > > >> > >> > >> > > > the >> > >> > >> > > > > >> > >> > >> > > > > > Java side as well. Can you please >> > >> > >> elaborate on >> > >> > >> > > > this? >> > >> > >> > > > > >> > >> > >> > > > > > >> > >> > >> > > > > >> > >> > >> > > > > > Ismael >> > >> > >> > > > > >> > >> > >> > > > > > >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > >> > >> > >> > > > > >> > >> > >> > > > >> > >> > >> > > > > >> > >> > >> > > > >> > >> > >> > > > > >> > >> > >> > > > -- >> > >> > >> > > > > >> > >> > >> > > > Grant Henke >> > >> > >> > > > > >> > >> > >> > > > Software Engineer | Cloudera >> > >> > >> > > > > >> > >> > >> > > > gr...@cloudera.com <javascript:;> | >> > >> > >> > > > twitter.com/gchenke >> > >> > >> > > > > | >> > >> > >> > > > > >> > >> > >> linkedin.com/in/granthenke >> > >> > >> > > > > >> > >> > >> > > > >> > >> > >> > > > > >> > >> > >> > > >> > >> > >> > > > > >> > >> > >> > >> > >> > >> > > > > >> > >> > >> > >> > >> > >> > > > > >> > >> > >> > >> > >> > >> > > > > >> > >> > >> > -- >> > >> > >> > > > > >> > >> > >> > Grant Henke >> > >> > >> > > > > >> > >> > >> > Software Engineer | Cloudera >> > >> > >> > > > > >> > >> > >> > gr...@cloudera.com <javascript:;> | >> > >> > >> > twitter.com/gchenke >> > >> > >> > > | >> > >> > >> > > > > >> > >> linkedin.com/in/granthenke >> > >> > >> > > > > >> > >> > >> > >> > >> > >> > > > > >> > >> > >> >> > >> > >> > > > > >> > >> > > >> > >> > >> > > > > >> > >> > > >> > >> > >> > > > > >> > >> > > >> > >> > >> > > > > >> > >> > > -- >> > >> > >> > > > > >> > >> > > >> > >> > >> > > > > >> > >> > > Regards, >> > >> > >> > > > > >> > >> > > Ashish >> > >> > >> > > > > >> > >> > > >> > >> > >> > > > > >> > >> > >> > >> > >> > > > > >> > >> > >> > >> > >> > > > > >> > >> > >> > >> > >> > > > > >> > >> > -- >> > >> > >> > > > > >> > >> > >> > >> > >> > > > > >> > >> > Regards, >> > >> > >> > > > > >> > >> > Ashish >> > >> > >> > > > > >> > >> >> > >> > >> > > > > >> > > >> > >> > >> > > > > >> > >-- >> > >> > >> > > > > >> > > >> > >> > >> > > > > >> > >Regards, >> > >> > >> > > > > >> > >Ashish >> > >> > >> > > > > >> > >> > >> > >> > > > > >> > >> > >> > >> > > > > >> >> > >> > >> > > > > > >> > >> > >> > > > > > >> > >> > >> > > > > > >> > >> > >> > > > > >-- >> > >> > >> > > > > > >> > >> > >> > > > > >Regards, >> > >> > >> > > > > >Ashish >> > >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > >> > >> > >> > > > -- >> > >> > >> > > > Ashish 🎤h >> > >> > >> > > > >> > >> > >> > > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > -- >> > >> > >> > >> > >> > >> > Regards, >> > >> > >> > Ashish >> > >> > >> > >> > >> > >> >> > >> > > >> > >> > > >> > >> > > >> > >> > > -- >> > >> > > >> > >> > > Regards, >> > >> > > Ashish >> > >> > > >> > >> > >> > >> > >> > >> > >> > >> > -- >> > >> > >> > >> > Regards, >> > >> > Ashish >> > >> > >> > >> >> > >> >> > >> >> > >> -- >> > >> Grant Henke >> > >> Software Engineer | Cloudera >> > >> gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke >> > >> >> > > >> > > >> > > >> > > -- >> > > >> > > Regards, >> > > Ashish >> > > >> > >> > >> > >> > -- >> > >> > Regards, >> > Ashish >> > >> > > > > -- > > Regards, > Ashish
