We could do this but I think its too simplistic plus now we are adding authorization related options in CLI which I thought everyone wants to avoid.
When I say its too simplistic I mean there are missing options like —hosts, what happens when we start supporting group now we will probably end up adding "—grant —groups”. I think we will just endup polluting kafka create CLI with all the different acl options or we will have 2 CLIs one for the basic stuff and for anything advance you will have to use a different tool. It might be better to just have a single separate ACL management CLI. Thanks Parth On 4/17/15, 10:42 AM, "Gwen Shapira" <gshap...@cloudera.com> wrote: >I've probably been a DBA for too long, but I imagined something like: >kafka-topic --topic t1 --grant user --action action >kafka-topic --topic t1 --revoke user --action action >(i.e. the commandline equivalent of "grant select on table1 to >gwenshap" and "revoke select on table2 from gwenshap") > >When you need gazillion of them, you generate a script with gazillion >of those and execute. > >Maybe it just looks reasonable to me because I'm used to it, though :) > >Or maybe including the json parsing code in TopicCommand is not so bad? > > > >On Fri, Apr 17, 2015 at 10:30 AM, Parth Brahmbhatt ><pbrahmbh...@hortonworks.com> wrote: >> * Yes, Acl pretty much captures everything. Originally I had resource as >> part of Acls, we can go back to that. >> * The describe can call getAcl and I plan to do so. addAcl is tricky >> because the user will have to specify the acls through command lines, >> which will probably be a location to some file. Basically the CLI won¹t >> know how to parse user input and convert it to a principal/acl that the >> plugin understands. We could add an API in authorizer that can take a >>file >> as input if we want ‹acl as an option during create. >> * Yes also getAcls(Principal principal). >> >> Thanks >> Parth >> >> >> On 4/17/15, 10:05 AM, "Gwen Shapira" <gshap...@cloudera.com> wrote: >> >>>On Fri, Apr 17, 2015 at 9:31 AM, Parth Brahmbhatt >>><pbrahmbh...@hortonworks.com> wrote: >>>> I was following the storm model but I think this is a reasonable >>>>change. I recommend changing the API names to addAcls, removeAcls and >>>>getAcls. >>> >>>And they probably just need to get List<Acl> instead of everything I >>>specified? Looks like Acl encapsulates everything we need. >>> >>>> Couple of points to ensure we are on same page: >>>> * With this approach the kafka command line will not provide a way to >>>>add/edit acls during topic creation, neither it will provide a way to >>>>modify the acls. It will be up to the authorizer to either define a >>>>command line utility or to allow other means to add acls(CLI/UI/REST). >>>>For the default implementation we can provide CLI. >>> >>>You looked into this deeper than I did - is there a reason >>>TopicCommand can't invoke addACL and getACL? >>> >>>> * We probably want to add List<Acl> getAcls(Resource resource) so >>>>users >>>>can list all acls on a topic. >>> >>>Also getAcls(Principal princ)? >>> >>>> >>>> I haven¹t looked at how consumer offsets are currently stored so I >>>>will >>>>have to take a look but I think that is implementation detail. >>>> >>>> Gwen,Jun and other interested parties, do you have time to jump on a >>>>quick hangout so we can go over some of the lower level details? >>>> >>>> Thanks >>>> Parth >>>> From: Tong Li <liton...@us.ibm.com<mailto:liton...@us.ibm.com>> >>>> Reply-To: "dev@kafka.apache.org<mailto:dev@kafka.apache.org>" >>>><dev@kafka.apache.org<mailto:dev@kafka.apache.org>> >>>> Date: Friday, April 17, 2015 at 7:34 AM >>>> To: "dev@kafka.apache.org<mailto:dev@kafka.apache.org>" >>>><dev@kafka.apache.org<mailto:dev@kafka.apache.org>> >>>> Subject: Re: [DISCUSSION] KIP-11: ACL Management >>>> >>>> >>>> Gwen, >>>> There is one product called ElasticSearch which has been >>>>quite >>>>successful. They recently added security, what they actually did is >>>>quite nice. They really separated Authentication and Authorization >>>>which >>>>many people get really confused about and often mix them up. I looked >>>>through what they did and quite impressed by it, I think there are many >>>>things we can borrow from. Here is a link to it. >>>>http://www.elastic.co/guide/en/shield/current/architecture.html. The >>>>product name is called "shield" which is implemented as an >>>>ElasticSearch >>>>plugin. The promise here is that you can have a running ElasticSearch, >>>>then you install this plugin, configure it, then your ElasticSearch >>>>service is secured. The goal should be really the same for Kafka, you >>>>have a Kafka service running, you install a new plugin (in this case >>>>security plugin), configure it, then your Kafka service is secured. I >>>>think that the key here is that we should introduce a true pluggable >>>>framework in Kafka which allows security, quota, encryption, >>>>compression, serialization/deserialization all being developed as >>>>plugins which can be all easily added and configured onto a running >>>>Kafka service, then the functions/features provided by the plugins will >>>>start working. Once we have this framework in, how a security plugin >>>>works internally becomes the really the concern of that plugin, for >>>>example, how a new user gets registered, permission granted, revoked, >>>>all these will be the concern of that plugin, rest of the Kafka >>>>components should not really be concerned about them. This way we are >>>>really following the design principal (Separation of concerns). With >>>>all that, what I am proposing is a true pluggable framework >>>>introduction >>>>into Kafka which I have also talked about in a previous email. For >>>>security we can implement a simple file based security plugin, other >>>>plugins such as LDAP, AD for authentication can come later, plugin for >>>>authorization such as RBAC can also come later if people care so much >>>>about using them. >>>> >>>> Thanks. >>>> >>>> Tong Li >>>> OpenStack & Kafka Community Development >>>> Building 501/B205 >>>> liton...@us.ibm.com<mailto:liton...@us.ibm.com> >>>> >>>> [Inactive hide details for Gwen Shapira ---04/16/2015 12:44:54 PM---Hi >>>>Kafka Authorization Fans, I'm starting a new thread on a]Gwen Shapira >>>>---04/16/2015 12:44:54 PM---Hi Kafka Authorization Fans, I'm starting a >>>>new thread on a specific sub-topic of KIP-11, since >>>> >>>> From: Gwen Shapira >>>><gshap...@cloudera.com<mailto:gshap...@cloudera.com>> >>>> To: "dev@kafka.apache.org<mailto:dev@kafka.apache.org>" >>>><dev@kafka.apache.org<mailto:dev@kafka.apache.org>> >>>> Date: 04/16/2015 12:44 PM >>>> Subject: [DISCUSSION] KIP-11: ACL Management >>>> >>>> ________________________________ >>>> >>>> >>>> >>>> Hi Kafka Authorization Fans, >>>> >>>> I'm starting a new thread on a specific sub-topic of KIP-11, since >>>> this is a bit long :) >>>> >>>> Currently KIP-11, as I understand it, proposes: >>>> * Authorizers are pluggable, with Kafka providing DefaultAuthorizer. >>>> * Kafka tools allow adding / managing ACLs. >>>> * Those ACLs are stored in ZK and cached in a new TopicCache >>>> * Authorizers can either use the ACLs defined and stored in Kafka, or >>>> define and use their own. >>>> >>>> I am concerned of two possible issues with this design: >>>> 1. Separation of concerns - only authorizers should worry about ACLs, >>>> and therefore the less code for ACLs that exist in Kafka core, the >>>> better. >>>> 2. User confusion - It sounded like we can define ACLs in Kafka itself >>>> but authorizers can also define their own, so "kafka-topics >>>> --describe" may show an ACL different than the one in use. This can be >>>> super confusing for admins. >>>> >>>> My alternative suggestion: >>>> * Authorizer API will include: >>>> grantPrivilege(List<Principals>, List<Privilege>) >>>> revokePrivilege(List<Principals>, List<Privilege>), >>>> getPrivilegesByPrincipal(Principal, Resource) >>>> .... >>>> (The exact API can be discussed in detail, but you get the idea) >>>> * Kafka tools will simply invoke these APIs when topics are added / >>>> modified / described. >>>> * Each authorizer (including the default one) will be responsible for >>>> storing, caching and using those ACLs. >>>> >>>> This way, we keep almost all ACL code with the Authorizer, where it >>>> belongs and users get a nice unified interface that reflects what is >>>> actually getting used in the system. >>>> This is pretty much how Sqoop and Hive implement their authorization >>>>APIs. >>>> >>>> What do you think? >>>> >>>> Gwen >>>> >>
