Re: [DISCUSS] KIP-11- Authorization design for kafka security
Hey everyone, Sorry to jump in on the conversation so late. I'm new to Kafka. I'll apologize in advance if you have already covered some of my questions. I read through the wiki and had some comments and questions. 1) public enum Operation needs EDIT changed to ALTER 2) Does the Authorizer class need a setAcls? Rather then just add to be able to set to explicit list and overwrite what was there? I see the kafka-acl.sh lists a removeall so I guess you could do removeall and then add. I also don't see a removeall in the Authorizer class, is it going to loop through them all to remove each one? 3) Can someone tell me what the use case to do acls based on the hosts? I can see some possibilities just wondering if we can concrete ones where one user is allowed from one host but not another. 4) I'm a bit unclear how the resource works in the Authorizer class. From what I see we have 2 resources - topics and cluster. If I want to add an acl to allow joe to CREATE for the cluster then I call addAcls with Acl(user: joe, ALLOW, Set(*), Set(CREATE)) and cluster? What if I want to call addAcls for DESCRIBE on a topic? Is the resource then topic or is it the topic name? 5) reassigning partitions is a CLUSTER_ACTION or superuser? Its not totally clear to me the differences between these. what about increasing # of partitions? 6) groups are mentioned, are we supporting right away or is that a follow on item? (is there going to be a kafka.supergroups) 7) Are there config options for setting acls when I create my topic? Or do I have to create my topic and then run the kafka-acl.sh script to set them? Although its very small, there would be possible race there that someone could start producing to topic before acls are set. 8) are there configs for cluster level acl defaults? Or does it default to superusers on bringing up new cluster and you have to modify with cli. thanks,Tom On Tuesday, April 21, 2015 7:10 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: I have added the notes to KIP-11 Open question sections. Thanks Parth On 4/21/15, 4:49 PM, Gwen Shapira gshap...@cloudera.com wrote: Adding my notes from today's call to the thread: ** Deny or Allow all by default? We will add a configuration to control this. The configuration will default to “allow” for backward compatibility. Security admins can set it to deny ** Storing ACLs for default authorizers: We'll store them in ZK. We'll support pointing the authorizer to any ZK. The use of ZK will be internal to the default authorizer. Authorizer reads ACLs from cache every hour. We proposed having mechanism (possibly via new ZK node) to tell broker to refresh the cache immediately. ** Support deny as permission type - we agreed to keep this. ** Mapping operations to API: We may need to add Group as a resource, with JoinGroup and OffsetCommit require privilege on the consumer group. This can be something we pass now and authorizers can support in future. - Jay will write specifics to the mailing list discussion. On Tue, Apr 21, 2015 at 4:32 PM, Jay Kreps jay.kr...@gmail.com wrote: Following up on the KIP discussion. Two options for authorizing consumers to read topic t as part of group g: 1. READ permission on resource /topic/t 2. READ permission on resource /topic/t AND WRITE permission on /group/g The advantage of (1) is that it is simpler. The disadvantage is that any member of any group that reads from t can commit offsets as any other member of a different group. This doesn't effect data security (who can access what) but it is a bit of a management issue--a malicious person can cause data loss or duplicates for another consumer by committing offset. I think I favor (2) but it's worth it to think it through. -Jay On Tue, Apr 21, 2015 at 2:43 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hey Jun, Yes and we support wild cards for all acl entities principal, hosts and operation. Thanks Parth On 4/21/15, 9:06 AM, Jun Rao j...@confluent.io wrote: Harsha, Parth, Thanks for the clarification. This makes sense. Perhaps we can clarify the meaning of those rules in the wiki. Related to this, it seems that we need to support wildcard in cli/request protocol for topics? Jun On Mon, Apr 20, 2015 at 9:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: The iptables on unix supports the DENY operator, not that it should matter. The deny operator can also be used to specify ³allow user1 to READ from topic1 from all hosts but host1,host2². Again we could add a host group semantic and extra complexity around that, not sure if its worth it. In addition with DENY operator you are now not forced to create a special group just to support the authorization use case. I am not convinced that the operator it self is really all that confusing. There are 3 practical use cases: - Resource with no acl what so ever - allow access to everyone ( just
Re: [DISCUSS] KIP-11- Authorization design for kafka security
FYI, I have modified the KIP to include group as resource. In order to access “joinGroup” and “commitOFfset” APIs the user will need a read permission on topic and WRITE permission on group. I plan to open a VOTE thread by noon if there are no more concerns. Thanks Parth On 4/22/15, 9:03 AM, Tom Graves tgraves...@yahoo.com.INVALID wrote: Hey everyone, Sorry to jump in on the conversation so late. I'm new to Kafka. I'll apologize in advance if you have already covered some of my questions. I read through the wiki and had some comments and questions. 1) public enum Operation needs EDIT changed to ALTER Done. 2) Does the Authorizer class need a setAcls? Rather then just add to be able to set to explicit list and overwrite what was there? I see the kafka-acl.sh lists a removeall so I guess you could do removeall and then add. I also don't see a removeall in the Authorizer class, is it going to loop through them all to remove each one? There is an overloaded version of removeAcls in the interface that takes in resource as the only input and as described in the javadoc all the acls attached to that resource will be deleted. To cover the setAcl use case the caller can first call remove and then add. 3) Can someone tell me what the use case to do acls based on the hosts? I can see some possibilities just wondering if we can concrete ones where one user is allowed from one host but not another. I am not sure if I understand the question given the use case you described in your question is what we are trying to cover with use of hosts in Acl. There are some additional use cases like “allow access to any user from host1,host2” but I think primarily it gives the admins the ability to define acls at a more granular level. 4) I'm a bit unclear how the resource works in the Authorizer class. From what I see we have 2 resources - topics and cluster. If I want to add an acl to allow joe to CREATE for the cluster then I call addAcls with Acl(user: joe, ALLOW, Set(*), Set(CREATE)) and cluster? What if I want to call addAcls for DESCRIBE on a topic? Is the resource then topic or is it the topic name? We now have 3 resources(added group), please see the updated doc. The CREATE acl that you described is correct. For any topic operation you should use topic name as the resource name and for group the user will provide groupId as resource name. 5) reassigning partitions is a CLUSTER_ACTION or superuser? Its not totally clear to me the differences between these. what about increasing # of partitions? I see this as an alter topic operation so it is at topic level and the user must have alter permissions on topic. 6) groups are mentioned, are we supporting right away or is that a follow on item? (is there going to be a kafka.supergroups) I think it can be a separate jira just for braking down the code review in smaller chunk. We will support it in first version but I think if we can not do it for any reason that should not block a release with all the other authZ work. We made deliberate design choices (like introducing a principalType in KafkaPrinciapl) to allow supporting groups as an incremental change. 7) Are there config options for setting acls when I create my topic? Or do I have to create my topic and then run the kafka-acl.sh script to set them? Although its very small, there would be possible race there that someone could start producing to topic before acls are set. We discussed this yesterday and we agreed to go with kafka-acl.sh. Yes there is a very very small window of vulnerability but I think that really does not warrant to change the decision in this case. 8) are there configs for cluster level acl defaults? Or does it default to superusers on bringing up new cluster and you have to modify with cli. thanks,Tom No defaults, the default is superusers will have full access. I don’t think making assumptions about ones security requirement should be our burden. On Tuesday, April 21, 2015 7:10 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: I have added the notes to KIP-11 Open question sections. Thanks Parth On 4/21/15, 4:49 PM, Gwen Shapira gshap...@cloudera.com wrote: Adding my notes from today's call to the thread: ** Deny or Allow all by default? We will add a configuration to control this. The configuration will default to “allow” for backward compatibility. Security admins can set it to deny ** Storing ACLs for default authorizers: We'll store them in ZK. We'll support pointing the authorizer to any ZK. The use of ZK will be internal to the default authorizer. Authorizer reads ACLs from cache every hour. We proposed having mechanism (possibly via new ZK node) to tell broker to refresh the cache immediately. ** Support deny as permission type - we agreed to keep this. ** Mapping operations to API: We may need to add Group as a resource, with JoinGroup and OffsetCommit require privilege
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Thanks for the explanations Parth. On the configs questions, the way I see it is its more likely to accidentally give everyone access, especially since you have to run a separate command to change the acls. If there was some config for defaults, a cluster admin could change that to be nobody or certain set of users, then grant others permissions. This would also remove the race between commands. This is something you can always add later though if people request it. So in kafka-acl.sh how do I actually tell it what the operation is?kafka-acl.sh --topic testtopic --add --grandprincipal user:joe,user:kate where does READ, WRITE, etc go? Can specify as a list so I don't have to run this a bunch of times for each. Do you want to have a --host option for --list so that admins could see what acls apply to specific host(s)? Tom On Wednesday, April 22, 2015 11:38 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: FYI, I have modified the KIP to include group as resource. In order to access “joinGroup” and “commitOFfset” APIs the user will need a read permission on topic and WRITE permission on group. I plan to open a VOTE thread by noon if there are no more concerns. Thanks Parth On 4/22/15, 9:03 AM, Tom Graves tgraves...@yahoo.com.INVALID wrote: Hey everyone, Sorry to jump in on the conversation so late. I'm new to Kafka. I'll apologize in advance if you have already covered some of my questions. I read through the wiki and had some comments and questions. 1) public enum Operation needs EDIT changed to ALTER Done. 2) Does the Authorizer class need a setAcls? Rather then just add to be able to set to explicit list and overwrite what was there? I see the kafka-acl.sh lists a removeall so I guess you could do removeall and then add. I also don't see a removeall in the Authorizer class, is it going to loop through them all to remove each one? There is an overloaded version of removeAcls in the interface that takes in resource as the only input and as described in the javadoc all the acls attached to that resource will be deleted. To cover the setAcl use case the caller can first call remove and then add. 3) Can someone tell me what the use case to do acls based on the hosts? I can see some possibilities just wondering if we can concrete ones where one user is allowed from one host but not another. I am not sure if I understand the question given the use case you described in your question is what we are trying to cover with use of hosts in Acl. There are some additional use cases like “allow access to any user from host1,host2” but I think primarily it gives the admins the ability to define acls at a more granular level. 4) I'm a bit unclear how the resource works in the Authorizer class. From what I see we have 2 resources - topics and cluster. If I want to add an acl to allow joe to CREATE for the cluster then I call addAcls with Acl(user: joe, ALLOW, Set(*), Set(CREATE)) and cluster? What if I want to call addAcls for DESCRIBE on a topic? Is the resource then topic or is it the topic name? We now have 3 resources(added group), please see the updated doc. The CREATE acl that you described is correct. For any topic operation you should use topic name as the resource name and for group the user will provide groupId as resource name. 5) reassigning partitions is a CLUSTER_ACTION or superuser? Its not totally clear to me the differences between these. what about increasing # of partitions? I see this as an alter topic operation so it is at topic level and the user must have alter permissions on topic. 6) groups are mentioned, are we supporting right away or is that a follow on item? (is there going to be a kafka.supergroups) I think it can be a separate jira just for braking down the code review in smaller chunk. We will support it in first version but I think if we can not do it for any reason that should not block a release with all the other authZ work. We made deliberate design choices (like introducing a principalType in KafkaPrinciapl) to allow supporting groups as an incremental change. 7) Are there config options for setting acls when I create my topic? Or do I have to create my topic and then run the kafka-acl.sh script to set them? Although its very small, there would be possible race there that someone could start producing to topic before acls are set. We discussed this yesterday and we agreed to go with kafka-acl.sh. Yes there is a very very small window of vulnerability but I think that really does not warrant to change the decision in this case. 8) are there configs for cluster level acl defaults? Or does it default to superusers on bringing up new cluster and you have to modify with cli. thanks,Tom No defaults, the default is superusers will have full access. I don’t think making assumptions about ones security requirement should be our burden. On Tuesday, April 21, 2015 7:10 PM, Parth Brahmbhatt
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Please see all the available options here https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface#KIP-11-AuthorizationInterface-AclManagement(CLI) . I think it covers both hosts and operations and allows to specify a list for both. Thanks Parth From: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Reply-To: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Date: Wednesday, April 22, 2015 at 11:02 AM To: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com, dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-11- Authorization design for kafka security Thanks for the explanations Parth. On the configs questions, the way I see it is its more likely to accidentally give everyone access, especially since you have to run a separate command to change the acls. If there was some config for defaults, a cluster admin could change that to be nobody or certain set of users, then grant others permissions. This would also remove the race between commands. This is something you can always add later though if people request it. So in kafka-acl.sh how do I actually tell it what the operation is? kafka-acl.sh --topic testtopic --add --grandprincipal user:joe,user:kate where does READ, WRITE, etc go? Can specify as a list so I don't have to run this a bunch of times for each. Do you want to have a --host option for --list so that admins could see what acls apply to specific host(s)? Tom On Wednesday, April 22, 2015 11:38 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com wrote: FYI, I have modified the KIP to include group as resource. In order to access “joinGroup” and “commitOFfset” APIs the user will need a read permission on topic and WRITE permission on group. I plan to open a VOTE thread by noon if there are no more concerns. Thanks Parth On 4/22/15, 9:03 AM, Tom Graves tgraves...@yahoo.com.INVALIDmailto:tgraves...@yahoo.com.INVALID wrote: Hey everyone, Sorry to jump in on the conversation so late. I'm new to Kafka. I'll apologize in advance if you have already covered some of my questions. I read through the wiki and had some comments and questions. 1) public enum Operation needs EDIT changed to ALTER Done. 2) Does the Authorizer class need a setAcls? Rather then just add to be able to set to explicit list and overwrite what was there? I see the kafka-acl.sh lists a removeall so I guess you could do removeall and then add. I also don't see a removeall in the Authorizer class, is it going to loop through them all to remove each one? There is an overloaded version of removeAcls in the interface that takes in resource as the only input and as described in the javadoc all the acls attached to that resource will be deleted. To cover the setAcl use case the caller can first call remove and then add. 3) Can someone tell me what the use case to do acls based on the hosts? I can see some possibilities just wondering if we can concrete ones where one user is allowed from one host but not another. I am not sure if I understand the question given the use case you described in your question is what we are trying to cover with use of hosts in Acl. There are some additional use cases like “allow access to any user from host1,host2” but I think primarily it gives the admins the ability to define acls at a more granular level. 4) I'm a bit unclear how the resource works in the Authorizer class. From what I see we have 2 resources - topics and cluster. If I want to add an acl to allow joe to CREATE for the cluster then I call addAcls with Acl(user: joe, ALLOW, Set(*), Set(CREATE)) and cluster? What if I want to call addAcls for DESCRIBE on a topic? Is the resource then topic or is it the topic name? We now have 3 resources(added group), please see the updated doc. The CREATE acl that you described is correct. For any topic operation you should use topic name as the resource name and for group the user will provide groupId as resource name. 5) reassigning partitions is a CLUSTER_ACTION or superuser? Its not totally clear to me the differences between these. what about increasing # of partitions? I see this as an alter topic operation so it is at topic level and the user must have alter permissions on topic. 6) groups are mentioned, are we supporting right away or is that a follow on item? (is there going to be a kafka.supergroups) I think it can be a separate jira just for braking down the code review in smaller chunk. We will support it in first version but I think if we can not do it for any reason that should not block a release with all the other authZ work. We made deliberate design choices (like introducing a principalType in KafkaPrinciapl) to allow supporting groups as an incremental change. 7) Are there config options for setting acls when I create my topic? Or do I have
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Sorry I must be missing it, that is the doc I'm looking at. I don't see how you specify the operation? I don't see a normal --host that goes with list. I see revokehost and granthost, neither I would think apply to list since --principal. says it applies to --list. Copying text here in case I'm looking at the wrong thing: - kafka-acl.sh --help A tool to manage acls for kafka cluster and topics. Following are the options --topic topicname name of topic whose acls you want to add/remove. --cluster this will add/edit cluster level acls where you can control which users can create topics or list all topics for the cluster or send control messages to brokers. --add indicates you are trying to add Acl --remove indicates you are trying to remove acl --grantprincipal principalType: principalName a comma separated list of principalType: principalName where currently supported principalType is user. When this option is not present but --granthost is specified it will default to * : * which is wild card that matches all types and all users. --granthost host1,host2 a comma separated list of hosts from which the --grantpricipal is allowed the actions. When this option is not present but --grantprincipal is specified, it defaults to * which is wildcard matching all hosts. --revokeprincipal principalType: principalName a comma separated list of principalType: principalName where currently supported principalType is user. When this option is not present but --revokehost is specified , it defaults to * : * which is wild card that matches all types and all users. Revoke should only be used to limit the access added by some other grants. A revoke without an accompanying grant is meaningless as by default only principals in grant lists are allowed access based on their acls and everyone else is denied. revokehost host1,host2 a comma separated list of hosts from which the --revokeprincipal will be denied actions. When this option is not present but --revokeprincipal is specified, it defaults to * which is wildcard matching all hosts. --removeAll removes all acls for a topic, only works with topic and not with cluster. --list list all acls --principal principalType: principalName Used only with --list to list all acls for a principal On Wednesday, April 22, 2015 1:08 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Please see all the available options here https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface#KIP-11-AuthorizationInterface-AclManagement(CLI) . I think it covers both hosts and operations and allows to specify a list for both. ThanksParth From: Tom Graves tgraves...@yahoo.com Reply-To: Tom Graves tgraves...@yahoo.com Date: Wednesday, April 22, 2015 at 11:02 AM To: Parth Brahmbhatt pbrahmbh...@hortonworks.com, dev@kafka.apache.org dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-11- Authorization design for kafka security Thanks for the explanations Parth. On the configs questions, the way I see it is its more likely to accidentally give everyone access, especially since you have to run a separate command to change the acls. If there was some config for defaults, a cluster admin could change that to be nobody or certain set of users, then grant others permissions. This would also remove the race between commands. This is something you can always add later though if people request it. So in kafka-acl.sh how do I actually tell it what the operation is?kafka-acl.sh --topic testtopic --add --grandprincipal user:joe,user:kate where does READ, WRITE, etc go? Can specify as a list so I don't have to run this a bunch of times for each. Do you want to have a --host option for --list so that admins could see what acls apply to specific host(s)? Tom On Wednesday, April 22, 2015 11:38 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: FYI, I have modified the KIP to include group as resource. In order to access “joinGroup” and “commitOFfset” APIs the user will need a read permission on topic and WRITE permission on group. I plan to open a VOTE thread by noon if there are no more concerns. Thanks Parth On 4/22/15, 9:03 AM, Tom Graves tgraves...@yahoo.com.INVALID wrote: Hey everyone, Sorry to jump in on the conversation so late. I'm new to Kafka. I'll apologize in advance if you have already covered some of my questions. I read through the wiki and had some comments and questions. 1) public enum Operation needs EDIT changed to ALTER Done. 2) Does the Authorizer class need a setAcls? Rather then just add to be able to set to explicit list and overwrite what was there? I see the kafka-acl.sh lists a removeall so I guess you could do removeall and then add. I also don't see a removeall in the Authorizer class, is it going to loop through them all to remove each one? There is an overloaded version of removeAcls in the interface that takes in resource as the only
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Sorry I missed your last questions. I am +0 on adding ―host option for ―list, we could add it for symmetry. Again if this is only a CLI change it can be added later if you mean adding this in authorizer interface then we should make a decision now. Given a choice I would like to actually keep only one option which is resource based get (remove even the get based on principal). I see those (getAcl for principal or host) as special filtering case which can easily be achieved by a third party tool by doing list all topics and calling getAcls for each topic and applying filtering logic on that. I really don’t see the need to make those first class citizens of the authorizer interface given these kind of queries will be issued outside of broker JVM so they will not benefit from the caching and because the storage will be indexed on resource both these options even as a first class API will just scan all topic acls and apply filtering logic. Thanks Parth On 4/22/15, 11:08 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Please see all the available options here https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+I nterface#KIP-11-AuthorizationInterface-AclManagement(CLI) . I think it covers both hosts and operations and allows to specify a list for both. Thanks Parth From: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Reply-To: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Date: Wednesday, April 22, 2015 at 11:02 AM To: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com, dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-11- Authorization design for kafka security Thanks for the explanations Parth. On the configs questions, the way I see it is its more likely to accidentally give everyone access, especially since you have to run a separate command to change the acls. If there was some config for defaults, a cluster admin could change that to be nobody or certain set of users, then grant others permissions. This would also remove the race between commands. This is something you can always add later though if people request it. So in kafka-acl.sh how do I actually tell it what the operation is? kafka-acl.sh --topic testtopic --add --grandprincipal user:joe,user:kate where does READ, WRITE, etc go? Can specify as a list so I don't have to run this a bunch of times for each. Do you want to have a --host option for --list so that admins could see what acls apply to specific host(s)? Tom On Wednesday, April 22, 2015 11:38 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com wrote: FYI, I have modified the KIP to include group as resource. In order to access “joinGroup” and “commitOFfset” APIs the user will need a read permission on topic and WRITE permission on group. I plan to open a VOTE thread by noon if there are no more concerns. Thanks Parth On 4/22/15, 9:03 AM, Tom Graves tgraves...@yahoo.com.INVALIDmailto:tgraves...@yahoo.com.INVALID wrote: Hey everyone, Sorry to jump in on the conversation so late. I'm new to Kafka. I'll apologize in advance if you have already covered some of my questions. I read through the wiki and had some comments and questions. 1) public enum Operation needs EDIT changed to ALTER Done. 2) Does the Authorizer class need a setAcls? Rather then just add to be able to set to explicit list and overwrite what was there? I see the kafka-acl.sh lists a removeall so I guess you could do removeall and then add. I also don't see a removeall in the Authorizer class, is it going to loop through them all to remove each one? There is an overloaded version of removeAcls in the interface that takes in resource as the only input and as described in the javadoc all the acls attached to that resource will be deleted. To cover the setAcl use case the caller can first call remove and then add. 3) Can someone tell me what the use case to do acls based on the hosts? I can see some possibilities just wondering if we can concrete ones where one user is allowed from one host but not another. I am not sure if I understand the question given the use case you described in your question is what we are trying to cover with use of hosts in Acl. There are some additional use cases like “allow access to any user from host1,host2” but I think primarily it gives the admins the ability to define acls at a more granular level. 4) I'm a bit unclear how the resource works in the Authorizer class. From what I see we have 2 resources - topics and cluster. If I want to add an acl to allow joe to CREATE for the cluster then I call addAcls with Acl(user: joe, ALLOW, Set(*), Set(CREATE)) and cluster? What if I want to call addAcls for DESCRIBE on a topic? Is the resource then topic or is it the topic name? We now have 3 resources(added group), please see the updated doc. The CREATE acl that you
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Thanks! Makes sense. Tom On Wednesday, April 22, 2015 1:25 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: You are right , I forgot to mention the ―operation option in CLI , I just added it. Sorry for about the confusion. Thanks Parth On 4/22/15, 11:22 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Sorry I missed your last questions. I am +0 on adding ―host option for ―list, we could add it for symmetry. Again if this is only a CLI change it can be added later if you mean adding this in authorizer interface then we should make a decision now. Given a choice I would like to actually keep only one option which is resource based get (remove even the get based on principal). I see those (getAcl for principal or host) as special filtering case which can easily be achieved by a third party tool by doing list all topics and calling getAcls for each topic and applying filtering logic on that. I really don’t see the need to make those first class citizens of the authorizer interface given these kind of queries will be issued outside of broker JVM so they will not benefit from the caching and because the storage will be indexed on resource both these options even as a first class API will just scan all topic acls and apply filtering logic. Thanks Parth On 4/22/15, 11:08 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Please see all the available options here https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+ I nterface#KIP-11-AuthorizationInterface-AclManagement(CLI) . I think it covers both hosts and operations and allows to specify a list for both. Thanks Parth From: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Reply-To: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Date: Wednesday, April 22, 2015 at 11:02 AM To: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com, dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-11- Authorization design for kafka security Thanks for the explanations Parth. On the configs questions, the way I see it is its more likely to accidentally give everyone access, especially since you have to run a separate command to change the acls. If there was some config for defaults, a cluster admin could change that to be nobody or certain set of users, then grant others permissions. This would also remove the race between commands. This is something you can always add later though if people request it. So in kafka-acl.sh how do I actually tell it what the operation is? kafka-acl.sh --topic testtopic --add --grandprincipal user:joe,user:kate where does READ, WRITE, etc go? Can specify as a list so I don't have to run this a bunch of times for each. Do you want to have a --host option for --list so that admins could see what acls apply to specific host(s)? Tom On Wednesday, April 22, 2015 11:38 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com wrote: FYI, I have modified the KIP to include group as resource. In order to access “joinGroup” and “commitOFfset” APIs the user will need a read permission on topic and WRITE permission on group. I plan to open a VOTE thread by noon if there are no more concerns. Thanks Parth On 4/22/15, 9:03 AM, Tom Graves tgraves...@yahoo.com.INVALIDmailto:tgraves...@yahoo.com.INVALID wrote: Hey everyone, Sorry to jump in on the conversation so late. I'm new to Kafka. I'll apologize in advance if you have already covered some of my questions. I read through the wiki and had some comments and questions. 1) public enum Operation needs EDIT changed to ALTER Done. 2) Does the Authorizer class need a setAcls? Rather then just add to be able to set to explicit list and overwrite what was there? I see the kafka-acl.sh lists a removeall so I guess you could do removeall and then add. I also don't see a removeall in the Authorizer class, is it going to loop through them all to remove each one? There is an overloaded version of removeAcls in the interface that takes in resource as the only input and as described in the javadoc all the acls attached to that resource will be deleted. To cover the setAcl use case the caller can first call remove and then add. 3) Can someone tell me what the use case to do acls based on the hosts? I can see some possibilities just wondering if we can concrete ones where one user is allowed from one host but not another. I am not sure if I understand the question given the use case you described in your question is what we are trying to cover with use of hosts in Acl. There are some additional use cases like “allow access to any user from host1,host2” but I think primarily it gives the admins the ability to define acls at a more granular level. 4) I'm a bit unclear how the resource works in the Authorizer class. From what I see we have 2 resources - topics and cluster. If I want
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Parth, This is a long thread, so trying to keep up here, sorry if this has been covered before. First, great job on the KIP proposal and work so far. Are we sure that we want to tie host level access to a given user? My understanding is that the ACL will be (omitting some fields) user_a, host1, host2, host3 user_b, host1, host2, host3 So there would potentially be a lot of redundancy in the configs. Does it make sense to have hosts be at the same level as principal in the hierarchy? This way you could just blanket the allowed / denied hosts and only have to worry about the users. So if you follow this, then we can wildcard the user so we can have a separate list of just host-based access. What's the order that the perms would be evaluated if a there was more than one match on a principal ? Is the thought that there wouldn't usually be much overlap on hosts? I guess I can imagine a scenario where I want to offline/online access to a particular hosts or set of hosts and if there was overlap, I'm doing a bunch of alter commands for just a single host. Maybe this is too contrived an example? I agree that having this level of granularity gives flexibility but I wonder if people will actually use it and not just * the hosts for a given user and create separate global list as i mentioned above? The only other system I know of that ties users with hosts for access is MySql and I don't love that model. Companies usually standardize on group authorization anyway, are we complicating that issue with the inclusion of hosts attached to users? Additionally I worry about the debt of big JSON configs in the first place, most non-developers find them non-intuitive already, so anything to ease this I think would be beneficial. Thanks Jeff On Wed, Apr 22, 2015 at 2:22 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Sorry I missed your last questions. I am +0 on adding ―host option for ―list, we could add it for symmetry. Again if this is only a CLI change it can be added later if you mean adding this in authorizer interface then we should make a decision now. Given a choice I would like to actually keep only one option which is resource based get (remove even the get based on principal). I see those (getAcl for principal or host) as special filtering case which can easily be achieved by a third party tool by doing list all topics and calling getAcls for each topic and applying filtering logic on that. I really don’t see the need to make those first class citizens of the authorizer interface given these kind of queries will be issued outside of broker JVM so they will not benefit from the caching and because the storage will be indexed on resource both these options even as a first class API will just scan all topic acls and apply filtering logic. Thanks Parth On 4/22/15, 11:08 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Please see all the available options here https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+I nterface#KIP-11-AuthorizationInterface-AclManagement(CLI) . I think it covers both hosts and operations and allows to specify a list for both. Thanks Parth From: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Reply-To: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Date: Wednesday, April 22, 2015 at 11:02 AM To: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com, dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-11- Authorization design for kafka security Thanks for the explanations Parth. On the configs questions, the way I see it is its more likely to accidentally give everyone access, especially since you have to run a separate command to change the acls. If there was some config for defaults, a cluster admin could change that to be nobody or certain set of users, then grant others permissions. This would also remove the race between commands. This is something you can always add later though if people request it. So in kafka-acl.sh how do I actually tell it what the operation is? kafka-acl.sh --topic testtopic --add --grandprincipal user:joe,user:kate where does READ, WRITE, etc go? Can specify as a list so I don't have to run this a bunch of times for each. Do you want to have a --host option for --list so that admins could see what acls apply to specific host(s)? Tom On Wednesday, April 22, 2015 11:38 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com wrote: FYI, I have modified the KIP to include group as resource. In order to access “joinGroup” and “commitOFfset” APIs the user will need a read permission on topic and WRITE permission on group. I plan to open a VOTE thread by noon if there are no more concerns. Thanks Parth On 4/22/15, 9:03 AM, Tom Graves tgraves...@yahoo.com.INVALIDmailto:tgraves
Re: [DISCUSS] KIP-11- Authorization design for kafka security
You are right , I forgot to mention the ―operation option in CLI , I just added it. Sorry for about the confusion. Thanks Parth On 4/22/15, 11:22 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Sorry I missed your last questions. I am +0 on adding ―host option for ―list, we could add it for symmetry. Again if this is only a CLI change it can be added later if you mean adding this in authorizer interface then we should make a decision now. Given a choice I would like to actually keep only one option which is resource based get (remove even the get based on principal). I see those (getAcl for principal or host) as special filtering case which can easily be achieved by a third party tool by doing list all topics and calling getAcls for each topic and applying filtering logic on that. I really don’t see the need to make those first class citizens of the authorizer interface given these kind of queries will be issued outside of broker JVM so they will not benefit from the caching and because the storage will be indexed on resource both these options even as a first class API will just scan all topic acls and apply filtering logic. Thanks Parth On 4/22/15, 11:08 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Please see all the available options here https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+ I nterface#KIP-11-AuthorizationInterface-AclManagement(CLI) . I think it covers both hosts and operations and allows to specify a list for both. Thanks Parth From: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Reply-To: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Date: Wednesday, April 22, 2015 at 11:02 AM To: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com, dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-11- Authorization design for kafka security Thanks for the explanations Parth. On the configs questions, the way I see it is its more likely to accidentally give everyone access, especially since you have to run a separate command to change the acls. If there was some config for defaults, a cluster admin could change that to be nobody or certain set of users, then grant others permissions. This would also remove the race between commands. This is something you can always add later though if people request it. So in kafka-acl.sh how do I actually tell it what the operation is? kafka-acl.sh --topic testtopic --add --grandprincipal user:joe,user:kate where does READ, WRITE, etc go? Can specify as a list so I don't have to run this a bunch of times for each. Do you want to have a --host option for --list so that admins could see what acls apply to specific host(s)? Tom On Wednesday, April 22, 2015 11:38 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com wrote: FYI, I have modified the KIP to include group as resource. In order to access “joinGroup” and “commitOFfset” APIs the user will need a read permission on topic and WRITE permission on group. I plan to open a VOTE thread by noon if there are no more concerns. Thanks Parth On 4/22/15, 9:03 AM, Tom Graves tgraves...@yahoo.com.INVALIDmailto:tgraves...@yahoo.com.INVALID wrote: Hey everyone, Sorry to jump in on the conversation so late. I'm new to Kafka. I'll apologize in advance if you have already covered some of my questions. I read through the wiki and had some comments and questions. 1) public enum Operation needs EDIT changed to ALTER Done. 2) Does the Authorizer class need a setAcls? Rather then just add to be able to set to explicit list and overwrite what was there? I see the kafka-acl.sh lists a removeall so I guess you could do removeall and then add. I also don't see a removeall in the Authorizer class, is it going to loop through them all to remove each one? There is an overloaded version of removeAcls in the interface that takes in resource as the only input and as described in the javadoc all the acls attached to that resource will be deleted. To cover the setAcl use case the caller can first call remove and then add. 3) Can someone tell me what the use case to do acls based on the hosts? I can see some possibilities just wondering if we can concrete ones where one user is allowed from one host but not another. I am not sure if I understand the question given the use case you described in your question is what we are trying to cover with use of hosts in Acl. There are some additional use cases like “allow access to any user from host1,host2” but I think primarily it gives the admins the ability to define acls at a more granular level. 4) I'm a bit unclear how the resource works in the Authorizer class. From what I see we have 2 resources - topics and cluster. If I want to add an acl to allow joe to CREATE for the cluster then I call addAcls with Acl(user: joe, ALLOW, Set(*), Set(CREATE)) and cluster
Re: [DISCUSS] KIP-11- Authorization design for kafka security
What about specifying the operation when I go to add or remove? Tom On Wednesday, April 22, 2015 1:23 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Sorry I missed your last questions. I am +0 on adding ―host option for ―list, we could add it for symmetry. Again if this is only a CLI change it can be added later if you mean adding this in authorizer interface then we should make a decision now. Given a choice I would like to actually keep only one option which is resource based get (remove even the get based on principal). I see those (getAcl for principal or host) as special filtering case which can easily be achieved by a third party tool by doing list all topics and calling getAcls for each topic and applying filtering logic on that. I really don’t see the need to make those first class citizens of the authorizer interface given these kind of queries will be issued outside of broker JVM so they will not benefit from the caching and because the storage will be indexed on resource both these options even as a first class API will just scan all topic acls and apply filtering logic. Thanks Parth On 4/22/15, 11:08 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Please see all the available options here https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+I nterface#KIP-11-AuthorizationInterface-AclManagement(CLI) . I think it covers both hosts and operations and allows to specify a list for both. Thanks Parth From: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Reply-To: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Date: Wednesday, April 22, 2015 at 11:02 AM To: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com, dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-11- Authorization design for kafka security Thanks for the explanations Parth. On the configs questions, the way I see it is its more likely to accidentally give everyone access, especially since you have to run a separate command to change the acls. If there was some config for defaults, a cluster admin could change that to be nobody or certain set of users, then grant others permissions. This would also remove the race between commands. This is something you can always add later though if people request it. So in kafka-acl.sh how do I actually tell it what the operation is? kafka-acl.sh --topic testtopic --add --grandprincipal user:joe,user:kate where does READ, WRITE, etc go? Can specify as a list so I don't have to run this a bunch of times for each. Do you want to have a --host option for --list so that admins could see what acls apply to specific host(s)? Tom On Wednesday, April 22, 2015 11:38 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com wrote: FYI, I have modified the KIP to include group as resource. In order to access “joinGroup” and “commitOFfset” APIs the user will need a read permission on topic and WRITE permission on group. I plan to open a VOTE thread by noon if there are no more concerns. Thanks Parth On 4/22/15, 9:03 AM, Tom Graves tgraves...@yahoo.com.INVALIDmailto:tgraves...@yahoo.com.INVALID wrote: Hey everyone, Sorry to jump in on the conversation so late. I'm new to Kafka. I'll apologize in advance if you have already covered some of my questions. I read through the wiki and had some comments and questions. 1) public enum Operation needs EDIT changed to ALTER Done. 2) Does the Authorizer class need a setAcls? Rather then just add to be able to set to explicit list and overwrite what was there? I see the kafka-acl.sh lists a removeall so I guess you could do removeall and then add. I also don't see a removeall in the Authorizer class, is it going to loop through them all to remove each one? There is an overloaded version of removeAcls in the interface that takes in resource as the only input and as described in the javadoc all the acls attached to that resource will be deleted. To cover the setAcl use case the caller can first call remove and then add. 3) Can someone tell me what the use case to do acls based on the hosts? I can see some possibilities just wondering if we can concrete ones where one user is allowed from one host but not another. I am not sure if I understand the question given the use case you described in your question is what we are trying to cover with use of hosts in Acl. There are some additional use cases like “allow access to any user from host1,host2” but I think primarily it gives the admins the ability to define acls at a more granular level. 4) I'm a bit unclear how the resource works in the Authorizer class. From what I see we have 2 resources - topics and cluster. If I want to add an acl to allow joe to CREATE for the cluster then I call addAcls with Acl(user: joe, ALLOW, Set(*), Set(CREATE)) and cluster? What if I want to call addAcls
Re: [DISCUSS] KIP-11- Authorization design for kafka security
https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+ I nterface#KIP-11-AuthorizationInterface-AclManagement(CLI) . I think it covers both hosts and operations and allows to specify a list for both. Thanks Parth From: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Reply-To: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Date: Wednesday, April 22, 2015 at 11:02 AM To: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com, dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-11- Authorization design for kafka security Thanks for the explanations Parth. On the configs questions, the way I see it is its more likely to accidentally give everyone access, especially since you have to run a separate command to change the acls. If there was some config for defaults, a cluster admin could change that to be nobody or certain set of users, then grant others permissions. This would also remove the race between commands. This is something you can always add later though if people request it. So in kafka-acl.sh how do I actually tell it what the operation is? kafka-acl.sh --topic testtopic --add --grandprincipal user:joe,user:kate where does READ, WRITE, etc go? Can specify as a list so I don't have to run this a bunch of times for each. Do you want to have a --host option for --list so that admins could see what acls apply to specific host(s)? Tom On Wednesday, April 22, 2015 11:38 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com wrote: FYI, I have modified the KIP to include group as resource. In order to access “joinGroup” and “commitOFfset” APIs the user will need a read permission on topic and WRITE permission on group. I plan to open a VOTE thread by noon if there are no more concerns. Thanks Parth On 4/22/15, 9:03 AM, Tom Graves tgraves...@yahoo.com.INVALIDmailto:tgraves...@yahoo.com.INVALID wrote: Hey everyone, Sorry to jump in on the conversation so late. I'm new to Kafka. I'll apologize in advance if you have already covered some of my questions. I read through the wiki and had some comments and questions. 1) public enum Operation needs EDIT changed to ALTER Done. 2) Does the Authorizer class need a setAcls? Rather then just add to be able to set to explicit list and overwrite what was there? I see the kafka-acl.sh lists a removeall so I guess you could do removeall and then add. I also don't see a removeall in the Authorizer class, is it going to loop through them all to remove each one? There is an overloaded version of removeAcls in the interface that takes in resource as the only input and as described in the javadoc all the acls attached to that resource will be deleted. To cover the setAcl use case the caller can first call remove and then add. 3) Can someone tell me what the use case to do acls based on the hosts? I can see some possibilities just wondering if we can concrete ones where one user is allowed from one host but not another. I am not sure if I understand the question given the use case you described in your question is what we are trying to cover with use of hosts in Acl. There are some additional use cases like “allow access to any user from host1,host2” but I think primarily it gives the admins the ability to define acls at a more granular level. 4) I'm a bit unclear how the resource works in the Authorizer class. From what I see we have 2 resources - topics and cluster. If I want to add an acl to allow joe to CREATE for the cluster then I call addAcls with Acl(user: joe, ALLOW, Set(*), Set(CREATE)) and cluster? What if I want to call addAcls for DESCRIBE on a topic? Is the resource then topic or is it the topic name? We now have 3 resources(added group), please see the updated doc. The CREATE acl that you described is correct. For any topic operation you should use topic name as the resource name and for group the user will provide groupId as resource name. 5) reassigning partitions is a CLUSTER_ACTION or superuser? Its not totally clear to me the differences between these. what about increasing # of partitions? I see this as an alter topic operation so it is at topic level and the user must have alter permissions on topic. 6) groups are mentioned, are we supporting right away or is that a follow on item? (is there going to be a kafka.supergroups) I think it can be a separate jira just for braking down the code review in smaller chunk. We will support it in first version but I think if we can not do it for any reason that should not block a release with all the other authZ work. We made deliberate design choices (like introducing a principalType in KafkaPrinciapl) to allow supporting groups as an incremental change. 7) Are there config
Re: [DISCUSS] KIP-11- Authorization design for kafka security
same question here as well on hosts. If I would like to secure a topic, regardless where the topic is, I would like it to be secured. It is hard to imagine that one would like a topic to be secured on one host, but not on the other unless a topic spreads over multiple hosts really meant to be totally different things. Then it will be really needed. Thanks. Tong Li OpenStack Kafka Community Development Building 501/B205 liton...@us.ibm.com From: Jeff Holoman jholo...@cloudera.com To: dev@kafka.apache.org Cc: Tom Graves tgraves...@yahoo.com Date: 04/22/2015 02:40 PM Subject:Re: [DISCUSS] KIP-11- Authorization design for kafka security Parth, This is a long thread, so trying to keep up here, sorry if this has been covered before. First, great job on the KIP proposal and work so far. Are we sure that we want to tie host level access to a given user? My understanding is that the ACL will be (omitting some fields) user_a, host1, host2, host3 user_b, host1, host2, host3 So there would potentially be a lot of redundancy in the configs. Does it make sense to have hosts be at the same level as principal in the hierarchy? This way you could just blanket the allowed / denied hosts and only have to worry about the users. So if you follow this, then we can wildcard the user so we can have a separate list of just host-based access. What's the order that the perms would be evaluated if a there was more than one match on a principal ? Is the thought that there wouldn't usually be much overlap on hosts? I guess I can imagine a scenario where I want to offline/online access to a particular hosts or set of hosts and if there was overlap, I'm doing a bunch of alter commands for just a single host. Maybe this is too contrived an example? I agree that having this level of granularity gives flexibility but I wonder if people will actually use it and not just * the hosts for a given user and create separate global list as i mentioned above? The only other system I know of that ties users with hosts for access is MySql and I don't love that model. Companies usually standardize on group authorization anyway, are we complicating that issue with the inclusion of hosts attached to users? Additionally I worry about the debt of big JSON configs in the first place, most non-developers find them non-intuitive already, so anything to ease this I think would be beneficial. Thanks Jeff On Wed, Apr 22, 2015 at 2:22 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Sorry I missed your last questions. I am +0 on adding ―host option for ―list, we could add it for symmetry. Again if this is only a CLI change it can be added later if you mean adding this in authorizer interface then we should make a decision now. Given a choice I would like to actually keep only one option which is resource based get (remove even the get based on principal). I see those (getAcl for principal or host) as special filtering case which can easily be achieved by a third party tool by doing list all topics and calling getAcls for each topic and applying filtering logic on that. I really don’t see the need to make those first class citizens of the authorizer interface given these kind of queries will be issued outside of broker JVM so they will not benefit from the caching and because the storage will be indexed on resource both these options even as a first class API will just scan all topic acls and apply filtering logic. Thanks Parth On 4/22/15, 11:08 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Please see all the available options here https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization +I nterface#KIP-11-AuthorizationInterface-AclManagement(CLI) . I think it covers both hosts and operations and allows to specify a list for both. Thanks Parth From: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Reply-To: Tom Graves tgraves...@yahoo.commailto:tgraves...@yahoo.com Date: Wednesday, April 22, 2015 at 11:02 AM To: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com, dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-11- Authorization design for kafka security Thanks for the explanations Parth. On the configs questions, the way I see it is its more likely to accidentally give everyone access, especially since you have to run a separate command to change the acls. If there was some config for defaults, a cluster admin could change that to be nobody or certain set of users, then grant others permissions. This would also remove the race between commands. This is something you can always add later though if people request it. So in kafka-acl.sh how do I actually tell it what the operation is? kafka-acl.sh --topic testtopic --add --grandprincipal user:joe,user:kate where does READ, WRITE, etc go? Can specify as a list so I
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Harsha, Parth, Thanks for the clarification. This makes sense. Perhaps we can clarify the meaning of those rules in the wiki. Related to this, it seems that we need to support wildcard in cli/request protocol for topics? Jun On Mon, Apr 20, 2015 at 9:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: The iptables on unix supports the DENY operator, not that it should matter. The deny operator can also be used to specify ³allow user1 to READ from topic1 from all hosts but host1,host2². Again we could add a host group semantic and extra complexity around that, not sure if its worth it. In addition with DENY operator you are now not forced to create a special group just to support the authorization use case. I am not convinced that the operator it self is really all that confusing. There are 3 practical use cases: - Resource with no acl what so ever - allow access to everyone ( just for backward compatibility, I would much rather fail close and force users to explicitly grant acls that allows access to all users.) - Resource with some acl attached - only users that have a matching allow acl are allowed (i.e. ³allow READ access to topic1 to user1 from all hosts², only user1 has READ access and no other user has access of any kind) - Resource with some allow and some deny acl attached - users are allowed to perform operation only when they satisfy allow acl and do not have conflicting deny acl. Users that have no acl(allow or deny) will still not have any access. (i.e. ³allow READ access to topic1 to user1 from all hosts except host1 and host², only user1 has access but not from host1 an host2) I think we need to make a decision on deny primarily because with introduction of acl management API, Acl is now a public class that will be used by Ranger/Santry and other authroization providers. In Current design the acl has a permissionType enum field with possible values of Allow and Deny. If we chose to remove deny we can assume all acls to be of allow type and remove the permissionType field completely. Thanks Parth On 4/20/15, 6:12 PM, Gwen Shapira gshap...@cloudera.com wrote: I think thats how its done in pretty much any system I can think of.
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Also, I think I may have missed this but does READ imply you also have DESCRIBE? A reader will need access to both read offsets (to determine their own initial position) as well as commit offsets. Currently, though fetching offsets is under DESCRIBE only and commit offsets is under READ. If READ=DESCRIBE are there any other implied permissions like that? -Jay On Tue, Apr 21, 2015 at 1:59 PM, Jay Kreps jay.kr...@gmail.com wrote: Hey Parth, Great write-up! One super minor thing: could we change the EDIT permission to be called ALTER? The request name in KIP-4 is Alter and the command line tool has always been alter (or we could go the other way and change those to EDIT). Not sure that one is any better than the other but consistency is always nice. -Jay On Tue, Apr 21, 2015 at 9:06 AM, Jun Rao j...@confluent.io wrote: Harsha, Parth, Thanks for the clarification. This makes sense. Perhaps we can clarify the meaning of those rules in the wiki. Related to this, it seems that we need to support wildcard in cli/request protocol for topics? Jun On Mon, Apr 20, 2015 at 9:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: The iptables on unix supports the DENY operator, not that it should matter. The deny operator can also be used to specify ³allow user1 to READ from topic1 from all hosts but host1,host2². Again we could add a host group semantic and extra complexity around that, not sure if its worth it. In addition with DENY operator you are now not forced to create a special group just to support the authorization use case. I am not convinced that the operator it self is really all that confusing. There are 3 practical use cases: - Resource with no acl what so ever - allow access to everyone ( just for backward compatibility, I would much rather fail close and force users to explicitly grant acls that allows access to all users.) - Resource with some acl attached - only users that have a matching allow acl are allowed (i.e. ³allow READ access to topic1 to user1 from all hosts², only user1 has READ access and no other user has access of any kind) - Resource with some allow and some deny acl attached - users are allowed to perform operation only when they satisfy allow acl and do not have conflicting deny acl. Users that have no acl(allow or deny) will still not have any access. (i.e. ³allow READ access to topic1 to user1 from all hosts except host1 and host², only user1 has access but not from host1 an host2) I think we need to make a decision on deny primarily because with introduction of acl management API, Acl is now a public class that will be used by Ranger/Santry and other authroization providers. In Current design the acl has a permissionType enum field with possible values of Allow and Deny. If we chose to remove deny we can assume all acls to be of allow type and remove the permissionType field completely. Thanks Parth On 4/20/15, 6:12 PM, Gwen Shapira gshap...@cloudera.com wrote: I think thats how its done in pretty much any system I can think of.
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Changed Edit to Alter. I did not think about it that way but Sriharsha raised the same point in a private conversation. I did not think about it that way but I agree it makes sense. If no one objects I think in default implementation we can infer that if user have READ or WRITE access he gets DESCRIBE for free. Thanks Parth On 4/21/15, 2:04 PM, Jay Kreps jay.kr...@gmail.com wrote: Also, I think I may have missed this but does READ imply you also have DESCRIBE? A reader will need access to both read offsets (to determine their own initial position) as well as commit offsets. Currently, though fetching offsets is under DESCRIBE only and commit offsets is under READ. If READ=DESCRIBE are there any other implied permissions like that? -Jay On Tue, Apr 21, 2015 at 1:59 PM, Jay Kreps jay.kr...@gmail.com wrote: Hey Parth, Great write-up! One super minor thing: could we change the EDIT permission to be called ALTER? The request name in KIP-4 is Alter and the command line tool has always been alter (or we could go the other way and change those to EDIT). Not sure that one is any better than the other but consistency is always nice. -Jay On Tue, Apr 21, 2015 at 9:06 AM, Jun Rao j...@confluent.io wrote: Harsha, Parth, Thanks for the clarification. This makes sense. Perhaps we can clarify the meaning of those rules in the wiki. Related to this, it seems that we need to support wildcard in cli/request protocol for topics? Jun On Mon, Apr 20, 2015 at 9:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: The iptables on unix supports the DENY operator, not that it should matter. The deny operator can also be used to specify ³allow user1 to READ from topic1 from all hosts but host1,host2². Again we could add a host group semantic and extra complexity around that, not sure if its worth it. In addition with DENY operator you are now not forced to create a special group just to support the authorization use case. I am not convinced that the operator it self is really all that confusing. There are 3 practical use cases: - Resource with no acl what so ever - allow access to everyone ( just for backward compatibility, I would much rather fail close and force users to explicitly grant acls that allows access to all users.) - Resource with some acl attached - only users that have a matching allow acl are allowed (i.e. ³allow READ access to topic1 to user1 from all hosts², only user1 has READ access and no other user has access of any kind) - Resource with some allow and some deny acl attached - users are allowed to perform operation only when they satisfy allow acl and do not have conflicting deny acl. Users that have no acl(allow or deny) will still not have any access. (i.e. ³allow READ access to topic1 to user1 from all hosts except host1 and host², only user1 has access but not from host1 an host2) I think we need to make a decision on deny primarily because with introduction of acl management API, Acl is now a public class that will be used by Ranger/Santry and other authroization providers. In Current design the acl has a permissionType enum field with possible values of Allow and Deny. If we chose to remove deny we can assume all acls to be of allow type and remove the permissionType field completely. Thanks Parth On 4/20/15, 6:12 PM, Gwen Shapira gshap...@cloudera.com wrote: I think thats how its done in pretty much any system I can think of.
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Hey Parth, Great write-up! One super minor thing: could we change the EDIT permission to be called ALTER? The request name in KIP-4 is Alter and the command line tool has always been alter (or we could go the other way and change those to EDIT). Not sure that one is any better than the other but consistency is always nice. -Jay On Tue, Apr 21, 2015 at 9:06 AM, Jun Rao j...@confluent.io wrote: Harsha, Parth, Thanks for the clarification. This makes sense. Perhaps we can clarify the meaning of those rules in the wiki. Related to this, it seems that we need to support wildcard in cli/request protocol for topics? Jun On Mon, Apr 20, 2015 at 9:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: The iptables on unix supports the DENY operator, not that it should matter. The deny operator can also be used to specify ³allow user1 to READ from topic1 from all hosts but host1,host2². Again we could add a host group semantic and extra complexity around that, not sure if its worth it. In addition with DENY operator you are now not forced to create a special group just to support the authorization use case. I am not convinced that the operator it self is really all that confusing. There are 3 practical use cases: - Resource with no acl what so ever - allow access to everyone ( just for backward compatibility, I would much rather fail close and force users to explicitly grant acls that allows access to all users.) - Resource with some acl attached - only users that have a matching allow acl are allowed (i.e. ³allow READ access to topic1 to user1 from all hosts², only user1 has READ access and no other user has access of any kind) - Resource with some allow and some deny acl attached - users are allowed to perform operation only when they satisfy allow acl and do not have conflicting deny acl. Users that have no acl(allow or deny) will still not have any access. (i.e. ³allow READ access to topic1 to user1 from all hosts except host1 and host², only user1 has access but not from host1 an host2) I think we need to make a decision on deny primarily because with introduction of acl management API, Acl is now a public class that will be used by Ranger/Santry and other authroization providers. In Current design the acl has a permissionType enum field with possible values of Allow and Deny. If we chose to remove deny we can assume all acls to be of allow type and remove the permissionType field completely. Thanks Parth On 4/20/15, 6:12 PM, Gwen Shapira gshap...@cloudera.com wrote: I think thats how its done in pretty much any system I can think of.
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Hey Jun, Yes and we support wild cards for all acl entities principal, hosts and operation. Thanks Parth On 4/21/15, 9:06 AM, Jun Rao j...@confluent.io wrote: Harsha, Parth, Thanks for the clarification. This makes sense. Perhaps we can clarify the meaning of those rules in the wiki. Related to this, it seems that we need to support wildcard in cli/request protocol for topics? Jun On Mon, Apr 20, 2015 at 9:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: The iptables on unix supports the DENY operator, not that it should matter. The deny operator can also be used to specify ³allow user1 to READ from topic1 from all hosts but host1,host2². Again we could add a host group semantic and extra complexity around that, not sure if its worth it. In addition with DENY operator you are now not forced to create a special group just to support the authorization use case. I am not convinced that the operator it self is really all that confusing. There are 3 practical use cases: - Resource with no acl what so ever - allow access to everyone ( just for backward compatibility, I would much rather fail close and force users to explicitly grant acls that allows access to all users.) - Resource with some acl attached - only users that have a matching allow acl are allowed (i.e. ³allow READ access to topic1 to user1 from all hosts², only user1 has READ access and no other user has access of any kind) - Resource with some allow and some deny acl attached - users are allowed to perform operation only when they satisfy allow acl and do not have conflicting deny acl. Users that have no acl(allow or deny) will still not have any access. (i.e. ³allow READ access to topic1 to user1 from all hosts except host1 and host², only user1 has access but not from host1 an host2) I think we need to make a decision on deny primarily because with introduction of acl management API, Acl is now a public class that will be used by Ranger/Santry and other authroization providers. In Current design the acl has a permissionType enum field with possible values of Allow and Deny. If we chose to remove deny we can assume all acls to be of allow type and remove the permissionType field completely. Thanks Parth On 4/20/15, 6:12 PM, Gwen Shapira gshap...@cloudera.com wrote: I think thats how its done in pretty much any system I can think of.
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Following up on the KIP discussion. Two options for authorizing consumers to read topic t as part of group g: 1. READ permission on resource /topic/t 2. READ permission on resource /topic/t AND WRITE permission on /group/g The advantage of (1) is that it is simpler. The disadvantage is that any member of any group that reads from t can commit offsets as any other member of a different group. This doesn't effect data security (who can access what) but it is a bit of a management issue--a malicious person can cause data loss or duplicates for another consumer by committing offset. I think I favor (2) but it's worth it to think it through. -Jay On Tue, Apr 21, 2015 at 2:43 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hey Jun, Yes and we support wild cards for all acl entities principal, hosts and operation. Thanks Parth On 4/21/15, 9:06 AM, Jun Rao j...@confluent.io wrote: Harsha, Parth, Thanks for the clarification. This makes sense. Perhaps we can clarify the meaning of those rules in the wiki. Related to this, it seems that we need to support wildcard in cli/request protocol for topics? Jun On Mon, Apr 20, 2015 at 9:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: The iptables on unix supports the DENY operator, not that it should matter. The deny operator can also be used to specify ³allow user1 to READ from topic1 from all hosts but host1,host2². Again we could add a host group semantic and extra complexity around that, not sure if its worth it. In addition with DENY operator you are now not forced to create a special group just to support the authorization use case. I am not convinced that the operator it self is really all that confusing. There are 3 practical use cases: - Resource with no acl what so ever - allow access to everyone ( just for backward compatibility, I would much rather fail close and force users to explicitly grant acls that allows access to all users.) - Resource with some acl attached - only users that have a matching allow acl are allowed (i.e. ³allow READ access to topic1 to user1 from all hosts², only user1 has READ access and no other user has access of any kind) - Resource with some allow and some deny acl attached - users are allowed to perform operation only when they satisfy allow acl and do not have conflicting deny acl. Users that have no acl(allow or deny) will still not have any access. (i.e. ³allow READ access to topic1 to user1 from all hosts except host1 and host², only user1 has access but not from host1 an host2) I think we need to make a decision on deny primarily because with introduction of acl management API, Acl is now a public class that will be used by Ranger/Santry and other authroization providers. In Current design the acl has a permissionType enum field with possible values of Allow and Deny. If we chose to remove deny we can assume all acls to be of allow type and remove the permissionType field completely. Thanks Parth On 4/20/15, 6:12 PM, Gwen Shapira gshap...@cloudera.com wrote: I think thats how its done in pretty much any system I can think of.
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Adding my notes from today's call to the thread: ** Deny or Allow all by default? We will add a configuration to control this. The configuration will default to “allow” for backward compatibility. Security admins can set it to deny ** Storing ACLs for default authorizers: We'll store them in ZK. We'll support pointing the authorizer to any ZK. The use of ZK will be internal to the default authorizer. Authorizer reads ACLs from cache every hour. We proposed having mechanism (possibly via new ZK node) to tell broker to refresh the cache immediately. ** Support deny as permission type - we agreed to keep this. ** Mapping operations to API: We may need to add Group as a resource, with JoinGroup and OffsetCommit require privilege on the consumer group. This can be something we pass now and authorizers can support in future. - Jay will write specifics to the mailing list discussion. On Tue, Apr 21, 2015 at 4:32 PM, Jay Kreps jay.kr...@gmail.com wrote: Following up on the KIP discussion. Two options for authorizing consumers to read topic t as part of group g: 1. READ permission on resource /topic/t 2. READ permission on resource /topic/t AND WRITE permission on /group/g The advantage of (1) is that it is simpler. The disadvantage is that any member of any group that reads from t can commit offsets as any other member of a different group. This doesn't effect data security (who can access what) but it is a bit of a management issue--a malicious person can cause data loss or duplicates for another consumer by committing offset. I think I favor (2) but it's worth it to think it through. -Jay On Tue, Apr 21, 2015 at 2:43 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hey Jun, Yes and we support wild cards for all acl entities principal, hosts and operation. Thanks Parth On 4/21/15, 9:06 AM, Jun Rao j...@confluent.io wrote: Harsha, Parth, Thanks for the clarification. This makes sense. Perhaps we can clarify the meaning of those rules in the wiki. Related to this, it seems that we need to support wildcard in cli/request protocol for topics? Jun On Mon, Apr 20, 2015 at 9:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: The iptables on unix supports the DENY operator, not that it should matter. The deny operator can also be used to specify ³allow user1 to READ from topic1 from all hosts but host1,host2². Again we could add a host group semantic and extra complexity around that, not sure if its worth it. In addition with DENY operator you are now not forced to create a special group just to support the authorization use case. I am not convinced that the operator it self is really all that confusing. There are 3 practical use cases: - Resource with no acl what so ever - allow access to everyone ( just for backward compatibility, I would much rather fail close and force users to explicitly grant acls that allows access to all users.) - Resource with some acl attached - only users that have a matching allow acl are allowed (i.e. ³allow READ access to topic1 to user1 from all hosts², only user1 has READ access and no other user has access of any kind) - Resource with some allow and some deny acl attached - users are allowed to perform operation only when they satisfy allow acl and do not have conflicting deny acl. Users that have no acl(allow or deny) will still not have any access. (i.e. ³allow READ access to topic1 to user1 from all hosts except host1 and host², only user1 has access but not from host1 an host2) I think we need to make a decision on deny primarily because with introduction of acl management API, Acl is now a public class that will be used by Ranger/Santry and other authroization providers. In Current design the acl has a permissionType enum field with possible values of Allow and Deny. If we chose to remove deny we can assume all acls to be of allow type and remove the permissionType field completely. Thanks Parth On 4/20/15, 6:12 PM, Gwen Shapira gshap...@cloudera.com wrote: I think thats how its done in pretty much any system I can think of.
Re: [DISCUSS] KIP-11- Authorization design for kafka security
I have added the notes to KIP-11 Open question sections. Thanks Parth On 4/21/15, 4:49 PM, Gwen Shapira gshap...@cloudera.com wrote: Adding my notes from today's call to the thread: ** Deny or Allow all by default? We will add a configuration to control this. The configuration will default to “allow” for backward compatibility. Security admins can set it to deny ** Storing ACLs for default authorizers: We'll store them in ZK. We'll support pointing the authorizer to any ZK. The use of ZK will be internal to the default authorizer. Authorizer reads ACLs from cache every hour. We proposed having mechanism (possibly via new ZK node) to tell broker to refresh the cache immediately. ** Support deny as permission type - we agreed to keep this. ** Mapping operations to API: We may need to add Group as a resource, with JoinGroup and OffsetCommit require privilege on the consumer group. This can be something we pass now and authorizers can support in future. - Jay will write specifics to the mailing list discussion. On Tue, Apr 21, 2015 at 4:32 PM, Jay Kreps jay.kr...@gmail.com wrote: Following up on the KIP discussion. Two options for authorizing consumers to read topic t as part of group g: 1. READ permission on resource /topic/t 2. READ permission on resource /topic/t AND WRITE permission on /group/g The advantage of (1) is that it is simpler. The disadvantage is that any member of any group that reads from t can commit offsets as any other member of a different group. This doesn't effect data security (who can access what) but it is a bit of a management issue--a malicious person can cause data loss or duplicates for another consumer by committing offset. I think I favor (2) but it's worth it to think it through. -Jay On Tue, Apr 21, 2015 at 2:43 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hey Jun, Yes and we support wild cards for all acl entities principal, hosts and operation. Thanks Parth On 4/21/15, 9:06 AM, Jun Rao j...@confluent.io wrote: Harsha, Parth, Thanks for the clarification. This makes sense. Perhaps we can clarify the meaning of those rules in the wiki. Related to this, it seems that we need to support wildcard in cli/request protocol for topics? Jun On Mon, Apr 20, 2015 at 9:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: The iptables on unix supports the DENY operator, not that it should matter. The deny operator can also be used to specify ³allow user1 to READ from topic1 from all hosts but host1,host2². Again we could add a host group semantic and extra complexity around that, not sure if its worth it. In addition with DENY operator you are now not forced to create a special group just to support the authorization use case. I am not convinced that the operator it self is really all that confusing. There are 3 practical use cases: - Resource with no acl what so ever - allow access to everyone ( just for backward compatibility, I would much rather fail close and force users to explicitly grant acls that allows access to all users.) - Resource with some acl attached - only users that have a matching allow acl are allowed (i.e. ³allow READ access to topic1 to user1 from all hosts², only user1 has READ access and no other user has access of any kind) - Resource with some allow and some deny acl attached - users are allowed to perform operation only when they satisfy allow acl and do not have conflicting deny acl. Users that have no acl(allow or deny) will still not have any access. (i.e. ³allow READ access to topic1 to user1 from all hosts except host1 and host², only user1 has access but not from host1 an host2) I think we need to make a decision on deny primarily because with introduction of acl management API, Acl is now a public class that will be used by Ranger/Santry and other authroization providers. In Current design the acl has a permissionType enum field with possible values of Allow and Deny. If we chose to remove deny we can assume all acls to be of allow type and remove the permissionType field completely. Thanks Parth On 4/20/15, 6:12 PM, Gwen Shapira gshap...@cloudera.com wrote: I think thats how its done in pretty much any system I can think of.
Re: [DISCUSS] KIP-11- Authorization design for kafka security
decides they want to represent their ACLs differently, and swap out the authorizer implementation, will that work? I guess what I’m asking is whether there’s any code in the Kafka codebase that will interpret that JSON, or does that logic live exclusively in the authorizer? On 4/14/15, 10:56 PM, Don Bosco Durai bo...@apache.orgmailto:bo...@apache.orgmailto:bo...@apache.org wrote: I also feel, having just IP would be more appropriate. Host lookup will unnecessary slow things down and would be insecure as you pointed out. With IP, it will be also able to setup policies (in future if needed) with ranges or netmasks and it would be more scalable. Bosco On 4/14/15, 1:40 PM, Michael Herstine mherst...@linkedin.com.INVALIDmailto:mherst...@linkedin.com.INVALID mailto:mherst...@linkedin.com.INVALID wrote: Hi Parth, Sorry to chime in so late, but I’ve got a minor question on the KIP. Several methods take a parameter named “host” of type String. Is that intended to be a hostname, or an IP address? If the former, I’m curious as to how that’s found (in my experience, when accepting an incoming socket connection, you only know the IP address, and there isn’t a way to map that to a hostname without a round trip to a DNS server, which is insecure anyway). On 3/25/15, 1:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto : pbrahmbh...@hortonworks.com wrote: Hi all, I have modified the KIP to reflect the recent change request from the reviewers. I have been working on the code and I have the server side code for authorization ready. I am now modifying the command line utilities. I would really appreciate if some of the committers can spend sometime to review the KIP so we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALIDmailto:mherst...@linkedin.com.INVALID mailto:mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto:bo...@apache.orgmailto:bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io mailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto : pbrahmbh...@hortonworks.commailto:p b rahmbh...@hortonworks.commailto:rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Parth, Sorry to chime in so late, but I’ve got a minor question on the KIP. Several methods take a parameter named “host” of type String. Is that intended to be a hostname, or an IP address? If the former, I’m curious as to how that’s found (in my experience, when accepting an incoming socket connection, you only know the IP address, and there isn’t a way to map that to a hostname without a round trip to a DNS server, which is insecure anyway). On 3/25/15, 1:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto : pbrahmbh...@hortonworks.com wrote: Hi all, I have modified the KIP to reflect the recent change request from the reviewers. I have been working on the code and I have the server side code for authorization ready. I am now modifying the command line utilities. I would really appreciate if some of the committers can spend sometime to review the KIP so we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALIDmailto:mherst...@linkedin.com.INVALID mailto:mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto:bo...@apache.orgmailto:bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io mailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto : pbrahmbh...@hortonworks.commailto:p b rahmbh...@hortonworks.commailto:rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
/14/15, 10:56 PM, Don Bosco Durai bo...@apache.orgmailto:bo...@apache.orgmailto:bo...@apache.org wrote: I also feel, having just IP would be more appropriate. Host lookup will unnecessary slow things down and would be insecure as you pointed out. With IP, it will be also able to setup policies (in future if needed) with ranges or netmasks and it would be more scalable. Bosco On 4/14/15, 1:40 PM, Michael Herstine mherst...@linkedin.com.INVALIDmailto:mherst...@linkedin.com.INVALID mailto:mherst...@linkedin.com.INVALID wrote: Hi Parth, Sorry to chime in so late, but I’ve got a minor question on the KIP. Several methods take a parameter named “host” of type String. Is that intended to be a hostname, or an IP address? If the former, I’m curious as to how that’s found (in my experience, when accepting an incoming socket connection, you only know the IP address, and there isn’t a way to map that to a hostname without a round trip to a DNS server, which is insecure anyway). On 3/25/15, 1:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto : pbrahmbh...@hortonworks.com wrote: Hi all, I have modified the KIP to reflect the recent change request from the reviewers. I have been working on the code and I have the server side code for authorization ready. I am now modifying the command line utilities. I would really appreciate if some of the committers can spend sometime to review the KIP so we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALIDmailto:mherst...@linkedin.com.INVALID mailto:mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto:bo...@apache.orgmailto:bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io mailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto : pbrahmbh...@hortonworks.commailto:p b rahmbh...@hortonworks.commailto:rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
socket connection, you only know the IP address, and there isn’t a way to map that to a hostname without a round trip to a DNS server, which is insecure anyway). On 3/25/15, 1:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com wrote: Hi all, I have modified the KIP to reflect the recent change request from the reviewers. I have been working on the code and I have the server side code for authorization ready. I am now modifying the command line utilities. I would really appreciate if some of the committers can spend sometime to review the KIP so we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALIDmailto:mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto:bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.commailto: p b rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:d...@kafka.apac h e .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:d...@kafka.apac h e .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io mailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto : pbrahmbh...@hortonworks.commailto:p b rahmbh...@hortonworks.commailto:rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto:bo...@apache.orgmailto:bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io mailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto : pbrahmbh...@hortonworks.commailto:p b rahmbh...@hortonworks.commailto:rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io mailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto : pbrahmbh...@hortonworks.commailto:p b rahmbh...@hortonworks.commailto:rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
for authorization ready. I am now modifying the command line utilities. I would really appreciate if some of the committers can spend sometime to review the KIP so we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALIDmailto:mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto:bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.commailto: p b rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:d...@kafka.apac h e .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:d...@kafka.apac h e .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
I tend to agree with Parth's point here. Most ACL systems I run into have deny and allow. In general, you have a default policy of allow, then you follow your rules stopping at the first line that matches. If you would like a default deny policy, you have a bunch of allow rules and your last rule is deny all. This says everyone listed is allowed. Everyone else is denied. If you instead want a default allow, you have a list of deny rules and the last rule is allow all. This says everyone listed is denied. Everyone else is allowed. I think leaving out a full rule set would be a mistake, as it makes the assumption that you know what all the use cases are. I think all it will really mean is that we will see a KIP before long to fix it. -Todd On Apr 20, 2015, at 7:13 PM, Gwen Shapira gshap...@cloudera.com wrote: Thanks for clarifying the logic. I'm +0 on the deny thing. IMO, its not really needed, but if you think its important, I don't object to having it in. Gwen On Mon, Apr 20, 2015 at 7:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: The iptables on unix supports the DENY operator, not that it should matter. The deny operator can also be used to specify ³allow user1 to READ from topic1 from all hosts but host1,host2². Again we could add a host group semantic and extra complexity around that, not sure if its worth it. In addition with DENY operator you are now not forced to create a special group just to support the authorization use case. I am not convinced that the operator it self is really all that confusing. There are 3 practical use cases: - Resource with no acl what so ever - allow access to everyone ( just for backward compatibility, I would much rather fail close and force users to explicitly grant acls that allows access to all users.) - Resource with some acl attached - only users that have a matching allow acl are allowed (i.e. ³allow READ access to topic1 to user1 from all hosts², only user1 has READ access and no other user has access of any kind) - Resource with some allow and some deny acl attached - users are allowed to perform operation only when they satisfy allow acl and do not have conflicting deny acl. Users that have no acl(allow or deny) will still not have any access. (i.e. ³allow READ access to topic1 to user1 from all hosts except host1 and host², only user1 has access but not from host1 an host2) I think we need to make a decision on deny primarily because with introduction of acl management API, Acl is now a public class that will be used by Ranger/Santry and other authroization providers. In Current design the acl has a permissionType enum field with possible values of Allow and Deny. If we chose to remove deny we can assume all acls to be of allow type and remove the permissionType field completely. Thanks Parth On 4/20/15, 6:12 PM, Gwen Shapira gshap...@cloudera.com wrote: I think thats how its done in pretty much any system I can think of.
Re: [DISCUSS] KIP-11- Authorization design for kafka security
The iptables on unix supports the DENY operator, not that it should matter. The deny operator can also be used to specify ³allow user1 to READ from topic1 from all hosts but host1,host2². Again we could add a host group semantic and extra complexity around that, not sure if its worth it. In addition with DENY operator you are now not forced to create a special group just to support the authorization use case. I am not convinced that the operator it self is really all that confusing. There are 3 practical use cases: - Resource with no acl what so ever - allow access to everyone ( just for backward compatibility, I would much rather fail close and force users to explicitly grant acls that allows access to all users.) - Resource with some acl attached - only users that have a matching allow acl are allowed (i.e. ³allow READ access to topic1 to user1 from all hosts², only user1 has READ access and no other user has access of any kind) - Resource with some allow and some deny acl attached - users are allowed to perform operation only when they satisfy allow acl and do not have conflicting deny acl. Users that have no acl(allow or deny) will still not have any access. (i.e. ³allow READ access to topic1 to user1 from all hosts except host1 and host², only user1 has access but not from host1 an host2) I think we need to make a decision on deny primarily because with introduction of acl management API, Acl is now a public class that will be used by Ranger/Santry and other authroization providers. In Current design the acl has a permissionType enum field with possible values of Allow and Deny. If we chose to remove deny we can assume all acls to be of allow type and remove the permissionType field completely. Thanks Parth On 4/20/15, 6:12 PM, Gwen Shapira gshap...@cloudera.com wrote: I think thats how its done in pretty much any system I can think of.
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Thanks for clarifying the logic. I'm +0 on the deny thing. IMO, its not really needed, but if you think its important, I don't object to having it in. Gwen On Mon, Apr 20, 2015 at 7:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: The iptables on unix supports the DENY operator, not that it should matter. The deny operator can also be used to specify ³allow user1 to READ from topic1 from all hosts but host1,host2². Again we could add a host group semantic and extra complexity around that, not sure if its worth it. In addition with DENY operator you are now not forced to create a special group just to support the authorization use case. I am not convinced that the operator it self is really all that confusing. There are 3 practical use cases: - Resource with no acl what so ever - allow access to everyone ( just for backward compatibility, I would much rather fail close and force users to explicitly grant acls that allows access to all users.) - Resource with some acl attached - only users that have a matching allow acl are allowed (i.e. ³allow READ access to topic1 to user1 from all hosts², only user1 has READ access and no other user has access of any kind) - Resource with some allow and some deny acl attached - users are allowed to perform operation only when they satisfy allow acl and do not have conflicting deny acl. Users that have no acl(allow or deny) will still not have any access. (i.e. ³allow READ access to topic1 to user1 from all hosts except host1 and host², only user1 has access but not from host1 an host2) I think we need to make a decision on deny primarily because with introduction of acl management API, Acl is now a public class that will be used by Ranger/Santry and other authroization providers. In Current design the acl has a permissionType enum field with possible values of Allow and Deny. If we chose to remove deny we can assume all acls to be of allow type and remove the permissionType field completely. Thanks Parth On 4/20/15, 6:12 PM, Gwen Shapira gshap...@cloudera.com wrote: I think thats how its done in pretty much any system I can think of.
Re: [DISCUSS] KIP-11- Authorization design for kafka security
it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io mailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.commailto : pbrahmbh...@hortonworks.commailto:p b rahmbh...@hortonworks.commailto:rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto:bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.commailto:p b rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apach e .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apach e .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io mailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.commailto: pbrahmbh...@hortonworks.commailto:p b rahmbh...@hortonworks.commailto:rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apach e .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.commailto:p b rahmbh...@hortonworks.commailto:rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apach e .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apach e .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Currently the authorizer does not perform any dns lookups and uses the hostname it receives as part of request.session as is. So in a way we are allowing only ip addresses. The only match is equality based so no ip ranges yet but that is easy to add. However, I think it is ok to allow for both ip addresses and hostnames and we should allow both. I am not sure why would I want to secure dns lookups and the host lookups extending to dns server are only necessary when the dns cache does not have the entry or the cache entry expires. This can be controlled by setting networkaddress.cache.ttl setting in jvm. Thanks Parth On 4/14/15, 10:56 PM, Don Bosco Durai bo...@apache.org wrote: I also feel, having just IP would be more appropriate. Host lookup will unnecessary slow things down and would be insecure as you pointed out. With IP, it will be also able to setup policies (in future if needed) with ranges or netmasks and it would be more scalable. Bosco On 4/14/15, 1:40 PM, Michael Herstine mherst...@linkedin.com.INVALID wrote: Hi Parth, Sorry to chime in so late, but I’ve got a minor question on the KIP. Several methods take a parameter named “host” of type String. Is that intended to be a hostname, or an IP address? If the former, I’m curious as to how that’s found (in my experience, when accepting an incoming socket connection, you only know the IP address, and there isn’t a way to map that to a hostname without a round trip to a DNS server, which is insecure anyway). On 3/25/15, 1:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hi all, I have modified the KIP to reflect the recent change request from the reviewers. I have been working on the code and I have the server side code for authorization ready. I am now modifying the command line utilities. I would really appreciate if some of the committers can spend sometime to review the KIP so we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
+ Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.commailto:pb rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apache .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apache .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALIDmailto:mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto:bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.commailto:p b rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apach e .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apach e .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto:bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authoriza t i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.commailto:p b rahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apach e .org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apach e .org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Hi Parth, Sorry to chime in so late, but I’ve got a minor question on the KIP. Several methods take a parameter named “host” of type String. Is that intended to be a hostname, or an IP address? If the former, I’m curious as to how that’s found (in my experience, when accepting an incoming socket connection, you only know the IP address, and there isn’t a way to map that to a hostname without a round trip to a DNS server, which is insecure anyway). On 3/25/15, 1:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hi all, I have modified the KIP to reflect the recent change request from the reviewers. I have been working on the code and I have the server side code for authorization ready. I am now modifying the command line utilities. I would really appreciate if some of the committers can spend sometime to review the KIP so we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizati o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
I also feel, having just IP would be more appropriate. Host lookup will unnecessary slow things down and would be insecure as you pointed out. With IP, it will be also able to setup policies (in future if needed) with ranges or netmasks and it would be more scalable. Bosco On 4/14/15, 1:40 PM, Michael Herstine mherst...@linkedin.com.INVALID wrote: Hi Parth, Sorry to chime in so late, but I’ve got a minor question on the KIP. Several methods take a parameter named “host” of type String. Is that intended to be a hostname, or an IP address? If the former, I’m curious as to how that’s found (in my experience, when accepting an incoming socket connection, you only know the IP address, and there isn’t a way to map that to a hostname without a round trip to a DNS server, which is insecure anyway). On 3/25/15, 1:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hi all, I have modified the KIP to reflect the recent change request from the reviewers. I have been working on the code and I have the server side code for authorization ready. I am now modifying the command line utilities. I would really appreciate if some of the committers can spend sometime to review the KIP so we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizat i o n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizatio n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth -- Thanks, Neha
Re: [DISCUSS] KIP-11- Authorization design for kafka security
it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizatio n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth -- Thanks, Neha
Re: [DISCUSS] KIP-11- Authorization design for kafka security
the KIP so we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALIDmailto:mherst...@linkedin.com.INVALID mailto:mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto:bo...@apache.orgmailto: bo...@apache.orgmailto:bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.iomailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizatio n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth -- Thanks, Neha
Re: [DISCUSS] KIP-11- Authorization design for kafka security
READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto: bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizatio n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth -- Thanks, Neha
Re: [DISCUSS] KIP-11- Authorization design for kafka security
as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto: bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizatio n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth -- Thanks, Neha
Re: [DISCUSS] KIP-11- Authorization design for kafka security
. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizatio n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth -- Thanks, Neha
Re: [DISCUSS] KIP-11- Authorization design for kafka security
and configuration Uis like (Ranger-Argus not sure what are they calling it now). On Wed, Mar 25, 2015 at 9:26 PM, Neha Narkhede n...@confluent.iomailto: n...@confluent.io wrote: Parth, We can make some 15 mins or so to discuss this at the next KIP hangout. Thanks, Neha On Wed, Mar 25, 2015 at 1:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com wrote: Hi all, I have modified the KIP to reflect the recent change request from the reviewers. I have been working on the code and I have the server side code for authorization ready. I am now modifying the command line utilities. I would really appreciate if some of the committers can spend sometime to review the KIP so we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALID mailto:mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto: bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizatio n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth -- Thanks, Neha
Re: [DISCUSS] KIP-11- Authorization design for kafka security
this at the next KIP hangout. Thanks, Neha On Wed, Mar 25, 2015 at 1:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.commailto: pbrahmbh...@hortonworks.com wrote: Hi all, I have modified the KIP to reflect the recent change request from the reviewers. I have been working on the code and I have the server side code for authorization ready. I am now modifying the command line utilities. I would really appreciate if some of the committers can spend sometime to review the KIP so we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALID mailto:mherst...@linkedin.com.INVALID mailto:mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto: bo...@apache.orgmailto: bo...@apache.orgmailto:bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io mailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizatio n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com mailto:pbrahmbh...@hortonworks.com mailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto: dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth -- Thanks, Neha
Re: [DISCUSS] KIP-11- Authorization design for kafka security
to Kerberos authentication AFAIK.) * Yes, again I think this can be a separate issue for now and can only be worked on after KIP-4 is delivered. 12. Do we want to support group acls as part of this authorizer? Do we want to support principal to local user mapping? If yes we need to add plugins for UserToGroupMapper and PrincipalToUserMapper.” - Sentry uses Groups for authorizing, so we need to support that. I figured that as long as the API specifies Principal, it typically contains both user and group, so nothing else is needed. Did I miss anything? * Once we support group acls we will need someway to indicate if a principal is of type group or user(as part of acl) or we can have group acls and user acls stored separately with topic config. We will also need a way to map an authenticated user to list of groups the user belongs to. 13. It looks like the Authorizer stores the ACLs and not Kafka. So we need an API for Kafka to notify Authorizer when a topic is added and when ACLs are modified, right? I didn’t see that. * ACLs will be stored under /topic/config at time of topic creation in json format. Authorizer will get these acls using newly introduced TopicConfigCache which gets updated anytime topic config changes. 14. Are we going to have any API for Kafka to give out the ACLs on a topic? Or we leave this to the Authorizer? * I think it is better to leave this out side of Kafka. Mainly because most 3rd party authorizers will have their own ACL stores and configuration Uis like (Ranger-Argus not sure what are they calling it now). On Wed, Mar 25, 2015 at 9:26 PM, Neha Narkhede n...@confluent.iomailto:n...@confluent.io wrote: Parth, We can make some 15 mins or so to discuss this at the next KIP hangout. Thanks, Neha On Wed, Mar 25, 2015 at 1:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com wrote: Hi all, I have modified the KIP to reflect the recent change request from the reviewers. I have been working on the code and I have the server side code for authorization ready. I am now modifying the command line utilities. I would really appreciate if some of the committers can spend sometime to review the KIP so we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALIDmailto:mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.orgmailto:bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.iomailto:ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizatio n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth -- Thanks, Neha
Re: [DISCUSS] KIP-11- Authorization design for kafka security
want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizatio n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth -- Thanks, Neha
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Hi all, I have modified the KIP to reflect the recent change request from the reviewers. I have been working on the code and I have the server side code for authorization ready. I am now modifying the command line utilities. I would really appreciate if some of the committers can spend sometime to review the KIP so we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizatio n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Parth, We can make some 15 mins or so to discuss this at the next KIP hangout. Thanks, Neha On Wed, Mar 25, 2015 at 1:07 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hi all, I have modified the KIP to reflect the recent change request from the reviewers. I have been working on the code and I have the server side code for authorization ready. I am now modifying the command line utilities. I would really appreciate if some of the committers can spend sometime to review the KIP so we can make progress on this. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizatio n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth -- Thanks, Neha
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Hi Michael, Thanks for taking the time to review. Currently I did not plan on adding “Deny” but I guess it can’t hurt except for adding more constructs would probably make the acls more complex. When a topic is created with no acls provided , I was planning to add a default ACL which would allow access to everyone from all hosts. I am assuming you are referring to principal in Acl and acls were supposed to be provided in property files, stored in zk so I thought it is better to just refer to a string. We will always be using session.principal.getName to get the actual principal name. Thanks Parth On 3/18/15, 2:20 PM, Michael Herstine mherst...@linkedin.com.INVALID wrote: Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorizatio n + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Hi Parth, Thanks! A few questions: 1. Do you want to permit rules in your ACLs that DENY access as well as ALLOW? This can be handy setting up rules that have exceptions. E.g. “Allow principal P to READ resource R from all hosts” with “Deny principal P READ access to resource R from host H1” in combination would allow P to READ R from all hosts *except* H1. 2. When a topic is newly created, will there be an ACL created for it? If not, would that not deny subsequent access to it? (nit) Maybe use Principal instead of String to represent principals? On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Parth Overall it is looking good. Couple of questions - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+ Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Hi Bosco, Thanks for taking the time to review. I will update the doc with a policy example and I will add CONNECT as an operation. For Admin APIs, I agree the best thing to do right now is just wait for KIP-4 to be submitted. I will update the doc to reflect the same. Thanks Parth On 3/9/15, 11:48 AM, Don Bosco Durai bo...@apache.org wrote: Parth Overall it is looking good. Couple of questionsŠ - Can you give an example how the policies will look like in the default implementation? - In the operations, can we support ³CONNECT² also? This can be used during Session connection - Regarding access control for ³Topic Creation², since we can¹t do it on the server side, can we de-scope it for? And plan it as a future feature request? Thanks Bosco On 3/6/15, 8:10 AM, Harsha ka...@harsha.io wrote: Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization + Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Hi Parth, Thanks for putting this together. Overall it looks good to me. Although AdminUtils is a concern KIP-4 can probably fix that part. Thanks, Harsha On Thu, Mar 5, 2015, at 10:39 AM, Parth Brahmbhatt wrote: Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
[DISCUSS] KIP-11- Authorization design for kafka security
Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
Re: [DISCUSS] KIP-11- Authorization design for kafka security
Forgot to add links to wiki and jira. Link to wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth From: Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh...@hortonworks.com Date: Thursday, March 5, 2015 at 10:33 AM To: dev@kafka.apache.orgmailto:dev@kafka.apache.org dev@kafka.apache.orgmailto:dev@kafka.apache.org Subject: [DISCUSS] KIP-11- Authorization design for kafka security Hi, KIP-11 is open for discussion , I have updated the wiki with the design and open questions. Thanks Parth
