Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-21 Thread Jun Rao
Parth, Thanks for driving this. Could you update the status of the KIP in the wiki? Thanks, Jun On Wed, May 20, 2015 at 2:37 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: This vote is now Closed with 4 binding +1s and 4 non binding +1s. Thanks Parth On 5/20/15, 12:04 PM, Joel

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-21 Thread Parth Brahmbhatt
I am sorry to be ignorant about this but what is the new state? Adopted seems too early given we are still in code review process. Should I just make it ³Code review²? Thanks Parth On 5/21/15, 8:43 AM, Jun Rao j...@confluent.io wrote: Parth, Thanks for driving this. Could you update the status

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-21 Thread Gwen Shapira
The KIP and design were accepted, so the WIKI should say accepted or something similar. Specific patch status is reflected in the JIRA. On Thu, May 21, 2015 at 8:37 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: I am sorry to be ignorant about this but what is the new state? Adopted

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-20 Thread Parth Brahmbhatt
This vote is now Closed with 4 binding +1s and 4 non binding +1s. Thanks Parth On 5/20/15, 12:04 PM, Joel Koshy jjkosh...@gmail.com wrote: +1 On Fri, May 15, 2015 at 04:18:49PM +, Parth Brahmbhatt wrote: Hi, Opening the voting thread for KIP-11. Link to the KIP:

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-20 Thread Joel Koshy
+1 On Fri, May 15, 2015 at 04:18:49PM +, Parth Brahmbhatt wrote: Hi, Opening the voting thread for KIP-11. Link to the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-18 Thread Joe Stein
+1 ~ Joe Stein - - - - - - - - - - - - - - - - - http://www.stealth.ly - - - - - - - - - - - - - - - - - On Fri, May 15, 2015 at 7:35 PM, Jun Rao j...@confluent.io wrote: +1 Thanks, Jun On Fri, May 15, 2015 at 9:18 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hi,

[Vote] KIP-11 Authorization design for kafka security

2015-05-15 Thread Parth Brahmbhatt
Hi, Opening the voting thread for KIP-11. Link to the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface Link to Jira: https://issues.apache.org/jira/browse/KAFKA-1688 Thanks Parth

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-15 Thread Jun Rao
+1 Thanks, Jun On Fri, May 15, 2015 at 9:18 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hi, Opening the voting thread for KIP-11. Link to the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface Link to Jira:

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-15 Thread Harsha
+1 non-binding On Fri, May 15, 2015 at 9:18 AM -0700, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hi, Opening the voting thread for KIP-11. Link to the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface Link to Jira:

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-15 Thread Jay Kreps
+1 -Jay On Fri, May 15, 2015 at 9:18 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hi, Opening the voting thread for KIP-11. Link to the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface Link to Jira:

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-15 Thread Don Bosco Durai
+1 non-binding On 5/15/15, 11:43 AM, Gwen Shapira gshap...@cloudera.com wrote: +1 non-binding On Fri, May 15, 2015 at 9:12 PM, Harsha harsh...@fastmail.fm wrote: +1 non-binding On Fri, May 15, 2015 at 9:18 AM -0700, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hi,

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-15 Thread Gwen Shapira
+1 non-binding On Fri, May 15, 2015 at 9:12 PM, Harsha harsh...@fastmail.fm wrote: +1 non-binding On Fri, May 15, 2015 at 9:18 AM -0700, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hi, Opening the voting thread for KIP-11. Link to the KIP:

Re: [Vote] KIP-11 Authorization design for kafka security

2015-05-15 Thread Tom Graves
+1 non-binding. Tom Graves On Friday, May 15, 2015 2:00 PM, Don Bosco Durai bo...@apache.org wrote: +1 non-binding On 5/15/15, 11:43 AM, Gwen Shapira gshap...@cloudera.com wrote: +1 non-binding On Fri, May 15, 2015 at 9:12 PM, Harsha harsh...@fastmail.fm wrote: +1 non-binding

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-05-01 Thread Jun Rao
...@cloudera.commailto:gshap...@cloudera.com Sent: Thursday, April 30, 2015 5:32 PM Subject: Re: [VOTE] KIP-11- Authorization design for kafka security To: dev@kafka.apache.orgmailto:dev@kafka.apache.org On Thu, Apr 30, 2015 at 4:39 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Joe Stein
@kafka.apache.org Subject: Re: [VOTE] KIP-11- Authorization design for kafka security While I see the advantage of being able to say something like: deny user X from hosts h1...h200 also allow user X from host h189, there are two issues here: 1. Complex rule systems can be difficult to reason about

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Jun Rao
@kafka.apache.org Subject: RE: [VOTE] KIP-11- Authorization design for kafka security Thank you for your rapid reply, Parth. * I think the wiki already describes the precedence order as Deny taking precedence over allow when conflicting acls are found https

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Gwen Shapira
...@cloudera.com] Sent: Tuesday, April 28, 2015 1:31 PM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-11- Authorization design for kafka security While I see the advantage of being able to say something like: deny user X from hosts h1...h200 also allow user X from host h189

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Joe Stein
...@cloudera.com] Sent: Tuesday, April 28, 2015 1:31 PM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-11- Authorization design for kafka security While I see the advantage of being able to say something like: deny user X from hosts h1...h200 also allow user X from host h189

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Sriharsha Chintalapani
smoothly. Regards Dapeng -Original Message- From: Gwen Shapira [mailto:gshap...@cloudera.com] Sent: Tuesday, April 28, 2015 1:31 PM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-11- Authorization design for kafka security While I

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Joe Stein
Dapeng -Original Message- From: Gwen Shapira [mailto:gshap...@cloudera.com] Sent: Tuesday, April 28, 2015 1:31 PM To: dev@kafka.apache.org Subject: Re [VOTE] KIP-11- Authorization design for kafka security While I see the advantage of being

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Joe Stein
:14 AM Subject: Re: [VOTE] KIP-11- Authorization design for kafka security To: dev@kafka.apache.orgmailto:dev@kafka.apache.org * Regarding additional authorizers: Prasad, who is a PMC on Apache Sentry reviewed the design and confirmed Sentry can integrate with the current APIs. Dapeng Sun

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Don Bosco Durai
: Gwen Shapira gshap...@cloudera.commailto:gshap...@cloudera.com Sent: Thursday, April 30, 2015 10:14 AM Subject: Re: [VOTE] KIP-11- Authorization design for kafka security To: dev@kafka.apache.orgmailto:dev@kafka.apache.org * Regarding additional authorizers: Prasad, who is a PMC on Apache

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Suresh Srinivas
://docs.aws.amazon.com/kinesis/latest/APIReference/CommonErrors.html From: Gwen Shapira gshap...@cloudera.com Sent: Thursday, April 30, 2015 6:05 PM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-11- Authorization design for kafka security I think Kafka's behavior

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Gwen Shapira
...@cloudera.commailto:gshap...@cloudera.com Sent: Thursday, April 30, 2015 5:32 PM Subject: Re: [VOTE] KIP-11- Authorization design for kafka security To: dev@kafka.apache.orgmailto:dev@kafka.apache.org On Thu, Apr 30, 2015 at 4:39 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.commailto:pbrahmbh

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Don Bosco Durai
Dapeng -Original Message- From: Gwen Shapira [mailto:gshap...@cloudera.com] Sent: Tuesday, April 28, 2015 1:31 PM To: dev@kafka.apache.org Subject: Re [VOTE] KIP-11- Authorization design for kafka security While I see the advantage of being able to say something

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Suresh Srinivas
_ From: Gwen Shapira gshap...@cloudera.commailto:gshap...@cloudera.com Sent: Thursday, April 30, 2015 10:14 AM Subject: Re: [VOTE] KIP-11- Authorization design for kafka security To: dev@kafka.apache.orgmailto:dev@kafka.apache.org * Regarding additional authorizers: Prasad, who is a PMC on Apache

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Suresh Srinivas
, 2015 4:12 PM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-11- Authorization design for kafka security I kind of thought of the authorization module as something that happens in handle(request: RequestChannel.Reuqest) in the request.requestId match If the request doesn't do what it is allowed too

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Gwen Shapira
...@cloudera.com Sent: Thursday, April 30, 2015 10:14 AM Subject: Re: [VOTE] KIP-11- Authorization design for kafka security To: dev@kafka.apache.orgmailto:dev@kafka.apache.org * Regarding additional authorizers: Prasad, who is a PMC on Apache Sentry reviewed the design and confirmed Sentry can

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Joe Stein
, access etc.? Regards, Suresh Sent from phone _ From: Joe Stein joe.st...@stealth.lymailto:joe.st...@stealth.ly Sent: Thursday, April 30, 2015 3:27 PM Subject: Re: [VOTE] KIP-11- Authorization design for kafka security To: dev@kafka.apache.orgmailto:dev

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-30 Thread Gwen Shapira
(and for whom), what operations were denied. From: Joe Stein joe.st...@tealth.ly Sent: Thursday, April 30, 2015 4:12 PM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-11- Authorization design for kafka security I kind of thought of the authorization module

RE: [VOTE] KIP-11- Authorization design for kafka security

2015-04-28 Thread Sun, Dapeng
@kafka.apache.org Subject: Re: [VOTE] KIP-11- Authorization design for kafka security While I see the advantage of being able to say something like: deny user X from hosts h1...h200 also allow user X from host h189, there are two issues here: 1. Complex rule systems can be difficult to reason about

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-27 Thread Jun Rao
Parth, I was thinking that in a multi-tenant environment, an admin may want to carve out some topic space to a user. For example, allow user X to create any topic of X_*. Not sure how critical it is though. Also, with the current api, what would the admin do to replicate the acls from one

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-27 Thread Parth Brahmbhatt
* We are not supporting regex matching to any of the strings (host,resource,principal) yet but this can be added. We have a special wild card (*) to refer to ALL but there is no other regex matching going on right now. We can associate CREATE with topics as you are proposing once KIP-4 is merged I

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-27 Thread Parth Brahmbhatt
Thanks for your comments Jun. * Renamed the resource to consumer-group in wiki. * I don’t see a use case where admins/users would want to reserve topic names in advance. Can you describe why this would be needed. Thanks Parth On 4/26/15, 2:01 PM, Jun Rao j...@confluent.io wrote: A few more

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-27 Thread Parth Brahmbhatt
@kafka.apache.org Subject: Re: [VOTE] KIP-11- Authorization design for kafka security A few more minor comments. 100. To make it clear, perhaps we should rename the resource group to consumer-group. We can probably make the same change in CLI as well so that it's not confused with user group. 101

RE: [VOTE] KIP-11- Authorization design for kafka security

2015-04-27 Thread Sun, Dapeng
Attach the image. https://raw.githubusercontent.com/sundapeng/attachment/master/kafka-acl1.png Regards Dapeng From: Sun, Dapeng [mailto:dapeng@intel.com] Sent: Tuesday, April 28, 2015 11:44 AM To: dev@kafka.apache.org Subject: RE: [VOTE] KIP-11- Authorization design for kafka security

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-27 Thread Gwen Shapira
the image. https://raw.githubusercontent.com/sundapeng/attachment/master/kafka-acl1.png Regards Dapeng From: Sun, Dapeng [mailto:dapeng@intel.com] Sent: Tuesday, April 28, 2015 11:44 AM To: dev@kafka.apache.org Subject: RE: [VOTE] KIP-11- Authorization design for kafka security Thank you

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-26 Thread Jun Rao
A few more minor comments. 100. To make it clear, perhaps we should rename the resource group to consumer-group. We can probably make the same change in CLI as well so that it's not confused with user group. 101. Currently, create is only at the cluster level. Should it also be at topic level?

RE: [VOTE] KIP-11- Authorization design for kafka security

2015-04-26 Thread Sun, Dapeng
meaning and make acl management easily. Regards Dapeng -Original Message- From: Jun Rao [mailto:j...@confluent.io] Sent: Monday, April 27, 2015 5:02 AM To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-11- Authorization design for kafka security A few more minor comments. 100. To make

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
I see Groups as something we can add incrementally in the current model. The acls take principalType: name so groups can be represented as group: groupName. We are not managing group memberships anywhere in kafka and I don’t see the need to do so. So for a topic1 using the CLI an admin can add an

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
We are not talking about same Groups :) I meant, Groups of consumers (which KIP-11 lists as a separate resource in the Privilege table) On Fri, Apr 24, 2015 at 11:31 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: I see Groups as something we can add incrementally in the current model.

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
Sorry, for the confusion. I'm not sure my last email is clear enough either... Consumers will have a Principal which may belong to a group. But consumer configuration also have a group.id, which controls how partitions are shared between consumers and how offsets are committed. I'm talking about

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
Thanks for your comments Gari. My responses are inline. Thanks Parth On 4/24/15, 10:36 AM, Gari Singh gari.r.si...@gmail.com wrote: Sorry - fat fingered send ... Not sure if my newbie vote will count, but I think you are getting pretty close here. Couple of things: 1) I know the Session

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
+1 (non-binding) Two nitpicks for the wiki: * Heartbeat is probably a READ and not CLUSTER operation. I'm pretty sure new consumers need it to be part of a consumer group. * Can you clearly separate which parts are the API (common to every Authorizer) and which parts are DefaultAuthorizer

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
You are right, moved it to the default implementation section. Thanks Parth On 4/24/15, 9:52 AM, Gwen Shapira gshap...@cloudera.com wrote: Sample ACL JSON and Zookeeper is in public API, but I thought it is part of DefaultAuthorizer (Since Sentry and Argus won't be using Zookeeper). Am I wrong?

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gari Singh
Not sure if my newbie vote will count, but I think you are getting pretty close here. Couple of things: 1) I know the Session object is from a different JIRA, but I think that Session should take a Subject rather than just a single Principal. The reason for this is because a Subject can have

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
Hi, I would like to open KIP-11 for voting. Thanks Parth On 4/22/15, 1:56 PM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Hi Jeff, Thanks a lot for the review. I think you have a valid point about acls being duplicated and the simplest solution would be to modify acls class so they

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
Sample ACL JSON and Zookeeper is in public API, but I thought it is part of DefaultAuthorizer (Since Sentry and Argus won't be using Zookeeper). Am I wrong? Or is it the KIP? On Fri, Apr 24, 2015 at 9:49 AM, Parth Brahmbhatt pbrahmbh...@hortonworks.com wrote: Thanks for clarifying Gwen, KIP

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Sriharsha Chintalapani
+1 (non-binding) --  Harsha On April 24, 2015 at 9:59:09 AM, Parth Brahmbhatt (pbrahmbh...@hortonworks.com) wrote: You are right, moved it to the default implementation section. Thanks Parth On 4/24/15, 9:52 AM, Gwen Shapira gshap...@cloudera.com wrote: Sample ACL JSON and

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
Thanks for clarifying Gwen, KIP updated. I tried to make the distinction by creating a section for all public APIs https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+In terface#KIP-11-AuthorizationInterface-PublicInterfacesandclasses Let me know if you think there is a

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gari Singh
Sorry - fat fingered send ... Not sure if my newbie vote will count, but I think you are getting pretty close here. Couple of things: 1) I know the Session object is from a different JIRA, but I think that Session should take a Subject rather than just a single Principal. The reason for this

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gwen Shapira
Thanks. One more thing I'm missing in the KIP is details on the Group resource (I think we discussed this and it was just not fully updated): * Does everyone have the privilege to create a new Group and use it to consume from Topics he's already privileged on? * Will the CLI tool be used to

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Parth Brahmbhatt
Sorry Gwen, completely misunderstood the question :-). * Does everyone have the privilege to create a new Group and use it to consume from Topics he's already privileged on? Yes in current proposal. I did not see an API to create group but if you have a READ permission on a TOPIC and

Re: [VOTE] KIP-11- Authorization design for kafka security

2015-04-24 Thread Gari Singh
I will move the comments about subject versus principal wrt session to the PR above. The comments around keys, etc are more appropriate there. If I tie this together with my comments in the thread on SASL / Kerberos, what I am having a hard time figuring out are the pluggable framework for both