Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-03-24 Thread Jun Rao
Hi, Mayuresh, Please find my replies inlined below. On Wed, Mar 22, 2017 at 8:07 PM, Mayuresh Gharat wrote: > Hi Jun, > > Please find the replies inline. > > One reason to have KafkaPrincipal in ACL is that we can extend it to > support group in the future. Have you thought about how to suppor

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-03-24 Thread radai
on logging - i think the best we can do is call principal.toString() and log that. the built-in one prints "type/name" but in some contexts that might not always make sense (using the certificate addr might make more sense if my security is cert-based) on string representation - again this is the b

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-03-22 Thread Mayuresh Gharat
Hi Jun, Please find the replies inline. One reason to have KafkaPrincipal in ACL is that we can extend it to support group in the future. Have you thought about how to support that in your new proposal? ---> This is a feature of PrincipalBuilder and Authorizer, which are pluggable. The type of Pr

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-03-20 Thread Jun Rao
Hi, Mayuresh, One reason to have KafkaPrincipal in ACL is that we can extend it to support group in the future. Have you thought about how to support that in your new proposal? Another reason that we had KafkaPrincipal is simplicity. It can be constructed from a simple string and makes matching e

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-03-15 Thread Mayuresh Gharat
Hi Jun, Sorry for the delayed reply. I agree that the easiest thing will be to add an additional field in the Session class and we should be OK. But having a KafkaPrincipal and java Principal with in the same class looks little weird. So we can do this and slowly deprecate the usage of KafkaPrinc

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-28 Thread Mayuresh Gharat
Hi Jun, Sure. I had an offline discussion with Joel on how we can deprecate the KafkaPrincipal from Session and Authorizer. I will update the KIP to see if we can address all the concerns here. If not we can keep the KafkaPrincipal. Thanks, Mayuresh On Tue, Feb 28, 2017 at 1:53 PM, Jun Rao wr

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-28 Thread Jun Rao
Hi, Joel, Good point on the getAcls() method. KafkaPrincipal is also tied to ACL, which is used in pretty much every method in Authorizer. Now, I am not sure if it's easy to deprecate KafkaPrincipal. Hi, Mayuresh, Given the above, it seems that the easiest thing is to add a new Principal field i

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-28 Thread Joel Koshy
If we deprecate KafkaPrincipal, then the Authorizer interface will also need to change - i.e., deprecate the getAcls(KafkaPrincipal) method. On Tue, Feb 28, 2017 at 10:11 AM, Mayuresh Gharat < gharatmayures...@gmail.com> wrote: > Hi Jun/Ismael, > > Thanks for the comments. > > I agree. > What I w

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-28 Thread Mayuresh Gharat
Hi Jun/Ismael, Thanks for the comments. I agree. What I was thinking was, we get the KIP passed now and wait till major kafka version release. We can then make this change, but for now we can wait. Does that work? If there are concerns, we can make the addition of extra field of type Principal t

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-28 Thread Jun Rao
Hi, Ismael, Good point on compatibility. Hi, Mayuresh, Given that, it seems that it's better to just add the raw principal as a new field in Session for now and deprecate the KafkaPrincipal field in the future if needed? Thanks, Jun On Mon, Feb 27, 2017 at 5:05 PM, Ismael Juma wrote: > Brea

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-27 Thread Ismael Juma
Breaking clients without a deprecation period is something we only do as a last resort. Is there strong justification for doing it here? Ismael On Mon, Feb 27, 2017 at 11:28 PM, Mayuresh Gharat < gharatmayures...@gmail.com> wrote: > Hi Ismael, > > Yeah. I agree that it might break the clients if

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-27 Thread Mayuresh Gharat
Hi Ismael, Yeah. I agree that it might break the clients if the user is using the kafkaPrincipal directly. But since KafkaPrincipal is also a Java Principal and I think, it would be a right thing to do replace the kafkaPrincipal with Java Principal at this stage than later. We can mention in the

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-27 Thread Ismael Juma
Hi Mayuresh, Sorry for the delay. The updated KIP states that there is no compatibility impact, but that doesn't seem right. The fact that we changed the type of Session.principal to `Principal` means that any code that expects it to be `KafkaPrincipal` will break. Either because of declared types

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-27 Thread Mayuresh Gharat
Hi Ismael, Joel, Becket Would you mind taking a look at this. We require 2 more binding votes for the KIP to pass. Thanks, Mayuresh On Thu, Feb 23, 2017 at 10:57 AM, Dong Lin wrote: > +1 (non-binding) > > On Wed, Feb 22, 2017 at 10:52 PM, Manikumar > wrote: > > > +1 (non-binding) > > > > On

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-23 Thread Dong Lin
+1 (non-binding) On Wed, Feb 22, 2017 at 10:52 PM, Manikumar wrote: > +1 (non-binding) > > On Thu, Feb 23, 2017 at 3:27 AM, Mayuresh Gharat < > gharatmayures...@gmail.com > > wrote: > > > Hi Jun, > > > > Thanks a lot for the comments and reviews. > > I agree we should log the username. > > What

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-22 Thread Manikumar
+1 (non-binding) On Thu, Feb 23, 2017 at 3:27 AM, Mayuresh Gharat wrote: > Hi Jun, > > Thanks a lot for the comments and reviews. > I agree we should log the username. > What I meant by creating KafkaPrincipal was, after this KIP we would not be > required to create KafkaPrincipal and if we want

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-22 Thread Mayuresh Gharat
Hi Jun, Thanks a lot for the comments and reviews. I agree we should log the username. What I meant by creating KafkaPrincipal was, after this KIP we would not be required to create KafkaPrincipal and if we want to maintain the old logging, we will have to create it as we do today. I will take car

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-22 Thread Jun Rao
Hi, Mayuresh, For logging the user name, we could do either way. We just need to make sure the expected user name is logged. Also, currently, we are already creating a KafkaPrincipal on every request. +1 on the latest KIP. Thanks, Jun On Tue, Feb 21, 2017 at 8:05 PM, Mayuresh Gharat wrote: >

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-21 Thread Mayuresh Gharat
Hi Jun, Thanks for the comments. I will mention in the KIP : how this change doesn't affect the default authorizer implementation. Regarding, Currently, we log the principal name in the request log in RequestChannel, which has the format of "principalType + SEPARATOR + name;". It would be good i

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-20 Thread Jun Rao
Hi, Mayuresh, Thanks for the updated KIP. A couple of more comments. 1. Do we convert java.security.Principal to KafkaPrincipal for authorization check in SimpleAclAuthorizer? If so, it would be useful to mention that in the wiki so that people can understand how this change doesn't affect the de

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-17 Thread Mayuresh Gharat
Hi Jun, I have updated the KIP. Would you mind taking another look? Thanks, Mayuresh On Fri, Feb 17, 2017 at 4:42 PM, Mayuresh Gharat wrote: > Hi Jun, > > Sure sounds good to me. > > Thanks, > > Mayuresh > > On Fri, Feb 17, 2017 at 1:54 PM, Jun Rao wrote: > >> Hi, Mani, >> >> Good point on u

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-17 Thread Mayuresh Gharat
Hi Jun, Sure sounds good to me. Thanks, Mayuresh On Fri, Feb 17, 2017 at 1:54 PM, Jun Rao wrote: > Hi, Mani, > > Good point on using PrincipalBuilder for SASL. It seems that > PrincipalBuilder already has access to Authenticator. So, we could just > enable that in SaslChannelBuilder. We proba

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-17 Thread Jun Rao
Hi, Mani, Good point on using PrincipalBuilder for SASL. It seems that PrincipalBuilder already has access to Authenticator. So, we could just enable that in SaslChannelBuilder. We probably could do that in a separate KIP? Hi, Mayuresh, If you don't think there is a concrete use case for using P

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-17 Thread Mayuresh Gharat
@Manikumar, Can you give an example how you are planning to use PrincipalBuilder? @Jun Yes, that is right. To give a brief overview, we just extract the cert and hand it over to a third party library for creating a Principal. So we cannot create a Principal from just a string. The main motive beh

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-16 Thread Manikumar
Hi Jun, yes, we can just customize rules to send full principal name. I was just thinking to use PrinciplaBuilder interface for implementing SASL rules also. So that the interface will be consistent across protocols. Thanks On Fri, Feb 17, 2017 at 1:07 AM, Jun Rao wrote: > Hi, Radai, Mayuresh

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-16 Thread Jun Rao
Hi, Radai, Mayuresh, Thanks for the explanation. Good point on a pluggable authorizer can customize how acls are added. However, earlier, Mayuresh was saying that in LinkedIn's customized authorizer, it's not possible to create a principal from string. If that's the case, will adding the principal

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-15 Thread Mayuresh Gharat
@Jun thanks for the comments.Please see the replies inline. Currently kafka-acl.sh just creates an ACL path in ZK with the principal name string. > Yes, the kafka-acl.sh calls the addAcl() on the inbuilt SimpleAclAuthorizer which in turn creates an ACL in ZK with the Principal name string. Thi

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-15 Thread radai
@jun: "Currently kafka-acl.sh just creates an ACL path in ZK with the principal name string" - yes, but not directly. all it actually does it spin-up the Authorizer and call Authorizer.addAcl() on it. the vanilla Authorizer goes to ZK. but generally speaking, users can plug in their own Authorizers

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-15 Thread Manikumar
Sorry, I am late to this discussion. PrincipalBuilder is only used for SSL Protocol. For SASL, we use "sasl.kerberos.principal.to.local.rules" config to map SASL principal names to short names. To make it consistent, Do we also need to pass the SASL full principal name to authorizer ? We may need

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-14 Thread Jun Rao
Hi, Radai, Currently kafka-acl.sh just creates an ACL path in ZK with the principal name string. The authorizer module in the broker reads the principal name string from the acl path in ZK and creates the expected KafkaPrincipal for matching. As you can see, the expected principal is created on th

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-13 Thread radai
if i understand correctly, kafka-acls.sh spins up an instance of (the custom, in our case) Authorizer, and calls things like addAcls(acls: Set[Acl], resource: Resource) on it, which are defined in the interface, hence expected to be "extensible". (side note: if Authorizer and PrincipalBuilder are

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-13 Thread Jun Rao
Hi, Mayuresh, I seems to me that there are two common use cases of authorizer. (1) Use the default SimpleAuthorizer and the kafka-acl to do authorization. (2) Use a customized authorizer and an external tool for authorization. Do you think there is a use case for a customized authorizer and kafka-

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-13 Thread Mayuresh Gharat
Hi Jun, Thanks for the review and comments. Please find the replies inline : This is so that in the future, we can extend to types like group. ---> Yep, I did think the same. But since the SocketServer was always creating User type, it wasn't actually used. If we go ahead with changes in this KIP

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-09 Thread Jun Rao
Hi, Mayuresh, Thanks for the reply. A few more comments below. On Wed, Feb 8, 2017 at 9:14 PM, Mayuresh Gharat wrote: > Hi Jun, > > Thanks for the review. Please find the responses inline. > > 1. It seems the problem that you are trying to address is that java > principal returned from KafkaCha

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-08 Thread Mayuresh Gharat
Hi Jun, Thanks for the review. Please find the responses inline. 1. It seems the problem that you are trying to address is that java principal returned from KafkaChannel may have additional fields than name that are needed during authorization. Have you considered a customized PrincipleBuilder th

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-08 Thread Jun Rao
Hi, Mayuresh, Thanks for the KIP. A few comments below. 1. It seems the problem that you are trying to address is that java principal returned from KafkaChannel may have additional fields than name that are needed during authorization. Have you considered a customized PrincipleBuilder that extrac

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-07 Thread Mayuresh Gharat
Bumping up this thread. Thanks, Mayuresh On Fri, Feb 3, 2017 at 5:09 PM, radai wrote: > +1 > > On Fri, Feb 3, 2017 at 11:25 AM, Mayuresh Gharat < > gharatmayures...@gmail.com > > wrote: > > > Hi All, > > > > It seems that there is no further concern with the KIP-111. At this point > > we would

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-03 Thread radai
+1 On Fri, Feb 3, 2017 at 11:25 AM, Mayuresh Gharat wrote: > Hi All, > > It seems that there is no further concern with the KIP-111. At this point > we would like to start the voting process. The KIP can be found at > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=67638388 > >

[VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

2017-02-03 Thread Mayuresh Gharat
Hi All, It seems that there is no further concern with the KIP-111. At this point we would like to start the voting process. The KIP can be found at https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=67638388 Thanks, Mayuresh