Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Thank you all for your votes and feedback. The vote has passed with 4 binding votes(Gwen, Jun, Grant, Harsha) and 2 non-binding votes(Roger, Dong Lin). I have updated the relevant wiki pages. Thanks Manikumar On Tue, Feb 14, 2017 at 12:02 AM, Dong Linwrote: > +1 (non-binding) > > On Mon, Feb 13, 2017 at 10:21 AM, Harsha Chintalapani > wrote: > > > +1. > > -Harsha > > > > On Fri, Feb 10, 2017 at 11:12 PM Manikumar > > wrote: > > > > > Yes, owners and the renewers can always describe their own tokens. > > Updated > > > the KIP. > > > > > > On Sat, Feb 11, 2017 at 3:12 AM, Jun Rao wrote: > > > > > > > Hi, Mani, > > > > > > > > Thanks for the update. Just a minor comment below. Otherwise, +1 from > > me. > > > > > > > > > > > > > > > > > > > > > > > > > 116. Could you document the ACL rules associated with those new > > > > requests? > > > > > > For example, do we allow any one to create, delete, describe > > > delegation > > > > > > tokens? > > > > > > > > > > > > > > > > > Currently we only allow a owner to create delegation token for that > > > owner > > > > > only. > > > > > Any thing the owner has permission to do, delegation tokens should > be > > > > > allowed to do as well. We can also check renew and expire requests > > are > > > > > coming > > > > > from owner or renewers of the token. So we may not need ACLs for > > > > > create/renew/expire requests. > > > > > > > > > > For describe, we can add DESCRIBE operation on TOKEN Resource. In > > > future, > > > > > when we extend > > > > > the support to allow a user to acquire delegation tokens for other > > > users, > > > > > then we can enable > > > > > CREATE/DELETE operations. Updated the KIP. > > > > > > > > > > > > > > This sounds good. I guess the owner and the renewer can always > describe > > > > their own tokens? > > > > > > > > Jun > > > > > > > > > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
+1 (non-binding) On Mon, Feb 13, 2017 at 10:21 AM, Harsha Chintalapaniwrote: > +1. > -Harsha > > On Fri, Feb 10, 2017 at 11:12 PM Manikumar > wrote: > > > Yes, owners and the renewers can always describe their own tokens. > Updated > > the KIP. > > > > On Sat, Feb 11, 2017 at 3:12 AM, Jun Rao wrote: > > > > > Hi, Mani, > > > > > > Thanks for the update. Just a minor comment below. Otherwise, +1 from > me. > > > > > > > > > > > > > > > > > > > > 116. Could you document the ACL rules associated with those new > > > requests? > > > > > For example, do we allow any one to create, delete, describe > > delegation > > > > > tokens? > > > > > > > > > > > > > > Currently we only allow a owner to create delegation token for that > > owner > > > > only. > > > > Any thing the owner has permission to do, delegation tokens should be > > > > allowed to do as well. We can also check renew and expire requests > are > > > > coming > > > > from owner or renewers of the token. So we may not need ACLs for > > > > create/renew/expire requests. > > > > > > > > For describe, we can add DESCRIBE operation on TOKEN Resource. In > > future, > > > > when we extend > > > > the support to allow a user to acquire delegation tokens for other > > users, > > > > then we can enable > > > > CREATE/DELETE operations. Updated the KIP. > > > > > > > > > > > This sounds good. I guess the owner and the renewer can always describe > > > their own tokens? > > > > > > Jun > > > > > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
+1. -Harsha On Fri, Feb 10, 2017 at 11:12 PM Manikumarwrote: > Yes, owners and the renewers can always describe their own tokens. Updated > the KIP. > > On Sat, Feb 11, 2017 at 3:12 AM, Jun Rao wrote: > > > Hi, Mani, > > > > Thanks for the update. Just a minor comment below. Otherwise, +1 from me. > > > > > > > > > > > > > > > 116. Could you document the ACL rules associated with those new > > requests? > > > > For example, do we allow any one to create, delete, describe > delegation > > > > tokens? > > > > > > > > > > > Currently we only allow a owner to create delegation token for that > owner > > > only. > > > Any thing the owner has permission to do, delegation tokens should be > > > allowed to do as well. We can also check renew and expire requests are > > > coming > > > from owner or renewers of the token. So we may not need ACLs for > > > create/renew/expire requests. > > > > > > For describe, we can add DESCRIBE operation on TOKEN Resource. In > future, > > > when we extend > > > the support to allow a user to acquire delegation tokens for other > users, > > > then we can enable > > > CREATE/DELETE operations. Updated the KIP. > > > > > > > > This sounds good. I guess the owner and the renewer can always describe > > their own tokens? > > > > Jun > > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Yes, owners and the renewers can always describe their own tokens. Updated the KIP. On Sat, Feb 11, 2017 at 3:12 AM, Jun Raowrote: > Hi, Mani, > > Thanks for the update. Just a minor comment below. Otherwise, +1 from me. > > > > > > > > > > 116. Could you document the ACL rules associated with those new > requests? > > > For example, do we allow any one to create, delete, describe delegation > > > tokens? > > > > > > > > Currently we only allow a owner to create delegation token for that owner > > only. > > Any thing the owner has permission to do, delegation tokens should be > > allowed to do as well. We can also check renew and expire requests are > > coming > > from owner or renewers of the token. So we may not need ACLs for > > create/renew/expire requests. > > > > For describe, we can add DESCRIBE operation on TOKEN Resource. In future, > > when we extend > > the support to allow a user to acquire delegation tokens for other users, > > then we can enable > > CREATE/DELETE operations. Updated the KIP. > > > > > This sounds good. I guess the owner and the renewer can always describe > their own tokens? > > Jun >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi, Mani, Thanks for the update. Just a minor comment below. Otherwise, +1 from me. > > > > > 116. Could you document the ACL rules associated with those new requests? > > For example, do we allow any one to create, delete, describe delegation > > tokens? > > > > > Currently we only allow a owner to create delegation token for that owner > only. > Any thing the owner has permission to do, delegation tokens should be > allowed to do as well. We can also check renew and expire requests are > coming > from owner or renewers of the token. So we may not need ACLs for > create/renew/expire requests. > > For describe, we can add DESCRIBE operation on TOKEN Resource. In future, > when we extend > the support to allow a user to acquire delegation tokens for other users, > then we can enable > CREATE/DELETE operations. Updated the KIP. > > This sounds good. I guess the owner and the renewer can always describe their own tokens? Jun
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi Jun, Thanks for the detailed review. Pl see the replies inline > 101.2/101.3. Could we just remove owner and renewer from > DelegationTokenResponse if we don't have a use case? > > Removed owner/renewer fileds from DelegationTokenResponse. Updated the KIP. > 111. ExpireTokenResponse: Should we return the new expiration time in the > response? > > Yes. expiration time is also included in RenewToken response . > 112. DescribeTokenRequest: A common use case is probably to see a list of > tokens associated with a particular owner. Would it be useful to include a > list of owners in the request? We can use the same convention in other > requests such that if the list is set to null (i.e., length is -1), return > all tokens. > > Agree. Updated the KIP. > 113. delegation.token.master.key: It seems that needs to be the same across > brokers? Perhaps we can mention that in the wiki. > Updated the KIP. > > 114. Could we document the procedure of manually rotating the secret? Does > one have to do sth like: expire all existing tokens, rotate the secret, and > generate new tokens? > > Thanks for the suggestion. Updated the suggested steps. During rotation process, already connected clients may continue to work. But any new connection requests and renew/expire requests with old tokens can fail. 115. Could we also include in the command line the ability to describe > tokens? > Updated the KIP. > > 116. Could you document the ACL rules associated with those new requests? > For example, do we allow any one to create, delete, describe delegation > tokens? > > Currently we only allow a owner to create delegation token for that owner only. Any thing the owner has permission to do, delegation tokens should be allowed to do as well. We can also check renew and expire requests are coming from owner or renewers of the token. So we may not need ACLs for create/renew/expire requests. For describe, we can add DESCRIBE operation on TOKEN Resource. In future, when we extend the support to allow a user to acquire delegation tokens for other users, then we can enable CREATE/DELETE operations. Updated the KIP. Thanks, Manikumar > On Wed, Feb 8, 2017 at 1:35 AM, Manikumar> wrote: > > > Hi Jun, > > > > > > > If a token expires, then every broker will potentially try to delete it > > > around the same time, but only one will succeed. So, we will have to > deal > > > with failures in that case? Another way is to let just one broker (say, > > the > > > controller) deletes expired tokens. > > > > > > > > Agree, we can run the token expiry check thread as part of controller > > broker. > > WIll update the KIP. > > > > > > Thanks, > > Manikumar > > > > > > > > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar > > > wrote: > > > > > > > Hi Jun, > > > > > > > > Please see the replies inline. > > > > > > > > > > > > > > > > > > > > Only one broker does the deletion. Broker updates the expiration > in > > > its > > > > > > local cache > > > > > > and on zookeeper so other brokers also get notified and their > cache > > > > > > statuses are updated as well. > > > > > > > > > > > > > > > > > Which broker does the deletion? > > > > > > > > > > > > > Any broker can handle the create/expire/renew/describe > delegationtoken > > > > requests. > > > > changes are propagated through zk notifications. Every broker is > > > > responsible for > > > > expiring the tokens. This check be can done during request handling > > time > > > > and/or > > > > during token authentication time. > > > > > > > > > > > > > > > > > > > > > > > 110. The diagrams in the wiki still show MD5 digest. Could you > change > > > it > > > > to > > > > > SCRAM? > > > > > > > > > > > > > > Updated the diagram. > > > > > > > > > > > > > > > > Thanks, > > > > Manikumar > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks. > > > > > > Manikumar > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar < > > > > manikumar.re...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > I would like to initiate the vote on KIP-48: > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ > > > > > > > > Delegation+token+support+for+Kafka > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Manikumar > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi, Mani, Thanks for the responses. A few more comments. 101.2/101.3. Could we just remove owner and renewer from DelegationTokenResponse if we don't have a use case? 111. ExpireTokenResponse: Should we return the new expiration time in the response? 112. DescribeTokenRequest: A common use case is probably to see a list of tokens associated with a particular owner. Would it be useful to include a list of owners in the request? We can use the same convention in other requests such that if the list is set to null (i.e., length is -1), return all tokens. 113. delegation.token.master.key: It seems that needs to be the same across brokers? Perhaps we can mention that in the wiki. 114. Could we document the procedure of manually rotating the secret? Does one have to do sth like: expire all existing tokens, rotate the secret, and generate new tokens? 115. Could we also include in the command line the ability to describe tokens? 116. Could you document the ACL rules associated with those new requests? For example, do we allow any one to create, delete, describe delegation tokens? Thanks, Jun On Wed, Feb 8, 2017 at 1:35 AM, Manikumarwrote: > Hi Jun, > > > > If a token expires, then every broker will potentially try to delete it > > around the same time, but only one will succeed. So, we will have to deal > > with failures in that case? Another way is to let just one broker (say, > the > > controller) deletes expired tokens. > > > > > Agree, we can run the token expiry check thread as part of controller > broker. > WIll update the KIP. > > > Thanks, > Manikumar > > > > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar > > wrote: > > > > > Hi Jun, > > > > > > Please see the replies inline. > > > > > > > > > > > > > > > > Only one broker does the deletion. Broker updates the expiration in > > its > > > > > local cache > > > > > and on zookeeper so other brokers also get notified and their cache > > > > > statuses are updated as well. > > > > > > > > > > > > > > Which broker does the deletion? > > > > > > > > > > Any broker can handle the create/expire/renew/describe delegationtoken > > > requests. > > > changes are propagated through zk notifications. Every broker is > > > responsible for > > > expiring the tokens. This check be can done during request handling > time > > > and/or > > > during token authentication time. > > > > > > > > > > > > > > > > > > 110. The diagrams in the wiki still show MD5 digest. Could you change > > it > > > to > > > > SCRAM? > > > > > > > > > > > Updated the diagram. > > > > > > > > > > > > Thanks, > > > Manikumar > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks. > > > > > Manikumar > > > > > > > > > > > > > > > > > > > > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar < > > > manikumar.re...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > I would like to initiate the vote on KIP-48: > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ > > > > > > > Delegation+token+support+for+Kafka > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > Manikumar > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Thanks. I found the other discussion thread. Sorry for being behind on this. I'm interested in the future impersonation use cases. This seems to get us closer. +1 (non-binding) On Wed, Feb 8, 2017 at 4:41 AM, Manikumarwrote: > Hi Roger, > > In the current proposal, we only allow a user to get delegation token for > that user only. > Anyone who gets that token can impersonate the user on the broker. > > Yes, In future we can extend the support to allow a user to acquire > delegation tokens for > other users. > > Pl refer discuss mail thread for impersonation related discussion. > > Thanks, > Manikumar > > On Wed, Feb 8, 2017 at 8:37 AM, Roger Hoover > wrote: > > > Hi Jun, > > > > How does it allow impersonation at the connection level? Looking at the > > KIP, the DelegationTokenRequest does not have an "Owner" field that can > be > > set. The owner field of the DelegationTokenResponse says it's the > "Kakfa > > Principal which requested the delegation token". For impersonation, > don't > > we need to be able to get tokens for other users besides the one making > the > > request? > > > > Thanks, > > > > Roger > > > > On Tue, Feb 7, 2017 at 6:45 PM, Jun Rao wrote: > > > > > Hi, Roger, > > > > > > Just to clarify. This KIP already allows you to do impersonation at the > > > connection level. Are you talking about impersonation at the request > > level? > > > > > > Thanks, > > > > > > Jun > > > > > > On Tue, Feb 7, 2017 at 5:53 PM, Roger Hoover > > > wrote: > > > > > > > Just wondering...how difficult would be it be to later add > > impersonation > > > ( > > > > https://issues.apache.org/jira/browse/KAFKA-3712)? One use case > would > > > be > > > > a > > > > Kafka admin UI that would take action on the cluster on behalf > > different > > > > users.I suppose we could later add an "effectiveUserId" (in Unix > > > > terminology) to the token details? > > > > > > > > On Tue, Feb 7, 2017 at 5:25 PM, Grant Henke > > wrote: > > > > > > > > > +1 from me as well. > > > > > > > > > > On Tue, Feb 7, 2017 at 7:10 PM, Jason Gustafson < > ja...@confluent.io> > > > > > wrote: > > > > > > > > > > > Looks like a great proposal! I noticed that key rotation is not > > > > included. > > > > > > That may be reasonable for the initial work, but it might be nice > > to > > > > > share > > > > > > some thoughts on how that might work in the future. For example, > I > > > > could > > > > > > imagine delegation.token.master.key could be a list, which would > > > allow > > > > > > users to support both a new and old key at the same time while > > > clients > > > > > are > > > > > > upgrading keys. > > > > > > > > > > > > -Jason > > > > > > > > > > > > On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira > > > > wrote: > > > > > > > > > > > > > Read the KIP again and I think it looks good. > > > > > > > > > > > > > > +1 from me. > > > > > > > > > > > > > > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao > > wrote: > > > > > > > > Hi, Mani, > > > > > > > > > > > > > > > > If a token expires, then every broker will potentially try to > > > > delete > > > > > it > > > > > > > > around the same time, but only one will succeed. So, we will > > have > > > > to > > > > > > deal > > > > > > > > with failures in that case? Another way is to let just one > > broker > > > > > (say, > > > > > > > the > > > > > > > > controller) deletes expired tokens. > > > > > > > > > > > > > > > > It would also be helpful for others to give feedback on this > > KIP. > > > > > > Rajini, > > > > > > > > Gwen, Ismael? > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar < > > > > manikumar.re...@gmail.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > >> Hi Jun, > > > > > > > >> > > > > > > > >> Please see the replies inline. > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > > > >> > > Only one broker does the deletion. Broker updates the > > > > expiration > > > > > > in > > > > > > > its > > > > > > > >> > > local cache > > > > > > > >> > > and on zookeeper so other brokers also get notified and > > > their > > > > > > cache > > > > > > > >> > > statuses are updated as well. > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > Which broker does the deletion? > > > > > > > >> > > > > > > > > >> > > > > > > > >> Any broker can handle the create/expire/renew/describe > > > > > delegationtoken > > > > > > > >> requests. > > > > > > > >> changes are propagated through zk notifications. Every > broker > > > is > > > > > > > >> responsible for > > > > > > > >> expiring the tokens. This check be can done during request > > > > handling > > > > > > time > > > > > > > >> and/or > > > > > > > >> during token authentication time. > > > > > > > >> > > > > > > > >> > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi Jason, As noticed by you, the current proposal does not support rotation of secret. We also discussed about maintaining a list of secret keys. Other option could be using the controller to generate and rotate secret and distribute it to all brokers. I will update the possible alternatives to future work section of KIP. On Wed, Feb 8, 2017 at 6:40 AM, Jason Gustafsonwrote: > Looks like a great proposal! I noticed that key rotation is not included. > That may be reasonable for the initial work, but it might be nice to share > some thoughts on how that might work in the future. For example, I could > imagine delegation.token.master.key could be a list, which would allow > users to support both a new and old key at the same time while clients are > upgrading keys. > > -Jason > > On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira wrote: > > > Read the KIP again and I think it looks good. > > > > +1 from me. > > > > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao wrote: > > > Hi, Mani, > > > > > > If a token expires, then every broker will potentially try to delete it > > > around the same time, but only one will succeed. So, we will have to > deal > > > with failures in that case? Another way is to let just one broker (say, > > the > > > controller) deletes expired tokens. > > > > > > It would also be helpful for others to give feedback on this KIP. > Rajini, > > > Gwen, Ismael? > > > > > > Thanks, > > > > > > Jun > > > > > > > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar > > wrote: > > > > > >> Hi Jun, > > >> > > >> Please see the replies inline. > > >> > > >> > > >> > > > > >> > > Only one broker does the deletion. Broker updates the expiration > in > > its > > >> > > local cache > > >> > > and on zookeeper so other brokers also get notified and their > cache > > >> > > statuses are updated as well. > > >> > > > > >> > > > > >> > Which broker does the deletion? > > >> > > > >> > > >> Any broker can handle the create/expire/renew/describe delegationtoken > > >> requests. > > >> changes are propagated through zk notifications. Every broker is > > >> responsible for > > >> expiring the tokens. This check be can done during request handling > time > > >> and/or > > >> during token authentication time. > > >> > > >> > > >> > > > >> > > > >> > 110. The diagrams in the wiki still show MD5 digest. Could you > change > > it > > >> to > > >> > SCRAM? > > >> > > > >> > > > >> Updated the diagram. > > >> > > >> > > >> > > >> Thanks, > > >> Manikumar > > >> > > >> > > >> > > >> > > >> > > > >> > > > >> > > > > >> > > Thanks. > > >> > > Manikumar > > >> > > > > >> > > > > >> > > > > > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar < > > >> manikumar.re...@gmail.com> > > >> > > > wrote: > > >> > > > > > >> > > > > Hi, > > >> > > > > > > >> > > > > I would like to initiate the vote on KIP-48: > > >> > > > > > > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ > > >> > > > > Delegation+token+support+for+Kafka > > >> > > > > > > >> > > > > > > >> > > > > Thanks, > > >> > > > > Manikumar > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > > > > -- > > Gwen Shapira > > Product Manager | Confluent > > 650.450.2760 | @gwenshap > > Follow us: Twitter | blog > > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi Roger, In the current proposal, we only allow a user to get delegation token for that user only. Anyone who gets that token can impersonate the user on the broker. Yes, In future we can extend the support to allow a user to acquire delegation tokens for other users. Pl refer discuss mail thread for impersonation related discussion. Thanks, Manikumar On Wed, Feb 8, 2017 at 8:37 AM, Roger Hooverwrote: > Hi Jun, > > How does it allow impersonation at the connection level? Looking at the > KIP, the DelegationTokenRequest does not have an "Owner" field that can be > set. The owner field of the DelegationTokenResponse says it's the "Kakfa > Principal which requested the delegation token". For impersonation, don't > we need to be able to get tokens for other users besides the one making the > request? > > Thanks, > > Roger > > On Tue, Feb 7, 2017 at 6:45 PM, Jun Rao wrote: > > > Hi, Roger, > > > > Just to clarify. This KIP already allows you to do impersonation at the > > connection level. Are you talking about impersonation at the request > level? > > > > Thanks, > > > > Jun > > > > On Tue, Feb 7, 2017 at 5:53 PM, Roger Hoover > > wrote: > > > > > Just wondering...how difficult would be it be to later add > impersonation > > ( > > > https://issues.apache.org/jira/browse/KAFKA-3712)? One use case would > > be > > > a > > > Kafka admin UI that would take action on the cluster on behalf > different > > > users.I suppose we could later add an "effectiveUserId" (in Unix > > > terminology) to the token details? > > > > > > On Tue, Feb 7, 2017 at 5:25 PM, Grant Henke > wrote: > > > > > > > +1 from me as well. > > > > > > > > On Tue, Feb 7, 2017 at 7:10 PM, Jason Gustafson > > > > wrote: > > > > > > > > > Looks like a great proposal! I noticed that key rotation is not > > > included. > > > > > That may be reasonable for the initial work, but it might be nice > to > > > > share > > > > > some thoughts on how that might work in the future. For example, I > > > could > > > > > imagine delegation.token.master.key could be a list, which would > > allow > > > > > users to support both a new and old key at the same time while > > clients > > > > are > > > > > upgrading keys. > > > > > > > > > > -Jason > > > > > > > > > > On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira > > > wrote: > > > > > > > > > > > Read the KIP again and I think it looks good. > > > > > > > > > > > > +1 from me. > > > > > > > > > > > > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao > wrote: > > > > > > > Hi, Mani, > > > > > > > > > > > > > > If a token expires, then every broker will potentially try to > > > delete > > > > it > > > > > > > around the same time, but only one will succeed. So, we will > have > > > to > > > > > deal > > > > > > > with failures in that case? Another way is to let just one > broker > > > > (say, > > > > > > the > > > > > > > controller) deletes expired tokens. > > > > > > > > > > > > > > It would also be helpful for others to give feedback on this > KIP. > > > > > Rajini, > > > > > > > Gwen, Ismael? > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar < > > > manikumar.re...@gmail.com > > > > > > > > > > > wrote: > > > > > > > > > > > > > >> Hi Jun, > > > > > > >> > > > > > > >> Please see the replies inline. > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > > > >> > > Only one broker does the deletion. Broker updates the > > > expiration > > > > > in > > > > > > its > > > > > > >> > > local cache > > > > > > >> > > and on zookeeper so other brokers also get notified and > > their > > > > > cache > > > > > > >> > > statuses are updated as well. > > > > > > >> > > > > > > > > >> > > > > > > > > >> > Which broker does the deletion? > > > > > > >> > > > > > > > >> > > > > > > >> Any broker can handle the create/expire/renew/describe > > > > delegationtoken > > > > > > >> requests. > > > > > > >> changes are propagated through zk notifications. Every broker > > is > > > > > > >> responsible for > > > > > > >> expiring the tokens. This check be can done during request > > > handling > > > > > time > > > > > > >> and/or > > > > > > >> during token authentication time. > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > > >> > > > > > > > >> > 110. The diagrams in the wiki still show MD5 digest. Could > you > > > > > change > > > > > > it > > > > > > >> to > > > > > > >> > SCRAM? > > > > > > >> > > > > > > > >> > > > > > > > >> Updated the diagram. > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> Thanks, > > > > > > >> Manikumar > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > Thanks. > > > > > > >> > > Manikumar > > > > > > >> > > > > > > > > >> > > > > > > > > >> > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi Jun, > If a token expires, then every broker will potentially try to delete it > around the same time, but only one will succeed. So, we will have to deal > with failures in that case? Another way is to let just one broker (say, the > controller) deletes expired tokens. > > Agree, we can run the token expiry check thread as part of controller broker. WIll update the KIP. Thanks, Manikumar > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar> wrote: > > > Hi Jun, > > > > Please see the replies inline. > > > > > > > > > > > > Only one broker does the deletion. Broker updates the expiration in > its > > > > local cache > > > > and on zookeeper so other brokers also get notified and their cache > > > > statuses are updated as well. > > > > > > > > > > > Which broker does the deletion? > > > > > > > Any broker can handle the create/expire/renew/describe delegationtoken > > requests. > > changes are propagated through zk notifications. Every broker is > > responsible for > > expiring the tokens. This check be can done during request handling time > > and/or > > during token authentication time. > > > > > > > > > > > > > 110. The diagrams in the wiki still show MD5 digest. Could you change > it > > to > > > SCRAM? > > > > > > > > Updated the diagram. > > > > > > > > Thanks, > > Manikumar > > > > > > > > > > > > > > > > > > > > > > Thanks. > > > > Manikumar > > > > > > > > > > > > > > > > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar < > > manikumar.re...@gmail.com> > > > > > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > I would like to initiate the vote on KIP-48: > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ > > > > > > Delegation+token+support+for+Kafka > > > > > > > > > > > > > > > > > > Thanks, > > > > > > Manikumar > > > > > > > > > > > > > > > > > > > > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi Jun, How does it allow impersonation at the connection level? Looking at the KIP, the DelegationTokenRequest does not have an "Owner" field that can be set. The owner field of the DelegationTokenResponse says it's the "Kakfa Principal which requested the delegation token". For impersonation, don't we need to be able to get tokens for other users besides the one making the request? Thanks, Roger On Tue, Feb 7, 2017 at 6:45 PM, Jun Raowrote: > Hi, Roger, > > Just to clarify. This KIP already allows you to do impersonation at the > connection level. Are you talking about impersonation at the request level? > > Thanks, > > Jun > > On Tue, Feb 7, 2017 at 5:53 PM, Roger Hoover > wrote: > > > Just wondering...how difficult would be it be to later add impersonation > ( > > https://issues.apache.org/jira/browse/KAFKA-3712)? One use case would > be > > a > > Kafka admin UI that would take action on the cluster on behalf different > > users.I suppose we could later add an "effectiveUserId" (in Unix > > terminology) to the token details? > > > > On Tue, Feb 7, 2017 at 5:25 PM, Grant Henke wrote: > > > > > +1 from me as well. > > > > > > On Tue, Feb 7, 2017 at 7:10 PM, Jason Gustafson > > > wrote: > > > > > > > Looks like a great proposal! I noticed that key rotation is not > > included. > > > > That may be reasonable for the initial work, but it might be nice to > > > share > > > > some thoughts on how that might work in the future. For example, I > > could > > > > imagine delegation.token.master.key could be a list, which would > allow > > > > users to support both a new and old key at the same time while > clients > > > are > > > > upgrading keys. > > > > > > > > -Jason > > > > > > > > On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira > > wrote: > > > > > > > > > Read the KIP again and I think it looks good. > > > > > > > > > > +1 from me. > > > > > > > > > > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao wrote: > > > > > > Hi, Mani, > > > > > > > > > > > > If a token expires, then every broker will potentially try to > > delete > > > it > > > > > > around the same time, but only one will succeed. So, we will have > > to > > > > deal > > > > > > with failures in that case? Another way is to let just one broker > > > (say, > > > > > the > > > > > > controller) deletes expired tokens. > > > > > > > > > > > > It would also be helpful for others to give feedback on this KIP. > > > > Rajini, > > > > > > Gwen, Ismael? > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar < > > manikumar.re...@gmail.com > > > > > > > > > wrote: > > > > > > > > > > > >> Hi Jun, > > > > > >> > > > > > >> Please see the replies inline. > > > > > >> > > > > > >> > > > > > >> > > > > > > > >> > > Only one broker does the deletion. Broker updates the > > expiration > > > > in > > > > > its > > > > > >> > > local cache > > > > > >> > > and on zookeeper so other brokers also get notified and > their > > > > cache > > > > > >> > > statuses are updated as well. > > > > > >> > > > > > > > >> > > > > > > > >> > Which broker does the deletion? > > > > > >> > > > > > > >> > > > > > >> Any broker can handle the create/expire/renew/describe > > > delegationtoken > > > > > >> requests. > > > > > >> changes are propagated through zk notifications. Every broker > is > > > > > >> responsible for > > > > > >> expiring the tokens. This check be can done during request > > handling > > > > time > > > > > >> and/or > > > > > >> during token authentication time. > > > > > >> > > > > > >> > > > > > >> > > > > > > >> > > > > > > >> > 110. The diagrams in the wiki still show MD5 digest. Could you > > > > change > > > > > it > > > > > >> to > > > > > >> > SCRAM? > > > > > >> > > > > > > >> > > > > > > >> Updated the diagram. > > > > > >> > > > > > >> > > > > > >> > > > > > >> Thanks, > > > > > >> Manikumar > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > > >> > > > > > > >> > > > > > > > >> > > Thanks. > > > > > >> > > Manikumar > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar < > > > > > >> manikumar.re...@gmail.com> > > > > > >> > > > wrote: > > > > > >> > > > > > > > > >> > > > > Hi, > > > > > >> > > > > > > > > > >> > > > > I would like to initiate the vote on KIP-48: > > > > > >> > > > > > > > > > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 48+ > > > > > >> > > > > Delegation+token+support+for+Kafka > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > Thanks, > > > > > >> > > > > Manikumar > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > -- > > > > > Gwen Shapira > > > > > Product Manager | Confluent > > > > > 650.450.2760 | @gwenshap > > > > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi, Roger, Just to clarify. This KIP already allows you to do impersonation at the connection level. Are you talking about impersonation at the request level? Thanks, Jun On Tue, Feb 7, 2017 at 5:53 PM, Roger Hooverwrote: > Just wondering...how difficult would be it be to later add impersonation ( > https://issues.apache.org/jira/browse/KAFKA-3712)? One use case would be > a > Kafka admin UI that would take action on the cluster on behalf different > users.I suppose we could later add an "effectiveUserId" (in Unix > terminology) to the token details? > > On Tue, Feb 7, 2017 at 5:25 PM, Grant Henke wrote: > > > +1 from me as well. > > > > On Tue, Feb 7, 2017 at 7:10 PM, Jason Gustafson > > wrote: > > > > > Looks like a great proposal! I noticed that key rotation is not > included. > > > That may be reasonable for the initial work, but it might be nice to > > share > > > some thoughts on how that might work in the future. For example, I > could > > > imagine delegation.token.master.key could be a list, which would allow > > > users to support both a new and old key at the same time while clients > > are > > > upgrading keys. > > > > > > -Jason > > > > > > On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira > wrote: > > > > > > > Read the KIP again and I think it looks good. > > > > > > > > +1 from me. > > > > > > > > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao wrote: > > > > > Hi, Mani, > > > > > > > > > > If a token expires, then every broker will potentially try to > delete > > it > > > > > around the same time, but only one will succeed. So, we will have > to > > > deal > > > > > with failures in that case? Another way is to let just one broker > > (say, > > > > the > > > > > controller) deletes expired tokens. > > > > > > > > > > It would also be helpful for others to give feedback on this KIP. > > > Rajini, > > > > > Gwen, Ismael? > > > > > > > > > > Thanks, > > > > > > > > > > Jun > > > > > > > > > > > > > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar < > manikumar.re...@gmail.com > > > > > > > wrote: > > > > > > > > > >> Hi Jun, > > > > >> > > > > >> Please see the replies inline. > > > > >> > > > > >> > > > > >> > > > > > > >> > > Only one broker does the deletion. Broker updates the > expiration > > > in > > > > its > > > > >> > > local cache > > > > >> > > and on zookeeper so other brokers also get notified and their > > > cache > > > > >> > > statuses are updated as well. > > > > >> > > > > > > >> > > > > > > >> > Which broker does the deletion? > > > > >> > > > > > >> > > > > >> Any broker can handle the create/expire/renew/describe > > delegationtoken > > > > >> requests. > > > > >> changes are propagated through zk notifications. Every broker is > > > > >> responsible for > > > > >> expiring the tokens. This check be can done during request > handling > > > time > > > > >> and/or > > > > >> during token authentication time. > > > > >> > > > > >> > > > > >> > > > > > >> > > > > > >> > 110. The diagrams in the wiki still show MD5 digest. Could you > > > change > > > > it > > > > >> to > > > > >> > SCRAM? > > > > >> > > > > > >> > > > > > >> Updated the diagram. > > > > >> > > > > >> > > > > >> > > > > >> Thanks, > > > > >> Manikumar > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > > >> > > > > > >> > > > > > > >> > > Thanks. > > > > >> > > Manikumar > > > > >> > > > > > > >> > > > > > > >> > > > > > > > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar < > > > > >> manikumar.re...@gmail.com> > > > > >> > > > wrote: > > > > >> > > > > > > > >> > > > > Hi, > > > > >> > > > > > > > > >> > > > > I would like to initiate the vote on KIP-48: > > > > >> > > > > > > > > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ > > > > >> > > > > Delegation+token+support+for+Kafka > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > Thanks, > > > > >> > > > > Manikumar > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > -- > > > > Gwen Shapira > > > > Product Manager | Confluent > > > > 650.450.2760 | @gwenshap > > > > Follow us: Twitter | blog > > > > > > > > > > > > > > > -- > > Grant Henke > > Software Engineer | Cloudera > > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke > > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Just wondering...how difficult would be it be to later add impersonation ( https://issues.apache.org/jira/browse/KAFKA-3712)? One use case would be a Kafka admin UI that would take action on the cluster on behalf different users.I suppose we could later add an "effectiveUserId" (in Unix terminology) to the token details? On Tue, Feb 7, 2017 at 5:25 PM, Grant Henkewrote: > +1 from me as well. > > On Tue, Feb 7, 2017 at 7:10 PM, Jason Gustafson > wrote: > > > Looks like a great proposal! I noticed that key rotation is not included. > > That may be reasonable for the initial work, but it might be nice to > share > > some thoughts on how that might work in the future. For example, I could > > imagine delegation.token.master.key could be a list, which would allow > > users to support both a new and old key at the same time while clients > are > > upgrading keys. > > > > -Jason > > > > On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira wrote: > > > > > Read the KIP again and I think it looks good. > > > > > > +1 from me. > > > > > > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao wrote: > > > > Hi, Mani, > > > > > > > > If a token expires, then every broker will potentially try to delete > it > > > > around the same time, but only one will succeed. So, we will have to > > deal > > > > with failures in that case? Another way is to let just one broker > (say, > > > the > > > > controller) deletes expired tokens. > > > > > > > > It would also be helpful for others to give feedback on this KIP. > > Rajini, > > > > Gwen, Ismael? > > > > > > > > Thanks, > > > > > > > > Jun > > > > > > > > > > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar > > > > wrote: > > > > > > > >> Hi Jun, > > > >> > > > >> Please see the replies inline. > > > >> > > > >> > > > >> > > > > > >> > > Only one broker does the deletion. Broker updates the expiration > > in > > > its > > > >> > > local cache > > > >> > > and on zookeeper so other brokers also get notified and their > > cache > > > >> > > statuses are updated as well. > > > >> > > > > > >> > > > > > >> > Which broker does the deletion? > > > >> > > > > >> > > > >> Any broker can handle the create/expire/renew/describe > delegationtoken > > > >> requests. > > > >> changes are propagated through zk notifications. Every broker is > > > >> responsible for > > > >> expiring the tokens. This check be can done during request handling > > time > > > >> and/or > > > >> during token authentication time. > > > >> > > > >> > > > >> > > > > >> > > > > >> > 110. The diagrams in the wiki still show MD5 digest. Could you > > change > > > it > > > >> to > > > >> > SCRAM? > > > >> > > > > >> > > > > >> Updated the diagram. > > > >> > > > >> > > > >> > > > >> Thanks, > > > >> Manikumar > > > >> > > > >> > > > >> > > > >> > > > >> > > > > >> > > > > >> > > > > > >> > > Thanks. > > > >> > > Manikumar > > > >> > > > > > >> > > > > > >> > > > > > > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar < > > > >> manikumar.re...@gmail.com> > > > >> > > > wrote: > > > >> > > > > > > >> > > > > Hi, > > > >> > > > > > > > >> > > > > I would like to initiate the vote on KIP-48: > > > >> > > > > > > > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ > > > >> > > > > Delegation+token+support+for+Kafka > > > >> > > > > > > > >> > > > > > > > >> > > > > Thanks, > > > >> > > > > Manikumar > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > > > > > > > -- > > > Gwen Shapira > > > Product Manager | Confluent > > > 650.450.2760 | @gwenshap > > > Follow us: Twitter | blog > > > > > > > > > -- > Grant Henke > Software Engineer | Cloudera > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
+1 from me as well. On Tue, Feb 7, 2017 at 7:10 PM, Jason Gustafsonwrote: > Looks like a great proposal! I noticed that key rotation is not included. > That may be reasonable for the initial work, but it might be nice to share > some thoughts on how that might work in the future. For example, I could > imagine delegation.token.master.key could be a list, which would allow > users to support both a new and old key at the same time while clients are > upgrading keys. > > -Jason > > On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapira wrote: > > > Read the KIP again and I think it looks good. > > > > +1 from me. > > > > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao wrote: > > > Hi, Mani, > > > > > > If a token expires, then every broker will potentially try to delete it > > > around the same time, but only one will succeed. So, we will have to > deal > > > with failures in that case? Another way is to let just one broker (say, > > the > > > controller) deletes expired tokens. > > > > > > It would also be helpful for others to give feedback on this KIP. > Rajini, > > > Gwen, Ismael? > > > > > > Thanks, > > > > > > Jun > > > > > > > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar > > wrote: > > > > > >> Hi Jun, > > >> > > >> Please see the replies inline. > > >> > > >> > > >> > > > > >> > > Only one broker does the deletion. Broker updates the expiration > in > > its > > >> > > local cache > > >> > > and on zookeeper so other brokers also get notified and their > cache > > >> > > statuses are updated as well. > > >> > > > > >> > > > > >> > Which broker does the deletion? > > >> > > > >> > > >> Any broker can handle the create/expire/renew/describe delegationtoken > > >> requests. > > >> changes are propagated through zk notifications. Every broker is > > >> responsible for > > >> expiring the tokens. This check be can done during request handling > time > > >> and/or > > >> during token authentication time. > > >> > > >> > > >> > > > >> > > > >> > 110. The diagrams in the wiki still show MD5 digest. Could you > change > > it > > >> to > > >> > SCRAM? > > >> > > > >> > > > >> Updated the diagram. > > >> > > >> > > >> > > >> Thanks, > > >> Manikumar > > >> > > >> > > >> > > >> > > >> > > > >> > > > >> > > > > >> > > Thanks. > > >> > > Manikumar > > >> > > > > >> > > > > >> > > > > > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar < > > >> manikumar.re...@gmail.com> > > >> > > > wrote: > > >> > > > > > >> > > > > Hi, > > >> > > > > > > >> > > > > I would like to initiate the vote on KIP-48: > > >> > > > > > > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ > > >> > > > > Delegation+token+support+for+Kafka > > >> > > > > > > >> > > > > > > >> > > > > Thanks, > > >> > > > > Manikumar > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > > > > -- > > Gwen Shapira > > Product Manager | Confluent > > 650.450.2760 | @gwenshap > > Follow us: Twitter | blog > > > -- Grant Henke Software Engineer | Cloudera gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Looks like a great proposal! I noticed that key rotation is not included. That may be reasonable for the initial work, but it might be nice to share some thoughts on how that might work in the future. For example, I could imagine delegation.token.master.key could be a list, which would allow users to support both a new and old key at the same time while clients are upgrading keys. -Jason On Tue, Feb 7, 2017 at 4:42 PM, Gwen Shapirawrote: > Read the KIP again and I think it looks good. > > +1 from me. > > On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao wrote: > > Hi, Mani, > > > > If a token expires, then every broker will potentially try to delete it > > around the same time, but only one will succeed. So, we will have to deal > > with failures in that case? Another way is to let just one broker (say, > the > > controller) deletes expired tokens. > > > > It would also be helpful for others to give feedback on this KIP. Rajini, > > Gwen, Ismael? > > > > Thanks, > > > > Jun > > > > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar > wrote: > > > >> Hi Jun, > >> > >> Please see the replies inline. > >> > >> > >> > > > >> > > Only one broker does the deletion. Broker updates the expiration in > its > >> > > local cache > >> > > and on zookeeper so other brokers also get notified and their cache > >> > > statuses are updated as well. > >> > > > >> > > > >> > Which broker does the deletion? > >> > > >> > >> Any broker can handle the create/expire/renew/describe delegationtoken > >> requests. > >> changes are propagated through zk notifications. Every broker is > >> responsible for > >> expiring the tokens. This check be can done during request handling time > >> and/or > >> during token authentication time. > >> > >> > >> > > >> > > >> > 110. The diagrams in the wiki still show MD5 digest. Could you change > it > >> to > >> > SCRAM? > >> > > >> > > >> Updated the diagram. > >> > >> > >> > >> Thanks, > >> Manikumar > >> > >> > >> > >> > >> > > >> > > >> > > > >> > > Thanks. > >> > > Manikumar > >> > > > >> > > > >> > > > > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar < > >> manikumar.re...@gmail.com> > >> > > > wrote: > >> > > > > >> > > > > Hi, > >> > > > > > >> > > > > I would like to initiate the vote on KIP-48: > >> > > > > > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ > >> > > > > Delegation+token+support+for+Kafka > >> > > > > > >> > > > > > >> > > > > Thanks, > >> > > > > Manikumar > >> > > > > > >> > > > > >> > > > >> > > >> > > > > -- > Gwen Shapira > Product Manager | Confluent > 650.450.2760 | @gwenshap > Follow us: Twitter | blog >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Read the KIP again and I think it looks good. +1 from me. On Tue, Feb 7, 2017 at 3:05 PM, Jun Raowrote: > Hi, Mani, > > If a token expires, then every broker will potentially try to delete it > around the same time, but only one will succeed. So, we will have to deal > with failures in that case? Another way is to let just one broker (say, the > controller) deletes expired tokens. > > It would also be helpful for others to give feedback on this KIP. Rajini, > Gwen, Ismael? > > Thanks, > > Jun > > > On Sun, Feb 5, 2017 at 9:54 AM, Manikumar wrote: > >> Hi Jun, >> >> Please see the replies inline. >> >> >> > > >> > > Only one broker does the deletion. Broker updates the expiration in its >> > > local cache >> > > and on zookeeper so other brokers also get notified and their cache >> > > statuses are updated as well. >> > > >> > > >> > Which broker does the deletion? >> > >> >> Any broker can handle the create/expire/renew/describe delegationtoken >> requests. >> changes are propagated through zk notifications. Every broker is >> responsible for >> expiring the tokens. This check be can done during request handling time >> and/or >> during token authentication time. >> >> >> > >> > >> > 110. The diagrams in the wiki still show MD5 digest. Could you change it >> to >> > SCRAM? >> > >> > >> Updated the diagram. >> >> >> >> Thanks, >> Manikumar >> >> >> >> >> > >> > >> > > >> > > Thanks. >> > > Manikumar >> > > >> > > >> > > > >> > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar < >> manikumar.re...@gmail.com> >> > > > wrote: >> > > > >> > > > > Hi, >> > > > > >> > > > > I would like to initiate the vote on KIP-48: >> > > > > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ >> > > > > Delegation+token+support+for+Kafka >> > > > > >> > > > > >> > > > > Thanks, >> > > > > Manikumar >> > > > > >> > > > >> > > >> > >> -- Gwen Shapira Product Manager | Confluent 650.450.2760 | @gwenshap Follow us: Twitter | blog
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi, Mani, If a token expires, then every broker will potentially try to delete it around the same time, but only one will succeed. So, we will have to deal with failures in that case? Another way is to let just one broker (say, the controller) deletes expired tokens. It would also be helpful for others to give feedback on this KIP. Rajini, Gwen, Ismael? Thanks, Jun On Sun, Feb 5, 2017 at 9:54 AM, Manikumarwrote: > Hi Jun, > > Please see the replies inline. > > > > > > > > Only one broker does the deletion. Broker updates the expiration in its > > > local cache > > > and on zookeeper so other brokers also get notified and their cache > > > statuses are updated as well. > > > > > > > > Which broker does the deletion? > > > > Any broker can handle the create/expire/renew/describe delegationtoken > requests. > changes are propagated through zk notifications. Every broker is > responsible for > expiring the tokens. This check be can done during request handling time > and/or > during token authentication time. > > > > > > > > 110. The diagrams in the wiki still show MD5 digest. Could you change it > to > > SCRAM? > > > > > Updated the diagram. > > > > Thanks, > Manikumar > > > > > > > > > > > > > > Thanks. > > > Manikumar > > > > > > > > > > > > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar < > manikumar.re...@gmail.com> > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > I would like to initiate the vote on KIP-48: > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ > > > > > Delegation+token+support+for+Kafka > > > > > > > > > > > > > > > Thanks, > > > > > Manikumar > > > > > > > > > > > > > > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi Jun, Please see the replies inline. > > > > Only one broker does the deletion. Broker updates the expiration in its > > local cache > > and on zookeeper so other brokers also get notified and their cache > > statuses are updated as well. > > > > > Which broker does the deletion? > Any broker can handle the create/expire/renew/describe delegationtoken requests. changes are propagated through zk notifications. Every broker is responsible for expiring the tokens. This check be can done during request handling time and/or during token authentication time. > > > 110. The diagrams in the wiki still show MD5 digest. Could you change it to > SCRAM? > > Updated the diagram. Thanks, Manikumar > > > > > > Thanks. > > Manikumar > > > > > > > > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar> > > wrote: > > > > > > > Hi, > > > > > > > > I would like to initiate the vote on KIP-48: > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ > > > > Delegation+token+support+for+Kafka > > > > > > > > > > > > Thanks, > > > > Manikumar > > > > > > > > > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi, Mani, Sorry for the late response. A couple of more comments below. > 107.4 How is token deletion handled? Does every broker delete the same > > token independently or only one broker does the deletion? > > > > Only one broker does the deletion. Broker updates the expiration in its > local cache > and on zookeeper so other brokers also get notified and their cache > statuses are updated as well. > > Which broker does the deletion? 110. The diagrams in the wiki still show MD5 digest. Could you change it to SCRAM? Thanks, Jun > > Thanks. > Manikumar > > > > > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar> > wrote: > > > > > Hi, > > > > > > I would like to initiate the vote on KIP-48: > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ > > > Delegation+token+support+for+Kafka > > > > > > > > > Thanks, > > > Manikumar > > > > > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi Jun, Thanks for the review. Sorry for the delayed response. Please see the replies inline. > 101. DelegationTokenRequest/DelegationTokenResponse > 101.1 I am wondering if renewer should be principalType + name to match > what we have in kafka-acls tool? > Yes, renewer should be principalType + name. Will Update the KIP. > 101.2 Do we need to return owner in DelegationTokenResponse? > 101.3 Do we need to return renewer in DelegationTokenResponse? > > The idea is to return complete token details. If there is no use, then we can omit them. @Ashish Since you are implementing this part of the KIP, Pl reply if you have any concerns. > 102. RenewDelegationTokenRequest/RenewDelegationTokenResponse > 102.1 In RenewDelegationTokenRequest, is the renewal time bounded > by delegation.token.expiry.time.sec ? > renewal time is bounded by delegation.token.max.lifetime.ms. delegation.token.expiry.time.sec is used as default renew period. It will be used to compute expiryDate for first time and used if renew period is not provided in RenewDelegationTokenRequest token expiryDate = min(currentTime + renewPeriod, maxDate) 102.2 Do we need to include TokenDetails in RenewDelegationTokenResponse? > > After renewal we are only updating token expiry time. So we may not need to return complete TokenDetails. Will Update the KIP. > 103. Do we need a DescribeDelegationTokenRequest and a corresponding tool > to list/describe existing delegation tokens? > > Yes, we can add token describe request and tool. will update the KIP > 104. KafkaConfig: Do we need a new config to disable/enable delegation > tokens? > > Currently we need masterKey (delegation.token.master.key config property) to generate delegation tokens. This masterKey needs to be configured with all the brokers. If this config property is not set, then brokers will disable the delegation token support. 105. Could we be more consistent with the naming in configs, the request > protocol fields and ZK structure with respect to time > (e.g. delegation.token.max.lifetime.sec => delegation.token.max.lifetime. > ms > )? > > All the time related configs/fileds changed to ms. > 106. Using SASL SCRAM for delegation token > 106.1 What should be set for username? Does it matter? > We will provide "tokenID(sequence number)" as username. This will uniquely identifies the token. > 106.2 Could you provide a bit more details on how to distinguish it from > regular SASL SCRAM? > > High level approach is given below: On Client Side: 1. add a config property "tokenauth=true" to jass config. ScramLoginModule loads this property if exists. 2. add this property to optional extension attributes of Scram ClientFirstMessage. On Server Side: 1. Update CredentialProvider class to handle tokenCredentialCache & add TokenCredentialCallback 2. read the ClientFirstMessage if tokenauth=true exists then do token validations get credentials from tokenCredentialCache else get credentials from regular CredentialCache 3. Continue Scram handshake > 107. ZK structure > 107.1 Could you provide more details on what SCRAM credentials are stored > in ZK path /tokenauth/tokens/? > We will store similar to regular scram credentials. These will get generated during token creation. Here we will use password=HMAC, iterations=4096 to generate credentials. { "version":1, "owner" : "owner", "renewer" : "renewer", "issueDate" : "issueDate", "tokenID" : "sequence-number", //Store SCRAM credentials as per KIP-84 "credentials": { "SCRAM-SHA-512" : "salt=,stored_key=,server_key=,iterations=", "SCRAM-SHA-256" : "salt=,stored_key=,server_key=,iterations= } }; 107.2 Do we need to somehow normalize tokeUID before using it > in /tokenauth/tokens/ (e.g., what happens if tokenUUID includes > '/')? > We will use a a sequence number as tokenUID. Will update the terminology tokenUID => tokenID. > 107.3 How does each broker know about changes in tokens? Does each broker > register a watcher for /tokenauth/tokens? > Yes, the changes are updated through zk notifications. > 107.4 How is token deletion handled? Does every broker delete the same > token independently or only one broker does the deletion? > Only one broker does the deletion. Broker updates the expiration in its local cache and on zookeeper so other brokers also get notified and their cache statuses are updated as well. Thanks. Manikumar > > On Fri, Dec 23, 2016 at 9:26 AM, Manikumar> wrote: > > > Hi, > > > > I would like to initiate the vote on KIP-48: > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ > > Delegation+token+support+for+Kafka > > > > > > Thanks, > > Manikumar > > >
Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi, Mani Thanks for the proposal. Looks good at the high level. A few comments below. 101. DelegationTokenRequest/DelegationTokenResponse 101.1 I am wondering if renewer should be principalType + name to match what we have in kafka-acls tool? 101.2 Do we need to return owner in DelegationTokenResponse? 101.3 Do we need to return renewer in DelegationTokenResponse? 102. RenewDelegationTokenRequest/RenewDelegationTokenResponse 102.1 In RenewDelegationTokenRequest, is the renewal time bounded by delegation.token.expiry.time.sec ? 102.2 Do we need to include TokenDetails in RenewDelegationTokenResponse? 103. Do we need a DescribeDelegationTokenRequest and a corresponding tool to list/describe existing delegation tokens? 104. KafkaConfig: Do we need a new config to disable/enable delegation tokens? 105. Could we be more consistent with the naming in configs, the request protocol fields and ZK structure with respect to time (e.g. delegation.token.max.lifetime.sec => delegation.token.max.lifetime.ms )? 106. Using SASL SCRAM for delegation token 106.1 What should be set for username? Does it matter? 106.2 Could you provide a bit more details on how to distinguish it from regular SASL SCRAM? 107. ZK structure 107.1 Could you provide more details on what SCRAM credentials are stored in ZK path /tokenauth/tokens/? 107.2 Do we need to somehow normalize tokeUID before using it in /tokenauth/tokens/ (e.g., what happens if tokenUUID includes '/')? 107.3 How does each broker know about changes in tokens? Does each broker register a watcher for /tokenauth/tokens? 107.4 How is token deletion handled? Does every broker delete the same token independently or only one broker does the deletion? Thanks, Jun On Fri, Dec 23, 2016 at 9:26 AM, Manikumarwrote: > Hi, > > I would like to initiate the vote on KIP-48: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ > Delegation+token+support+for+Kafka > > > Thanks, > Manikumar >
[VOTE] KIP-48 Support for delegation tokens as an authentication mechanism
Hi, I would like to initiate the vote on KIP-48: https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+ Delegation+token+support+for+Kafka Thanks, Manikumar