>From what I can see, remaining questions are: 1. Who / how are tokens renewed? By original requester only? or using Kerberos auth only? 2. Are tokens stored on each broker or in ZK? 3. How are tokens invalidated / expired? 4. Which encryption algorithm is used? 5. What is the impersonation proposal (it wasn't in the KIP but was discussed in this thread)? 6. Do we need new ACLs, if so - for what actions?
Gwen On Thu, Jun 9, 2016 at 7:48 PM, Harsha <ka...@harsha.io> wrote: > Jun & Ismael, > Unfortunately I couldn't attend the KIP meeting > when delegation tokens discussed. Appreciate if > you can update the thread if you have any > further questions. > Thanks, > Harsha > > On Tue, May 24, 2016, at 11:32 AM, Liquan Pei wrote: >> It seems that the links to images in the KIP are broken. >> >> Liquan >> >> On Tue, May 24, 2016 at 9:33 AM, parth brahmbhatt < >> brahmbhatt.pa...@gmail.com> wrote: >> >> > 110. What does getDelegationTokenAs mean? >> > In the current proposal we only allow a user to get delegation token for >> > the identity that it authenticated as using another mechanism, i.e. A user >> > that authenticate using a keytab for principal us...@example.com will get >> > delegation tokens for that user only. In future I think we will have to >> > extend support such that we allow some set of users ( >> > kafka-rest-u...@example.com, storm-nim...@example.com) to acquire >> > delegation tokens on behalf of other users whose identity they have >> > verified independently. Kafka brokers will have ACLs to control which >> > users are allowed to impersonate other users and get tokens on behalf of >> > them. Overall Impersonation is a whole different problem in my opinion and >> > I think we can tackle it in separate KIP. >> > >> > 111. What's the typical rate of getting and renewing delegation tokens? >> > Typically this should be very very low, 1 request per minute is a >> > relatively high estimate. However it depends on the token expiration. I am >> > less worried about the extra load it puts on controller vs the added >> > complexity and the value it offers. >> > >> > Thanks >> > Parth >> > >> > >> > >> > On Tue, May 24, 2016 at 7:30 AM, Ismael Juma <ism...@juma.me.uk> wrote: >> > >> > > Thanks Rajini. It would probably require a separate KIP as it will >> > > introduce user visible changes. We could also update KIP-48 to have this >> > > information, but it seems cleaner to do it separately. We can discuss >> > that >> > > in the KIP call today. >> > > >> > > Ismael >> > > >> > > On Tue, May 24, 2016 at 3:19 PM, Rajini Sivaram < >> > > rajinisiva...@googlemail.com> wrote: >> > > >> > > > Ismael, >> > > > >> > > > I have created a JIRA ( >> > https://issues.apache.org/jira/browse/KAFKA-3751) >> > > > for adding SCRAM as a SASL mechanism. Would that need another KIP? If >> > > > KIP-48 will use this mechanism, can this just be a JIRA that gets >> > > reviewed >> > > > when the PR is ready? >> > > > >> > > > Thank you, >> > > > >> > > > Rajini >> > > > >> > > > On Tue, May 24, 2016 at 2:46 PM, Ismael Juma <ism...@juma.me.uk> >> > wrote: >> > > > >> > > > > Thanks Rajini, SCRAM seems like a good candidate. >> > > > > >> > > > > Gwen had independently mentioned this as a SASL mechanism that might >> > be >> > > > > useful for Kafka and I have been meaning to check it in more detail. >> > > Good >> > > > > to know that you are willing to contribute an implementation. Maybe >> > we >> > > > > should file a separate JIRA for this? >> > > > > >> > > > > Ismael >> > > > > >> > > > > On Tue, May 24, 2016 at 2:12 PM, Rajini Sivaram < >> > > > > rajinisiva...@googlemail.com> wrote: >> > > > > >> > > > > > SCRAM (Salted Challenge Response Authentication Mechanism) is a >> > > better >> > > > > > mechanism than Digest-MD5. Java doesn't come with a built-in SCRAM >> > > > > > SaslServer or SaslClient, but I will be happy to add support in >> > Kafka >> > > > > since >> > > > > > it would be a useful mechanism to support anyway. >> > > > > > https://tools.ietf.org/html/rfc7677 describes the protocol for >> > > > > > SCRAM-SHA-256. >> > > > > > >> > > > > > On Tue, May 24, 2016 at 2:37 AM, Jun Rao <j...@confluent.io> wrote: >> > > > > > >> > > > > > > Parth, >> > > > > > > >> > > > > > > Thanks for the explanation. A couple of more questions. >> > > > > > > >> > > > > > > 110. What does getDelegationTokenAs mean? >> > > > > > > >> > > > > > > 111. What's the typical rate of getting and renewing delegation >> > > > tokens? >> > > > > > > That may have an impact on whether they should be directed to the >> > > > > > > controller. >> > > > > > > >> > > > > > > Jun >> > > > > > > >> > > > > > > On Mon, May 23, 2016 at 1:19 PM, parth brahmbhatt < >> > > > > > > brahmbhatt.pa...@gmail.com> wrote: >> > > > > > > >> > > > > > > > Hi Jun, >> > > > > > > > >> > > > > > > > Thanks for reviewing. >> > > > > > > > >> > > > > > > > * We could add a Cluster action to add acls on who can request >> > > > > > delegation >> > > > > > > > tokens. I don't see the use case for that yet but down the line >> > > > when >> > > > > we >> > > > > > > > start supporting getDelegationTokenAs it will be necessary. >> > > > > > > > * Yes we recommend tokens to be only used/distributed over >> > secure >> > > > > > > channels. >> > > > > > > > * Depending on what design we end up choosing Invalidation will >> > > be >> > > > > > > > responsibility of every broker or controller. >> > > > > > > > * I am not sure if I documented somewhere that invalidation >> > will >> > > > > > directly >> > > > > > > > go through zookeeper but that is not the intent. Invalidation >> > > will >> > > > > > either >> > > > > > > > be request based or due to expiration. No direct zookeeper >> > > > > interaction >> > > > > > > from >> > > > > > > > any client. >> > > > > > > > * "Broker also stores the DelegationToken without the hmac in >> > the >> > > > > > > > zookeeper." : Sorry about the confusion. The sole purpose of >> > > > > zookeeper >> > > > > > in >> > > > > > > > this design is as distribution channel for tokens between all >> > > > brokers >> > > > > > > and a >> > > > > > > > layer that ensures only tokens that were generated by making a >> > > > > request >> > > > > > > to a >> > > > > > > > broker will be accepted (more on this in second paragraph). The >> > > > token >> > > > > > > > consists of few elements (owner, renewer, uuid , expiration, >> > > hmac) >> > > > , >> > > > > > one >> > > > > > > of >> > > > > > > > which is the finally generated hmac but hmac it self is >> > derivable >> > > > if >> > > > > > you >> > > > > > > > have all the other elements of the token + secret key to >> > generate >> > > > > hmac. >> > > > > > > > Given zookeeper does not provide SSL support we do not want the >> > > > > entire >> > > > > > > > token to be wire transferred to zookeeper as that will be an >> > > > insecure >> > > > > > > wire >> > > > > > > > transfer. Instead we only store all the other elements of a >> > > > > delegation >> > > > > > > > tokens. Brokers can read these elements and because they also >> > > have >> > > > > > access >> > > > > > > > to secret key they will be able to generate hmac on their end. >> > > > > > > > >> > > > > > > > One of the alternative proposed is to avoid zookeeper >> > > altogether. A >> > > > > > > Client >> > > > > > > > will call broker with required information (owner, renwer, >> > > > > expiration) >> > > > > > > and >> > > > > > > > get back (signed hmac, uuid). Broker won't store this in >> > > zookeeper. >> > > > > > From >> > > > > > > > this point a client can contact any broker with all the >> > > delegation >> > > > > > token >> > > > > > > > info (owner, rewner, expiration, hmac, uuid) the borker will >> > > > > regenerate >> > > > > > > the >> > > > > > > > hmac and as long as it matches with hmac presented by client , >> > > > broker >> > > > > > > will >> > > > > > > > allow the request to authenticate. Only problem with this >> > > approach >> > > > > is >> > > > > > if >> > > > > > > > the secret key is compromised any client can now generate >> > random >> > > > > tokens >> > > > > > > and >> > > > > > > > they will still be able to authenticate as any user they like. >> > > with >> > > > > > > > zookeeper we guarantee that only tokens acquired via a broker >> > > > (using >> > > > > > some >> > > > > > > > auth scheme other than delegation token) will be accepted. We >> > > need >> > > > to >> > > > > > > > discuss which proposal makes more sense and we can go over it >> > in >> > > > > > > tomorrow's >> > > > > > > > meeting. >> > > > > > > > >> > > > > > > > Also, can you forward the invite to me? >> > > > > > > > >> > > > > > > > Thanks >> > > > > > > > Parth >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > On Mon, May 23, 2016 at 10:35 AM, Jun Rao <j...@confluent.io> >> > > > wrote: >> > > > > > > > >> > > > > > > > > Thanks for the KIP. A few comments. >> > > > > > > > > >> > > > > > > > > 100. This potentially can be useful for Kafka Connect and >> > Kafka >> > > > > rest >> > > > > > > > proxy >> > > > > > > > > where a worker agent will need to run a task on behalf of a >> > > > client. >> > > > > > We >> > > > > > > > will >> > > > > > > > > likely need to change how those services use Kafka clients >> > > > > > > > > (producer/consumer). Instead of a shared client per worker, >> > we >> > > > will >> > > > > > > need >> > > > > > > > a >> > > > > > > > > client per user task since the authentication happens at the >> > > > > > connection >> > > > > > > > > level. For Kafka Connect, the renewer will be the workers. >> > So, >> > > we >> > > > > > > > probably >> > > > > > > > > need to allow multiple renewers. For Kafka rest proxy, the >> > > > renewer >> > > > > > can >> > > > > > > > > probably just be the creator of the token. >> > > > > > > > > >> > > > > > > > > 101. Do we need new acl on who can request delegation tokens? >> > > > > > > > > >> > > > > > > > > 102. Do we recommend people to send delegation tokens in an >> > > > > encrypted >> > > > > > > > > channel? >> > > > > > > > > >> > > > > > > > > 103. Who is responsible for expiring tokens, every broker? >> > > > > > > > > >> > > > > > > > > 104. For invalidating tokens, would it be better to do it in >> > a >> > > > > > request >> > > > > > > > > instead of going to ZK directly? >> > > > > > > > > >> > > > > > > > > 105. The terminology of client in the wiki sometimes refers >> > to >> > > > the >> > > > > > end >> > > > > > > > > client and some other times refers to the client using the >> > > > > delegation >> > > > > > > > > tokens. It would be useful to distinguish between the two. >> > > > > > > > > >> > > > > > > > > 106. Could you explain the sentence "Broker also stores the >> > > > > > > > DelegationToken >> > > > > > > > > without the hmac in the zookeeper." a bit more? I thought the >> > > > > > > delegation >> > > > > > > > > token is the hmac. >> > > > > > > > > >> > > > > > > > > Thanks, >> > > > > > > > > >> > > > > > > > > Jun >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > On Mon, May 23, 2016 at 9:22 AM, Jun Rao <j...@confluent.io> >> > > > wrote: >> > > > > > > > > >> > > > > > > > > > Hi, Harsha, >> > > > > > > > > > >> > > > > > > > > > Just sent out a KIP meeting invite. We can discuss this in >> > > the >> > > > > > > meeting >> > > > > > > > > > tomorrow. >> > > > > > > > > > >> > > > > > > > > > Thanks, >> > > > > > > > > > >> > > > > > > > > > Jun >> > > > > > > > > > >> > > > > > > > > > On Thu, May 19, 2016 at 8:47 AM, Harsha <ka...@harsha.io> >> > > > wrote: >> > > > > > > > > > >> > > > > > > > > >> Hi All, >> > > > > > > > > >> Can we have a KIP meeting around this. The KIP >> > is >> > > > up >> > > > > > for >> > > > > > > > > >> sometime and if there are any questions lets >> > > > quickly >> > > > > > hash >> > > > > > > > out >> > > > > > > > > >> details. >> > > > > > > > > >> >> > > > > > > > > >> Thanks, >> > > > > > > > > >> Harsha >> > > > > > > > > >> >> > > > > > > > > >> On Thu, May 19, 2016, at 08:40 AM, parth brahmbhatt wrote: >> > > > > > > > > >> > That is what the hadoop echo system uses so no good >> > reason >> > > > > > really. >> > > > > > > > We >> > > > > > > > > >> > could >> > > > > > > > > >> > change it to whatever is the newest recommended standard >> > > is. >> > > > > > > > > >> > >> > > > > > > > > >> > Thanks >> > > > > > > > > >> > Parth >> > > > > > > > > >> > >> > > > > > > > > >> > On Thu, May 19, 2016 at 3:33 AM, Ismael Juma < >> > > > > ism...@juma.me.uk >> > > > > > > >> > > > > > > > > wrote: >> > > > > > > > > >> > >> > > > > > > > > >> > > Hi Parth, >> > > > > > > > > >> > > >> > > > > > > > > >> > > Thanks for the KIP. I only started reviewing this and >> > > may >> > > > > have >> > > > > > > > > >> additional >> > > > > > > > > >> > > questions later. The immediate question that came to >> > > mind >> > > > is >> > > > > > our >> > > > > > > > > >> choice of >> > > > > > > > > >> > > "DIGEST-MD5" even though it's marked as OBSOLETE in >> > the >> > > > IANA >> > > > > > > > > Registry >> > > > > > > > > >> of >> > > > > > > > > >> > > SASL mechanisms and the original RFC (2831) has been >> > > moved >> > > > > to >> > > > > > > > > Historic >> > > > > > > > > >> > > status: >> > > > > > > > > >> > > >> > > > > > > > > >> > > https://tools.ietf.org/html/rfc6331 >> > > > > > > > > >> > > >> > > > > > > > > >> > > > > > >> > > http://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml >> > > > > > > > > >> > > >> > > > > > > > > >> > > What is the reasoning behind that choice? >> > > > > > > > > >> > > >> > > > > > > > > >> > > Thanks, >> > > > > > > > > >> > > Ismael >> > > > > > > > > >> > > >> > > > > > > > > >> > > On Fri, May 13, 2016 at 11:29 PM, Gwen Shapira < >> > > > > > > g...@confluent.io >> > > > > > > > > >> > > > > > > > > >> wrote: >> > > > > > > > > >> > > >> > > > > > > > > >> > > > Also comments inline :) >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > > * I want to emphasize that even though delegation >> > > > tokens >> > > > > > > are a >> > > > > > > > > >> Hadoop >> > > > > > > > > >> > > > > innovation, I feel very strongly about not adding >> > > > > > dependency >> > > > > > > > on >> > > > > > > > > >> Hadoop >> > > > > > > > > >> > > > > when implementing delegation tokens for Kafka. The >> > > KIP >> > > > > > > doesn't >> > > > > > > > > >> imply >> > > > > > > > > >> > > > > such dependency, but if you can clarify... >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > *No hadoop dependency.* >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > Yay! Just add this to the KIP so no one will read >> > the >> > > > KIP >> > > > > > and >> > > > > > > > > panic >> > > > > > > > > >> > > > three weeks before the next release... >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > > * Can we get delegation token at any time after >> > > > > > > > authenticating? >> > > > > > > > > >> only >> > > > > > > > > >> > > > > immediately after? >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > *As long as you are authenticated you can get >> > > > delegation >> > > > > > > > tokens. >> > > > > > > > > >> We >> > > > > > > > > >> > > need >> > > > > > > > > >> > > > to >> > > > > > > > > >> > > > > discuss if a client authenticated using delegation >> > > > > token, >> > > > > > > can >> > > > > > > > > also >> > > > > > > > > >> > > > acquire >> > > > > > > > > >> > > > > delegation token again or not. Also there is the >> > > > > question >> > > > > > of >> > > > > > > > do >> > > > > > > > > we >> > > > > > > > > >> > > allow >> > > > > > > > > >> > > > > anyone to acquire delegation token or we want >> > > specific >> > > > > > ACLs >> > > > > > > (I >> > > > > > > > > >> think >> > > > > > > > > >> > > its >> > > > > > > > > >> > > > an >> > > > > > > > > >> > > > > overkill.)* >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > I agree that ACLs is an overkill. >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > I think we are debating two options: Either require >> > > > > Kerberos >> > > > > > > > auth >> > > > > > > > > >> for >> > > > > > > > > >> > > > renewal or require non-owners to renew. >> > > > > > > > > >> > > > I *think* the latter is simpler (it basically >> > require >> > > a >> > > > > "job >> > > > > > > > > master" >> > > > > > > > > >> > > > to take responsibility for the renewal, it will have >> > > its >> > > > > own >> > > > > > > > > >> identity >> > > > > > > > > >> > > > anyway and I think this is the correct design >> > pattern >> > > > > > anyway. >> > > > > > > > For >> > > > > > > > > >> > > > storm, I'd expect Nimbus to coordinate renewals?), >> > but >> > > > it >> > > > > is >> > > > > > > > hard >> > > > > > > > > to >> > > > > > > > > >> > > > debate simplicity without looking at the code >> > changes >> > > > > > > required. >> > > > > > > > If >> > > > > > > > > >> you >> > > > > > > > > >> > > > have a draft of how the "require Kerberos" will look >> > > in >> > > > > > Kafka >> > > > > > > > > code, >> > > > > > > > > >> > > > I'll be happy to take a look. >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > > * My understanding is that tokens will propagate >> > via >> > > > ZK >> > > > > > but >> > > > > > > > > >> without >> > > > > > > > > >> > > > > additional changes to UpdateMetadata protocol, >> > > > correct? >> > > > > > > > Clients >> > > > > > > > > >> > > > > currently don't retry on SASL auth failure (IIRC), >> > > but >> > > > > > since >> > > > > > > > the >> > > > > > > > > >> > > > > tokens propagate between brokers asynch, we will >> > > need >> > > > to >> > > > > > > > retry a >> > > > > > > > > >> bit >> > > > > > > > > >> > > > > to avoid clients failing auth due to timing >> > issues. >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > *I am considering 2 alternatives right now. The >> > > > current >> > > > > > > > > documented >> > > > > > > > > >> > > > approach >> > > > > > > > > >> > > > > is zookeeper based and it does not require any >> > > changes >> > > > > to >> > > > > > > > > >> > > UpdateMetadata >> > > > > > > > > >> > > > > protocol. An alternative approach can remove >> > > zookeeper >> > > > > > > > > dependency >> > > > > > > > > >> as >> > > > > > > > > >> > > well >> > > > > > > > > >> > > > > but we can discuss that in KIP discussion call.* >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > Oooh! Sounds interesting. Do you want to ping Jun to >> > > > > > arrange a >> > > > > > > > > call? >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > > * I liked Ashish's suggestion of having just the >> > > > > > controller >> > > > > > > > > issue >> > > > > > > > > >> the >> > > > > > > > > >> > > > > delegation tokens, to avoid syncing a shared >> > secret. >> > > > Not >> > > > > > > sure >> > > > > > > > if >> > > > > > > > > >> we >> > > > > > > > > >> > > > > want to continue the discussion here or on the >> > > wiki. I >> > > > > > think >> > > > > > > > > that >> > > > > > > > > >> we >> > > > > > > > > >> > > > > can decouple the problem of "token distribution" >> > > from >> > > > > > > "shared >> > > > > > > > > >> secret >> > > > > > > > > >> > > > > distribution" and use the controller as the only >> > > token >> > > > > > > > generator >> > > > > > > > > >> to >> > > > > > > > > >> > > > > solve the second issue, while still using ZK async >> > > to >> > > > > > > > distribute >> > > > > > > > > >> > > > > tokens. >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > *As mentioned in the previous Email I am fine with >> > > > that >> > > > > > > > approach >> > > > > > > > > >> as >> > > > > > > > > >> > > long >> > > > > > > > > >> > > > as >> > > > > > > > > >> > > > > we agree that the extra complexity of >> > > adding/updating >> > > > > APIS >> > > > > > > > adds >> > > > > > > > > >> enough >> > > > > > > > > >> > > > > value. The advantage with the controller approach >> > is >> > > > > > secret >> > > > > > > > > >> rotation >> > > > > > > > > >> > > can >> > > > > > > > > >> > > > be >> > > > > > > > > >> > > > > automated,frequent and would not require >> > > deployment. * >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > Can you detail the extra complexity (or point me to >> > > the >> > > > > > email >> > > > > > > I >> > > > > > > > > >> > > > missed?) - which Apis are required? >> > > > > > > > > >> > > > As far as I can tell, clients can already find the >> > > > > > controller >> > > > > > > > from >> > > > > > > > > >> > > > metadata. I'm a bit more concerned about controller >> > > > load. >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > * While I like the idea of forcing kerberos auth >> > for >> > > > > > > renwal, I >> > > > > > > > > >> think >> > > > > > > > > >> > > > > it mixes the transport layer the the request >> > content >> > > > in >> > > > > a >> > > > > > > > pretty >> > > > > > > > > >> ugly >> > > > > > > > > >> > > > > way. Perhaps limiting renewer to non-owner is >> > > better. >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > *I feel this is a necessary evil. While this will >> > > make >> > > > > the >> > > > > > > > kafka >> > > > > > > > > >> code >> > > > > > > > > >> > > > > pretty straight forward , forcing renewer to >> > > > non-owner >> > > > > > > pushes >> > > > > > > > > >> the code >> > > > > > > > > >> > > > > ugliness to client and makes it even harder to >> > > > > > integrate. * >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > As mentioned before, I don't think the "renewal by >> > > > other" >> > > > > > > > approach >> > > > > > > > > >> is >> > > > > > > > > >> > > > that ugly for the clients we expect to use >> > delegation >> > > > > tokens >> > > > > > > > since >> > > > > > > > > >> > > > they will have an app-master of some sort who >> > > requested >> > > > > the >> > > > > > > > token >> > > > > > > > > to >> > > > > > > > > >> > > > begin with. >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > The response for my question on how multiple >> > > > identities >> > > > > > will >> > > > > > > > be >> > > > > > > > > >> > > > > handled wasn't super clear to me - AFAIK, we don't >> > > > > > > > authenticate >> > > > > > > > > >> each >> > > > > > > > > >> > > > > request, we authenticate connections. >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > *We authenticate connections, and only when they >> > are >> > > > > being >> > > > > > > > > >> established. >> > > > > > > > > >> > > > Let >> > > > > > > > > >> > > > > me try to phrase this as a question, in absence of >> > > > > > > delegation >> > > > > > > > > >> tokens if >> > > > > > > > > >> > > > we >> > > > > > > > > >> > > > > had to support the use case using user TGT's how >> > > would >> > > > > we >> > > > > > do >> > > > > > > > it? >> > > > > > > > > >> My >> > > > > > > > > >> > > point >> > > > > > > > > >> > > > > was it would be no different with delegation >> > tokens. >> > > > The >> > > > > > use >> > > > > > > > > case >> > > > > > > > > >> you >> > > > > > > > > >> > > are >> > > > > > > > > >> > > > > describing seems more like impersonation.* >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > Yeah, I thought that one of the things that >> > delegation >> > > > > > tokens >> > > > > > > > > >> handled. >> > > > > > > > > >> > > > Maybe I got it wrong :) >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > Thanks for the detailed answers. >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > Gwen >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> > > > > > > > > >> > > > > Thanks >> > > > > > > > > >> > > > > Parth >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > > On Fri, May 13, 2016 at 12:19 AM, Gwen Shapira < >> > > > > > > > > g...@confluent.io >> > > > > > > > > >> > >> > > > > > > > > >> > > > wrote: >> > > > > > > > > >> > > > > >> > > > > > > > > >> > > > >> Hi Parth and Harsha, >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> Few more comments: >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> * The API / RequestResponse section doesn't seem >> > to >> > > > > have >> > > > > > > good >> > > > > > > > > >> > > > >> description of the changes to the Kafka Protocol. >> > > > > Sounds >> > > > > > > like >> > > > > > > > > >> you are >> > > > > > > > > >> > > > >> proposing new DelegationTokenRequest and >> > > > > > RenewTokenRequest >> > > > > > > > (and >> > > > > > > > > >> > > > >> matching responses), without detailing the >> > contents >> > > > of >> > > > > > the >> > > > > > > > > >> requests >> > > > > > > > > >> > > > >> and responses? Or rather, you show the class >> > > > interface, >> > > > > > but >> > > > > > > > not >> > > > > > > > > >> the >> > > > > > > > > >> > > > >> underlying protocol. This could be seen as an >> > > > > > > implementation >> > > > > > > > > >> detail, >> > > > > > > > > >> > > > >> but since the binary protocol is what we provide >> > to >> > > > > > > non-Java >> > > > > > > > > >> clients, >> > > > > > > > > >> > > > >> we need to show the changes there. >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> * getDelegationToken sounds like >> > > > > > > > delegationTokenRequestHandler? >> > > > > > > > > >> Is it >> > > > > > > > > >> > > > >> planned to be part of KafkaApi? or Client? Its >> > > > > unclear... >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> * I want to emphasize that even though delegation >> > > > > tokens >> > > > > > > are >> > > > > > > > a >> > > > > > > > > >> Hadoop >> > > > > > > > > >> > > > >> innovation, I feel very strongly about not adding >> > > > > > > dependency >> > > > > > > > on >> > > > > > > > > >> Hadoop >> > > > > > > > > >> > > > >> when implementing delegation tokens for Kafka. >> > The >> > > > KIP >> > > > > > > > doesn't >> > > > > > > > > >> imply >> > > > > > > > > >> > > > >> such dependency, but if you can clarify... >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> * Can we get delegation token at any time after >> > > > > > > > authenticating? >> > > > > > > > > >> only >> > > > > > > > > >> > > > >> immediately after? >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> * My understanding is that tokens will propagate >> > > via >> > > > ZK >> > > > > > but >> > > > > > > > > >> without >> > > > > > > > > >> > > > >> additional changes to UpdateMetadata protocol, >> > > > correct? >> > > > > > > > Clients >> > > > > > > > > >> > > > >> currently don't retry on SASL auth failure >> > (IIRC), >> > > > but >> > > > > > > since >> > > > > > > > > the >> > > > > > > > > >> > > > >> tokens propagate between brokers asynch, we will >> > > need >> > > > > to >> > > > > > > > retry >> > > > > > > > > a >> > > > > > > > > >> bit >> > > > > > > > > >> > > > >> to avoid clients failing auth due to timing >> > issues. >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> * Strongly agreeing on clients not touching ZK >> > > > directly >> > > > > > :) >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> * I liked Ashish's suggestion of having just the >> > > > > > controller >> > > > > > > > > >> issue the >> > > > > > > > > >> > > > >> delegation tokens, to avoid syncing a shared >> > > secret. >> > > > > Not >> > > > > > > sure >> > > > > > > > > if >> > > > > > > > > >> we >> > > > > > > > > >> > > > >> want to continue the discussion here or on the >> > > wiki. >> > > > I >> > > > > > > think >> > > > > > > > > >> that we >> > > > > > > > > >> > > > >> can decouple the problem of "token distribution" >> > > from >> > > > > > > "shared >> > > > > > > > > >> secret >> > > > > > > > > >> > > > >> distribution" and use the controller as the only >> > > > token >> > > > > > > > > generator >> > > > > > > > > >> to >> > > > > > > > > >> > > > >> solve the second issue, while still using ZK >> > async >> > > to >> > > > > > > > > distribute >> > > > > > > > > >> > > > >> tokens. >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> * I am also uncomfortable with infinite lifetime >> > of >> > > > > > tokens >> > > > > > > > (and >> > > > > > > > > >> hoped >> > > > > > > > > >> > > > >> to hear from others in the community) - but >> > having >> > > > the >> > > > > > > option >> > > > > > > > > and >> > > > > > > > > >> > > > >> documenting it as less secure, allows users to >> > > > > configure >> > > > > > > > their >> > > > > > > > > >> system >> > > > > > > > > >> > > > >> as the wish. >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> * While I like the idea of forcing kerberos auth >> > > for >> > > > > > > renwal, >> > > > > > > > I >> > > > > > > > > >> think >> > > > > > > > > >> > > > >> it mixes the transport layer the the request >> > > content >> > > > > in a >> > > > > > > > > pretty >> > > > > > > > > >> ugly >> > > > > > > > > >> > > > >> way. Perhaps limiting renewer to non-owner is >> > > better. >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> Things I'd still like to see: >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> * More detailed explanation on what we plan to do >> > > for >> > > > > the >> > > > > > > > java >> > > > > > > > > >> clients >> > > > > > > > > >> > > > >> specifically - new configuration? new APIs? >> > > > > > > > > >> > > > >> The response for my question on how multiple >> > > > identities >> > > > > > > will >> > > > > > > > be >> > > > > > > > > >> > > > >> handled wasn't super clear to me - AFAIK, we >> > don't >> > > > > > > > authenticate >> > > > > > > > > >> each >> > > > > > > > > >> > > > >> request, we authenticate connections. >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> * Alternatives: Delegation tokens are only used >> > in >> > > > the >> > > > > > > Hadoop >> > > > > > > > > >> > > > >> ecosystem. I'm wondering if there are >> > alternatives >> > > in >> > > > > > other >> > > > > > > > > >> ecosystems >> > > > > > > > > >> > > > >> (Mesos? Tachyon? Cassandra?) and whether there >> > are >> > > > some >> > > > > > > > > >> advantages >> > > > > > > > > >> > > > >> there. >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> Gwen >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> On Thu, May 12, 2016 at 1:05 PM, Harsha < >> > > > > ka...@harsha.io >> > > > > > > >> > > > > > > > > wrote: >> > > > > > > > > >> > > > >> > Hi Gwen, >> > > > > > > > > >> > > > >> > Can you look at Parth's last reply. >> > > Does >> > > > > it >> > > > > > > > answer >> > > > > > > > > >> your >> > > > > > > > > >> > > > >> > concerns. >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> > Thanks, >> > > > > > > > > >> > > > >> > Harsha >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> > On Wed, May 4, 2016, at 09:25 AM, parth >> > > brahmbhatt >> > > > > > wrote: >> > > > > > > > > >> > > > >> >> Thanks for reviewing Gwen. The wiki already >> > has >> > > > > > details >> > > > > > > on >> > > > > > > > > >> token >> > > > > > > > > >> > > > >> >> expiration >> > > > > > > > > >> > > > >> >> under token acquisition process >> > > > > > > > > >> > > > >> >> < >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> > > > > > > > > >> > > >> > > > > > > > > >> >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka#KIP-48DelegationtokensupportforKafka-Tokenacquisition >> > > > > > > > > >> > > > >> >. >> > > > > > > > > >> > > > >> >> Current proposal is that tokens will expire >> > > based >> > > > > on a >> > > > > > > > > server >> > > > > > > > > >> side >> > > > > > > > > >> > > > >> >> configuration (default 24 hours) unless >> > renewed. >> > > > > > Renewal >> > > > > > > > is >> > > > > > > > > >> only >> > > > > > > > > >> > > > allowed >> > > > > > > > > >> > > > >> >> until the max life time of token. >> > Alternatively >> > > we >> > > > > > could >> > > > > > > > > also >> > > > > > > > > >> make >> > > > > > > > > >> > > > that >> > > > > > > > > >> > > > >> >> an >> > > > > > > > > >> > > > >> >> optional param and the server side default can >> > > > serve >> > > > > > as >> > > > > > > > the >> > > > > > > > > >> upper >> > > > > > > > > >> > > > bound. >> > > > > > > > > >> > > > >> >> >> > > > > > > > > >> > > > >> >> To your second point it will be done exactly >> > the >> > > > > same >> > > > > > > way >> > > > > > > > we >> > > > > > > > > >> would >> > > > > > > > > >> > > > >> >> support >> > > > > > > > > >> > > > >> >> multiple keytabs. The calling client will have >> > > to >> > > > > put >> > > > > > > the >> > > > > > > > > >> tokens it >> > > > > > > > > >> > > > >> wants >> > > > > > > > > >> > > > >> >> to use in the subject instance and call >> > > > > > produce/consume >> > > > > > > > > inside >> > > > > > > > > >> > > > >> >> subject.doas. Each caller will have to keep >> > > track >> > > > of >> > > > > > its >> > > > > > > > own >> > > > > > > > > >> > > > subject. I >> > > > > > > > > >> > > > >> >> will have to look at the code to see if we >> > > support >> > > > > > this >> > > > > > > > > >> feature >> > > > > > > > > >> > > right >> > > > > > > > > >> > > > >> now >> > > > > > > > > >> > > > >> >> but my understanding is delegation token >> > > shouldn't >> > > > > > need >> > > > > > > > any >> > > > > > > > > >> special >> > > > > > > > > >> > > > >> >> treatment as its just another type of >> > Credential >> > > > in >> > > > > > the >> > > > > > > > > >> subject. >> > > > > > > > > >> > > > >> >> >> > > > > > > > > >> > > > >> >> I would also like to know what is your opinion >> > > > about >> > > > > > > > > infinite >> > > > > > > > > >> > > renewal >> > > > > > > > > >> > > > >> (my >> > > > > > > > > >> > > > >> >> recommendation is to not support this), tokens >> > > > > > renewing >> > > > > > > > them >> > > > > > > > > >> > > self(my >> > > > > > > > > >> > > > >> >> recommendation is to not support this) and >> > most >> > > > > > > > importantly >> > > > > > > > > >> your >> > > > > > > > > >> > > > choice >> > > > > > > > > >> > > > >> >> between the alternatives listed on this thread >> > > > > > > > > >> > > > >> >> < >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> > > > > > > > > >> > > >> > > > > > > > > >> >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > http://apache.markmail.org/message/ca3iakt3m6c4yygp?q=KIP-48+Support+for+delegation+tokens+as+an+authentication+mechanism >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> >> ( I am leaning towards the alternative-2 minus >> > > > > > > controller >> > > > > > > > > >> > > > distributing >> > > > > > > > > >> > > > >> >> secret). Thanks again for reviewing. >> > > > > > > > > >> > > > >> >> >> > > > > > > > > >> > > > >> >> Thanks >> > > > > > > > > >> > > > >> >> Parth >> > > > > > > > > >> > > > >> >> >> > > > > > > > > >> > > > >> >> >> > > > > > > > > >> > > > >> >> >> > > > > > > > > >> > > > >> >> On Wed, May 4, 2016 at 6:17 AM, Gwen Shapira < >> > > > > > > > > >> g...@confluent.io> >> > > > > > > > > >> > > > wrote: >> > > > > > > > > >> > > > >> >> >> > > > > > > > > >> > > > >> >> > Harsha, >> > > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> > I was thinking of the Rest Proxy. I didn't >> > see >> > > > > your >> > > > > > > > design >> > > > > > > > > >> yet, >> > > > > > > > > >> > > > but in >> > > > > > > > > >> > > > >> >> > our proxy, we have a set of producers, which >> > > > will >> > > > > > > serve >> > > > > > > > > >> multiple >> > > > > > > > > >> > > > users >> > > > > > > > > >> > > > >> >> > going through the proxy. Since these users >> > > will >> > > > > have >> > > > > > > > > >> different >> > > > > > > > > >> > > > >> >> > privileges, they'll need to authenticate >> > > > > separately, >> > > > > > > and >> > > > > > > > > >> can't >> > > > > > > > > >> > > > share a >> > > > > > > > > >> > > > >> >> > token. >> > > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> > Am I missing anything? >> > > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> > Gwen >> > > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> > On Tue, May 3, 2016 at 2:11 PM, Harsha < >> > > > > > > ka...@harsha.io >> > > > > > > > > >> > > > > > > > > >> wrote: >> > > > > > > > > >> > > > >> >> > > Gwen, >> > > > > > > > > >> > > > >> >> > > On your second point. Can you >> > > > > describe >> > > > > > a >> > > > > > > > > >> usecase >> > > > > > > > > >> > > where >> > > > > > > > > >> > > > >> >> > > mutliple clients ended up >> > > sharing a >> > > > > > > > producer >> > > > > > > > > >> and >> > > > > > > > > >> > > even >> > > > > > > > > >> > > > if >> > > > > > > > > >> > > > >> they >> > > > > > > > > >> > > > >> >> > > do why can't they not use >> > single >> > > > > token >> > > > > > > that >> > > > > > > > > >> producer >> > > > > > > > > >> > > > >> >> > > captures. Why would we need >> > > > multiple >> > > > > > > > clients >> > > > > > > > > >> with >> > > > > > > > > >> > > > >> different >> > > > > > > > > >> > > > >> >> > > tokens sharing a single >> > instance >> > > of >> > > > > > > > producer. >> > > > > > > > > >> Also >> > > > > > > > > >> > > in >> > > > > > > > > >> > > > >> this >> > > > > > > > > >> > > > >> >> > > case other clients have access >> > > all >> > > > > the >> > > > > > > > tokens >> > > > > > > > > >> no? >> > > > > > > > > >> > > > >> >> > > >> > > > > > > > > >> > > > >> >> > > Thanks, >> > > > > > > > > >> > > > >> >> > > Harsha >> > > > > > > > > >> > > > >> >> > > >> > > > > > > > > >> > > > >> >> > > >> > > > > > > > > >> > > > >> >> > > On Tue, May 3, 2016, at 11:49 AM, Gwen >> > > Shapira >> > > > > > > wrote: >> > > > > > > > > >> > > > >> >> > >> Sorry for the delay: >> > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > >> > > > >> >> > >> Two questions that we didn't see in the >> > > wiki: >> > > > > > > > > >> > > > >> >> > >> 1. Is there an expiration for delegation >> > > > > tokens? >> > > > > > > > > >> Renewal? How >> > > > > > > > > >> > > > do we >> > > > > > > > > >> > > > >> >> > >> revoke them? >> > > > > > > > > >> > > > >> >> > >> 2. If we want to use delegation tokens >> > for >> > > > > > "do-as" >> > > > > > > > > (say, >> > > > > > > > > >> > > submit >> > > > > > > > > >> > > > >> Storm >> > > > > > > > > >> > > > >> >> > >> job as my user), we will need a producer >> > > for >> > > > > > every >> > > > > > > > job >> > > > > > > > > >> (we >> > > > > > > > > >> > > can't >> > > > > > > > > >> > > > >> share >> > > > > > > > > >> > > > >> >> > >> them between multiple jobs running on >> > same >> > > > > node), >> > > > > > > > since >> > > > > > > > > >> we >> > > > > > > > > >> > > only >> > > > > > > > > >> > > > >> >> > >> authenticate when connecting. Is there a >> > > plan >> > > > > to >> > > > > > > > change >> > > > > > > > > >> this >> > > > > > > > > >> > > for >> > > > > > > > > >> > > > >> >> > >> delegation tokens, in order to allow >> > > multiple >> > > > > > users >> > > > > > > > > with >> > > > > > > > > >> > > > different >> > > > > > > > > >> > > > >> >> > >> tokens to share a client? >> > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > >> > > > >> >> > >> Gwen >> > > > > > > > > >> > > > >> >> > >> >> > > > > > > > > >> > > > >> >> > >> On Tue, May 3, 2016 at 9:12 AM, parth >> > > > > brahmbhatt >> > > > > > > > > >> > > > >> >> > >> <brahmbhatt.pa...@gmail.com> wrote: >> > > > > > > > > >> > > > >> >> > >> > Bumping this up one more time, can >> > other >> > > > > > > committers >> > > > > > > > > >> review? >> > > > > > > > > >> > > > >> >> > >> > >> > > > > > > > > >> > > > >> >> > >> > Thanks >> > > > > > > > > >> > > > >> >> > >> > Parth >> > > > > > > > > >> > > > >> >> > >> > >> > > > > > > > > >> > > > >> >> > >> > On Tue, Apr 26, 2016 at 9:07 AM, >> > Harsha < >> > > > > > > > > >> ka...@harsha.io> >> > > > > > > > > >> > > > wrote: >> > > > > > > > > >> > > > >> >> > >> > >> > > > > > > > > >> > > > >> >> > >> >> Parth, >> > > > > > > > > >> > > > >> >> > >> >> Overall current design looks >> > > > good >> > > > > to >> > > > > > > > me. I >> > > > > > > > > >> am +1 >> > > > > > > > > >> > > on >> > > > > > > > > >> > > > >> the >> > > > > > > > > >> > > > >> >> > KIP. >> > > > > > > > > >> > > > >> >> > >> >> >> > > > > > > > > >> > > > >> >> > >> >> Gwen , Jun can you review this as >> > well. >> > > > > > > > > >> > > > >> >> > >> >> >> > > > > > > > > >> > > > >> >> > >> >> -Harsha >> > > > > > > > > >> > > > >> >> > >> >> >> > > > > > > > > >> > > > >> >> > >> >> On Tue, Apr 19, 2016, at 09:57 AM, >> > parth >> > > > > > > > brahmbhatt >> > > > > > > > > >> wrote: >> > > > > > > > > >> > > > >> >> > >> >> > Thanks for review Jitendra. >> > > > > > > > > >> > > > >> >> > >> >> > >> > > > > > > > > >> > > > >> >> > >> >> > I don't like the idea of infinite >> > > > lifetime >> > > > > > > but I >> > > > > > > > > >> see the >> > > > > > > > > >> > > > >> Streaming >> > > > > > > > > >> > > > >> >> > use >> > > > > > > > > >> > > > >> >> > >> >> > case. Even for Streaming use case I >> > > was >> > > > > > hoping >> > > > > > > > > >> there will >> > > > > > > > > >> > > > be >> > > > > > > > > >> > > > >> some >> > > > > > > > > >> > > > >> >> > notion >> > > > > > > > > >> > > > >> >> > >> >> > of >> > > > > > > > > >> > > > >> >> > >> >> > master/driver that can get new >> > > > delegation >> > > > > > > tokens >> > > > > > > > > at >> > > > > > > > > >> fixed >> > > > > > > > > >> > > > >> interval >> > > > > > > > > >> > > > >> >> > and >> > > > > > > > > >> > > > >> >> > >> >> > distribute to workers. If that is >> > not >> > > > the >> > > > > > case >> > > > > > > > for >> > > > > > > > > >> we can >> > > > > > > > > >> > > > >> discuss >> > > > > > > > > >> > > > >> >> > >> >> > delegation tokens renewing them self >> > > and >> > > > > the >> > > > > > > > > >> security >> > > > > > > > > >> > > > >> implications >> > > > > > > > > >> > > > >> >> > of the >> > > > > > > > > >> > > > >> >> > >> >> > same. >> > > > > > > > > >> > > > >> >> > >> >> > >> > > > > > > > > >> > > > >> >> > >> >> > I did not want clients to fetch >> > tokens >> > > > > from >> > > > > > > > > >> zookeeper, >> > > > > > > > > >> > > > >> overall I >> > > > > > > > > >> > > > >> >> > think >> > > > > > > > > >> > > > >> >> > >> >> > its >> > > > > > > > > >> > > > >> >> > >> >> > better if clients don't rely on our >> > > > > metadata >> > > > > > > > store >> > > > > > > > > >> and I >> > > > > > > > > >> > > > >> think we >> > > > > > > > > >> > > > >> >> > are >> > > > > > > > > >> > > > >> >> > >> >> > moving in that direction with all >> > the >> > > > > KIP-4 >> > > > > > > > > >> improvements. >> > > > > > > > > >> > > > I >> > > > > > > > > >> > > > >> chose >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as in this case the client >> > > > will >> > > > > > > still >> > > > > > > > > >> just talk >> > > > > > > > > >> > > > to >> > > > > > > > > >> > > > >> >> > broker , its >> > > > > > > > > >> > > > >> >> > >> >> > the brokers that will use zookeeper >> > > > which >> > > > > we >> > > > > > > > > >> already do >> > > > > > > > > >> > > > for a >> > > > > > > > > >> > > > >> lot >> > > > > > > > > >> > > > >> >> > of >> > > > > > > > > >> > > > >> >> > >> >> > other >> > > > > > > > > >> > > > >> >> > >> >> > usecases + ease of development + and >> > > the >> > > > > > > ability >> > > > > > > > > so >> > > > > > > > > >> > > tokens >> > > > > > > > > >> > > > >> will >> > > > > > > > > >> > > > >> >> > survive >> > > > > > > > > >> > > > >> >> > >> >> > even a rolling restart/cluster >> > > failure. >> > > > > if a >> > > > > > > > > >> majority >> > > > > > > > > >> > > > agrees >> > > > > > > > > >> > > > >> the >> > > > > > > > > >> > > > >> >> > added >> > > > > > > > > >> > > > >> >> > >> >> > complexity to have controller >> > > forwarding >> > > > > > keys >> > > > > > > to >> > > > > > > > > all >> > > > > > > > > >> > > > broker is >> > > > > > > > > >> > > > >> >> > justified >> > > > > > > > > >> > > > >> >> > >> >> > as >> > > > > > > > > >> > > > >> >> > >> >> > it provides tighter security , I am >> > > fine >> > > > > > with >> > > > > > > > that >> > > > > > > > > >> option >> > > > > > > > > >> > > > too. >> > > > > > > > > >> > > > >> >> > >> >> > >> > > > > > > > > >> > > > >> >> > >> >> > Given zookeeper does not support SSL >> > > we >> > > > > can >> > > > > > > not >> > > > > > > > > >> store >> > > > > > > > > >> > > > master >> > > > > > > > > >> > > > >> keys >> > > > > > > > > >> > > > >> >> > in >> > > > > > > > > >> > > > >> >> > >> >> > zookeeper as master keys will be >> > > exposed >> > > > > on >> > > > > > > > wire. >> > > > > > > > > To >> > > > > > > > > >> > > > support >> > > > > > > > > >> > > > >> >> > rotation >> > > > > > > > > >> > > > >> >> > >> >> > without affecting current clients is >> > > > > > > something I >> > > > > > > > > >> need to >> > > > > > > > > >> > > > put >> > > > > > > > > >> > > > >> more >> > > > > > > > > >> > > > >> >> > thought >> > > > > > > > > >> > > > >> >> > >> >> > in. My current proposal assumes the >> > > > > rotation >> > > > > > > > will >> > > > > > > > > >> > > > invalidate >> > > > > > > > > >> > > > >> all >> > > > > > > > > >> > > > >> >> > current >> > > > > > > > > >> > > > >> >> > >> >> > tokens. >> > > > > > > > > >> > > > >> >> > >> >> > >> > > > > > > > > >> > > > >> >> > >> >> > I request committers to also review >> > > and >> > > > > post >> > > > > > > > their >> > > > > > > > > >> > > comments >> > > > > > > > > >> > > > >> so we >> > > > > > > > > >> > > > >> >> > can >> > > > > > > > > >> > > > >> >> > >> >> > make >> > > > > > > > > >> > > > >> >> > >> >> > progress on this KIP. >> > > > > > > > > >> > > > >> >> > >> >> > >> > > > > > > > > >> > > > >> >> > >> >> > Thanks >> > > > > > > > > >> > > > >> >> > >> >> > Parth >> > > > > > > > > >> > > > >> >> > >> >> > >> > > > > > > > > >> > > > >> >> > >> >> > On Tue, Apr 19, 2016 at 8:39 AM, >> > > Ashish >> > > > > > Singh >> > > > > > > < >> > > > > > > > > >> > > > >> asi...@cloudera.com >> > > > > > > > > >> > > > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > wrote: >> > > > > > > > > >> > > > >> >> > >> >> > >> > > > > > > > > >> > > > >> >> > >> >> > > On Mon, Apr 18, 2016 at 11:26 AM, >> > > > > Harsha < >> > > > > > > > > >> > > > ka...@harsha.io> >> > > > > > > > > >> > > > >> >> > wrote: >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > Unifying the two discussion >> > > threads >> > > > on >> > > > > > > this >> > > > > > > > > KIP. >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > Here is the response from >> > Jitendra >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > "The need for a large number of >> > > > > clients >> > > > > > > that >> > > > > > > > > are >> > > > > > > > > >> > > > running >> > > > > > > > > >> > > > >> all >> > > > > > > > > >> > > > >> >> > over the >> > > > > > > > > >> > > > >> >> > >> >> > > > cluster that authenticate with >> > > Kafka >> > > > > > > > brokers, >> > > > > > > > > >> is very >> > > > > > > > > >> > > > >> similar >> > > > > > > > > >> > > > >> >> > to the >> > > > > > > > > >> > > > >> >> > >> >> > > > Hadoop use case of large number >> > of >> > > > > tasks >> > > > > > > > > running >> > > > > > > > > >> > > across >> > > > > > > > > >> > > > >> the >> > > > > > > > > >> > > > >> >> > cluster >> > > > > > > > > >> > > > >> >> > >> >> that >> > > > > > > > > >> > > > >> >> > >> >> > > > need authentication to Hdfs >> > > > Namenode. >> > > > > > > > > >> Therefore, the >> > > > > > > > > >> > > > >> >> > delegation token >> > > > > > > > > >> > > > >> >> > >> >> > > > approach does seem like a good >> > fit >> > > > for >> > > > > > > this >> > > > > > > > > use >> > > > > > > > > >> case >> > > > > > > > > >> > > > as we >> > > > > > > > > >> > > > >> >> > have seen >> > > > > > > > > >> > > > >> >> > >> >> it >> > > > > > > > > >> > > > >> >> > >> >> > > > working at large scale in HDFS >> > and >> > > > > YARN. >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > The proposed design is very >> > much >> > > > > > inline >> > > > > > > > with >> > > > > > > > > >> Hadoop >> > > > > > > > > >> > > > >> >> > approach. A few >> > > > > > > > > >> > > > >> >> > >> >> > > > comments: >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > 1) Why do you guys want to allow >> > > > > > infinite >> > > > > > > > > >> renewable >> > > > > > > > > >> > > > >> lifetime >> > > > > > > > > >> > > > >> >> > for a >> > > > > > > > > >> > > > >> >> > >> >> > > > token? HDFS restricts a token >> > to a >> > > > max >> > > > > > > life >> > > > > > > > > time >> > > > > > > > > >> > > > (default >> > > > > > > > > >> > > > >> 7 >> > > > > > > > > >> > > > >> >> > days). A >> > > > > > > > > >> > > > >> >> > >> >> > > > token's vulnerability is >> > believed >> > > to >> > > > > > > > increase >> > > > > > > > > >> with >> > > > > > > > > >> > > > time. >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > I agree that having infinite >> > > lifetime >> > > > > > might >> > > > > > > > not >> > > > > > > > > >> be the >> > > > > > > > > >> > > > best >> > > > > > > > > >> > > > >> idea. >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > 2) As I understand the tokens >> > are >> > > > > stored >> > > > > > > in >> > > > > > > > > >> zookeeper >> > > > > > > > > >> > > > as >> > > > > > > > > >> > > > >> well, >> > > > > > > > > >> > > > >> >> > and >> > > > > > > > > >> > > > >> >> > >> >> can >> > > > > > > > > >> > > > >> >> > >> >> > > > be updated there. This is clever >> > > as >> > > > it >> > > > > > can >> > > > > > > > > allow >> > > > > > > > > >> > > > >> replacing the >> > > > > > > > > >> > > > >> >> > tokens >> > > > > > > > > >> > > > >> >> > >> >> > > > once they run out of max life >> > > time, >> > > > > and >> > > > > > > > > clients >> > > > > > > > > >> can >> > > > > > > > > >> > > > >> download >> > > > > > > > > >> > > > >> >> > new >> > > > > > > > > >> > > > >> >> > >> >> tokens >> > > > > > > > > >> > > > >> >> > >> >> > > > from zookeeper. It shouldn't be >> > a >> > > > big >> > > > > > load >> > > > > > > > on >> > > > > > > > > >> > > zookeeper >> > > > > > > > > >> > > > >> as a >> > > > > > > > > >> > > > >> >> > client >> > > > > > > > > >> > > > >> >> > >> >> will >> > > > > > > > > >> > > > >> >> > >> >> > > > need to get a new token once in >> > > > > several >> > > > > > > > days. >> > > > > > > > > >> In this >> > > > > > > > > >> > > > >> approach >> > > > > > > > > >> > > > >> >> > you >> > > > > > > > > >> > > > >> >> > >> >> don't >> > > > > > > > > >> > > > >> >> > >> >> > > > need infinite lifetime on the >> > > token >> > > > > even >> > > > > > > for >> > > > > > > > > >> long >> > > > > > > > > >> > > > running >> > > > > > > > > >> > > > >> >> > clients. >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > 3) The token password are >> > > generated >> > > > > > using >> > > > > > > a >> > > > > > > > > >> master >> > > > > > > > > >> > > key. >> > > > > > > > > >> > > > >> The >> > > > > > > > > >> > > > >> >> > master >> > > > > > > > > >> > > > >> >> > >> >> key >> > > > > > > > > >> > > > >> >> > >> >> > > > should also be periodically >> > > changed. >> > > > > In >> > > > > > > > > Hadoop, >> > > > > > > > > >> the >> > > > > > > > > >> > > > >> default >> > > > > > > > > >> > > > >> >> > renewal >> > > > > > > > > >> > > > >> >> > >> >> > > > period is 1 day.? >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > IIUC, this will require brokers >> > > > > > maintaining >> > > > > > > a >> > > > > > > > > >> list of X >> > > > > > > > > >> > > > most >> > > > > > > > > >> > > > >> >> > recent >> > > > > > > > > >> > > > >> >> > >> >> master >> > > > > > > > > >> > > > >> >> > >> >> > > keys. This list will have to be >> > > > > persisted >> > > > > > > > > >> somewhere, as >> > > > > > > > > >> > > > if a >> > > > > > > > > >> > > > >> >> > broker >> > > > > > > > > >> > > > >> >> > >> >> goes >> > > > > > > > > >> > > > >> >> > >> >> > > down it will have to get that list >> > > > again >> > > > > > and >> > > > > > > > > >> storing >> > > > > > > > > >> > > > master >> > > > > > > > > >> > > > >> keys >> > > > > > > > > >> > > > >> >> > on ZK >> > > > > > > > > >> > > > >> >> > >> >> is >> > > > > > > > > >> > > > >> >> > >> >> > > not the best idea. However, if a >> > > > broker >> > > > > > goes >> > > > > > > > > down >> > > > > > > > > >> then >> > > > > > > > > >> > > we >> > > > > > > > > >> > > > >> have >> > > > > > > > > >> > > > >> >> > much >> > > > > > > > > >> > > > >> >> > >> >> bigger >> > > > > > > > > >> > > > >> >> > >> >> > > issue to deal with and client can >> > > > always >> > > > > > > > > >> > > re-authenticate >> > > > > > > > > >> > > > is >> > > > > > > > > >> > > > >> such >> > > > > > > > > >> > > > >> >> > >> >> events. >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > Did you happen to take a look at >> > > other >> > > > > > > > > >> alternatives >> > > > > > > > > >> > > this >> > > > > > > > > >> > > > >> list has >> > > > > > > > > >> > > > >> >> > >> >> > > suggested? >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > Thanks for a thorough proposal, >> > > > great >> > > > > > > work!" >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > On Mon, Mar 7, 2016, at 10:28 >> > PM, >> > > > Gwen >> > > > > > > > Shapira >> > > > > > > > > >> wrote: >> > > > > > > > > >> > > > >> >> > >> >> > > > > Makes sense to me. Thanks! >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > On Mon, Mar 7, 2016 at 9:25 >> > PM, >> > > > > > Harsha < >> > > > > > > > > >> > > > ka...@harsha.io >> > > > > > > > > >> > > > >> > >> > > > > > > > > >> > > > >> >> > wrote: >> > > > > > > > > >> > > > >> >> > >> >> > > > > > It doesn't need any release >> > > > > vehicle >> > > > > > > but >> > > > > > > > > >> still the >> > > > > > > > > >> > > > >> work can >> > > > > > > > > >> > > > >> >> > move >> > > > > > > > > >> > > > >> >> > >> >> > > > forward. >> > > > > > > > > >> > > > >> >> > >> >> > > > > > If anyone is interested in >> > the >> > > > KIP >> > > > > > > > please >> > > > > > > > > >> do the >> > > > > > > > > >> > > > >> review and >> > > > > > > > > >> > > > >> >> > >> >> provide >> > > > > > > > > >> > > > >> >> > >> >> > > the >> > > > > > > > > >> > > > >> >> > >> >> > > > > > comments. >> > > > > > > > > >> > > > >> >> > >> >> > > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > > -Harsha >> > > > > > > > > >> > > > >> >> > >> >> > > > > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > > On Mon, Mar 7, 2016, at >> > 04:59 >> > > > PM, >> > > > > > > Ismael >> > > > > > > > > >> Juma >> > > > > > > > > >> > > > wrote: >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> I agree that it would be >> > good >> > > > to >> > > > > > have >> > > > > > > > > more >> > > > > > > > > >> time >> > > > > > > > > >> > > to >> > > > > > > > > >> > > > >> review >> > > > > > > > > >> > > > >> >> > and >> > > > > > > > > >> > > > >> >> > >> >> > > discuss >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> KIP-48. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> Ismael >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> On Tue, Mar 8, 2016 at >> > 12:55 >> > > > AM, >> > > > > > Gwen >> > > > > > > > > >> Shapira < >> > > > > > > > > >> > > > >> >> > >> >> g...@confluent.io> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote: >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Hi Team, >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Since KIP-48 depends on >> > > > KIP-43, >> > > > > > > which >> > > > > > > > > is >> > > > > > > > > >> > > > already a >> > > > > > > > > >> > > > >> bit >> > > > > > > > > >> > > > >> >> > of a >> > > > > > > > > >> > > > >> >> > >> >> risk >> > > > > > > > > >> > > > >> >> > >> >> > > for >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > the next release - any >> > > chance >> > > > > we >> > > > > > > can >> > > > > > > > > >> delay >> > > > > > > > > >> > > > >> delegation >> > > > > > > > > >> > > > >> >> > tokens >> > > > > > > > > >> > > > >> >> > >> >> to >> > > > > > > > > >> > > > >> >> > >> >> > > > Kafka >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > 0.10.1? >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > With the community >> > working >> > > > on a >> > > > > > > > release >> > > > > > > > > >> every >> > > > > > > > > >> > > 3 >> > > > > > > > > >> > > > >> month, >> > > > > > > > > >> > > > >> >> > this >> > > > > > > > > >> > > > >> >> > >> >> is not >> > > > > > > > > >> > > > >> >> > >> >> > > > a huge >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > delay. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > Gwen >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > On Fri, Feb 26, 2016 at >> > > 5:11 >> > > > > PM, >> > > > > > > > Ashish >> > > > > > > > > >> Singh >> > > > > > > > > >> > > < >> > > > > > > > > >> > > > >> >> > >> >> > > asi...@cloudera.com> >> > > > > > > > > >> > > > >> >> > >> >> > > > wrote: >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Parth, >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Thanks again for the >> > > > awesome >> > > > > > > write >> > > > > > > > > up. >> > > > > > > > > >> > > > Following >> > > > > > > > > >> > > > >> our >> > > > > > > > > >> > > > >> >> > >> >> discussion >> > > > > > > > > >> > > > >> >> > >> >> > > > from the >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > JIRA, I think it will >> > be >> > > > > easier >> > > > > > > to >> > > > > > > > > >> compare >> > > > > > > > > >> > > > >> various >> > > > > > > > > >> > > > >> >> > >> >> alternatives >> > > > > > > > > >> > > > >> >> > >> >> > > > if they >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > are >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > listed together. I am >> > > > stating >> > > > > > > > below a >> > > > > > > > > >> few >> > > > > > > > > >> > > > >> >> > alternatives along >> > > > > > > > > >> > > > >> >> > >> >> > > with >> > > > > > > > > >> > > > >> >> > >> >> > > > a the >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > current proposal. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Current proposal) >> > Store >> > > > > > > Delegation >> > > > > > > > > >> Token, >> > > > > > > > > >> > > DT, >> > > > > > > > > >> > > > >> on ZK. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client >> > > authenticates >> > > > > > with a >> > > > > > > > > >> broker. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a client is >> > > > > > > > authenticated, >> > > > > > > > > >> it >> > > > > > > > > >> > > will >> > > > > > > > > >> > > > >> make a >> > > > > > > > > >> > > > >> >> > broker >> > > > > > > > > >> > > > >> >> > >> >> side >> > > > > > > > > >> > > > >> >> > >> >> > > > call to >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a delegation >> > > > token. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The broker >> > > generates >> > > > a >> > > > > > > shared >> > > > > > > > > >> secret >> > > > > > > > > >> > > > based >> > > > > > > > > >> > > > >> on >> > > > > > > > > >> > > > >> >> > >> >> > > HMAC-SHA256(a >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Password/Secret >> > shared >> > > > > > between >> > > > > > > > all >> > > > > > > > > >> > > brokers, >> > > > > > > > > >> > > > >> >> > randomly >> > > > > > > > > >> > > > >> >> > >> >> > > generated >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > tokenId). >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Broker stores >> > this >> > > > > token >> > > > > > in >> > > > > > > > its >> > > > > > > > > >> in >> > > > > > > > > >> > > > memory >> > > > > > > > > >> > > > >> cache. >> > > > > > > > > >> > > > >> >> > >> >> Broker >> > > > > > > > > >> > > > >> >> > >> >> > > > also stores >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the DelegationToken >> > > > > without >> > > > > > > the >> > > > > > > > > >> hmac in >> > > > > > > > > >> > > the >> > > > > > > > > >> > > > >> >> > zookeeper. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. All brokers will >> > > > have a >> > > > > > > cache >> > > > > > > > > >> backed >> > > > > > > > > >> > > by >> > > > > > > > > >> > > > >> >> > zookeeper so >> > > > > > > > > >> > > > >> >> > >> >> they >> > > > > > > > > >> > > > >> >> > >> >> > > > will all >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > get notified >> > whenever >> > > a >> > > > > new >> > > > > > > > token >> > > > > > > > > is >> > > > > > > > > >> > > > >> generated and >> > > > > > > > > >> > > > >> >> > they >> > > > > > > > > >> > > > >> >> > >> >> will >> > > > > > > > > >> > > > >> >> > >> >> > > > update >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > local cache whenever >> > > > token >> > > > > > > state >> > > > > > > > > >> changes. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Broker returns >> > the >> > > > > token >> > > > > > to >> > > > > > > > > >> Client. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues and >> > fixes >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Probable race >> > > > > condition, >> > > > > > > > client >> > > > > > > > > >> tries >> > > > > > > > > >> > > to >> > > > > > > > > >> > > > >> >> > authenticate >> > > > > > > > > >> > > > >> >> > >> >> with >> > > > > > > > > >> > > > >> >> > >> >> > > > a broker >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > that is yet to be >> > > > updated >> > > > > > with >> > > > > > > > the >> > > > > > > > > >> newly >> > > > > > > > > >> > > > >> generated >> > > > > > > > > >> > > > >> >> > DT. >> > > > > > > > > >> > > > >> >> > >> >> This >> > > > > > > > > >> > > > >> >> > >> >> > > can >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > probably be >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > dealt with making >> > > > > dtRequest >> > > > > > > > block >> > > > > > > > > >> until >> > > > > > > > > >> > > all >> > > > > > > > > >> > > > >> >> > brokers have >> > > > > > > > > >> > > > >> >> > >> >> > > > updated >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > their DT >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > cache. Zk barrier or >> > > > > similar >> > > > > > > > > >> mechanism >> > > > > > > > > >> > > can >> > > > > > > > > >> > > > be >> > > > > > > > > >> > > > >> used. >> > > > > > > > > >> > > > >> >> > >> >> However, >> > > > > > > > > >> > > > >> >> > >> >> > > > all such >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > mechanisms will >> > > increase >> > > > > > > > > complexity. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Using a static >> > > secret >> > > > > key >> > > > > > > > from >> > > > > > > > > >> config >> > > > > > > > > >> > > > >> file. Will >> > > > > > > > > >> > > > >> >> > >> >> require >> > > > > > > > > >> > > > >> >> > >> >> > > yet >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > another >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > config and uses a >> > > static >> > > > > > > secret >> > > > > > > > > >> key. It >> > > > > > > > > >> > > is >> > > > > > > > > >> > > > >> advised >> > > > > > > > > >> > > > >> >> > to >> > > > > > > > > >> > > > >> >> > >> >> rotate >> > > > > > > > > >> > > > >> >> > >> >> > > > secret >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > keys >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically. This >> > can >> > > > be >> > > > > > > > avoided >> > > > > > > > > >> with >> > > > > > > > > >> > > > >> controller >> > > > > > > > > >> > > > >> >> > >> >> generating >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > secretKey and >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > passing to brokers >> > > > > > > periodically. >> > > > > > > > > >> However, >> > > > > > > > > >> > > > >> this will >> > > > > > > > > >> > > > >> >> > >> >> require >> > > > > > > > > >> > > > >> >> > >> >> > > > brokers to >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maintain certain >> > > counts >> > > > of >> > > > > > > > > >> secretKeys. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 1) Have >> > > > > controller >> > > > > > > > > >> generate >> > > > > > > > > >> > > > >> delegation >> > > > > > > > > >> > > > >> >> > token. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client >> > > authenticates >> > > > > > with a >> > > > > > > > > >> broker. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a client is >> > > > > > > > authenticated, >> > > > > > > > > >> it >> > > > > > > > > >> > > will >> > > > > > > > > >> > > > >> make a >> > > > > > > > > >> > > > >> >> > broker >> > > > > > > > > >> > > > >> >> > >> >> side >> > > > > > > > > >> > > > >> >> > >> >> > > > call to >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a delegation >> > > > token. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. Broker forwards >> > the >> > > > > > request >> > > > > > > > to >> > > > > > > > > >> > > > controller. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Controller >> > > generates >> > > > a >> > > > > DT >> > > > > > > and >> > > > > > > > > >> > > broadcasts >> > > > > > > > > >> > > > >> to all >> > > > > > > > > >> > > > >> >> > >> >> brokers. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 5. Broker stores >> > this >> > > > > token >> > > > > > in >> > > > > > > > its >> > > > > > > > > >> memory >> > > > > > > > > >> > > > >> cache. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 6. Controller >> > responds >> > > > to >> > > > > > > > broker’s >> > > > > > > > > >> DT >> > > > > > > > > >> > > req. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 7. Broker returns >> > the >> > > > > token >> > > > > > to >> > > > > > > > > >> Client. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues and >> > fixes >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. We will have to >> > add >> > > > new >> > > > > > > APIs >> > > > > > > > to >> > > > > > > > > >> > > support >> > > > > > > > > >> > > > >> >> > controller >> > > > > > > > > >> > > > >> >> > >> >> pushing >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > to >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > brokers on top of >> > the >> > > > > > minimal >> > > > > > > > APIs >> > > > > > > > > >> that >> > > > > > > > > >> > > are >> > > > > > > > > >> > > > >> >> > currently >> > > > > > > > > >> > > > >> >> > >> >> > > proposed. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. We will also have >> > > to >> > > > > add >> > > > > > > APIs >> > > > > > > > > to >> > > > > > > > > >> > > support >> > > > > > > > > >> > > > >> the >> > > > > > > > > >> > > > >> >> > >> >> bootstrapping >> > > > > > > > > >> > > > >> >> > >> >> > > > case, >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > i.e, >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > when a new broker >> > > comes >> > > > up >> > > > > > it >> > > > > > > > will >> > > > > > > > > >> have >> > > > > > > > > >> > > to >> > > > > > > > > >> > > > >> get all >> > > > > > > > > >> > > > >> >> > >> >> delegation >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > from >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > the controller. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. In catastrophic >> > > > > failures >> > > > > > > > where >> > > > > > > > > >> all >> > > > > > > > > >> > > > brokers >> > > > > > > > > >> > > > >> go >> > > > > > > > > >> > > > >> >> > down, >> > > > > > > > > >> > > > >> >> > >> >> the >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even if >> > > servers >> > > > > are >> > > > > > > > > >> restarted as >> > > > > > > > > >> > > > >> tokens >> > > > > > > > > >> > > > >> >> > are not >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this happens, >> > then >> > > > > there >> > > > > > > are >> > > > > > > > > more >> > > > > > > > > >> > > > important >> > > > > > > > > >> > > > >> >> > things to >> > > > > > > > > >> > > > >> >> > >> >> > > worry >> > > > > > > > > >> > > > >> >> > >> >> > > > about >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is better >> > to >> > > > > > > > > >> re-authenticate. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > (Alternative 2) Do not >> > > > > > distribute >> > > > > > > > DT >> > > > > > > > > to >> > > > > > > > > >> > > other >> > > > > > > > > >> > > > >> brokers >> > > > > > > > > >> > > > >> >> > at >> > > > > > > > > >> > > > >> >> > >> >> all. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. Client >> > > authenticates >> > > > > > with a >> > > > > > > > > >> broker. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. Once a client is >> > > > > > > > authenticated, >> > > > > > > > > >> it >> > > > > > > > > >> > > will >> > > > > > > > > >> > > > >> make a >> > > > > > > > > >> > > > >> >> > broker >> > > > > > > > > >> > > > >> >> > >> >> side >> > > > > > > > > >> > > > >> >> > >> >> > > > call to >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > issue a delegation >> > > > token. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 3. The broker >> > > generates >> > > > DT >> > > > > > of >> > > > > > > > > form, >> > > > > > > > > >> > > [hmac + >> > > > > > > > > >> > > > >> (owner, >> > > > > > > > > >> > > > >> >> > >> >> renewer, >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maxLifeTime, id, >> > hmac, >> > > > > > > > > >> expirationTime)] >> > > > > > > > > >> > > and >> > > > > > > > > >> > > > >> passes >> > > > > > > > > >> > > > >> >> > back >> > > > > > > > > >> > > > >> >> > >> >> this >> > > > > > > > > >> > > > >> >> > >> >> > > > DT to >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > client. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > hmac is generated >> > via >> > > > > > > > > >> {HMAC-SHA256(owner, >> > > > > > > > > >> > > > >> renewer, >> > > > > > > > > >> > > > >> >> > >> >> > > > maxLifeTime, id, >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac, >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > expirationTime) >> > using >> > > > > > > > SecretKey}. >> > > > > > > > > >> Note >> > > > > > > > > >> > > that >> > > > > > > > > >> > > > >> all >> > > > > > > > > >> > > > >> >> > brokers >> > > > > > > > > >> > > > >> >> > >> >> have >> > > > > > > > > >> > > > >> >> > >> >> > > > this >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > SecretKey. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 4. Client then goes >> > to >> > > > any >> > > > > > > > broker >> > > > > > > > > >> and to >> > > > > > > > > >> > > > >> >> > authenticate >> > > > > > > > > >> > > > >> >> > >> >> sends >> > > > > > > > > >> > > > >> >> > >> >> > > > the DT. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Broker recalculates >> > > hmac >> > > > > > using >> > > > > > > > > >> (owner, >> > > > > > > > > >> > > > >> renewer, >> > > > > > > > > >> > > > >> >> > >> >> maxLifeTime, >> > > > > > > > > >> > > > >> >> > >> >> > > > id, hmac, >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > expirationTime) info >> > > > from >> > > > > DT >> > > > > > > and >> > > > > > > > > its >> > > > > > > > > >> > > > >> SecretKey. If >> > > > > > > > > >> > > > >> >> > it >> > > > > > > > > >> > > > >> >> > >> >> matches >> > > > > > > > > >> > > > >> >> > >> >> > > > with >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > hmac of >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > DT, client is >> > > > > authenticated. >> > > > > > > > Yes, >> > > > > > > > > >> it will >> > > > > > > > > >> > > > do >> > > > > > > > > >> > > > >> other >> > > > > > > > > >> > > > >> >> > >> >> obvious >> > > > > > > > > >> > > > >> >> > >> >> > > > checks of >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > timestamp expiry and >> > > > such. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Note that secret key >> > will >> > > > be >> > > > > > > > > generated >> > > > > > > > > >> by >> > > > > > > > > >> > > > >> controller >> > > > > > > > > >> > > > >> >> > and >> > > > > > > > > >> > > > >> >> > >> >> passed >> > > > > > > > > >> > > > >> >> > >> >> > > to >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > brokers >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > periodically. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Probable issues and >> > fixes >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 1. How to delete a >> > DT? >> > > > > Yes, >> > > > > > > that >> > > > > > > > > is >> > > > > > > > > >> a >> > > > > > > > > >> > > > downside >> > > > > > > > > >> > > > >> >> > here. >> > > > > > > > > >> > > > >> >> > >> >> However, >> > > > > > > > > >> > > > >> >> > >> >> > > > this can >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be handled with >> > > brokers >> > > > > > > > > maintaining >> > > > > > > > > >> a >> > > > > > > > > >> > > > >> blacklist of >> > > > > > > > > >> > > > >> >> > DTs, >> > > > > > > > > >> > > > >> >> > >> >> DTs >> > > > > > > > > >> > > > >> >> > >> >> > > > from this >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > list >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > can be removed after >> > > > > expiry. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > 2. In catastrophic >> > > > > failures >> > > > > > > > where >> > > > > > > > > >> all >> > > > > > > > > >> > > > brokers >> > > > > > > > > >> > > > >> go >> > > > > > > > > >> > > > >> >> > down, >> > > > > > > > > >> > > > >> >> > >> >> the >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens will >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > be lost even if >> > > servers >> > > > > are >> > > > > > > > > >> restarted as >> > > > > > > > > >> > > > >> tokens >> > > > > > > > > >> > > > >> >> > are not >> > > > > > > > > >> > > > >> >> > >> >> > > > persisted >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > anywhere. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > If this happens, >> > then >> > > > > there >> > > > > > > are >> > > > > > > > > more >> > > > > > > > > >> > > > important >> > > > > > > > > >> > > > >> >> > things to >> > > > > > > > > >> > > > >> >> > >> >> > > worry >> > > > > > > > > >> > > > >> >> > >> >> > > > about >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > and >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > maybe it is better >> > to >> > > > > > > > > >> re-authenticate. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > On Fri, Feb 26, 2016 at >> > > > 1:58 >> > > > > > PM, >> > > > > > > > > Parth >> > > > > > > > > >> > > > >> Brahmbhatt < >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > pbrahmbh...@hortonworks.com> >> > > > > > > > wrote: >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Hi, >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> I have filed KIP-48 so >> > > we >> > > > > can >> > > > > > > > offer >> > > > > > > > > >> hadoop >> > > > > > > > > >> > > > like >> > > > > > > > > >> > > > >> >> > delegation >> > > > > > > > > >> > > > >> >> > >> >> > > > tokens in >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> kafka. You can review >> > > the >> > > > > > design >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> >> > > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> > > > > > > > > >> > > >> > > > > > > > > >> >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > . >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> This KIP depends on >> > > KIP-43 >> > > > > and >> > > > > > > we >> > > > > > > > > >> have also >> > > > > > > > > >> > > > >> >> > discussed an >> > > > > > > > > >> > > > >> >> > >> >> > > > alternative to >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> proposed design here< >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> >> > > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> > > > > > > > > >> > > >> > > > > > > > > >> >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > https://issues.apache.org/jira/browse/KAFKA-1696?focusedCommentId=15167800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167800 >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >. >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Thanks >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> Parth >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > -- >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Regards, >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > > Ashish >> > > > > > > > > >> > > > >> >> > >> >> > > > > >> > >> > > > > > > > > >> > > > >> >> > >> >> > > > >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > -- >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> > > Regards, >> > > > > > > > > >> > > > >> >> > >> >> > > Ashish >> > > > > > > > > >> > > > >> >> > >> >> > > >> > > > > > > > > >> > > > >> >> > >> >> >> > > > > > > > > >> > > > >> >> > >> > > > > > > > > >> > > > >> >> > > > > > > > > >> > > > >> > > > > > > > > >> > > >> > > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > -- >> > > > > > Regards, >> > > > > > >> > > > > > Rajini >> > > > > > >> > > > > >> > > > >> > > > >> > > > >> > > > -- >> > > > Regards, >> > > > >> > > > Rajini >> > > > >> > > >> > >> >> >> >> -- >> Liquan Pei >> Software Engineer, Confluent Inc
