Hi Rajini, The use case you outline is indeed the one I was thinking of. The concern is indeed what you pointed out, it could be a large change.
Ismael On Thu, Dec 15, 2016 at 3:34 AM, Rajini Sivaram <rsiva...@pivotal.io> wrote: > @Mani > > Can you add a sample Jaas configuration using delegation tokens to the KIP? > Since delegation tokens will be handled differently from other SCRAM > credentials, it should work anyway, but it will be good to see an example > of the configuration the user provides. It sounds like users provide both > tokenID and delegation token, but I wasn't sure. > > @Gwen, @Ismael, @Harsha, @Mani > > To make sure I have understood correctly, KAFKA-3712 is aimed at enabling a > superuser to impersonate another (single) user, say alice. A producer using > impersonation will authenticate with superuser credentials. All requests > from the producer will be run with the principal alice. But alice is not > involved in the authentication and alice's credentials are not actually > provided to the broker? > > The use case I was thinking of was the other one on Mani's list above. I > want to make sure that my understanding matches Gwen's and Ismael's for > this one. Using REST proxy as an example, users sending requests to the > REST proxy provide a delegation token as an auth token. REST proxy itself > authenticates as a different user, perhaps with access to read metadata of > all topics. But Kafka requests corresponding to each REST request should be > authenticated based on the authentication token provided in the request. > At the moment, the REST proxy has to create producers for each user (or > perform authentication and authorization for the user). Instead we want to > create a single producer which has the ability to send requests on behalf > of multiple users where the user is authenticated by Kafka and each request > from the producer is authorized based on the user who initiated the > request. > > 1a) I think Gwen's suggestion was something along the lines of: > producer.send(record1, delegationToken1); > producer.send(record2, delegationToken2); > 1b) I was thinking more along the lines of: > subject1 = producer.authenticate(user1Credentials); > subject2 = producer.authenticate(user2Credentials); > .... > Subject.doAs(subject1, (PrivilegedExceptionAction<Future<RecordMetadata>>) > () -> producer.send(record1)); > Subject.doAs(subject2, (PrivilegedExceptionAction<Future<RecordMetadata>>) > () -> producer.send(record2)); > > To make either approach work, each request needs to be associated with a > user. This would be delegation token in 1a) and user principal in 1b). And > both need to ensure that producers dont send records from multiple users in > one request. And if producers hold one metadata rather than one per-user, > calls like partitionsFor() shouldn't return metadata for unauthorized > topics. It is a very big change, so it will be good to know whether there > is a real requirement to support this. > > > On Thu, Dec 15, 2016 at 9:04 AM, Manikumar <manikumar.re...@gmail.com> > wrote: > > > @Gwen, @Rajini, > > > > As mentioned in the KIP, main motivation for this KIP is to reduce load > on > > Kerberos > > server on large kafka deployments with large number of clients. > > > > Also it looks like we are combining two overlapping concepts > > 1. Single client sending requests with multiple users/authentications > > 2. Impersonation > > > > Option 1, is definitely useful in some use cases and can be used to > > implement workaround for > > impersonation > > > > In Impersonation, a super user can send requests on behalf of another > > user(Alice) in a secured way. > > superuser has credentials but user Alice doesn't have any. The requests > are > > required > > to run as user Alice and accesses/ACLs on Broker are required to be done > as > > user Alice. > > It is required that user Alice can connect to the Broker on a connection > > authenticated with > > superuser's credentials. In other words superuser is impersonating the > user > > Alice. > > > > The approach mentioned by Harsha in previous mail is implemented in > hadoop, > > storm etc.. > > > > Some more details here: > > https://hadoop.apache.org/docs/r2.7.2/hadoop-project- > > dist/hadoop-common/Superusers.html > > > > > > @Rajini > > > > Thanks for your comments on SASL/SCRAM usage. I am thinking to send > > tokenHmac (salted-hashed version) > > as password for authentication and tokenID for retrial of tokenHmac at > > server side. > > Does above sound OK? > > > > > > Thanks, > > Manikumar > > > > On Wed, Dec 14, 2016 at 10:33 PM, Harsha Chintalapani <ka...@harsha.io> > > wrote: > > > > > @Gwen @Mani Not sure why we want to authenticate at every request. > Even > > if > > > the token exchange is cheap it still a few calls that need to go > through > > > round trip. Impersonation doesn't require authentication for every > > > request. > > > > > > "So a centralized app can create few producers, do the metadata request > > and > > > broker discovery with its own user auth, but then use delegation tokens > > to > > > allow performing produce/fetch requests as different users? Instead of > > > having to re-connect for each impersonated user?" > > > > > > Yes. But what we will have is this centralized user as impersonation > user > > > on behalf of other users. When it authenticates initially we will > create > > a > > > "Subject" and from there on wards centralized user can do > > > Subject.doAsPrivileged > > > on behalf, other users. > > > On the server side, we can retrieve two principals out of this one is > the > > > authenticated user (centralized user) and another is impersonated user. > > We > > > will first check if the authenticated user allowed to impersonate and > > then > > > move on to check if the user Alice has access to the topic "X" to > > > read/write. > > > > > > @Rajini Intention of this KIP is to support token auth via SASL/SCRAM, > > not > > > just with TLS. What you raised is a good point let me take a look and > > add > > > details. > > > > > > It will be easier to add impersonation once we reach agreement on this > > KIP. > > > > > > > > > On Wed, Dec 14, 2016 at 5:51 AM Ismael Juma <ism...@juma.me.uk> wrote: > > > > > > > Hi Rajini, > > > > > > > > I think it would definitely be valuable to have a KIP for > > impersonation. > > > > > > > > Ismael > > > > > > > > On Wed, Dec 14, 2016 at 4:03 AM, Rajini Sivaram <rsiva...@pivotal.io > > > > > > wrote: > > > > > > > > > It would clearly be very useful to enable clients to send requests > on > > > > > behalf of multiple users. A separate KIP makes sense, but it may be > > > worth > > > > > thinking through some of the implications now, especially if the > main > > > > > interest in delegation tokens comes from its potential to enable > > > > > impersonation. > > > > > > > > > > I understand that delegation tokens are only expected to be used > with > > > > TLS. > > > > > But the choice of SASL/SCRAM for authentication must be based on a > > > > > requirement to protect the tokenHmac - otherwise you could just use > > > > > SASL/PLAIN. With SASL/SCRAM the tokenHmac is never propagated > > > > on-the-wire, > > > > > only a salted-hashed version of it is used in the SASL > authentication > > > > > exchange. If impersonation is based on sending tokenHmac in > requests, > > > any > > > > > benefit of using SCRAM is lost. > > > > > > > > > > An alternative may be to allow clients to authenticate multiple > times > > > > using > > > > > SASL and include one of its authenticated principals in each > request > > > > > (optionally). I haven't thought it through yet, obviously. But if > the > > > > > approach is of interest and no one is working on a KIP for > > > impersonation > > > > at > > > > > the moment, I am happy to write one. It may provide something for > > > > > comparison at least. > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > On Wed, Dec 14, 2016 at 9:53 AM, Manikumar < > > manikumar.re...@gmail.com> > > > > > wrote: > > > > > > > > > > > That's a good idea. Authenticating every request with delegation > > > token > > > > > will > > > > > > be useful for > > > > > > impersonation use-cases. But as of now, we are thinking > delegation > > > > token > > > > > as > > > > > > just another way > > > > > > to authenticate the users. We haven't think through all the use > > cases > > > > > > related to > > > > > > impersonation or using delegation token for impersonation. We > want > > to > > > > > > handle impersonation > > > > > > (KAFKA-3712) as part of separate KIP. > > > > > > > > > > > > Will that be Ok? > > > > > > > > > > > > > > > > > > On Wed, Dec 14, 2016 at 8:09 AM, Gwen Shapira <g...@confluent.io > > > > > > wrote: > > > > > > > > > > > > > Thinking out loud here: > > > > > > > > > > > > > > It looks like authentication with a delegation token is going > to > > be > > > > > > > super-cheap, right? We just compare the token to a value in the > > > > broker > > > > > > > cache? > > > > > > > > > > > > > > If I understood the KIP correctly, right now it suggests that > > > > > > > authentication happens when establishing the client-broker > > > connection > > > > > (as > > > > > > > normal for Kafka. But perhaps we want to consider > authenticating > > > > every > > > > > > > request with delegation token (if exists)? > > > > > > > > > > > > > > So a centralized app can create few producers, do the metadata > > > > request > > > > > > and > > > > > > > broker discovery with its own user auth, but then use > delegation > > > > tokens > > > > > > to > > > > > > > allow performing produce/fetch requests as different users? > > Instead > > > > of > > > > > > > having to re-connect for each impersonated user? > > > > > > > > > > > > > > This may over-complicate things quite a bit (basically adding > > extra > > > > > > > information in every request), but maybe it will be useful for > > > > > > > impersonation use-cases (which seem to drive much of the > interest > > > in > > > > > this > > > > > > > KIP)? > > > > > > > Kafka Connect, NiFi and friends can probably use this to share > > > > clients > > > > > > > between multiple jobs, tasks, etc. > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > Gwen > > > > > > > > > > > > > > On Tue, Dec 13, 2016 at 12:43 AM, Manikumar < > > > > manikumar.re...@gmail.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Ashish, > > > > > > > > > > > > > > > > Thank you for reviewing the KIP. Please see the replies > > inline. > > > > > > > > > > > > > > > > > > > > > > > > > 1. How to disable delegation token authentication? > > > > > > > > > > > > > > > > > > This can be achieved in various ways, however I think > reusing > > > > > > > delegation > > > > > > > > > token secret config for this makes sense here. Avoids > > creating > > > > yet > > > > > > > > another > > > > > > > > > config and forces delegation token users to consciously set > > the > > > > > > secret. > > > > > > > > If > > > > > > > > > the secret is not set or set to empty string, brokers > should > > > turn > > > > > off > > > > > > > > > delegation token support. This will however require a new > > error > > > > > code > > > > > > to > > > > > > > > > indicate delegation token support is turned off on broker. > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the suggestion. Option to turnoff delegation > token > > > > > > > > authentication will be useful. > > > > > > > > I'll update the KIP. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2. ACLs on delegation token? > > > > > > > > > > > > > > > > > > Do we need to have ACLs defined for tokens? I do not think > it > > > > buys > > > > > us > > > > > > > > > anything, as delegation token can be treated as > impersonation > > > of > > > > > the > > > > > > > > owner. > > > > > > > > > Any thing the owner has permission to do, delegation tokens > > > > should > > > > > be > > > > > > > > > allowed to do as well. If so, we probably won't need to > > return > > > > > > > > > authorization exception error code while creating > delegation > > > > token. > > > > > > It > > > > > > > > > however would make sense to check renew and expire requests > > are > > > > > > coming > > > > > > > > from > > > > > > > > > owner or renewers of the token, but that does not require > > > > explicit > > > > > > > acls. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, We agreed to not have new acl on who can request > > delegation > > > > > token. > > > > > > > > I'll update the KIP. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 3. How to restrict max life time of a token? > > > > > > > > > > > > > > > > > > Admins might want to restrict max life time of tokens > created > > > on > > > > a > > > > > > > > cluster, > > > > > > > > > and this can very from cluster to cluster based on > use-cases. > > > > This > > > > > > > might > > > > > > > > > warrant a separate broker config. > > > > > > > > > > > > > > > > > > > > > > > > > > Currently we have "delegation.token.max.lifetime.sec" > server > > > > config > > > > > > > > property > > > > > > > > May be we can take min(User supplied MaxTime, Server MaxTime) > > as > > > > max > > > > > > life > > > > > > > > time. > > > > > > > > I am open to add new config property. > > > > > > > > > > > > > > > > Few more comments based on recent KIP update. > > > > > > > > > > > > > > > > > > 1. Do we need a separate {{InvalidateTokenRequest}}? Can't > we > > > use > > > > > > > > > {{ExpireTokenRequest}} with with expiryDate set to anything > > > > before > > > > > > > > current > > > > > > > > > date? > > > > > > > > > > > > > > > > > > > > > > > > > makes sense. we don't need special request to cancel the > token. > > > We > > > > > can > > > > > > > use > > > > > > > > ExpireTokenRequest. > > > > > > > > I'll update the KIP. > > > > > > > > > > > > > > > > > > > > > > > > > 2. Can we change time field names to indicate their unit is > > > > > > > milliseconds, > > > > > > > > > like, IssueDateMs, ExpiryDateMs, etc.? > > > > > > > > > > > > > > > > > > > > > > > > > > Done. > > > > > > > > > > > > > > > > > > > > > > > > > 3. Can we allow users to renew a token for a specified > amount > > > of > > > > > > time? > > > > > > > In > > > > > > > > > current version of KIP, renew request does not take time > as a > > > > > param, > > > > > > > not > > > > > > > > > sure what is expiry time set to after renewal. > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, we need to specify renew period. I'll update the KIP. > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Mankumar > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Dec 12, 2016 at 9:08 AM Manikumar < > > > > > manikumar.re...@gmail.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I would like to reinitiate the discussion on Delegation > > token > > > > > > support > > > > > > > > for > > > > > > > > > > > > > > > > > > > > Kafka. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Brief summary of the past discussion: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 1) Broker stores delegation tokens in zookeeper. All > > brokers > > > > > will > > > > > > > > have a > > > > > > > > > > > > > > > > > > > > cache backed by > > > > > > > > > > > > > > > > > > > > zookeeper so they will all get notified whenever a new > > > token > > > > > is > > > > > > > > > > > > > > > > > > > > generated and they will > > > > > > > > > > > > > > > > > > > > update their local cache whenever token state changes. > > > > > > > > > > > > > > > > > > > > 2) The current proposal does not support rotation of > secret > > > > > > > > > > > > > > > > > > > > 3) Only allow the renewal by users that authenticated > using > > > > *non* > > > > > > > > > > > > > > > > > > > > delegation token mechanism > > > > > > > > > > > > > > > > > > > > 4) KIP-84 proposes to support SASL SCRAM mechanisms. > Kafka > > > > > clients > > > > > > > can > > > > > > > > > > > > > > > > > > > > authenticate using > > > > > > > > > > > > > > > > > > > > SCRAM-SHA-256, providing the delegation token HMAC as > > > > > password. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Updated the KIP with the following: > > > > > > > > > > > > > > > > > > > > 1. Protocol and Config changes > > > > > > > > > > > > > > > > > > > > 2. format of the data stored in ZK. > > > > > > > > > > > > > > > > > > > > 3. Changes to Java Clients/Usage of SASL SCRAM mechanism > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > > > > > > > > > > > > > > > > > > > 48+Delegation+token+support+for+Kafka > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Jun, Ashish, Gwen, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Pl review the updated KIP. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > Manikumar > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Sep 29, 2016 at 9:56 PM, Ashish Singh < > > > > > asi...@cloudera.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Harsha/ Gwen, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > How do we proceed here? I am willing to help out with > > here. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Sep 23, 2016 at 11:41 AM, Gwen Shapira < > > > > > > g...@confluent.io> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Is it updated? are all concerns addressed? do you > want > > to > > > > > > start a > > > > > > > > > vote? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sorry for being pushy, I do appreciate that we are > all > > > > > > volunteers > > > > > > > > and > > > > > > > > > > > > > > > > > > > > > > finding time is difficult. This feature is important > > for > > > > > > anything > > > > > > > > > that > > > > > > > > > > > > > > > > > > > > > > integrates with Kafka (stream processors, Flume, > NiFi, > > > etc) > > > > > > and I > > > > > > > > > > > > > > > > > > > > > > don't want to see this getting stuck because we lack > > > > > > coordination > > > > > > > > > > > > > > > > > > > > > > within the community. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Sep 15, 2016 at 6:39 PM, Harsha Chintalapani > < > > > > > > > > > ka...@harsha.io> > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > The only pending update for the KIP is to write up > > the > > > > > > protocol > > > > > > > > > > changes > > > > > > > > > > > > > > > > > > > > > > like > > > > > > > > > > > > > > > > > > > > > > > we've it KIP-4. I'll update the wiki. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Sep 15, 2016 at 4:27 PM Ashish Singh < > > > > > > > > asi...@cloudera.com> > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > >> I think we decided to not support secret > rotation, I > > > > guess > > > > > > > this > > > > > > > > > can > > > > > > > > > > be > > > > > > > > > > > > > > > > > > > > > > >> stated clearly on the KIP. Also, more details on > how > > > > > clients > > > > > > > > will > > > > > > > > > > > > > > > > > > > > > > perform > > > > > > > > > > > > > > > > > > > > > > >> token distribution and how CLI will look like will > > be > > > > > > helpful. > > > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > >> On Thu, Sep 15, 2016 at 3:20 PM, Gwen Shapira < > > > > > > > > g...@confluent.io> > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > >> > Hi Guys, > > > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > >> > This discussion was dead for a while. Are there > > > still > > > > > > > > > contentious > > > > > > > > > > > > > > > > > > > > > > >> > points? If not, why are there no votes? > > > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > >> > On Tue, Aug 23, 2016 at 1:26 PM, Jun Rao < > > > > > > j...@confluent.io> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > >> > > Ashish, > > > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > >> > > Yes, I will send out a KIP invite for next > week > > to > > > > > > discuss > > > > > > > > > > KIP-48 > > > > > > > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > > > > > > >> > other > > > > > > > > > > > > > > > > > > > > > > >> > > remaining KIPs. > > > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > >> > > Thanks, > > > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > >> > > Jun > > > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > >> > > On Tue, Aug 23, 2016 at 1:22 PM, Ashish Singh > < > > > > > > > > > > > > > > > > > > > > > asi...@cloudera.com> > > > > > > > > > > > > > > > > > > > > > > >> > wrote: > > > > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > >> > >> Thanks Harsha! > > > > > > > > > > > > > > > > > > > > > > >> > >> > > > > > > > > > > > > > > > > > > > > > > >> > >> Jun, can we add KIP-48 to next KIP hangout's > > > > agenda. > > > > > > > Also, > > > > > > > > we > > > > > > > > > > did > > > > > > > > > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > > > > > > > > >> > >> actually make a call on when we should have > > next > > > > KIP > > > > > > > call. > > > > > > > > As > > > > > > > > > > > > > > > > > > > > > there > > > > > > > > > > > > > > > > > > > > > > >> > >> are > > > > > > > > > > > > > > > > > > > > > > >> > a > > > > > > > > > > > > > > > > > > > > > > >> > >> few outstanding KIPs that could not be > > discussed > > > > this > > > > > > > week, > > > > > > > > > can > > > > > > > > > > > > > > > > > > > > > we > > > > > > > > > > > > > > > > > > > > > > >> > >> have > > > > > > > > > > > > > > > > > > > > > > >> > a > > > > > > > > > > > > > > > > > > > > > > >> > >> KIP hangout call next week? > > > > > > > > > > > > > > > > > > > > > > >> > >> > > > > > > > > > > > > > > > > > > > > > > >> > >> On Tue, Aug 23, 2016 at 1:10 PM, Harsha > > > > Chintalapani > > > > > > > > > > > > > > > > > > > > > > >> > >> <ka...@harsha.io> > > > > > > > > > > > > > > > > > > > > > > >> > >> wrote: > > > > > > > > > > > > > > > > > > > > > > >> > >> > > > > > > > > > > > > > > > > > > > > > > >> > >>> Ashish, > > > > > > > > > > > > > > > > > > > > > > >> > >>> Yes we are working on it. Lets > discuss > > > in > > > > > the > > > > > > > next > > > > > > > > > KIP > > > > > > > > > > > > > > > > > > > > > > >> > >>> meeting. > > > > > > > > > > > > > > > > > > > > > > >> > >>> I'll join. > > > > > > > > > > > > > > > > > > > > > > >> > >>> -Harsha > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > >> > >>> On Tue, Aug 23, 2016 at 12:07 PM Ashish > Singh > > < > > > > > > > > > > > > > > > > > > > > > > asi...@cloudera.com> > > > > > > > > > > > > > > > > > > > > > > >> > >>> wrote: > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > >> > >>> > Hello Harsha, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > Are you still working on this? Wondering > if > > we > > > > can > > > > > > > > discuss > > > > > > > > > > > > > > > > > > > > > this > > > > > > > > > > > > > > > > > > > > > > in > > > > > > > > > > > > > > > > > > > > > > >> > next > > > > > > > > > > > > > > > > > > > > > > >> > >>> KIP > > > > > > > > > > > > > > > > > > > > > > >> > >>> > meeting, if you can join. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > On Mon, Jul 18, 2016 at 9:51 AM, Harsha > > > > > > Chintalapani < > > > > > > > > > > > > > > > > > > > > > > >> > ka...@harsha.io> > > > > > > > > > > > > > > > > > > > > > > >> > >>> > wrote: > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > Hi Grant, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > We are working on it. Will add > > the > > > > > > details > > > > > > > > to > > > > > > > > > > KIP > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > about > > > > > > > > > > > > > > > > > > > > > > >> > the > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > request protocol. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > Thanks, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > Harsha > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > On Mon, Jul 18, 2016 at 6:50 AM Grant > > Henke > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > <ghe...@cloudera.com> > > > > > > > > > > > > > > > > > > > > > > >> > >>> wrote: > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > Hi Parth, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > Are you still working on this? If you > > need > > > > any > > > > > > > help > > > > > > > > > > please > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > don't > > > > > > > > > > > > > > > > > > > > > > >> > >>> > hesitate > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > to ask. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > Grant > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > On Thu, Jun 30, 2016 at 4:35 PM, Jun > > Rao < > > > > > > > > > > > > > > > > > > > > > j...@confluent.io> > > > > > > > > > > > > > > > > > > > > > > >> > wrote: > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > Parth, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > Thanks for the reply. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > It makes sense to only allow the > > renewal > > > > by > > > > > > > users > > > > > > > > > that > > > > > > > > > > > > > > > > > > > > > > >> > >>> authenticated > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > using > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > *non* delegation token mechanism. > > Then, > > > > > should > > > > > > > we > > > > > > > > > make > > > > > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > >> > >>> renewal a > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > list? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > For example, in the case of rest > > proxy, > > > it > > > > > > will > > > > > > > be > > > > > > > > > > > > > > > > > > > > > useful > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > for > > > > > > > > > > > > > > > > > > > > > > >> > >>> every > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > instance of rest proxy to be able to > > > renew > > > > > the > > > > > > > > > tokens. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > It would be clearer if we can > document > > > the > > > > > > > request > > > > > > > > > > > > > > > > > > > > > > protocol > > > > > > > > > > > > > > > > > > > > > > >> > like > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl > > > > > > > > > uence/display/KAFKA/KIP- > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > 4+-+Command+line+and+ > > > > > centralized+administrative+ > > > > > > > > > > > > > > > > > > > > > > operations#KIP-4- > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > Commandlineandcentralizedadmin > > > > > > istrativeoperations- > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > CreateTopicsRequest(KAFKA- > > > > > > > > 2945):(VotedandPlannedforin0. > > > > > > > > > > > > > > > > > > > > > > 10.1.0) > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > . > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > It would also be useful to document > > the > > > > > client > > > > > > > > APIs. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > Jun > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > On Tue, Jun 28, 2016 at 2:55 PM, > parth > > > > > > > brahmbhatt > > > > > > > > < > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > brahmbhatt.pa...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > I am suggesting that we will only > > > allow > > > > > the > > > > > > > > > renewal > > > > > > > > > > by > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > users > > > > > > > > > > > > > > > > > > > > > > >> > >>> that > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > authenticated using *non* > delegation > > > > token > > > > > > > > > > mechanism. > > > > > > > > > > > > > > > > > > > > > > For > > > > > > > > > > > > > > > > > > > > > > >> > >>> example, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > If > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > user > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > Alice authenticated using kerberos > > and > > > > > > > requested > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > delegation > > > > > > > > > > > > > > > > > > > > > > >> > >>> tokens, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > only > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > user Alice authenticated via non > > > > > delegation > > > > > > > > token > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > mechanism > > > > > > > > > > > > > > > > > > > > > > >> > can > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > renew. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > Clients that have access to > > > delegation > > > > > > tokens > > > > > > > > can > > > > > > > > > > not > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > issue > > > > > > > > > > > > > > > > > > > > > > >> > >>> > renewal > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > request for renewing their own > token > > > and > > > > > > this > > > > > > > is > > > > > > > > > > > > > > > > > > > > > > primarily > > > > > > > > > > > > > > > > > > > > > > >> > >>> > important > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > to > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > reduce the time window for which a > > > > > > compromised > > > > > > > > > token > > > > > > > > > > > > > > > > > > > > > > will > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > be > > > > > > > > > > > > > > > > > > > > > > >> > >>> valid. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > To clarify, Yes any authenticated > > user > > > > can > > > > > > > > request > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > delegation > > > > > > > > > > > > > > > > > > > > > > >> > >>> > tokens > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > but > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > even here I would recommend to > avoid > > > > > > creating > > > > > > > a > > > > > > > > > > chain > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > where a > > > > > > > > > > > > > > > > > > > > > > >> > >>> > client > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > authenticated via delegation token > > > > request > > > > > > for > > > > > > > > > more > > > > > > > > > > > > > > > > > > > > > > >> > delegation > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > tokens. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > Basically anyone can request > > > delegation > > > > > > token, > > > > > > > > as > > > > > > > > > > long > > > > > > > > > > > > > > > > > > > > > > as > > > > > > > > > > > > > > > > > > > > > > >> > they > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > authenticate > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > via a non delegation token > > mechanism. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > Aren't classes listed here > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > < > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > https://cwiki.apache.org/confl > > > > > > > > > uence/display/KAFKA/KIP- > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > 48+Delegation+token+support+fo > > > > > > > > > > > > > > > > > > > > > r+Kafka#KIP-48Delegationtokens > > > > > > > > > > > > > > > > > > > > > > >> > >>> upportforKaf > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > ka-PublicInterfaces > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > sufficient? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > Parth > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > On Tue, Jun 21, 2016 at 4:33 PM, > Jun > > > Rao > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > <j...@confluent.io> > > > > > > > > > > > > > > > > > > > > > > >> > >>> wrote: > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > Parth, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > Thanks for the reply. A couple > of > > > > > comments > > > > > > > > > inline > > > > > > > > > > > > > > > > > > > > > > below. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > On Tue, Jun 21, 2016 at 10:36 > AM, > > > > parth > > > > > > > > > > brahmbhatt < > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > brahmbhatt.pa...@gmail.com> > > wrote: > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > 1. Who / how are tokens > renewed? > > > By > > > > > > > original > > > > > > > > > > > > > > > > > > > > > > requester > > > > > > > > > > > > > > > > > > > > > > >> > >>> only? or > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > using > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > Kerberos > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > auth only? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > My recommendation is to do > this > > > only > > > > > > using > > > > > > > > > > > > > > > > > > > > > Kerberos > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > auth > > > > > > > > > > > > > > > > > > > > > > >> > and > > > > > > > > > > > > > > > > > > > > > > >> > >>> > only > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > threw > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > the > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > renewer specified during the > > > > > acquisition > > > > > > > > > > request. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > Hmm, not sure that I follow > this. > > > Are > > > > > you > > > > > > > > saying > > > > > > > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > any > > > > > > > > > > > > > > > > > > > > > > >> > >>> client > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > authenticated with the > delegation > > > > token > > > > > > can > > > > > > > > > renew, > > > > > > > > > > > > > > > > > > > > > > i.e. > > > > > > > > > > > > > > > > > > > > > > >> > there > > > > > > > > > > > > > > > > > > > > > > >> > >>> is > > > > > > > > > > > > > > > > > > > > > > >> > >>> > no > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > renewer > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > needed? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > Also, just to be clear, any > > > > > authenticated > > > > > > > > client > > > > > > > > > > > > > > > > > > > > > > (either > > > > > > > > > > > > > > > > > > > > > > >> > >>> through > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > SASL > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > or > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > SSL) can request a delegation > > token > > > > for > > > > > > the > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > authenticated > > > > > > > > > > > > > > > > > > > > > > >> > >>> user, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > right? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > 2. Are tokens stored on each > > > broker > > > > or > > > > > > in > > > > > > > > ZK? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > My recommendation is still to > > > store > > > > in > > > > > > ZK > > > > > > > or > > > > > > > > > not > > > > > > > > > > > > > > > > > > > > > > store > > > > > > > > > > > > > > > > > > > > > > >> > them > > > > > > > > > > > > > > > > > > > > > > >> > >>> at > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > all. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > The > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > whole controller based > > > distribution > > > > is > > > > > > too > > > > > > > > > much > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > overhead > > > > > > > > > > > > > > > > > > > > > > >> > >>> with > > > > > > > > > > > > > > > > > > > > > > >> > >>> > not > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > much > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > to > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > achieve. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > 3. How are tokens invalidated > / > > > > > expired? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > Either by expiration time out > or > > > > > through > > > > > > > an > > > > > > > > > > > > > > > > > > > > > explicit > > > > > > > > > > > > > > > > > > > > > > >> > >>> request to > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > invalidate. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > 4. Which encryption algorithm > is > > > > used? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > SCRAM > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > 5. What is the impersonation > > > > proposal > > > > > > (it > > > > > > > > > wasn't > > > > > > > > > > > > > > > > > > > > > in > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > >> > KIP > > > > > > > > > > > > > > > > > > > > > > >> > >>> but > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > was > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > discussed > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > in this thread)? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > There is no imperonation > > > proposal. I > > > > > > tried > > > > > > > > and > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > explained > > > > > > > > > > > > > > > > > > > > > > >> > how > > > > > > > > > > > > > > > > > > > > > > >> > >>> > its > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > a > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > different problem and why its > > not > > > > > really > > > > > > > > > > necessary > > > > > > > > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > >> > >>> discuss > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > that > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > as > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > part > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > of this KIP. This KIP will > not > > > > > support > > > > > > > any > > > > > > > > > > > > > > > > > > > > > > >> > impersonation, > > > > > > > > > > > > > > > > > > > > > > >> > >>> it > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > will > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > just > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > be > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > another way to authenticate. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > 6. Do we need new ACLs, if so > - > > > for > > > > > what > > > > > > > > > > actions? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > We do not need new ACLs. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > Could we document the format of > > the > > > > new > > > > > > > > > > > > > > > > > > > > > > request/response > > > > > > > > > > > > > > > > > > > > > > >> > and > > > > > > > > > > > > > > > > > > > > > > >> > >>> > their > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > associated Resource and > Operation > > > for > > > > > ACL? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > 7. How would the delegation > > token > > > be > > > > > > > > > configured > > > > > > > > > > in > > > > > > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > >> > >>> client? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > Should be through config. I > > wasn't > > > > > > > planning > > > > > > > > on > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > supporting > > > > > > > > > > > > > > > > > > > > > > >> > >>> JAAS > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > for > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > tokens. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > I don't believe hadoop does > this > > > > > either. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > Parth > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > On Thu, Jun 16, 2016 at 4:03 > PM, > > > Jun > > > > > > Rao < > > > > > > > > > > > > > > > > > > > > > > >> > j...@confluent.io> > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > wrote: > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > Harsha, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > Another question. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > 9. How would the delegation > > > token > > > > be > > > > > > > > > > configured > > > > > > > > > > > > > > > > > > > > > in > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > >> > >>> > client? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > The > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > standard > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > way is to do this through > > JAAS. > > > > > > However, > > > > > > > > we > > > > > > > > > > will > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > need > > > > > > > > > > > > > > > > > > > > > > >> > to > > > > > > > > > > > > > > > > > > > > > > >> > >>> > think > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > through > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > if > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > this is convenient in a > shared > > > > > > > > environment. > > > > > > > > > > For > > > > > > > > > > > > > > > > > > > > > > >> > example, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > when a > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > new > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > task > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > is > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > added to a Storm worker > node, > > do > > > > we > > > > > > need > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > dynamically > > > > > > > > > > > > > > > > > > > > > > >> > >>> add a > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > new > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > section > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > in the JAAS file? It may be > > more > > > > > > > > convenient > > > > > > > > > if > > > > > > > > > > > > > > > > > > > > > we > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > can > > > > > > > > > > > > > > > > > > > > > > >> > >>> pass in > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > the > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > token > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > through the config directly > > w/o > > > > > going > > > > > > > > > through > > > > > > > > > > > > > > > > > > > > > > JAAS. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > Are you or Parth still > > actively > > > > > > working > > > > > > > on > > > > > > > > > > this > > > > > > > > > > > > > > > > > > > > > > KIP? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > On Sun, Jun 12, 2016 at 2:18 > > PM, > > > > Jun > > > > > > > Rao < > > > > > > > > > > > > > > > > > > > > > > >> > >>> j...@confluent.io> > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > wrote: > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > Just to add on that list. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > 2. It would be good to > > > document > > > > > the > > > > > > > > format > > > > > > > > > > of > > > > > > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > >> > data > > > > > > > > > > > > > > > > > > > > > > >> > >>> > stored > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > in > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > ZK. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > 7. Earlier, there was a > > > > discussion > > > > > > on > > > > > > > > > > whether > > > > > > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > >> > tokens > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > should > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > be > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > propagated through ZK like > > > > > > > > > config/acl/quota, > > > > > > > > > > > > > > > > > > > > > or > > > > > > > > > > > > > > > > > > > > > > >> > through > > > > > > > > > > > > > > > > > > > > > > >> > >>> the > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > controller. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > Currently, the controller > is > > > > only > > > > > > > > designed > > > > > > > > > > for > > > > > > > > > > > > > > > > > > > > > > >> > >>> propagating > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > topic > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > metadata, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > but not other data. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > 8. Should we use SCRAM to > > send > > > > the > > > > > > > token > > > > > > > > > > > > > > > > > > > > > instead > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > of > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > DIGEST-MD5 > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > since > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > it's > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > deprecated? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > Also, the images in the > wiki > > > > seem > > > > > > > > broken. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > On Fri, Jun 10, 2016 at > > 10:02 > > > > AM, > > > > > > Gwen > > > > > > > > > > > > > > > > > > > > > Shapira < > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > g...@confluent.io> > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> From what I can see, > > > remaining > > > > > > > > questions > > > > > > > > > > are: > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> 1. Who / how are tokens > > > > renewed? > > > > > By > > > > > > > > > > original > > > > > > > > > > > > > > > > > > > > > > >> > requester > > > > > > > > > > > > > > > > > > > > > > >> > >>> > only? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > or > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > using > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> Kerberos auth only? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> 2. Are tokens stored on > > each > > > > > broker > > > > > > > or > > > > > > > > in > > > > > > > > > > ZK? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> 3. How are tokens > > > invalidated / > > > > > > > > expired? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> 4. Which encryption > > algorithm > > > > is > > > > > > > used? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> 5. What is the > > impersonation > > > > > > proposal > > > > > > > > (it > > > > > > > > > > > > > > > > > > > > > > wasn't > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> in > > > > > > > > > > > > > > > > > > > > > > >> > the > > > > > > > > > > > > > > > > > > > > > > >> > >>> > KIP > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > but > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > was > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> discussed in this > thread)? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> 6. Do we need new ACLs, > if > > > so - > > > > > for > > > > > > > > what > > > > > > > > > > > > > > > > > > > > > > actions? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> Gwen > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> On Thu, Jun 9, 2016 at > 7:48 > > > PM, > > > > > > > Harsha > > > > > > > > < > > > > > > > > > > > > > > > > > > > > > > >> > >>> ka...@harsha.io> > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > wrote: > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> > Jun & Ismael, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> > > > > > > > > > Unfortunately > > > > > > > > > I > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> > couldn't > > > > > > > > > > > > > > > > > > > > > > >> > >>> attend > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > the > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > KIP > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > meeting > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> > > > > when > > > > > > > > > delegation > > > > > > > > > > > > > > > > > > > > > > tokens > > > > > > > > > > > > > > > > > > > > > > >> > >>> > discussed. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > Appreciate > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > if > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> > > > > you > > > > > can > > > > > > > > update > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > >> > thread if > > > > > > > > > > > > > > > > > > > > > > >> > >>> > you > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > have > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > any > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> > > > > > > further > > > > > > > > > > questions. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> > Thanks, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> > Harsha > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> > On Tue, May 24, 2016, > at > > > > 11:32 > > > > > > AM, > > > > > > > > > Liquan > > > > > > > > > > > > > > > > > > > > > Pei > > > > > > > > > > > > > > > > > > > > > > >> > wrote: > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> It seems that the > links > > to > > > > > > images > > > > > > > in > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > KIP > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> are > > > > > > > > > > > > > > > > > > > > > > >> > >>> > broken. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> Liquan > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> On Tue, May 24, 2016 > at > > > 9:33 > > > > > AM, > > > > > > > > parth > > > > > > > > > > > > > > > > > > > > > > >> > brahmbhatt < > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > brahmbhatt.pa...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > 110. What does > > > > > > > > getDelegationTokenAs > > > > > > > > > > > > > > > > > > > > > mean? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > In the current > > proposal > > > we > > > > > > only > > > > > > > > > allow > > > > > > > > > > a > > > > > > > > > > > > > > > > > > > > > > user > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > to > > > > > > > > > > > > > > > > > > > > > > >> > >>> get > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > delegation > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > token > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> for > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > the identity that it > > > > > > > authenticated > > > > > > > > > as > > > > > > > > > > > > > > > > > > > > > > using > > > > > > > > > > > > > > > > > > > > > > >> > >>> another > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > mechanism, > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > i.e. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> A user > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > that authenticate > > using > > > a > > > > > > keytab > > > > > > > > for > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > principal > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > us...@example.com > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> will get > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens > for > > > that > > > > > > user > > > > > > > > > only. > > > > > > > > > > In > > > > > > > > > > > > > > > > > > > > > > >> > future I > > > > > > > > > > > > > > > > > > > > > > >> > >>> > think > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > we > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > will > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > have > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> to > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > extend support such > > that > > > > we > > > > > > > allow > > > > > > > > > some > > > > > > > > > > > > > > > > > > > > > set > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > of > > > > > > > > > > > > > > > > > > > > > > >> > >>> users ( > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > kafka-rest-u...@example.com > > > > > , > > > > > > > > > > > > > > > > > > > > > > >> > >>> > storm-nim...@example.com) > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > to > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > acquire > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > delegation tokens on > > > > behalf > > > > > of > > > > > > > > other > > > > > > > > > > > > > > > > > > > > > users > > > > > > > > > > > > > > > > > > > > > > >> > whose > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > identity > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > they > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > have > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > verified > > independently. > > > > > Kafka > > > > > > > > > brokers > > > > > > > > > > > > > > > > > > > > > > will > > > > > > > > > > > > > > > > > > > > > > >> > have > > > > > > > > > > > > > > > > > > > > > > >> > >>> ACLs > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > to > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > control > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> which > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > users are allowed to > > > > > > impersonate > > > > > > > > > other > > > > > > > > > > > > > > > > > > > > > > users > > > > > > > > > > > > > > > > > > > > > > >> > and > > > > > > > > > > > > > > > > > > > > > > >> > >>> get > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > tokens > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > on > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> behalf of > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > them. Overall > > > > Impersonation > > > > > > is a > > > > > > > > > whole > > > > > > > > > > > > > > > > > > > > > > >> > different > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > problem > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > in > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > my > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> opinion and > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > I think we can > tackle > > it > > > > in > > > > > > > > separate > > > > > > > > > > > > > > > > > > > > > KIP. > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > 111. What's the > > typical > > > > rate > > > > > > of > > > > > > > > > > getting > > > > > > > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > > > > > > >> > >>> renewing > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > delegation > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> tokens? > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > Typically this > should > > be > > > > > very > > > > > > > very > > > > > > > > > > low, > > > > > > > > > > > > > > > > > > > > > 1 > > > > > > > > > > > > > > > > > > > > > > >> > request > > > > > > > > > > > > > > > > > > > > > > >> > >>> per > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > minute > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > is a > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > relatively high > > > estimate. > > > > > > > However > > > > > > > > it > > > > > > > > > > > > > > > > > > > > > > depends > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > > > > > > > >> >> > on > > > > > > > > > > > > > > > > > > > > > > >> > >>> the > > > > > > > > > > > > > > > > > > > > > > >> > >>> > > token > > > > > > > > > > > > > > > > > > > >
