[jira] [Comment Edited] (HDFS-13532) RBF: Adding security
[ https://issues.apache.org/jira/browse/HDFS-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623943#comment-16623943 ] CR Hota edited comment on HDFS-13532 at 9/21/18 5:48 PM: - [~brahmareddy] Thanks for sharing your comments. Updated the document with your review points. Yes Cons mentioned earlier in Approach 1 is invalid. Irrespective of using tokens for auth, even in kerberos route, a client side service ticket is cached and when a service ticket is presented to name node, the name node does NOT connect to KDC for verification. So increased load on KDC isn't a valid point as mentioned in my first document. [~ajayydv] had mentioned it when he had reviewed. Am working on submitting the design based on Approach 1 and meanwhile also looking at creating a quick prototype that can demonstrate approach 1's feasibility. was (Author: crh): [~brahmareddy] Updated the document with your comments. Yes Cons mentioned earlier in Approach 1 is invalid. Irrespective of using tokens for auth, even in kerberos route, a client side service ticket is cached and when a service ticket is presented to namenode, the namenode does NOT connect to KDC for verification. So increased load on KDC isn't a valid point as mentioned in my first document. Am working on submitting the design based on Approach 1 and meanwhile also looking at creating a quick prototype that can demonstrate approach 1's feasibility. > RBF: Adding security > > > Key: HDFS-13532 > URL: https://issues.apache.org/jira/browse/HDFS-13532 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Íñigo Goiri >Assignee: CR Hota >Priority: Major > Attachments: RBF _ Security delegation token thoughts.pdf, RBF _ > Security delegation token thoughts_updated.pdf, RBF _ Security delegation > token thoughts_updated_2.pdf, RBF-DelegationToken-Approach1b.pdf, RBF_ > Security delegation token thoughts_updated_3.pdf, Security_for_Router-based > Federation_design_doc.pdf > > > HDFS Router based federation should support security. This includes > authentication and delegation tokens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13532) RBF: Adding security
[
https://issues.apache.org/jira/browse/HDFS-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16617749#comment-16617749
]
Brahma Reddy Battula edited comment on HDFS-13532 at 9/17/18 4:14 PM:
--
[~crh] thanks for updating.
As discussed in call, Following Cons for approach 1 are still valid..?As Router
also have token(act as proxy user) so auth can be done through token.
{quote}Without delegation token use namenodes will end up putting all the load
on KDC for kerberos ticket verification. This will defeat one of the main
rationales behind why delegation tokens were introduced in namenode.
{quote}
{quote}bq. Performance of namenodes will deteriorate further as network calls
need to be made to kdc for ticket verification instead of in memory cache of
delegation tokens that is maintained currently.
{quote}
and once after updating in statestore then we can return ack to the client.
was (Author: brahmareddy):
[~crh] thanks for updating.
As discussed in call, Following Cons for approach 1 are still valid..?As Router
also have token(act as proxy user) so auth can be done through token.
{quote}Without delegation token use namenodes will end up putting all the load
on KDC for kerberos ticket verification. This will defeat one of the main
rationales behind why delegation tokens were introduced in namenode.
{quote}
bq.Performance of namenodes will deteriorate further as network calls need to
be made to kdc for ticket verification instead of in memory cache of delegation
tokens that is maintained currently.
and once after updating in statestore then we can return ack to the client.
> RBF: Adding security
>
>
> Key: HDFS-13532
> URL: https://issues.apache.org/jira/browse/HDFS-13532
> Project: Hadoop HDFS
> Issue Type: New Feature
>Reporter: Íñigo Goiri
>Assignee: CR Hota
>Priority: Major
> Attachments: RBF _ Security delegation token thoughts.pdf, RBF _
> Security delegation token thoughts_updated.pdf, RBF _ Security delegation
> token thoughts_updated_2.pdf, RBF-DelegationToken-Approach1b.pdf,
> Security_for_Router-based Federation_design_doc.pdf
>
>
> HDFS Router based federation should support security. This includes
> authentication and delegation tokens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13532) RBF: Adding security
[
https://issues.apache.org/jira/browse/HDFS-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16617749#comment-16617749
]
Brahma Reddy Battula edited comment on HDFS-13532 at 9/17/18 4:13 PM:
--
[~crh] thanks for updating.
As discussed in call, Following Cons for approach 1 are still valid..?As Router
also have token(act as proxy user) so auth can be done through token.
{quote}Without delegation token use namenodes will end up putting all the load
on KDC for kerberos ticket verification. This will defeat one of the main
rationales behind why delegation tokens were introduced in namenode.
{quote}
bq.Performance of namenodes will deteriorate further as network calls need to
be made to kdc for ticket verification instead of in memory cache of delegation
tokens that is maintained currently.
and once after updating in statestore then we can return ack to the client.
was (Author: brahmareddy):
[~crh] thanks for updating.
As discussed in call, Following Cons for approach 1 are still valid..?As Router
also have token(act as proxy user) so auth can be done through token.
{quote}Without delegation token use namenodes will end up putting all the load
on KDC for kerberos ticket verification. This will defeat one of the main
rationales behind why delegation tokens were introduced in namenode.
{quote}
bq. Performance of namenodes will deteriorate further as network calls need to
be made to kdc for ticket verification instead of in memory cache of delegation
tokens that is maintained currently.
and once after updating in statestore then we can return ack to the client.
> RBF: Adding security
>
>
> Key: HDFS-13532
> URL: https://issues.apache.org/jira/browse/HDFS-13532
> Project: Hadoop HDFS
> Issue Type: New Feature
>Reporter: Íñigo Goiri
>Assignee: CR Hota
>Priority: Major
> Attachments: RBF _ Security delegation token thoughts.pdf, RBF _
> Security delegation token thoughts_updated.pdf, RBF _ Security delegation
> token thoughts_updated_2.pdf, RBF-DelegationToken-Approach1b.pdf,
> Security_for_Router-based Federation_design_doc.pdf
>
>
> HDFS Router based federation should support security. This includes
> authentication and delegation tokens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13532) RBF: Adding security
[
https://issues.apache.org/jira/browse/HDFS-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16617749#comment-16617749
]
Brahma Reddy Battula edited comment on HDFS-13532 at 9/17/18 4:13 PM:
--
[~crh] thanks for updating.
As discussed in call, Following Cons for approach 1 are still valid..?As Router
also have token(act as proxy user) so auth can be done through token.
bq. Without delegation token use namenodes will end up putting all the load on
KDC for kerberos ticket verification. This will defeat one of the main
rationales behind why delegation tokens were introduced in namenode.
bq. Performance of namenodes will deteriorate further as network calls need
to be made to kdc for ticket verification instead of in memory cache of
delegation tokens that is maintained currently.
and once after updating in statestore then we can return ack to the client.
was (Author: brahmareddy):
[~crh] thanks for updating.
As discussed in call, Following Cons for approach 1 are still valid..?As Router
also have token(act as proxy user) so auth can be done through token.
{quote}{quote} Without delegation token use namenodes will end up putting all
the load on KDC for kerberos ticket verification. This will defeat one of the
main rationales behind why delegation tokens were introduced in namenode.
{quote}
{quote} Performance of namenodes will deteriorate further as network calls
need to be made to kdc for ticket verification instead of in memory cache of
delegation tokens that is maintained currently.
{quote}{quote}
and once after updating in statestore then we can return ack to the client.
> RBF: Adding security
>
>
> Key: HDFS-13532
> URL: https://issues.apache.org/jira/browse/HDFS-13532
> Project: Hadoop HDFS
> Issue Type: New Feature
>Reporter: Íñigo Goiri
>Assignee: CR Hota
>Priority: Major
> Attachments: RBF _ Security delegation token thoughts.pdf, RBF _
> Security delegation token thoughts_updated.pdf, RBF _ Security delegation
> token thoughts_updated_2.pdf, RBF-DelegationToken-Approach1b.pdf,
> Security_for_Router-based Federation_design_doc.pdf
>
>
> HDFS Router based federation should support security. This includes
> authentication and delegation tokens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13532) RBF: Adding security
[
https://issues.apache.org/jira/browse/HDFS-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16617749#comment-16617749
]
Brahma Reddy Battula edited comment on HDFS-13532 at 9/17/18 4:13 PM:
--
[~crh] thanks for updating.
As discussed in call, Following Cons for approach 1 are still valid..?As Router
also have token(act as proxy user) so auth can be done through token.
{quote}Without delegation token use namenodes will end up putting all the load
on KDC for kerberos ticket verification. This will defeat one of the main
rationales behind why delegation tokens were introduced in namenode.
{quote}
bq. Performance of namenodes will deteriorate further as network calls need to
be made to kdc for ticket verification instead of in memory cache of delegation
tokens that is maintained currently.
and once after updating in statestore then we can return ack to the client.
was (Author: brahmareddy):
[~crh] thanks for updating.
As discussed in call, Following Cons for approach 1 are still valid..?As Router
also have token(act as proxy user) so auth can be done through token.
bq. Without delegation token use namenodes will end up putting all the load on
KDC for kerberos ticket verification. This will defeat one of the main
rationales behind why delegation tokens were introduced in namenode.
bq. Performance of namenodes will deteriorate further as network calls need
to be made to kdc for ticket verification instead of in memory cache of
delegation tokens that is maintained currently.
and once after updating in statestore then we can return ack to the client.
> RBF: Adding security
>
>
> Key: HDFS-13532
> URL: https://issues.apache.org/jira/browse/HDFS-13532
> Project: Hadoop HDFS
> Issue Type: New Feature
>Reporter: Íñigo Goiri
>Assignee: CR Hota
>Priority: Major
> Attachments: RBF _ Security delegation token thoughts.pdf, RBF _
> Security delegation token thoughts_updated.pdf, RBF _ Security delegation
> token thoughts_updated_2.pdf, RBF-DelegationToken-Approach1b.pdf,
> Security_for_Router-based Federation_design_doc.pdf
>
>
> HDFS Router based federation should support security. This includes
> authentication and delegation tokens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13532) RBF: Adding security
[
https://issues.apache.org/jira/browse/HDFS-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16617749#comment-16617749
]
Brahma Reddy Battula edited comment on HDFS-13532 at 9/17/18 4:12 PM:
--
[~crh] thanks for updating.
As discussed in call, Following Cons for approach 1 are still valid..?As Router
also have token(act as proxy user) so auth can be done through token.
{quote}{quote} Without delegation token use namenodes will end up putting all
the load on KDC for kerberos ticket verification. This will defeat one of the
main rationales behind why delegation tokens were introduced in namenode.
{quote}
{quote} Performance of namenodes will deteriorate further as network calls
need to be made to kdc for ticket verification instead of in memory cache of
delegation tokens that is maintained currently.
{quote}{quote}
and once after updating in statestore then we can return ack to the client.
was (Author: brahmareddy):
[~crh] thanks for updating.
As discussed in call, Following Cons for approach 1 are still valid, as Router
also token(act as proxy user) so auth can be done through token.
{quote}bq. Without delegation token use namenodes will end up putting all the
load on KDC for kerberos ticket verification. This will defeat one of the main
rationales behind why delegation tokens were introduced in namenode.
bq. Performance of namenodes will deteriorate further as network calls need to
be made to kdc for ticket verification instead of in memory cache of delegation
tokens that is maintained currently.
{quote}
and once after updating in statestore then we can return ack to the client.
> RBF: Adding security
>
>
> Key: HDFS-13532
> URL: https://issues.apache.org/jira/browse/HDFS-13532
> Project: Hadoop HDFS
> Issue Type: New Feature
>Reporter: Íñigo Goiri
>Assignee: CR Hota
>Priority: Major
> Attachments: RBF _ Security delegation token thoughts.pdf, RBF _
> Security delegation token thoughts_updated.pdf, RBF _ Security delegation
> token thoughts_updated_2.pdf, RBF-DelegationToken-Approach1b.pdf,
> Security_for_Router-based Federation_design_doc.pdf
>
>
> HDFS Router based federation should support security. This includes
> authentication and delegation tokens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13532) RBF: Adding security
[ https://issues.apache.org/jira/browse/HDFS-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566108#comment-16566108 ] Ajay Kumar edited comment on HDFS-13532 at 8/1/18 10:38 PM: [~crh], i had a offline discussion with [~jnp], [~xyao] and [~arpitagarwal] on this. If ServiceTicket is cached, Router will not be hammering KDC for each request. If other security experts in community believe that this will be an issue may be we can tweak our approach 1 to mitigate this issue. Attached [RBF-DelegationToken-Approach1b.pdf|https://issues.apache.org/jira/secure/attachment/12933984/RBF-DelegationToken-Approach1b.pdf] to discuss an slightly modified approach. was (Author: ajayydv): [~crh], i had a offline discussion with [~jnp], [~xyao] and [~arpitagarwal] on this. If ServiceTicket is cached, Router will not be hammering KDC for each request. If other security experts in community believe that this will be an issue may be we can tweak our approach 1 to mitigate this issue. Attached [^RBF _ Security delegation token thoughts.pdf] to discuss an slightly modified approach. > RBF: Adding security > > > Key: HDFS-13532 > URL: https://issues.apache.org/jira/browse/HDFS-13532 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Íñigo Goiri >Assignee: Sherwood Zheng >Priority: Major > Attachments: RBF _ Security delegation token thoughts.pdf, > RBF-DelegationToken-Approach1b.pdf, Security_for_Router-based > Federation_design_doc.pdf > > > HDFS Router based federation should support security. This includes > authentication and delegation tokens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13532) RBF: Adding security
[ https://issues.apache.org/jira/browse/HDFS-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517575#comment-16517575 ] Xiao Chen edited comment on HDFS-13532 at 6/19/18 10:00 PM: Thanks for the work here [~zhengxg3] and all. The last page of the doc looks familiar. :) Some high level questions from the doc. I have not followed RBF closely and my apologies if these are stupid comments/questions... * I second what Inigo said above. It's not clear to me how DTr is used. * It looks like we'll add the same mechanism to the router, so clients can auth with kerberos, then get a delegation token for subsequent authentications. Is this understanding correct? * I'm not a very security person - the router proxying as client part seems fine. But IMO that should only work if the client auth'ed via kerberos; if client->router auth is dt, then router should not auth to NN via kerberos, but only via the provided DTnn. * Who's gonna renew the router tokens? Tokens from different NNs may have different expiration time, hence need to be renewed at different intervals. RM currently does this, it's kinda nice to reuse RM to handle the DTr token renewal / cancelation. * [~daryn] at one point mentioned he's working on some token issuer interface. Not sure if it will benefit/collide with the work here. was (Author: xiaochen): Thanks for the work here [~zhengxg3] and all. The last page of the doc looks familiar. :) Some high level questions from the doc. I have not followed RBF closely and my apologies if these are stupid questions... * I second what Inigo said above. It's not clear to me how DTr is used. * It looks like we'll add the same mechanism to the router, so clients can auth with kerberos, then get a delegation token for subsequent authentications. Is this understanding correct? * I'm not a very security person - the router proxying as client part seems fine. But IMO that should only work if the client auth'ed via kerberos; if client->router auth is dt, then router should not auth to NN via kerberos, but only via the provided DTnn. * Who's gonna renew the router tokens? Tokens from different NNs may have different expiration time, hence need to be renewed at different intervals. RM currently does this, it's kinda nice to reuse RM to handle the DTr token renewal / cancelation. * [~daryn] at one point mentioned he's working on some token issuer interface. Not sure if it will benefit/collide with the work here. > RBF: Adding security > > > Key: HDFS-13532 > URL: https://issues.apache.org/jira/browse/HDFS-13532 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Íñigo Goiri >Assignee: Sherwood Zheng >Priority: Major > Attachments: Security_for_Router-based Federation_design_doc.pdf > > > HDFS Router based federation should support security. This includes > authentication and delegation tokens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13532) RBF: Adding security
[ https://issues.apache.org/jira/browse/HDFS-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516050#comment-16516050 ] Íñigo Goiri edited comment on HDFS-13532 at 6/18/18 5:36 PM: - Thanks [~zhengxg3] for the document. Could you add more details on how the Router does the mapping between the DT from the Router (e.g., step 8) to the federated token? >From what we saw in the prototypes, we were getting a DT with a different >signature and we had to do guesses to map it to the DT stored. Can you give more details on how the federated token is created and managed? For example, how do we propagate the federated tokens across Routers, etc. In addition, can you add pointers to the particular JIRAs in the design doc? was (Author: elgoiri): Thanks [~zhengxg3] for the document. Could you add more details on how the Router does the mapping between the DT from the Router (e.g., step 8) to the federated token? >From what we saw in the prototypes, we were getting a DT with a different >signature and we had to do guesses to map it to the DT stored. Can you give more details on how the federated token is created and managed? For example, how do we propagate the federated tokens across Routers, etc. > RBF: Adding security > > > Key: HDFS-13532 > URL: https://issues.apache.org/jira/browse/HDFS-13532 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Íñigo Goiri >Assignee: Sherwood Zheng >Priority: Major > Attachments: Security_for_Router-based Federation_design_doc.pdf > > > HDFS Router based federation should support security. This includes > authentication and delegation tokens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13532) RBF: Adding security
[ https://issues.apache.org/jira/browse/HDFS-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466791#comment-16466791 ] Íñigo Goiri edited comment on HDFS-13532 at 5/8/18 3:00 AM: [~zhengxg3] do you mind adding a design doc here for this as [~daryn] asked in HDFS-13358? was (Author: elgoiri): [~zhengxg3] do you mind adding a design doc here for this as [~daryn] asked in HDFS-12284? > RBF: Adding security > > > Key: HDFS-13532 > URL: https://issues.apache.org/jira/browse/HDFS-13532 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Íñigo Goiri >Assignee: Sherwood Zheng >Priority: Major > > HDFS Router based federation should support security. This includes > authentication and delegation tokens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
