[
https://issues.apache.org/jira/browse/YARN-3053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671338#comment-15671338
]
Varun Saxena edited comment on YARN-3053 at 11/16/16 7:38 PM:
--------------------------------------------------------------
Well the reason I was talking about fetching delegation token was because of
the support in terms of web/HTTP filters in current hadoop common code.
Refer to DelegationTokenAuthenticationFilter and
DelegationTokenAuthenticatedURL. Delegation token allows AM to post entities to
timeline service if it cannot do kerberos login in a secure setup. Also we can
renew these tokens.
This is what we do in ATSv1 too.
As Timeline Client is web based, we can embed the delegation token and use it
for authentication. The filter I talked about above will automatically
intercept the token and verify the token using backing token secret manager.
When I said that collector can generate a delegation token and send it back to
NM, I meant we can use the same secret manager to generate the token which the
filter uses so that when AM sends it in HTTP request, its available with
collector manager. Is there any problem with it ?
Now the issue with timeline service v2 is that we do not know where the
collector is running and collector won't start till we send the request to a NM
to launch AM. Now if we want tokens to be set in ApplicationSubmissionContext
while launching AM, we cannot really do it when collector is launched. RM/NM
(approach 2 or 3 in doc) will have to generate a token per app on behalf of
collector before it launches and then inform the collector about this token.
But here, tokens typically use sequence numbers and they may clash and master
keys used in both the processes(if collector is running outside NM) will have
to be in sync as well.
If the solution suggested i.e. collector generating a token explicitly when it
starts and sending it back to NM is not agreed upon, one other option would be
for RM or NM to explicitly contact timeline service to get a delegation token
when a new timeline service address is reported to it. This would be similar to
what we do for ATSv1 at client side during job submission except that we will
have to use RM/NM login user for authentication (assumption being app has
already been authenticated at the time of submission).
One more issue is who renews these tokens and cancels them. Probably collector
manager can do it by itself as it is informed when app finishes. We will have
to handle failover scenarios as well i.e. store all the tokens in state store.
was (Author: varun_saxena):
Well the reason I was talking about fetching delegation token was because of
the support in terms of web/HTTP filters in current hadoop common code.
Refer to DelegationTokenAuthenticationFilter and
DelegationTokenAuthenticatedURL. Delegation token allows AM to post entities to
timeline service if it cannot do kerberos login in a secure setup. Also we can
renew these tokens.
This is what we do in ATSv1 too.
As Timeline Client is web based, we can embed the delegation token and use it
for authentication. The filter I talked about above will automatically
intercept the token and verify the token using backing token secret manager.
When I said that collector can generate a delegation token and send it back to
NM, I meant we can use the same secret manager to generate the token which the
filter uses so that when AM sends it in HTTP request, its available with
collector manager. Is there any problem with it ?
Now the issue with timeline service v2 is that we do not know where the
collector is running and collector won't start till we send the request to a NM
to launch AM. Now if we want tokens to be set in ApplicationSubmissionContext
while launching AM, we cannot really do it when collector is launched. RM/NM
(approach 2 or 3 in doc) will have to generate a token per app on behalf of
collector before it launches and then inform the collector about this token.
But here, tokens typically use sequence numbers and they may clash and master
keys used in both the processes(if collector is running outside NM) will have
to be in sync as well.
If the solution suggested i.e. collector generating a token explicitly when it
starts and sending it back to NM is not agreed upon, one other option would be
for RM or NM to explicitly contact timeline service to get a delegation token
when a new timeline service address is reported to it. This would be similar to
what we do for ATSv1 at client side during job submission except that we will
have to use RM/NM login user for authentication.
One more issue is who renews these tokens and cancels them. Probably collector
manager can do it by itself as it is informed when app finishes. We will have
to handle failover scenarios as well i.e. store all the tokens in state store.
> [Security] Review and implement security in ATS v.2
> ---------------------------------------------------
>
> Key: YARN-3053
> URL: https://issues.apache.org/jira/browse/YARN-3053
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Sangjin Lee
> Assignee: Varun Saxena
> Labels: YARN-5355
> Attachments: ATSv2Authentication(draft).pdf
>
>
> Per design in YARN-2928, we want to evaluate and review the system for
> security, and ensure proper security in the system.
> This includes proper authentication, token management, access control, and
> any other relevant security aspects.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
