[
https://issues.apache.org/jira/browse/YARN-3053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840182#comment-15840182
]
Varun Saxena edited comment on YARN-3053 at 1/26/17 6:38 PM:
-------------------------------------------------------------
[~jianhe], first of all whatever I mentioned above is based on how I think
off-app clients would work. We haven't yet finalized and brainstormed design of
off-app clients because its not in immediate scope. For app collectors I agree
with you that we do not need to store tokens in state store.
By off-app clients I mean frameworks like Oozie or Hive publishing their
entities i.e. some info before and after executing a workflow or a set of
queries which does not necessarily fit within the scope of individiual AMs'. In
other words, information which spans multiple YARN applications and hence
cannot be published by individual AMs'.
For such clients, collector address cannot be pushed like we do via RM-AM
protocol for individual AMs'. So most probably clients will have to get this
info from YARN (first when client comes up and later on connection loss if
collector goes down). This query will most probably go to RM. And this
communication needs to be kerberos authenticanted. We can probably open up a
channel to pull a token as well from RM (after off-app collector reports it to
RM via NM).
But an easier way would be to get it from collector via an explicit get
delegation token API. Because such clients would anyways need to do kerberos
authentication.
If we do so, we may need to store and recover tokens.
Anyways off-app clients require a little more thought. I can think of ways of
avoiding storing token even in this case but that may make other parts more
complex. Let us discuss and handle use-cases of off-app clients once we start
working on them. Security design for off app clients would depend largely on
how we decide on implementing them.
bq. I'm not sure the original collector design had accounted for unmanaged AM
in general case
Yes it hasn't been accounted for. We currently launch an app collector on a NM
where AM container is launched. So we do not launch any app collector for
unmanaged AM. Have raised YARN-6121 for that.
was (Author: varun_saxena):
[~jianhe], first of all whatever I mentioned above is based on how I think
off-app clients would work. We haven't yet finalized and brainstormed design of
off-app clients because its not in immediate scope. For app collectors I agree
with you that we do not need to store tokens in state store.
By off-app clients I mean frameworks like Oozie or Hive publishing their
entities i.e. some info before and after executing a workflow or a set of
queries which does not necessarily fit within the scope of individiual AMs'. In
other words, information which spans multiple YARN applications and hence
cannot be published by individual AMs'.
For such clients, collector address cannot be pushed like we do via RM-AM
protocol for individual AMs'. So most probably clients will have to get this
info from YARN (first when client comes up and later on connection loss if
collector goes down). This query will most probably go to RM. And this
communication needs to be kerberos authenticanted. We can probably open up a
channel to pull a token as well from RM (after off-app collector reports it to
RM via NM).
But an easier way would be to get it from collector via an explicit get
delegation token API. Because such clients would anyways need to do kerberos
authentication.
If we do so, we may need to store and recover tokens.
Anyways off-app clients require a little more thought. I can think of ways of
avoiding storing token even in this case but that may make other parts more
complex. Let us discuss and handle use-cases of off-app clients once we start
working on them.
bq. I'm not sure the original collector design had accounted for unmanaged AM
in general case
Yes it hasn't been accounted for. We currently launch an app collector on a NM
where AM container is launched. So we do not launch any app collector for
unmanaged AM. Have raised YARN-6121 for that.
> [Security] Review and implement authentication in ATS v.2
> ---------------------------------------------------------
>
> Key: YARN-3053
> URL: https://issues.apache.org/jira/browse/YARN-3053
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Sangjin Lee
> Assignee: Varun Saxena
> Labels: YARN-5355, yarn-5355-merge-blocker
> Attachments: ATSv2Authentication(draft).pdf
>
>
> Per design in YARN-2928, we want to evaluate and review the system for
> security, and ensure proper security in the system.
> This includes proper authentication, token management, access control, and
> any other relevant security aspects.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
