[
https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16424562#comment-16424562
]
Haibo Chen commented on YARN-6936:
----------------------------------
I have been thinking about this issue along with YARN-3401. Here is what I
think the problem is in an abstract form.
|*Scope*|*Entity Type*|*Source*|*Target Table*|
|{color:#cccccc}Flow{color}|{color:#cccccc}YARN_FLOW_RUN{color}|{color:#cccccc}YARN{color}|{color:#cccccc}FlowRunTable{color}|
|{color:#cccccc}Flow{color}|{color:#cccccc}YARN_FLOW_ACTIVITY{color}|{color:#cccccc}YARN{color}|{color:#cccccc}FlowActivityTable{color}|
|Application|{color:#9876aa}YARN_APPLICATION{color}|YARN|ApplicationTable [AM]|
|Application|YARN_APPLICATION_ATTEMPT|YARN|EntityTable|
|Application|YARN_CONTAINER|YARN|EntityTable|
|Application|Custom types (e.g. MR_TASK)|AM|EntityTable|
|SubApplication|Custom types (e.g. DAG)|AM|SubApplicationTable|
|{color:#cccccc}User{color}|{color:#cccccc}YARN_USER{color}|{color:#cccccc}YARN{color}|{color:#cccccc}TBD{color}|
|{color:#cccccc}Queue{color}|{color:#cccccc}YARN_QUEUE{color}|{color:#cccccc}YARN{color}|{color:#cccccc}TBD{color}|
In the context of YARN-3401, we should stop AMs from forging data/entities that
should come from YARN itself. But in the meantime, AMs can still post their
custom entities types, whether in the scope of a YARN application or a
SubApplication.
I don't think we rely on just the scope (Entities in Application can go to
either ApplicationTable or EntityTable) or just the type to achieve that. One
thing I am not sure about is, whether we want to allow AMs to write
YARN_APPLICATION entities. MapReduce is already doing that, but from a pure
security point of view, it allows MR AMs to tamper with data posted by YARN.
> [Atsv2] Retrospect storing entities into sub application table from client
> perspective
> --------------------------------------------------------------------------------------
>
> Key: YARN-6936
> URL: https://issues.apache.org/jira/browse/YARN-6936
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Rohith Sharma K S
> Assignee: Rohith Sharma K S
> Priority: Major
> Attachments: YARN-6936.000.patch, YARN-6936.001.patch,
> YARN-6936.002.patch
>
>
> Currently YARN-6734 stores entities into sub application table only if doAs
> user and submitted users are different. This holds good for Tez kind of use
> cases. But AM runs as same as submitted user like MR also need to store
> entities in sub application table so that it could read entities without
> application id.
> This would be a point of concern later stages when ATSv2 is deployed into
> production. This JIRA is to retrospect decision of storing entities into sub
> application table based on client side configuration driven rather than user
> driven.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
