[
https://issues.apache.org/jira/browse/YARN-1390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13818863#comment-13818863
]
Hitesh Shah commented on YARN-1390:
-----------------------------------
[~vinodkv] How is the application going to be identified from a application
history point of view?
There seems to be 2 different things which are required. Lineage to understand
how an application was submitted ( this could be multi-levels deep ) and the
other to identify the application itself. For example, what is the plan for a
oozie job that launches a pig script that in turn runs multiple mapreduce jobs?
I think applicationType as it stands today should not change and should remain
hardcoded by MR. For lineage, something else should be introduced but it
requires each and every layer to cooperate to augment the lineage data. I dont
think there is a quick fix here. This is something which can be introduced at
the hadoop layer but will need to traverse through the whole ecosystem for it
to work correctly.
> Provide a way to capture source of an application to be queried through REST
> or Java Client APIs
> ------------------------------------------------------------------------------------------------
>
> Key: YARN-1390
> URL: https://issues.apache.org/jira/browse/YARN-1390
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: api
> Affects Versions: 2.2.0
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
>
> In addition to other fields like application-type (added in YARN-563), it is
> useful to have an applicationSource field to track the source of an
> application. The application source can be useful in (1) fetching only those
> applications a user is interested in, (2) potentially adding source-specific
> optimizations in the future.
> Examples of sources are: User-defined project names, Pig, Hive, Oozie, Sqoop
> etc.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
