[
https://issues.apache.org/jira/browse/YARN-1390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13819992#comment-13819992
]
Hitesh Shah commented on YARN-1390:
-----------------------------------
Also, from an implementation point of view, I would assume this map would be
*not* be searchable. Free-form text or even a set of variable key-val pairs are
expensive to search. Only defined fields such as applicationType ( which would
contain only a single value ) should be searchable.
bq. Representing applicationType as a set should suffice.
Representing it as a set is fine. However, how do you expect Oozie to pass
source info to Pig which in turn will pass it to MR ? Are you assuming the
source info is just a simple well defined string such as "Oozie" or would Oozie
do something like "Oozie:workflowId=1234" ? I think lineage is something which
YARN does not need to know or understand at the moment. Better to support it
via the free-form map instead of introducing a new field which we are not sure
how we plan to use/handle/support.
> Provide a way to capture source of an application to be queried through REST
> or Java Client APIs
> ------------------------------------------------------------------------------------------------
>
> Key: YARN-1390
> URL: https://issues.apache.org/jira/browse/YARN-1390
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: api
> Affects Versions: 2.2.0
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
>
> In addition to other fields like application-type (added in YARN-563), it is
> useful to have an applicationSource field to track the source of an
> application. The application source can be useful in (1) fetching only those
> applications a user is interested in, (2) potentially adding source-specific
> optimizations in the future.
> Examples of sources are: User-defined project names, Pig, Hive, Oozie, Sqoop
> etc.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
