[
https://issues.apache.org/jira/browse/YARN-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15263511#comment-15263511
]
Varun Saxena commented on YARN-3959:
------------------------------------
bq. One advantage of having it in the YARN application entity is that it
enables querying (and filtering) for configs across different application types.
Thats correct. Let's have it in both places then. Similar to what Naga had done
in MAPREDUCE=6424. Thoughts ?
bq. Regarding handling the size, the following may be necessary
As discussed in the meeting, this might be doable even before 1st milestone if
we reach a consensus on what to do ? Have internal fixed limits or have
configurable limits ? We need to decide on limits too. Say 100kb or 200kb.
Number of configs per entity may not be a very good critieria IMO here. If we
have done something in hRaven, we can probably adopt the same approach if
conflicting opinions do not exist.
bq. If there is no generic way of doing this, I think we might want to re-word
this JIRA to make it specific to mapreduce
Yes, this JIRA Is intended for Job configurations only. Will move it to
MAPREDUCE.
Moreover, we might need some thought on how to do it from YARN framework. Maybe
provide some additional interface from RM/NM collector. Me and Naga did have a
discussion surrounding writing these metrics and configs as part of
YARN_APPLICATION a couple of days ago. Should we allow AMs' to write anything
for YARN specific entity types ? Spurious AMs' can possibly overwrite metric
values or config values or other things. If we are relying on this history data
for making some decision, it may be a problem.This may be more a case in public
cloud scenario instead of controlled environments. But this anyways goes into
the realm of ACLs' and who can write what and how, which we haven't given a
thought on as yet.
Will have a look at Li's suggestion on MAPREDUCE-6424 and move this JIRA to
MAPREDUCE.
> Store application related configurations in Timeline Service v2
> ---------------------------------------------------------------
>
> Key: YARN-3959
> URL: https://issues.apache.org/jira/browse/YARN-3959
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Junping Du
> Assignee: Varun Saxena
> Labels: yarn-2928-1st-milestone
> Attachments: YARN-3959-YARN-2928.01.patch
>
>
> We already have configuration field in HBase schema for application entity.
> We need to make sure AM write it out when it get launched.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
