[
https://issues.apache.org/jira/browse/YARN-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15337102#comment-15337102
]
Sangjin Lee commented on YARN-5174:
-----------------------------------
I am fine with the current statement. My point earlier was whether we should
draw more attention to this than is needed.
Individual frameworks can do either of the approaches you pointed out. I
suspect most frameworks already aggregate values at the app-equivalent level
and are capable of emitting already-aggregated metrics at the YARN application
entity. I hope the other option is more of an exception.
{quote}
Say, in case of Spark on YARN, multiple spark executors will be launched (with
each one executing multiple tasks). So, if they ever integrate with ATSv2 and
report say some executor level metrics, then we can make it clear that they
themselves will have to roll over these metrics to app level. This will not be
an immediate use case though, if ever.
{quote}
We'd like Spark to use timeline service v.2 as soon as feasible. :) Seriously,
I think it would be easier for them to emit aggregated metrics themselves?
Otherwise, they would need to identify a suitable entity of their own and use
this aggregation ops. Either should be possible.
> several updates/corrections to timeline service documentation
> -------------------------------------------------------------
>
> Key: YARN-5174
> URL: https://issues.apache.org/jira/browse/YARN-5174
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Sangjin Lee
> Assignee: Sangjin Lee
> Labels: yarn-2928-1st-milestone
> Attachments: Hierarchy.png,
> PublishingApplicationDatatoYARNTimelineServicev.pdf, The YARN Timeline
> Service v.2.pdf, YARN-5174-YARN-2928.01.patch, YARN-5174-YARN-2928.02.patch,
> YARN-5174-YARN-2928.03.patch, YARN-5174-YARN-2928.03.patch,
> YARN-5174-YARN-2928.04.patch, YARN-5174-YARN-2928.05.patch, flow_hierarchy.png
>
>
> One part that is missing in the documentation is the need to add
> {{hbase-site.xml}} on the client side (the client hadoop cluster). First, we
> need to arrive at the minimally required client setting to connect to the
> right hbase cluster. Then, we need to document it so that users know exactly
> what to do to configure the cluster to use the timeline service v.2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
