[
https://issues.apache.org/jira/browse/YARN-4821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15230813#comment-15230813
]
Vinod Kumar Vavilapalli commented on YARN-4821:
-----------------------------------------------
bq. This proposal is simply to use a different publishing interval just for the
timeline publishing
+1. We should completely decouple these two. If the publishing-interval is
configured to be not a multiple of the monitoring-interval, the publisher could
only look at the last N values from the monitor before the last cycle.
Can you also please have a read at YARN-3332 and see if you can organize code
in a bit of independent way?
A related data point for deciding the interval itself - the Hadoop Metrics
plugin pulls metrics from all of our daemons and pushes them out periodically -
with a default value of 10 sec IIRC. This is the periodicity for most of the
production clusters. Assuming adding container-metrics data to this still keep
the total outgoing data to the same or immediate order of magnitude (say 250
metrics per NM + (50 containers * 50 metrics)), we should be okay with the same
frequency. Anything more frequent will need careful benchmarking.
> Have a separate NM timeline publishing-interval
> -----------------------------------------------
>
> Key: YARN-4821
> URL: https://issues.apache.org/jira/browse/YARN-4821
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Sangjin Lee
> Assignee: Naganarasimha G R
> Labels: yarn-2928-1st-milestone
>
> Currently the interval with which NM publishes container CPU and memory
> metrics is tied to {{yarn.nodemanager.resource-monitor.interval-ms}} whose
> default is 3 seconds. This is too aggressive.
> There should be a separate configuration that controls how often
> {{NMTimelinePublisher}} publishes container metrics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
