[
https://issues.apache.org/jira/browse/YARN-6375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950347#comment-15950347
]
Vrushali C commented on YARN-6375:
----------------------------------
Hello Varun,
I am trying to understand the jira and I have some questions. I understand what
you are saying by aggregation could aggregate only those metrics that are
reported by entities in the current timeframe. I am trying to think of an
example where we need aggregation for only currently reported metrics vs
aggregation for lifetime metrics.
CPU may be a good example of aggregating currently reported metrics? For
instance, we might want to see what is the CPU usage while these containers are
running but once a container is done, we need not consider it's last reported
CPU for current CPU usage of this app.
But, for other metrics, like say HDFS_BYTES_READ, perhaps we do want to think
about how many bytes have been read across tasks? I am thinking out loud.
Also, for CPU, perhaps we might want to know across this application what was
the cpu used by all the containers in the lifetime of this app? What metrics
are you thinking of..
> App level aggregation should not consider metric values reported in the
> previous aggregation cycle
> --------------------------------------------------------------------------------------------------
>
> Key: YARN-6375
> URL: https://issues.apache.org/jira/browse/YARN-6375
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Varun Saxena
> Assignee: Varun Saxena
> Attachments: YARN-6375-YARN-5355.01.patch
>
>
> Currently app level aggregation is done every 15 seconds.
> And we consider last reported metric value for each entity belonging to an
> app for aggregation.
> We however merely update the corresponding metric values for the entity on
> put. We never remove the entries.
> But it is possible that multiple entities finish during lifetime of an
> application. We however continue to consider them till the end.
> We should however not consider metric values of entities unless reported
> within the 15 second period.
> Consider containers. For a long running app, several containers would start
> and end at various times during the lifetime of an app.
> To consider metrics for all the containers throughout the lifetime of app,
> hence wont be correct.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
