Li Lu commented on YARN-3816:

bq. Also I was wondering if TimelineMetric#aggregateTo should be moved to some 
util class. TimelineMetric is part of object model and exposed to client. And 
IIUC aggregateTo wont be called by client.
Sorry but I think putting the aggregateTo method here is fine. I don't really 
like the idea of putting these static methods to a util class just because they 
look like utils. This is more of a subjective topic, but I hope our util 
methods to be general enough for the entire module. Aggregating metrics is not 
like reverting integers in their binary implementations, which is general 
enough for the whole module as a general "util". Here, aggregate metrics is 
clean enough to be a general operation to timeline metrics. I didn't get the 
part of the "called by client" discussion: our object model is used by both 
ourselves and our clients, so why "not called by clients" is a problem of our 
object model (the offline aggregation, for example, will also use this 
aggregation method)? 

> [Aggregation] App-level Aggregation for YARN system metrics
> -----------------------------------------------------------
>                 Key: YARN-3816
>                 URL: https://issues.apache.org/jira/browse/YARN-3816
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Junping Du
>            Assignee: Junping Du
>         Attachments: Application Level Aggregation of Timeline Data.pdf, 
> YARN-3816-YARN-2928-v1.patch, YARN-3816-poc-v1.patch, YARN-3816-poc-v2.patch
> We need application level aggregation of Timeline data:
> - To present end user aggregated states for each application, include: 
> resource (CPU, Memory) consumption across all containers, number of 
> containers launched/completed/failed, etc. We need this for apps while they 
> are running as well as when they are done.
> - Also, framework specific metrics, e.g. HDFS_BYTES_READ, should be 
> aggregated to show details of states in framework level.
> - Other level (Flow/User/Queue) aggregation can be more efficient to be based 
> on Application-level aggregations rather than raw entity-level data as much 
> less raws need to scan (with filter out non-aggregated entities, like: 
> events, configurations, etc.).

This message was sent by Atlassian JIRA

Reply via email to