[
https://issues.apache.org/jira/browse/YARN-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612648#comment-14612648
]
Junping Du commented on YARN-3815:
----------------------------------
bq. Also, it would be GREAT if you could give a clear and compelling use case
(a real life example) on why such support would be crucial. Thanks!
The use case here should be obviously. A quick real life example here is Google
Borg - cluster management tools
(http://static.googleusercontent.com/media/research.google.com/en/us/pubs/archive/43438.pdf)
which aggregate per-task resource usage information for usage-based charging,
debugging job and long-term capacity planning.
> [Aggregation] Application/Flow/User/Queue Level Aggregations
> ------------------------------------------------------------
>
> Key: YARN-3815
> URL: https://issues.apache.org/jira/browse/YARN-3815
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Junping Du
> Assignee: Junping Du
> Priority: Critical
> Attachments: Timeline Service Nextgen Flow, User, Queue Level
> Aggregations (v1).pdf, aggregation-design-discussion.pdf,
> hbase-schema-proposal-for-aggregation.pdf
>
>
> Per previous discussions in some design documents for YARN-2928, the basic
> scenario is the query for stats can happen on:
> - Application level, expect return: an application with aggregated stats
> - Flow level, expect return: aggregated stats for a flow_run, flow_version
> and flow
> - User level, expect return: aggregated stats for applications submitted by
> user
> - Queue level, expect return: aggregated stats for applications within the
> Queue
> Application states is the basic building block for all other level
> aggregations. We can provide Flow/User/Queue level aggregated statistics info
> based on application states (a dedicated table for application states is
> needed which is missing from previous design documents like HBase/Phoenix
> schema design).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
