Sangjin Lee commented on YARN-3815:

Some of us ([~gtCarrera9], [~vinodkv], [~djp], [~zjshen], [~vrushalic], and 
[~sjlee0]) had a face-to-face design discussion on the aggregation. I am going 
to post the summary of that discussion along with a proposal for an expanded 
native HBase schema to support aggregation.

I believe we are much closer to a consensus on the aggregation design, but some 
important questions still remain. For the sake of public discussion and 
inviting more participants and comments, we should follow up here on this JIRA.

> [Aggregation] Application/Flow/User/Queue Level Aggregations
> ------------------------------------------------------------
>                 Key: YARN-3815
>                 URL: https://issues.apache.org/jira/browse/YARN-3815
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Junping Du
>            Assignee: Junping Du
>            Priority: Critical
>         Attachments: Timeline Service Nextgen Flow, User, Queue Level 
> Aggregations (v1).pdf
> Per previous discussions in some design documents for YARN-2928, the basic 
> scenario is the query for stats can happen on:
> - Application level, expect return: an application with aggregated stats
> - Flow level, expect return: aggregated stats for a flow_run, flow_version 
> and flow 
> - User level, expect return: aggregated stats for applications submitted by 
> user
> - Queue level, expect return: aggregated stats for applications within the 
> Queue
> Application states is the basic building block for all other level 
> aggregations. We can provide Flow/User/Queue level aggregated statistics info 
> based on application states (a dedicated table for application states is 
> needed which is missing from previous design documents like HBase/Phoenix 
> schema design). 

This message was sent by Atlassian JIRA

Reply via email to