[ 
https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14652140#comment-14652140
 ] 

Sangjin Lee commented on YARN-3049:
-----------------------------------

When {{HBaseTimelineWriterImpl}} processes events for writes, it could have a 
rule for those couple of special events (identified by the entity type = "yarn 
application", event type = "application created" or "application finished"), 
and trigger those events, right? I understand that it is bit unnatural for 
{{HBaseTimelineWriterImpl}} to recognize those events explicitly, but that 
could make this self-contained, right?

I think this is a rather important point because there are more tables that 
need to be written to on application creation and completed and also more data 
than the flow context. For example, the schema proposal calls for writing the 
application start time and the application end time upon receiving those events 
among others. We want to have a single point where all these are done. See 
https://issues.apache.org/jira/secure/attachment/12743391/hbase-schema-proposal-for-aggregation.pdf
 for more details.

I also think that doing it via the RM timeline collector is probably the best 
for this. The RM timeline collector is the one that's writing these events to 
begin with, and it can do that without worrying about the *app* timeline 
collector starting up in time, etc. Thoughts?

> [Storage Implementation] Implement storage reader interface to fetch raw data 
> from HBase backend
> ------------------------------------------------------------------------------------------------
>
>                 Key: YARN-3049
>                 URL: https://issues.apache.org/jira/browse/YARN-3049
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Zhijie Shen
>         Attachments: YARN-3049-WIP.1.patch, YARN-3049-WIP.2.patch, 
> YARN-3049-WIP.3.patch, YARN-3049-YARN-2928.2.patch, 
> YARN-3049-YARN-2928.3.patch, YARN-3049-YARN-2928.4.patch
>
>
> Implement existing ATS queries with the new ATS reader design.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to