[
https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14652140#comment-14652140
]
Sangjin Lee commented on YARN-3049:
-----------------------------------
When {{HBaseTimelineWriterImpl}} processes events for writes, it could have a
rule for those couple of special events (identified by the entity type = "yarn
application", event type = "application created" or "application finished"),
and trigger those events, right? I understand that it is bit unnatural for
{{HBaseTimelineWriterImpl}} to recognize those events explicitly, but that
could make this self-contained, right?
I think this is a rather important point because there are more tables that
need to be written to on application creation and completed and also more data
than the flow context. For example, the schema proposal calls for writing the
application start time and the application end time upon receiving those events
among others. We want to have a single point where all these are done. See
https://issues.apache.org/jira/secure/attachment/12743391/hbase-schema-proposal-for-aggregation.pdf
for more details.
I also think that doing it via the RM timeline collector is probably the best
for this. The RM timeline collector is the one that's writing these events to
begin with, and it can do that without worrying about the *app* timeline
collector starting up in time, etc. Thoughts?
> [Storage Implementation] Implement storage reader interface to fetch raw data
> from HBase backend
> ------------------------------------------------------------------------------------------------
>
> Key: YARN-3049
> URL: https://issues.apache.org/jira/browse/YARN-3049
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Sangjin Lee
> Assignee: Zhijie Shen
> Attachments: YARN-3049-WIP.1.patch, YARN-3049-WIP.2.patch,
> YARN-3049-WIP.3.patch, YARN-3049-YARN-2928.2.patch,
> YARN-3049-YARN-2928.3.patch, YARN-3049-YARN-2928.4.patch
>
>
> Implement existing ATS queries with the new ATS reader design.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
