[
https://issues.apache.org/jira/browse/YARN-3045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14641295#comment-14641295
]
Junping Du commented on YARN-3045:
----------------------------------
Thanks [~Naganarasimha] for updating the patch and sorry for coming late on
this.
I just go through the latest patch (06) and have some comments below:
1. I don't think we want to apply priority in container level to say which
containers are important while others are not important. What we want to
differentiate here is what kind of events are critical (so writer client in
TimelineCollector could flush to backend storage after written them) and what
kinds of events are not so critical.
>From an initiative thinking, some important app/container events include:
>INIT_APPLICATION, INIT_CONTAINER, FINISH_APPLICATION,
>APPLICATION_CONTAINER_FINISHED, APPLICATION_LOG_HANDLING_FAILED, while
>unimportant events could include: APPLICATION_INITED,
>APPLICATION_RESOURCES_CLEANEDUP, APPLICATION_LOG_HANDLING_INITED,
>APPLICATION_LOG_HANDLING_FINISHED, etc.
2. We should have some handy method to turn these app/container events to
TimelineEvent and publish these events in a consensus way rather than publish
one type of event with one method.
3. We don't need to create new container events but should log existing YARN
app/container events that happen in NM. If we really think some important
events are missing in YARN, we can have futher discussions later after timeline
service v2 in good shape.
4. It looks like NMTimelinePublisher should be used by ContainerManager,
Container, ResourceLocalizationService and Log Handler. Move it to NMContext
should be convenient to use for other components.
5. Container Resource Usage event may not be necessary given we already have
metrics update and will do aggregation according to metrics update.
> [Event producers] Implement NM writing container lifecycle events to ATS
> ------------------------------------------------------------------------
>
> Key: YARN-3045
> URL: https://issues.apache.org/jira/browse/YARN-3045
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Sangjin Lee
> Assignee: Naganarasimha G R
> Attachments: YARN-3045-YARN-2928.002.patch,
> YARN-3045-YARN-2928.003.patch, YARN-3045-YARN-2928.004.patch,
> YARN-3045-YARN-2928.005.patch, YARN-3045-YARN-2928.006.patch,
> YARN-3045.20150420-1.patch
>
>
> Per design in YARN-2928, implement NM writing container lifecycle events and
> container system metrics to ATS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
