[
https://issues.apache.org/jira/browse/YARN-3045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520902#comment-14520902
]
Naganarasimha G R commented on YARN-3045:
-----------------------------------------
Thanks for the reviews [~sjlee0] & [~vinodkv]
+1 for {{NMTimelinePublisher, NMTimelineEvent, NMTimelineEventType}}.
bq. The allocated memory/vcore/host/port/http-port/priority information per
container are also getting captured by the ResourceManager? Similarly the
exit-status, diagnostics etc. We definitely don't need these in two places.
IIUC from discussions in 3044, as suggested by [~djp] we have container metrics
published as optional feature in RM side and have also coded accordingly, and
even [~sjlee0] recent
[comment|https://issues.apache.org/jira/browse/YARN-3044?focusedCommentId=14518761&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14518761]
was inline with this idea. Hence earlier added some code in NM side to get all
the required information about container which are currently got it in RM.
bq. YARN-3334 added the notion of a timelineClient object embedded inside
Nodemanager's ApplicationImpl - we need to reconcile these things with what you
are doing here.
Well had some offline discussion with [~djp] regarding this. Currently [~djp]
had life cycle of timeline client and some code in NMContainerMonitor to
publish container resource metrics. So possible approaches here are :
# For other life cycle events, have inline code in transitions to get hold of
timelineclient from context and publish the timeline entity/event
# Gather all timeline publishing code in one place and have a single async
dispatcher taking caring of publishing timeline entities/events.
Advantage of later approach is no need to maintain thread pool (introduced as
part of YARN-3334). in each place where we want to publish timeline entities.
And felt it will be more organised if in one place all timeline publishing code
like RM.
[~djp]'s point was ??Using AsyncDispatcher is also a good way. However, I think
this is just a temporarily solution before YARN-3367 where we will do
non-blocking in TimelineClient later. So please feel free to choose
AsyncDispatcher or thread pool - whatever works end-2-end in current phase.
But I would recommend not trying to replace existing thread pool with
AsynDispatcher - no need to waste time to fix something which will get removed
later.??
So little confused with the scope of this jira if we conclude it then can come
up with the updated patch.
> [Event producers] Implement NM writing container lifecycle events to ATS
> ------------------------------------------------------------------------
>
> Key: YARN-3045
> URL: https://issues.apache.org/jira/browse/YARN-3045
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Sangjin Lee
> Assignee: Naganarasimha G R
> Attachments: YARN-3045.20150420-1.patch
>
>
> Per design in YARN-2928, implement NM writing container lifecycle events and
> container system metrics to ATS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
