[
https://issues.apache.org/jira/browse/YARN-3045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14646992#comment-14646992
]
Naganarasimha G R commented on YARN-3045:
-----------------------------------------
Thanks for the comments [~djp],
bq. We already have a new flush() API now for writer that checked in
YARN-3949... You are right that we are lacking of API to respect this
priority/policy in the whole data flow for writing. I will file another JIRA to
track this.
I went through the discussions and the patch of YARN-3949, i feel calling two
apis would be not so user friendly and how will the users of TimelineClient
call flush ? i think its not captured in YARN-3949
bq. Anyway, I would support the scope (container events + foundation work) you
proposed here in case you are comfortable with.
I am fine with single jira, but only trouble is as and when the scope increases
there will be more delay in the jira as more discussions will be required(in
this case which entity to publish NM App & localization events ) and also as
its been long since i am holding this jira so thought of getting the basic one
out and develop on top of it. I am ok if you want to avoid multiple jira's.
bq. That's a good question. My initative thinking is we could need something
like NodemanagerEntity to store application events, resource localizaiton
event, log aggregation handling events, configuration, etc. However, I would
like to hear you and other guys' ideas on this as well.
We had a discussion on this topic today in the meeting and [~sjlee0] was of the
opinion not to have another entity here. I think we need more discussions on
this as it involves querying too. Approach what i can think of is :
* For Applicationlevel events in NM can be under ApplicationEntity and EventID
can have Event Type
(INIT_APPLICATION/APPLICATION_FINISHED/APPLICATION_LOG_HANDLING_FAILED) and
NM_ID
* For Localization i feel it can be under ContainerEntity and the EventID can
have Event Type (REQUEST,LOCALIZED,LOCALIZATION_FAILED)and PATH of the
localized resource.
bq. IMO, the 2nd approach (hook to existing event dispatcher) looks simpler and
straightforward.
This approach is straight fwd but not sure it might have impact( just initial
apprehensions) but will start of implementing for container events and share
the initial patch based on this approach.
> [Event producers] Implement NM writing container lifecycle events to ATS
> ------------------------------------------------------------------------
>
> Key: YARN-3045
> URL: https://issues.apache.org/jira/browse/YARN-3045
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Sangjin Lee
> Assignee: Naganarasimha G R
> Attachments: YARN-3045-YARN-2928.002.patch,
> YARN-3045-YARN-2928.003.patch, YARN-3045-YARN-2928.004.patch,
> YARN-3045-YARN-2928.005.patch, YARN-3045-YARN-2928.006.patch,
> YARN-3045.20150420-1.patch
>
>
> Per design in YARN-2928, implement NM writing container lifecycle events and
> container system metrics to ATS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
