Naganarasimha G R commented on YARN-3044:

Thanks for the review comments [~zjshen],
bq. Can we use ContainerEntity. The events from RM are RM_XXXX_EVENT, and those 
from NM are NM_XXXX_EVENT.
This approach should be fine, will update in the next patch. 

bq. I think we may overestimate the performance impact of writing NM 
lifecycles. Perhaps a more reasonable performance metric is {{cost of writing 
lifecycle events per container / cost of managing lifecycle per container * 
100%}}. For example, if it is 2%, I guess it will probably be acceptable.
Well true we might be underestimating the RM's ability to handle publishing of 
Container Entity. But currently anyway have made it configurable to publish 
Container entities from RM side and while measuring performance we can enable 
this and check the performance, if fine then we can totally disable this 
configuration check and make RM publish always, your opinion ?

bq. I'm not sure if I understand this part correctly, but I incline that system 
timeline data (RM/NM) is controlled by cluster config and per cluster, while 
application data is controlled by framework or even per-application config. It 
may have some problem if the user is able to change the former config. For 
example, he can hide its application information from cluster admin.
may be i dint get this correctly, Is it that you intend to say that 
framework/cluster config (which can impact the application execution) should be 
logged by RM/NM and other application specific config can be logged by the AM ?

bq. Do you mean we should keep 
yarn.resourcemanager.system-metrics-publisher.enabled to control RM SMP, and 
and create yarn.nodemanager.system-metrics-publisher.enabled to control NM SMP?
No i meant this commment of [~djp] {{We can have different entity types, e.g. 
NM_CONTAINER_EVENT, RM_CONTAINER_EVENT, for containers' event get posted from 
NM or RM then we can fully understand how the world could be different from NM 
and RM (i.e. start time, end time, etc.}} & {{However, we can disable RM-side 
posting work in production environment by default.}}

> [Event producers] Implement RM writing app lifecycle events to ATS
> ------------------------------------------------------------------
>                 Key: YARN-3044
>                 URL: https://issues.apache.org/jira/browse/YARN-3044
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Naganarasimha G R
>         Attachments: YARN-3044.20150325-1.patch, YARN-3044.20150406-1.patch
> Per design in YARN-2928, implement RM writing app lifecycle events to ATS.

This message was sent by Atlassian JIRA

Reply via email to