Naganarasimha G R commented on YARN-3039:

Hi [~djp],
 bq. for security reason it make NM to take AMRMTokens for using TimelineClient 
in future which make less sense. To get rid of rack condition you mentioned 
above, we propose to use observer pattern to make TimelineClient can listen 
aggregator address update in AM or NM (wrap with retry logic to tolerant 
connection failure).
Even if we are not able to have "AMRMClient can be wrapped into TimelineClient" 
i feel other suggestion from vinod was right 
{{to add a blocking call in AMRMClient to get aggregator address directly from 
RM.}} instead of observer pattern @ the AM side.  thoughts ?

bq. There are other ways (check from diagram in YARN-3033) that app aggregators 
could be deployed in a separate process or an independent container which make 
less sense to have a protocol between AUX service and RM. I think now we should 
plan to add a protocol between aggregator and NM, and then notify RM through 
NM-RM heartbeat on registering/rebind for aggregator.
Yes i have gone through 3033, but earlier was trying to mention as our current 
approach was with NM AUX service. But anyway what i wanted was some kind of 
protocol between appAggregators with either NM or RM. Protocol between NM and 
appAgregator should suffice all other ways to launch AppAgregators.

bq. app aggregator should have logic to consolidate all messages (events and 
metrics) for one application into more complex and flexible new data model. If 
each NM do aggregation separately, then it still a writer (like old timeline 
service), but not an aggregator
Well if there is no logic/requirement to aggregate/consolidate all messages 
(events and metrics) for an App, then in my opinion it better not to have 
additional instances of aggregators and we can  keep it similar to old Timline 

bq. Will update proposal to reflect all these discussions (JIRA's and offline).
Thanks it will be more clear to implement if we have the proposals documented.

> [Aggregator wireup] Implement ATS app-appgregator service discovery
> -------------------------------------------------------------------
>                 Key: YARN-3039
>                 URL: https://issues.apache.org/jira/browse/YARN-3039
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Naganarasimha G R
>         Attachments: Service Binding for applicationaggregator of ATS 
> (draft).pdf, YARN-3039-no-test.patch
> Per design in YARN-2928, implement ATS writer service discovery. This is 
> essential for off-node clients to send writes to the right ATS writer. This 
> should also handle the case of AM failures.

This message was sent by Atlassian JIRA

Reply via email to