[
https://issues.apache.org/jira/browse/YARN-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347053#comment-14347053
]
Naganarasimha G R commented on YARN-3039:
-----------------------------------------
Hi [~djp],
bq. for security reason it make NM to take AMRMTokens for using TimelineClient
in future which make less sense. To get rid of rack condition you mentioned
above, we propose to use observer pattern to make TimelineClient can listen
aggregator address update in AM or NM (wrap with retry logic to tolerant
connection failure).
Even if we are not able to have "AMRMClient can be wrapped into TimelineClient"
i feel other suggestion from vinod was right
{{to add a blocking call in AMRMClient to get aggregator address directly from
RM.}} instead of observer pattern @ the AM side. thoughts ?
bq. There are other ways (check from diagram in YARN-3033) that app aggregators
could be deployed in a separate process or an independent container which make
less sense to have a protocol between AUX service and RM. I think now we should
plan to add a protocol between aggregator and NM, and then notify RM through
NM-RM heartbeat on registering/rebind for aggregator.
Yes i have gone through 3033, but earlier was trying to mention as our current
approach was with NM AUX service. But anyway what i wanted was some kind of
protocol between appAggregators with either NM or RM. Protocol between NM and
appAgregator should suffice all other ways to launch AppAgregators.
bq. app aggregator should have logic to consolidate all messages (events and
metrics) for one application into more complex and flexible new data model. If
each NM do aggregation separately, then it still a writer (like old timeline
service), but not an aggregator
Well if there is no logic/requirement to aggregate/consolidate all messages
(events and metrics) for an App, then in my opinion it better not to have
additional instances of aggregators and we can keep it similar to old Timline
service.
bq. Will update proposal to reflect all these discussions (JIRA's and offline).
Thanks it will be more clear to implement if we have the proposals documented.
> [Aggregator wireup] Implement ATS app-appgregator service discovery
> -------------------------------------------------------------------
>
> Key: YARN-3039
> URL: https://issues.apache.org/jira/browse/YARN-3039
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Sangjin Lee
> Assignee: Naganarasimha G R
> Attachments: Service Binding for applicationaggregator of ATS
> (draft).pdf, YARN-3039-no-test.patch
>
>
> Per design in YARN-2928, implement ATS writer service discovery. This is
> essential for off-node clients to send writes to the right ATS writer. This
> should also handle the case of AM failures.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
