[
https://issues.apache.org/jira/browse/YARN-3039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359441#comment-14359441
]
Zhijie Shen commented on YARN-3039:
-----------------------------------
bq. When AM get launched, NM auxiliary service will add a new aggregator
service to aggregatorCollection (per Node) for necessary binding work.
aggregatorCollection also has a client for AggregatorNodeManagerProtocol to
notify NM on new app aggregator registered and detailed address.
Hi Junping, thanks for creating the new patch. Sorry for raising the question
in late, but I'd like to think it out loudly about the first step. Nowadays,
app-level aggregator is started by the callback handler listening to the
container start even of NM. Given we are going to support stand-alone and
container mode, this approach may not work. As we're going to have IPC channel
between aggregator and NM, should we use an IPC call to invoke adding one
app-level aggregator.
So the protocol is that NM sends a request to the aggregator collection to
start a app-level aggregator, and collection responds with the aggregator
address. However, in this case, it may not be AggregatorNodemanagerProtocol,
but NodemanagerAggregatorProtocol instead. The benefit is to uniform the way of
starting app-level aggregator inside node-level aggregator (at lease it seem
that we need to something similar in YARN-3033), and further reducing the
dependency/assumption on aux service.
[~vinodkv] and [~sjlee0] how do you think about it?
> [Aggregator wireup] Implement ATS app-appgregator service discovery
> -------------------------------------------------------------------
>
> Key: YARN-3039
> URL: https://issues.apache.org/jira/browse/YARN-3039
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Sangjin Lee
> Assignee: Junping Du
> Attachments: Service Binding for applicationaggregator of ATS
> (draft).pdf, Service Discovery For Application Aggregator of ATS (v2).pdf,
> YARN-3039-no-test.patch, YARN-3039-v2-incomplete.patch,
> YARN-3039-v3-core-changes-only.patch, YARN-3039-v4.patch, YARN-3039-v5.patch
>
>
> Per design in YARN-2928, implement ATS writer service discovery. This is
> essential for off-node clients to send writes to the right ATS writer. This
> should also handle the case of AM failures.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
