Zhijie Shen commented on YARN-3039:

bq. When AM get launched, NM auxiliary service will add a new aggregator 
service to aggregatorCollection (per Node) for necessary binding work. 
aggregatorCollection also has a client for AggregatorNodeManagerProtocol to 
notify NM on new app aggregator registered and detailed address.

Hi Junping, thanks for creating the new patch. Sorry for raising the question 
in late, but I'd like to think it out loudly about the first step. Nowadays, 
app-level aggregator is started by the callback handler listening to the 
container start even of NM. Given we are going to support stand-alone and 
container mode, this approach may not work. As we're going to have IPC channel 
between aggregator and NM, should we use an IPC call to invoke adding one 
app-level aggregator.

So the protocol is that NM sends a request to the aggregator collection to 
start a app-level aggregator, and collection responds with the aggregator 
address. However, in this case, it may not be AggregatorNodemanagerProtocol, 
but NodemanagerAggregatorProtocol instead. The benefit is to uniform the way of 
starting app-level aggregator inside node-level aggregator (at lease it seem 
that we need to something similar in YARN-3033),  and further reducing the 
dependency/assumption on aux service.

[~vinodkv] and [~sjlee0] how do you think about it?

> [Aggregator wireup] Implement ATS app-appgregator service discovery
> -------------------------------------------------------------------
>                 Key: YARN-3039
>                 URL: https://issues.apache.org/jira/browse/YARN-3039
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Junping Du
>         Attachments: Service Binding for applicationaggregator of ATS 
> (draft).pdf, Service Discovery For Application Aggregator of ATS (v2).pdf, 
> YARN-3039-no-test.patch, YARN-3039-v2-incomplete.patch, 
> YARN-3039-v3-core-changes-only.patch, YARN-3039-v4.patch, YARN-3039-v5.patch
> Per design in YARN-2928, implement ATS writer service discovery. This is 
> essential for off-node clients to send writes to the right ATS writer. This 
> should also handle the case of AM failures.

This message was sent by Atlassian JIRA

Reply via email to