Zhijie Shen commented on YARN-3030:

Sangjin. Thanks for updating the patch. There're two more issues:

1. According to the proposal, the framework data flow is: task container -> AM 
container -> aggregator -> backend. However, in the lifecycle of per-node 
aggregator, the per-app aggregator will be constructed once the app lifecycle 
starts on the NM. It makes all the NMs that have the containers of this app 
have a per-node aggregator, which is not necessary. We should identify where 
the AM container is and only constructor the per-app aggregator there. The 
problem is that there may not be a reliable way to know a container is an AM 
container or not, other than parsing the container ID. We have to investigate 
more here.

2. The per-node aggregator has annotated the endpoint, but we still need an 
embedded web container to host this module. You can refer to 
ApplicationHistoryServer#startWebApp for the detail. One possible problem is 
whether NM can have more than one embedded web containers (NM already has one 
to host NM web-UI). I remember MiniYARNCluster has some problem of multiple web 
app in the same process. We need to double check.

> set up ATS writer with basic request serving structure and lifecycle
> --------------------------------------------------------------------
>                 Key: YARN-3030
>                 URL: https://issues.apache.org/jira/browse/YARN-3030
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Sangjin Lee
>         Attachments: YARN-3030.001.patch, YARN-3030.002.patch
> Per design in YARN-2928, create an ATS writer as a service, and implement the 
> basic service structure including the lifecycle management.
> Also, as part of this JIRA, we should come up with the ATS client API for 
> sending requests to this ATS writer.

This message was sent by Atlassian JIRA

Reply via email to