[jira] [Updated] (YARN-3030) set up ATS writer with basic request serving structure and lifecycle
[ https://issues.apache.org/jira/browse/YARN-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3030: -- Target Version/s: YARN-2928 Hadoop Flags: Reviewed > set up ATS writer with basic request serving structure and lifecycle > > > Key: YARN-3030 > URL: https://issues.apache.org/jira/browse/YARN-3030 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Sangjin Lee >Assignee: Sangjin Lee > Attachments: YARN-3030.001.patch, YARN-3030.002.patch, > YARN-3030.003.patch, YARN-3030.004.patch > > > Per design in YARN-2928, create an ATS writer as a service, and implement the > basic service structure including the lifecycle management. > Also, as part of this JIRA, we should come up with the ATS client API for > sending requests to this ATS writer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3030) set up ATS writer with basic request serving structure and lifecycle
[ https://issues.apache.org/jira/browse/YARN-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sangjin Lee updated YARN-3030: -- Attachment: YARN-3030.004.patch Posted patch v.4 to address Zhijie's suggestions. By adding the per-node aggregator web service directly to the node manager web app we're able to get around YARN-3087, but it required adding a dependency from node manager to timeline service (yuck). Also it was bit involved because we need to make sure we still retain the aux service behavior (to be able to leverage the aux service lifecycle) and make sure the same app-level service manager is shared between the aux service and the web service. Verified by adding it as an aux-service of the node manager. > set up ATS writer with basic request serving structure and lifecycle > > > Key: YARN-3030 > URL: https://issues.apache.org/jira/browse/YARN-3030 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Sangjin Lee >Assignee: Sangjin Lee > Attachments: YARN-3030.001.patch, YARN-3030.002.patch, > YARN-3030.003.patch, YARN-3030.004.patch > > > Per design in YARN-2928, create an ATS writer as a service, and implement the > basic service structure including the lifecycle management. > Also, as part of this JIRA, we should come up with the ATS client API for > sending requests to this ATS writer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3030) set up ATS writer with basic request serving structure and lifecycle
[
https://issues.apache.org/jira/browse/YARN-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sangjin Lee updated YARN-3030:
--
Attachment: YARN-3030.003.patch
Posted patch v.3.
Corrected the issue of using wrong lifecycle methods to initialize/remove app
level services. Also added the web server to be able to serve requests.
The server works fine in a standalone process. However, as an aux service to
the NM, the services comes up and initializes cleanly, but there are issues
serving REST requests. I filed YARN-3087 for that problem.
{color:green}+1 overall{color}.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version ) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
> set up ATS writer with basic request serving structure and lifecycle
>
>
> Key: YARN-3030
> URL: https://issues.apache.org/jira/browse/YARN-3030
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Attachments: YARN-3030.001.patch, YARN-3030.002.patch,
> YARN-3030.003.patch
>
>
> Per design in YARN-2928, create an ATS writer as a service, and implement the
> basic service structure including the lifecycle management.
> Also, as part of this JIRA, we should come up with the ATS client API for
> sending requests to this ATS writer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3030) set up ATS writer with basic request serving structure and lifecycle
[
https://issues.apache.org/jira/browse/YARN-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sangjin Lee updated YARN-3030:
--
Attachment: YARN-3030.002.patch
Posted patch v.2.
Basically switched PerNodeAggregator to AuxiliaryService. Ran the test-patch
script:
{color:green}+1 overall{color}.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version ) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
> set up ATS writer with basic request serving structure and lifecycle
>
>
> Key: YARN-3030
> URL: https://issues.apache.org/jira/browse/YARN-3030
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Attachments: YARN-3030.001.patch, YARN-3030.002.patch
>
>
> Per design in YARN-2928, create an ATS writer as a service, and implement the
> basic service structure including the lifecycle management.
> Also, as part of this JIRA, we should come up with the ATS client API for
> sending requests to this ATS writer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3030) set up ATS writer with basic request serving structure and lifecycle
[ https://issues.apache.org/jira/browse/YARN-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sangjin Lee updated YARN-3030: -- Attachment: YARN-3030.001.patch Posted patch v.1. (1) It sets up the basic skeleton of the aggregator service hierarchy. BaseAggregatorService sets up a core aggregator service. AppLevelAggregatorService is an extension that is instantiated per app. (2) I added a partial implementation of the PerNodeAggregator. It serves as a REST end point, and provides lifecycle methods so per-app in-memory aggregators can be created and destroyed according to the app lifecycle. (3) Added a few unit tests, and cleaned up the pom a little bit. This is still a skeleton. I simply used the existing common APIs from ATS to fill in the arguments and so on. We'll need to add more as our APIs jell. > set up ATS writer with basic request serving structure and lifecycle > > > Key: YARN-3030 > URL: https://issues.apache.org/jira/browse/YARN-3030 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Sangjin Lee >Assignee: Sangjin Lee > Attachments: YARN-3030.001.patch > > > Per design in YARN-2928, create an ATS writer as a service, and implement the > basic service structure including the lifecycle management. > Also, as part of this JIRA, we should come up with the ATS client API for > sending requests to this ATS writer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
