[jira] [Updated] (YARN-3030) set up ATS writer with basic request serving structure and lifecycle

2015-01-27 Thread Zhijie Shen (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhijie Shen updated YARN-3030:
--
Target Version/s: YARN-2928
Hadoop Flags: Reviewed

> set up ATS writer with basic request serving structure and lifecycle
> 
>
> Key: YARN-3030
> URL: https://issues.apache.org/jira/browse/YARN-3030
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: timelineserver
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Attachments: YARN-3030.001.patch, YARN-3030.002.patch, 
> YARN-3030.003.patch, YARN-3030.004.patch
>
>
> Per design in YARN-2928, create an ATS writer as a service, and implement the 
> basic service structure including the lifecycle management.
> Also, as part of this JIRA, we should come up with the ATS client API for 
> sending requests to this ATS writer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (YARN-3030) set up ATS writer with basic request serving structure and lifecycle

2015-01-26 Thread Sangjin Lee (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sangjin Lee updated YARN-3030:
--
Attachment: YARN-3030.004.patch

Posted patch v.4 to address Zhijie's suggestions.

By adding the per-node aggregator web service directly to the node manager web 
app we're able to get around YARN-3087, but it required adding a dependency 
from node manager to timeline service (yuck).

Also it was bit involved because we need to make sure we still retain the aux 
service behavior (to be able to leverage the aux service lifecycle) and make 
sure the same app-level service manager is shared between the aux service and 
the web service.

Verified by adding it as an aux-service of the node manager.

> set up ATS writer with basic request serving structure and lifecycle
> 
>
> Key: YARN-3030
> URL: https://issues.apache.org/jira/browse/YARN-3030
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: timelineserver
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Attachments: YARN-3030.001.patch, YARN-3030.002.patch, 
> YARN-3030.003.patch, YARN-3030.004.patch
>
>
> Per design in YARN-2928, create an ATS writer as a service, and implement the 
> basic service structure including the lifecycle management.
> Also, as part of this JIRA, we should come up with the ATS client API for 
> sending requests to this ATS writer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (YARN-3030) set up ATS writer with basic request serving structure and lifecycle

2015-01-22 Thread Sangjin Lee (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sangjin Lee updated YARN-3030:
--
Attachment: YARN-3030.003.patch

Posted patch v.3.

Corrected the issue of using wrong lifecycle methods to initialize/remove app 
level services. Also added the web server to be able to serve requests.

The server works fine in a standalone process. However, as an aux service to 
the NM, the services comes up and initializes cleanly, but there are issues 
serving REST requests. I filed YARN-3087 for that problem.

{color:green}+1 overall{color}.  

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 4 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version ) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.


> set up ATS writer with basic request serving structure and lifecycle
> 
>
> Key: YARN-3030
> URL: https://issues.apache.org/jira/browse/YARN-3030
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: timelineserver
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Attachments: YARN-3030.001.patch, YARN-3030.002.patch, 
> YARN-3030.003.patch
>
>
> Per design in YARN-2928, create an ATS writer as a service, and implement the 
> basic service structure including the lifecycle management.
> Also, as part of this JIRA, we should come up with the ATS client API for 
> sending requests to this ATS writer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (YARN-3030) set up ATS writer with basic request serving structure and lifecycle

2015-01-20 Thread Sangjin Lee (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sangjin Lee updated YARN-3030:
--
Attachment: YARN-3030.002.patch

Posted patch v.2.

Basically switched PerNodeAggregator to AuxiliaryService. Ran the test-patch 
script:

{color:green}+1 overall{color}.  

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version ) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.


> set up ATS writer with basic request serving structure and lifecycle
> 
>
> Key: YARN-3030
> URL: https://issues.apache.org/jira/browse/YARN-3030
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: timelineserver
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Attachments: YARN-3030.001.patch, YARN-3030.002.patch
>
>
> Per design in YARN-2928, create an ATS writer as a service, and implement the 
> basic service structure including the lifecycle management.
> Also, as part of this JIRA, we should come up with the ATS client API for 
> sending requests to this ATS writer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (YARN-3030) set up ATS writer with basic request serving structure and lifecycle

2015-01-16 Thread Sangjin Lee (JIRA)

 [ 
https://issues.apache.org/jira/browse/YARN-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sangjin Lee updated YARN-3030:
--
Attachment: YARN-3030.001.patch

Posted patch v.1.

(1)
It sets up the basic skeleton of the aggregator service hierarchy. 
BaseAggregatorService sets up a core aggregator service. 
AppLevelAggregatorService is an extension that is instantiated per app.

(2)
I added a partial implementation of the PerNodeAggregator. It serves as a REST 
end point, and provides lifecycle methods so per-app in-memory aggregators can 
be created and destroyed according to the app lifecycle.

(3)
Added a few unit tests, and cleaned up the pom a little bit.

This is still a skeleton. I simply used the existing common APIs from ATS to 
fill in the arguments and so on. We'll need to add more as our APIs jell.

> set up ATS writer with basic request serving structure and lifecycle
> 
>
> Key: YARN-3030
> URL: https://issues.apache.org/jira/browse/YARN-3030
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: timelineserver
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Attachments: YARN-3030.001.patch
>
>
> Per design in YARN-2928, create an ATS writer as a service, and implement the 
> basic service structure including the lifecycle management.
> Also, as part of this JIRA, we should come up with the ATS client API for 
> sending requests to this ATS writer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)