[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[ https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14092098#comment-14092098 ] Hudson commented on YARN-2302: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1859 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1859/]) YARN-2302. Refactor TimelineWebServices. (Contributed by Zhijie Shen) (junping_du: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1617055) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/AHSWebApp.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineDataManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/webapp/TimelineWebServices.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryClientService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/webapp/TestTimelineWebServices.java > Refactor TimelineWebServices > > > Key: YARN-2302 > URL: https://issues.apache.org/jira/browse/YARN-2302 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: 2.6.0 > > Attachments: YARN-2302.1.patch, YARN-2302.2.patch, YARN-2302.3.patch, > YARN-2302.4.patch > > > Now TimelineWebServices contains non-trivial logic to process the HTTP > requests, manipulate the data, check the access, and interact with the > timeline store. > I propose the move the data-oriented logic to a middle layer (so called > TimelineDataManager), and TimelineWebServices only processes the requests, > and call TimelineDataManager to complete the remaining tasks. > By doing this, we make the generic history module reuse TimelineDataManager > internally (YARN-2033), invoking the putting/getting methods directly. > Otherwise, we have to send the HTTP requests to TimelineWebServices to query > the generic history data, which is not an efficient way. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[ https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14092089#comment-14092089 ] Hudson commented on YARN-2302: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1833 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1833/]) YARN-2302. Refactor TimelineWebServices. (Contributed by Zhijie Shen) (junping_du: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1617055) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/AHSWebApp.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineDataManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/webapp/TimelineWebServices.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryClientService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/webapp/TestTimelineWebServices.java > Refactor TimelineWebServices > > > Key: YARN-2302 > URL: https://issues.apache.org/jira/browse/YARN-2302 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: 2.6.0 > > Attachments: YARN-2302.1.patch, YARN-2302.2.patch, YARN-2302.3.patch, > YARN-2302.4.patch > > > Now TimelineWebServices contains non-trivial logic to process the HTTP > requests, manipulate the data, check the access, and interact with the > timeline store. > I propose the move the data-oriented logic to a middle layer (so called > TimelineDataManager), and TimelineWebServices only processes the requests, > and call TimelineDataManager to complete the remaining tasks. > By doing this, we make the generic history module reuse TimelineDataManager > internally (YARN-2033), invoking the putting/getting methods directly. > Otherwise, we have to send the HTTP requests to TimelineWebServices to query > the generic history data, which is not an efficient way. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[ https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14092068#comment-14092068 ] Hudson commented on YARN-2302: -- FAILURE: Integrated in Hadoop-Yarn-trunk #640 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/640/]) YARN-2302. Refactor TimelineWebServices. (Contributed by Zhijie Shen) (junping_du: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1617055) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/AHSWebApp.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineDataManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/webapp/TimelineWebServices.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryClientService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/webapp/TestTimelineWebServices.java > Refactor TimelineWebServices > > > Key: YARN-2302 > URL: https://issues.apache.org/jira/browse/YARN-2302 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: 2.6.0 > > Attachments: YARN-2302.1.patch, YARN-2302.2.patch, YARN-2302.3.patch, > YARN-2302.4.patch > > > Now TimelineWebServices contains non-trivial logic to process the HTTP > requests, manipulate the data, check the access, and interact with the > timeline store. > I propose the move the data-oriented logic to a middle layer (so called > TimelineDataManager), and TimelineWebServices only processes the requests, > and call TimelineDataManager to complete the remaining tasks. > By doing this, we make the generic history module reuse TimelineDataManager > internally (YARN-2033), invoking the putting/getting methods directly. > Otherwise, we have to send the HTTP requests to TimelineWebServices to query > the generic history data, which is not an efficient way. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[ https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14092047#comment-14092047 ] Hudson commented on YARN-2302: -- FAILURE: Integrated in Hadoop-trunk-Commit #6044 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6044/]) YARN-2302. Refactor TimelineWebServices. (Contributed by Zhijie Shen) (junping_du: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1617055) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/AHSWebApp.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/TimelineDataManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/timeline/webapp/TimelineWebServices.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/TestApplicationHistoryClientService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/timeline/webapp/TestTimelineWebServices.java > Refactor TimelineWebServices > > > Key: YARN-2302 > URL: https://issues.apache.org/jira/browse/YARN-2302 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: 2.6.0 > > Attachments: YARN-2302.1.patch, YARN-2302.2.patch, YARN-2302.3.patch, > YARN-2302.4.patch > > > Now TimelineWebServices contains non-trivial logic to process the HTTP > requests, manipulate the data, check the access, and interact with the > timeline store. > I propose the move the data-oriented logic to a middle layer (so called > TimelineDataManager), and TimelineWebServices only processes the requests, > and call TimelineDataManager to complete the remaining tasks. > By doing this, we make the generic history module reuse TimelineDataManager > internally (YARN-2033), invoking the putting/getting methods directly. > Otherwise, we have to send the HTTP requests to TimelineWebServices to query > the generic history data, which is not an efficient way. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[
https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14092010#comment-14092010
]
Hadoop QA commented on YARN-2302:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12660836/YARN-2302.4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4576//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4576//console
This message is automatically generated.
> Refactor TimelineWebServices
>
>
> Key: YARN-2302
> URL: https://issues.apache.org/jira/browse/YARN-2302
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Zhijie Shen
>Assignee: Zhijie Shen
> Attachments: YARN-2302.1.patch, YARN-2302.2.patch, YARN-2302.3.patch,
> YARN-2302.4.patch
>
>
> Now TimelineWebServices contains non-trivial logic to process the HTTP
> requests, manipulate the data, check the access, and interact with the
> timeline store.
> I propose the move the data-oriented logic to a middle layer (so called
> TimelineDataManager), and TimelineWebServices only processes the requests,
> and call TimelineDataManager to complete the remaining tasks.
> By doing this, we make the generic history module reuse TimelineDataManager
> internally (YARN-2033), invoking the putting/getting methods directly.
> Otherwise, we have to send the HTTP requests to TimelineWebServices to query
> the generic history data, which is not an efficient way.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[ https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14092008#comment-14092008 ] Junping Du commented on YARN-2302: -- Latest patch LGTM. +1 pending on Jenkins. > Refactor TimelineWebServices > > > Key: YARN-2302 > URL: https://issues.apache.org/jira/browse/YARN-2302 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Attachments: YARN-2302.1.patch, YARN-2302.2.patch, YARN-2302.3.patch, > YARN-2302.4.patch > > > Now TimelineWebServices contains non-trivial logic to process the HTTP > requests, manipulate the data, check the access, and interact with the > timeline store. > I propose the move the data-oriented logic to a middle layer (so called > TimelineDataManager), and TimelineWebServices only processes the requests, > and call TimelineDataManager to complete the remaining tasks. > By doing this, we make the generic history module reuse TimelineDataManager > internally (YARN-2033), invoking the putting/getting methods directly. > Otherwise, we have to send the HTTP requests to TimelineWebServices to query > the generic history data, which is not an efficient way. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[ https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14091958#comment-14091958 ] Junping Du commented on YARN-2302: -- Sounds good. I don't have additional comments except fixing the javadoc warning. > Refactor TimelineWebServices > > > Key: YARN-2302 > URL: https://issues.apache.org/jira/browse/YARN-2302 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Attachments: YARN-2302.1.patch, YARN-2302.2.patch, YARN-2302.3.patch > > > Now TimelineWebServices contains non-trivial logic to process the HTTP > requests, manipulate the data, check the access, and interact with the > timeline store. > I propose the move the data-oriented logic to a middle layer (so called > TimelineDataManager), and TimelineWebServices only processes the requests, > and call TimelineDataManager to complete the remaining tasks. > By doing this, we make the generic history module reuse TimelineDataManager > internally (YARN-2033), invoking the putting/getting methods directly. > Otherwise, we have to send the HTTP requests to TimelineWebServices to query > the generic history data, which is not an efficient way. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[
https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14091856#comment-14091856
]
Hadoop QA commented on YARN-2302:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12660815/YARN-2302.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3
warning messages.
See
https://builds.apache.org/job/PreCommit-YARN-Build/4572//artifact/trunk/patchprocess/diffJavadocWarnings.txt
for details.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4572//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4572//console
This message is automatically generated.
> Refactor TimelineWebServices
>
>
> Key: YARN-2302
> URL: https://issues.apache.org/jira/browse/YARN-2302
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Zhijie Shen
>Assignee: Zhijie Shen
> Attachments: YARN-2302.1.patch, YARN-2302.2.patch, YARN-2302.3.patch
>
>
> Now TimelineWebServices contains non-trivial logic to process the HTTP
> requests, manipulate the data, check the access, and interact with the
> timeline store.
> I propose the move the data-oriented logic to a middle layer (so called
> TimelineDataManager), and TimelineWebServices only processes the requests,
> and call TimelineDataManager to complete the remaining tasks.
> By doing this, we make the generic history module reuse TimelineDataManager
> internally (YARN-2033), invoking the putting/getting methods directly.
> Otherwise, we have to send the HTTP requests to TimelineWebServices to query
> the generic history data, which is not an efficient way.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[ https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14091845#comment-14091845 ] Zhijie Shen commented on YARN-2302: --- Thanks for the input, Junping! Patch updated again. bq. Shall we rename this method to putEntities? This is a good question. AFAIK, PUT is used to update the entire resource and supposed to be idempotent. Therefore, I'm not sure it's an idea to change the method to PUT because once the entity is created, the following updates are actually appending more data to the existing one. The best fit should be PATCH, however, it requires the additional implementation at the web services side. Hence, somebody online suggested using POST for partial non-idempotent update as well. Anyway, i think it's not a simple code refactoring, I'll file another ticket to track the investigation of proper HTTP method for postEntities. bq. In addition, I think we should have javadoc for public methods in TimelineDataManager.java. As TimelineDataManager is not the user-oriented interface, I added some simple javadoc for the public methods, and link them to TimelineStoreReader for argument details. > Refactor TimelineWebServices > > > Key: YARN-2302 > URL: https://issues.apache.org/jira/browse/YARN-2302 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Attachments: YARN-2302.1.patch, YARN-2302.2.patch, YARN-2302.3.patch > > > Now TimelineWebServices contains non-trivial logic to process the HTTP > requests, manipulate the data, check the access, and interact with the > timeline store. > I propose the move the data-oriented logic to a middle layer (so called > TimelineDataManager), and TimelineWebServices only processes the requests, > and call TimelineDataManager to complete the remaining tasks. > By doing this, we make the generic history module reuse TimelineDataManager > internally (YARN-2033), invoking the putting/getting methods directly. > Otherwise, we have to send the HTTP requests to TimelineWebServices to query > the generic history data, which is not an efficient way. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[
https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14091539#comment-14091539
]
Hadoop QA commented on YARN-2302:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12660727/YARN-2302.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4566//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4566//console
This message is automatically generated.
> Refactor TimelineWebServices
>
>
> Key: YARN-2302
> URL: https://issues.apache.org/jira/browse/YARN-2302
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Zhijie Shen
>Assignee: Zhijie Shen
> Attachments: YARN-2302.1.patch, YARN-2302.2.patch
>
>
> Now TimelineWebServices contains non-trivial logic to process the HTTP
> requests, manipulate the data, check the access, and interact with the
> timeline store.
> I propose the move the data-oriented logic to a middle layer (so called
> TimelineDataManager), and TimelineWebServices only processes the requests,
> and call TimelineDataManager to complete the remaining tasks.
> By doing this, we make the generic history module reuse TimelineDataManager
> internally (YARN-2033), invoking the putting/getting methods directly.
> Otherwise, we have to send the HTTP requests to TimelineWebServices to query
> the generic history data, which is not an efficient way.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[
https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14091522#comment-14091522
]
Junping Du commented on YARN-2302:
--
Thanks for updating the patch, [~zjshen]! The patch looks good now in overall.
Some minor comments:
{code}
+ public TimelinePutResponse postEntities(
+ TimelineEntities entities,
+ UserGroupInformation callerUGI) throws YarnException, IOException {
+if (entities == null) {
{code}
Shall we rename this method to putEntities? There is slightly different between
put and post operation (in REST prospective) while post is "create" (first
time) but put is an "update". The internal behavior of the method is like
"update" and call put operation actually, so "put" could be more properly.
In addition, I think we should have javadoc for public methods in
TimelineDataManager.java.
Other looks fine.
> Refactor TimelineWebServices
>
>
> Key: YARN-2302
> URL: https://issues.apache.org/jira/browse/YARN-2302
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Zhijie Shen
>Assignee: Zhijie Shen
> Attachments: YARN-2302.1.patch, YARN-2302.2.patch
>
>
> Now TimelineWebServices contains non-trivial logic to process the HTTP
> requests, manipulate the data, check the access, and interact with the
> timeline store.
> I propose the move the data-oriented logic to a middle layer (so called
> TimelineDataManager), and TimelineWebServices only processes the requests,
> and call TimelineDataManager to complete the remaining tasks.
> By doing this, we make the generic history module reuse TimelineDataManager
> internally (YARN-2033), invoking the putting/getting methods directly.
> Otherwise, we have to send the HTTP requests to TimelineWebServices to query
> the generic history data, which is not an efficient way.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[
https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14090707#comment-14090707
]
Junping Du commented on YARN-2302:
--
Thanks for the patch, [~zjshen]! A couple of comments so far:
In ApplicationHistoryServer.java,
{code}
+ protected TimelineDataManager timelineDataManager;
{code}
Better to be private, as it only get consumed within ApplicationHistoryServer.
{code}
timelineACLsManager = createTimelineACLsManager(conf);
{code}
Looks like we don’t need timelineACLsManager anymore except initiating
TimeLineDataManager. We can completely remove it (method and variable) after
merge below
{code}
protected TimelineACLsManager createTimelineACLsManager(Configuration conf) {
return new TimelineACLsManager(conf);
}
protected TimelineDataManager createTimelineDataManager(Configuration conf) {
return new TimelineDataManager(timelineStore, timelineACLsManager);
}
{code}
to:
{code}
private TimelineDataManager createTimelineDataManager(Configuration conf) {
return new TimelineDataManager(timelineStore, new
TimelineACLsManager(conf));
}
{code}
The visibility of method should be private, as it is not get assumed outside of
class. There are also some similar unnecessary protected methods around in this
class, see if you want to do update here also or we can do it separately later.
In TimelineDataManager.java,
{code}
+ try {
+if (existingEntity == null) {
+ injectOwnerInfo(entity, callerUGI.getShortUserName());
+}
+ } catch (YarnException e) {
+// Skip the entity which messes up the primary filter and record the
+// error
+LOG.warn("Skip the timeline entity: " + entityID + ", because "
++ e.getMessage());
{code}
This exception sounds more serious than just a warn, so Log.error here may make
more sense?
> Refactor TimelineWebServices
>
>
> Key: YARN-2302
> URL: https://issues.apache.org/jira/browse/YARN-2302
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Zhijie Shen
>Assignee: Zhijie Shen
> Attachments: YARN-2302.1.patch
>
>
> Now TimelineWebServices contains non-trivial logic to process the HTTP
> requests, manipulate the data, check the access, and interact with the
> timeline store.
> I propose the move the data-oriented logic to a middle layer (so called
> TimelineDataManager), and TimelineWebServices only processes the requests,
> and call TimelineDataManager to complete the remaining tasks.
> By doing this, we make the generic history module reuse TimelineDataManager
> internally (YARN-2033), invoking the putting/getting methods directly.
> Otherwise, we have to send the HTTP requests to TimelineWebServices to query
> the generic history data, which is not an efficient way.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[
https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14064565#comment-14064565
]
Hadoop QA commented on YARN-2302:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12656216/YARN-2302.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4338//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4338//console
This message is automatically generated.
> Refactor TimelineWebServices
>
>
> Key: YARN-2302
> URL: https://issues.apache.org/jira/browse/YARN-2302
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Zhijie Shen
>Assignee: Zhijie Shen
> Attachments: YARN-2302.1.patch
>
>
> Now TimelineWebServices contains non-trivial logic to process the HTTP
> requests, manipulate the data, check the access, and interact with the
> timeline store.
> I propose the move the data-oriented logic to a middle layer (so called
> TimelineDataManager), and TimelineWebServices only processes the requests,
> and call TimelineDataManager to complete the remaining tasks.
> By doing this, we make the generic history module reuse TimelineDataManager
> internally (YARN-2033), invoking the putting/getting methods directly.
> Otherwise, we have to send the HTTP requests to TimelineWebServices to query
> the generic history data, which is not an efficient way.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[ https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14064533#comment-14064533 ] Zhijie Shen commented on YARN-2302: --- Compilation failure is caused by YARN-2219. Will re-kick jenkins once that problem is fixed. > Refactor TimelineWebServices > > > Key: YARN-2302 > URL: https://issues.apache.org/jira/browse/YARN-2302 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Attachments: YARN-2302.1.patch > > > Now TimelineWebServices contains non-trivial logic to process the HTTP > requests, manipulate the data, check the access, and interact with the > timeline store. > I propose the move the data-oriented logic to a middle layer (so called > TimelineDataManager), and TimelineWebServices only processes the requests, > and call TimelineDataManager to complete the remaining tasks. > By doing this, we make the generic history module reuse TimelineDataManager > internally (YARN-2033), invoking the putting/getting methods directly. > Otherwise, we have to send the HTTP requests to TimelineWebServices to query > the generic history data, which is not an efficient way. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2302) Refactor TimelineWebServices
[
https://issues.apache.org/jira/browse/YARN-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14064528#comment-14064528
]
Hadoop QA commented on YARN-2302:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12656211/YARN-2302.1.patch
against trunk revision .
{color:red}-1 patch{color}. Trunk compilation may be broken.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4336//console
This message is automatically generated.
> Refactor TimelineWebServices
>
>
> Key: YARN-2302
> URL: https://issues.apache.org/jira/browse/YARN-2302
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Zhijie Shen
>Assignee: Zhijie Shen
> Attachments: YARN-2302.1.patch
>
>
> Now TimelineWebServices contains non-trivial logic to process the HTTP
> requests, manipulate the data, check the access, and interact with the
> timeline store.
> I propose the move the data-oriented logic to a middle layer (so called
> TimelineDataManager), and TimelineWebServices only processes the requests,
> and call TimelineDataManager to complete the remaining tasks.
> By doing this, we make the generic history module reuse TimelineDataManager
> internally (YARN-2033), invoking the putting/getting methods directly.
> Otherwise, we have to send the HTTP requests to TimelineWebServices to query
> the generic history data, which is not an efficient way.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
