[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636400#comment-13636400 ] Hudson commented on YARN-514: - Integrated in Hadoop-Mapreduce-trunk #1404 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1404/]) YARN-585. Fix failure in TestFairScheduler#testNotAllowSubmitApplication caused by YARN-514. Contributed by Zhijie Shen. (Revision 1469492) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469492 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Fix For: 2.0.5-beta > > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch, > YARN-514.8.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636343#comment-13636343 ] Hudson commented on YARN-514: - Integrated in Hadoop-Hdfs-trunk #1377 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1377/]) YARN-585. Fix failure in TestFairScheduler#testNotAllowSubmitApplication caused by YARN-514. Contributed by Zhijie Shen. (Revision 1469492) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469492 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Fix For: 2.0.5-beta > > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch, > YARN-514.8.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636263#comment-13636263 ] Hudson commented on YARN-514: - Integrated in Hadoop-Yarn-trunk #188 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/188/]) YARN-585. Fix failure in TestFairScheduler#testNotAllowSubmitApplication caused by YARN-514. Contributed by Zhijie Shen. (Revision 1469492) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469492 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Fix For: 2.0.5-beta > > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch, > YARN-514.8.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635430#comment-13635430 ] Hudson commented on YARN-514: - Integrated in Hadoop-trunk-Commit #3627 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3627/]) YARN-585. Fix failure in TestFairScheduler#testNotAllowSubmitApplication caused by YARN-514. Contributed by Zhijie Shen. (Revision 1469492) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469492 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Fix For: 2.0.5-beta > > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch, > YARN-514.8.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635179#comment-13635179 ] Hudson commented on YARN-514: - Integrated in Hadoop-Mapreduce-trunk #1403 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1403/]) MAPREDUCE-5140. MR part of YARN-514 (Zhijie Shen via bikas) (Revision 1469060) YARN-514.Delayed store operations should not result in RM unavailability for app submission (Zhijie Shen via bikas) (Revision 1469059) Result = SUCCESS bikas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469060 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java bikas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469059 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/YarnApplicationState.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreAppEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreEventType.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppEventType.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppState.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppStoredEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmController.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-5
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635136#comment-13635136 ] Hudson commented on YARN-514: - Integrated in Hadoop-Hdfs-trunk #1376 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1376/]) MAPREDUCE-5140. MR part of YARN-514 (Zhijie Shen via bikas) (Revision 1469060) YARN-514.Delayed store operations should not result in RM unavailability for app submission (Zhijie Shen via bikas) (Revision 1469059) Result = FAILURE bikas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469060 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java bikas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469059 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/YarnApplicationState.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreAppEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreEventType.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppEventType.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppState.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppStoredEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmController.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635080#comment-13635080 ] Hudson commented on YARN-514: - Integrated in Hadoop-Yarn-trunk #187 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/187/]) MAPREDUCE-5140. MR part of YARN-514 (Zhijie Shen via bikas) (Revision 1469060) YARN-514.Delayed store operations should not result in RM unavailability for app submission (Zhijie Shen via bikas) (Revision 1469059) Result = FAILURE bikas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469060 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java bikas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469059 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/YarnApplicationState.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreAppEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreEventType.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppEventType.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppState.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppStoredEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmController.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch,
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[
https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634841#comment-13634841
]
Zhijie Shen commented on YARN-514:
--
I've run through the yarn test cases locally, and found no more failures.
I also find problem why the final assert still fails. As RMApp have one more
state in its life cycle, the direct call:
{code}
application.handle(new RMAppEvent(applicationId, RMAppEventType.START));
{code}
moves application from NEW to NEW_SAVING. We still need to give the application
some time to transit from NEW_SAVING to SUMBITTED, where the attempt is created
and RMAppAttemptEventType.START is scheduled. The current code in
TestFairScheduler#testNotAllowSubmitApplication results in that
RMAppAttemptEventType.APP_REJECTED comes earlier than
RMAppAttemptEventType.START. Therefore, the attempt gets APP_REJECTED event
when it is still in NEW. This transition happens not to be defined.
So, the quick test case fix is that add "Thread.sleep(100);" immediately after
"application.handle(new RMAppEvent(applicationId, RMAppEventType.START));" to
yield the process for asyncDispatcher to push application forward.
> Delayed store operations should not result in RM unavailability for app
> submission
> --
>
> Key: YARN-514
> URL: https://issues.apache.org/jira/browse/YARN-514
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Bikas Saha
>Assignee: Zhijie Shen
> Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch,
> YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch,
> YARN-514.8.patch
>
>
> Currently, app submission is the only store operation performed synchronously
> because the app must be stored before the request returns with success. This
> makes the RM susceptible to blocking all client threads on slow store
> operations, resulting in RM being perceived as unavailable by clients.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634810#comment-13634810 ] Vinod Kumar Vavilapalli commented on YARN-514: -- Zhijie, can you please run all the tests locally and see what else may be failing? I am running them too to cross-verify. If there are many failures we should revert the patch, otherwise we can just file a follow up ticket and fix it. > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch, > YARN-514.8.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634804#comment-13634804 ] Zhijie Shen commented on YARN-514: -- [~kkambatl], TestFairScheduler#testNotAllowSubmitApplication is buggy. "submissionContext.setApplicationId(applicationId);" needs to be added to prevent test crash. However, the final assert still fails. I'm looking into the problem. > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch, > YARN-514.8.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634799#comment-13634799 ] Karthik Kambatla commented on YARN-514: --- [~bikassaha], [~zjshen]: can one of you verify if this is indeed the case. I couldn't identify the root cause from cursory examination. > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch, > YARN-514.8.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634697#comment-13634697 ] Karthik Kambatla commented on YARN-514: --- Looks like this patch breaks TestFairScheduler (as Alejandro reports on YARN-482), I wonder why Hudson didn't catch it. The tests pass when I checkout the commit before this, and fail with this. > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch, > YARN-514.8.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634390#comment-13634390 ] Hudson commented on YARN-514: - Integrated in Hadoop-trunk-Commit #3623 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3623/]) MAPREDUCE-5140. MR part of YARN-514 (Zhijie Shen via bikas) (Revision 1469060) Result = SUCCESS bikas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469060 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch, > YARN-514.8.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634378#comment-13634378 ] Hudson commented on YARN-514: - Integrated in Hadoop-trunk-Commit #3622 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3622/]) YARN-514.Delayed store operations should not result in RM unavailability for app submission (Zhijie Shen via bikas) (Revision 1469059) Result = SUCCESS bikas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1469059 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/YarnApplicationState.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/MemoryRMStateStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreAppEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/RMStateStoreEventType.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppEventType.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppState.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppStoredEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RmController.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch, > YARN-514.8.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[
https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634338#comment-13634338
]
Hadoop QA commented on YARN-514:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579161/YARN-514.8.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/762//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/762//console
This message is automatically generated.
> Delayed store operations should not result in RM unavailability for app
> submission
> --
>
> Key: YARN-514
> URL: https://issues.apache.org/jira/browse/YARN-514
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Bikas Saha
>Assignee: Zhijie Shen
> Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch,
> YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch,
> YARN-514.8.patch
>
>
> Currently, app submission is the only store operation performed synchronously
> because the app must be stored before the request returns with success. This
> makes the RM susceptible to blocking all client threads on slow store
> operations, resulting in RM being perceived as unavailable by clients.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[
https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634210#comment-13634210
]
Bikas Saha commented on YARN-514:
-
Looks great. Thanks for being patient with my comments. One minor nit before I
commit this.
"app.rmContext.getRMApps().get(app.applicationId) == app" right?
{code}
+ LOG.info("Storing application with id " + app.applicationId);
+ app.rmContext.getStateStore().storeApplication(
+ app.rmContext.getRMApps().get(app.applicationId));
{code}
Could you also please check that the patch on the related MR jira still applies
cleanly. I will commit both these in succession.
> Delayed store operations should not result in RM unavailability for app
> submission
> --
>
> Key: YARN-514
> URL: https://issues.apache.org/jira/browse/YARN-514
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Bikas Saha
>Assignee: Zhijie Shen
> Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch,
> YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch
>
>
> Currently, app submission is the only store operation performed synchronously
> because the app must be stored before the request returns with success. This
> makes the RM susceptible to blocking all client threads on slow store
> operations, resulting in RM being perceived as unavailable by clients.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[
https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13633819#comment-13633819
]
Hadoop QA commented on YARN-514:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579098/YARN-514.7.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/759//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/759//console
This message is automatically generated.
> Delayed store operations should not result in RM unavailability for app
> submission
> --
>
> Key: YARN-514
> URL: https://issues.apache.org/jira/browse/YARN-514
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Bikas Saha
>Assignee: Zhijie Shen
> Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch,
> YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch
>
>
> Currently, app submission is the only store operation performed synchronously
> because the app must be stored before the request returns with success. This
> makes the RM susceptible to blocking all client threads on slow store
> operations, resulting in RM being perceived as unavailable by clients.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13633072#comment-13633072 ] Bikas Saha commented on YARN-514: - I am thinking of the following RMAppManager.recover can call submitapplication and send RECOVER event to RMApp instead of START. in RMApp we can transition from NEW->SUBMITTED when we receive RECOVER event. that may be less code change than saving recovered flag and then doing transitions dependent on a flag. and more clear also in having a clean recover transition than flag dependent transitions. currently in NEW->SUBMITTED on recover event, new app will be created. later for work preserving restart, on recover event, app could move from NEW->RUNNING based on the last running app attempt. this will also prevent us from having more bugs like storing the same app twice because we will go down recovery transition and not the normal transition. > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[
https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632975#comment-13632975
]
Bikas Saha commented on YARN-514:
-
StartAppAttemptTransition and RMAppStartingOrSavingTransition both can start an
app attempt and are currently duplicating the code to start the attempt.
IMO, some of the new tests seem redundant. eg.
testAppSubmittedNoRecoveryKill/testAppSubmittedRecoveryKill basically testing
transitions SUBMITTED->KILL. Do they depend on how the app came into the
SUBMITTED state? If no, then the tests are redundant and increase maintenance
burden. Same for other such tests for SUBMITTED state.
In new testAppFailedFailed(), in the comments you mean FAILED->FAILED and not
KILLED->KILLED right?
Javadoc is wrong since this method is not called by the derived class. In fact
its a private method. This looks like it was copied from the doc for
notify*Attempt. Please correct both javadocs.
{code}
+ * In (@link storeApplication}, derived class can call this method to notify
+ * the application about operation completion
+ * @param appId id of the application that has been saved
+ * @param storedException the exception that is thrown when storing the
+ * application
+ */
+ private void notifyDoneStoringApplication(ApplicationId appId,
{code}
> Delayed store operations should not result in RM unavailability for app
> submission
> --
>
> Key: YARN-514
> URL: https://issues.apache.org/jira/browse/YARN-514
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Bikas Saha
>Assignee: Zhijie Shen
> Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch,
> YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch
>
>
> Currently, app submission is the only store operation performed synchronously
> because the app must be stored before the request returns with success. This
> makes the RM susceptible to blocking all client threads on slow store
> operations, resulting in RM being perceived as unavailable by clients.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[
https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632655#comment-13632655
]
Hadoop QA commented on YARN-514:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578893/YARN-514.6.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/745//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/745//console
This message is automatically generated.
> Delayed store operations should not result in RM unavailability for app
> submission
> --
>
> Key: YARN-514
> URL: https://issues.apache.org/jira/browse/YARN-514
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Bikas Saha
>Assignee: Zhijie Shen
> Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch,
> YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch
>
>
> Currently, app submission is the only store operation performed synchronously
> because the app must be stored before the request returns with success. This
> makes the RM susceptible to blocking all client threads on slow store
> operations, resulting in RM being perceived as unavailable by clients.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632536#comment-13632536 ] Bikas Saha commented on YARN-514: - Sorry for not catching this earlier. RMAppEventType.APP_SAVED will need to be handled in the KILLED and FAILED states. RMApp can transition into KILLED/FAILED while waiting for the store operation. After the operation completes, the current code will crash. > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch, YARN-514.5.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[
https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632248#comment-13632248
]
Hadoop QA commented on YARN-514:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578816/YARN-514.5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/743//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/743//console
This message is automatically generated.
> Delayed store operations should not result in RM unavailability for app
> submission
> --
>
> Key: YARN-514
> URL: https://issues.apache.org/jira/browse/YARN-514
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Bikas Saha
>Assignee: Zhijie Shen
> Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch,
> YARN-514.4.patch, YARN-514.5.patch
>
>
> Currently, app submission is the only store operation performed synchronously
> because the app must be stored before the request returns with success. This
> makes the RM susceptible to blocking all client threads on slow store
> operations, resulting in RM being perceived as unavailable by clients.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[
https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632081#comment-13632081
]
Bikas Saha commented on YARN-514:
-
If we are going to change the enum values then we might as well get them in
order and consistent. ACCEPTED should not be last. Also could you please check
consistency with RMAppState.
{code}
enum YarnApplicationStateProto {
NEW = 1;
- SUBMITTED = 2;
- RUNNING = 3;
- FINISHED = 4;
- FAILED = 5;
- KILLED = 6;
- ACCEPTED = 7;
+ NEW_SAVING = 2;
+ SUBMITTED = 3;
+ RUNNING = 4;
+ FINISHED = 5;
+ FAILED = 6;
+ KILLED = 7;
+ ACCEPTED = 8;
}
{code}
> Delayed store operations should not result in RM unavailability for app
> submission
> --
>
> Key: YARN-514
> URL: https://issues.apache.org/jira/browse/YARN-514
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Bikas Saha
>Assignee: Zhijie Shen
> Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch,
> YARN-514.4.patch
>
>
> Currently, app submission is the only store operation performed synchronously
> because the app must be stored before the request returns with success. This
> makes the RM susceptible to blocking all client threads on slow store
> operations, resulting in RM being perceived as unavailable by clients.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13630320#comment-13630320 ] Bikas Saha commented on YARN-514: - you only need to add the new field in the enum. I dont think we should change the values of all existing enums. > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, > YARN-514.4.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[
https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13629818#comment-13629818
]
Hadoop QA commented on YARN-514:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578361/YARN-514.4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/724//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/724//console
This message is automatically generated.
> Delayed store operations should not result in RM unavailability for app
> submission
> --
>
> Key: YARN-514
> URL: https://issues.apache.org/jira/browse/YARN-514
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Bikas Saha
>Assignee: Zhijie Shen
> Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch,
> YARN-514.4.patch
>
>
> Currently, app submission is the only store operation performed synchronously
> because the app must be stored before the request returns with success. This
> makes the RM susceptible to blocking all client threads on slow store
> operations, resulting in RM being perceived as unavailable by clients.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13629782#comment-13629782 ] Zhijie Shen commented on YARN-514: -- @Biksa, the enum values in the proto needs to be changed because YarnApplicationStateProto will be used by application report. MR may also need it when doing state conversion from Yarn state to MR state. > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13629745#comment-13629745 ] Bikas Saha commented on YARN-514: - For MAPREDUCE-5140 please check for uses of both NEW and SUBMITTED in order to find out places where NEW_SAVING would need to be handled. > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13629742#comment-13629742 ] Bikas Saha commented on YARN-514: - Looks good overall. Minor tab issues in the patch. I dont think we want to change the enum values in the proto. Please prepare a MAPREDUCE side patch for MAPREDUCE-5140. These need to go in together. > Delayed store operations should not result in RM unavailability for app > submission > -- > > Key: YARN-514 > URL: https://issues.apache.org/jira/browse/YARN-514 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Bikas Saha >Assignee: Zhijie Shen > Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch > > > Currently, app submission is the only store operation performed synchronously > because the app must be stored before the request returns with success. This > makes the RM susceptible to blocking all client threads on slow store > operations, resulting in RM being perceived as unavailable by clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[
https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628699#comment-13628699
]
Hadoop QA commented on YARN-514:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578135/YARN-514.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/713//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/713//console
This message is automatically generated.
> Delayed store operations should not result in RM unavailability for app
> submission
> --
>
> Key: YARN-514
> URL: https://issues.apache.org/jira/browse/YARN-514
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Bikas Saha
>Assignee: Zhijie Shen
> Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch
>
>
> Currently, app submission is the only store operation performed synchronously
> because the app must be stored before the request returns with success. This
> makes the RM susceptible to blocking all client threads on slow store
> operations, resulting in RM being perceived as unavailable by clients.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[
https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628649#comment-13628649
]
Hadoop QA commented on YARN-514:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578135/YARN-514.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/712//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/712//console
This message is automatically generated.
> Delayed store operations should not result in RM unavailability for app
> submission
> --
>
> Key: YARN-514
> URL: https://issues.apache.org/jira/browse/YARN-514
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Bikas Saha
>Assignee: Zhijie Shen
> Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch
>
>
> Currently, app submission is the only store operation performed synchronously
> because the app must be stored before the request returns with success. This
> makes the RM susceptible to blocking all client threads on slow store
> operations, resulting in RM being perceived as unavailable by clients.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[
https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13627572#comment-13627572
]
Hadoop QA commented on YARN-514:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577966/YARN-514.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/701//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/701//console
This message is automatically generated.
> Delayed store operations should not result in RM unavailability for app
> submission
> --
>
> Key: YARN-514
> URL: https://issues.apache.org/jira/browse/YARN-514
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Bikas Saha
>Assignee: Zhijie Shen
> Attachments: YARN-514.1.patch, YARN-514.2.patch
>
>
> Currently, app submission is the only store operation performed synchronously
> because the app must be stored before the request returns with success. This
> makes the RM susceptible to blocking all client threads on slow store
> operations, resulting in RM being perceived as unavailable by clients.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission
[
https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13627489#comment-13627489
]
Hadoop QA commented on YARN-514:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577953/YARN-514.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/700//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/700//console
This message is automatically generated.
> Delayed store operations should not result in RM unavailability for app
> submission
> --
>
> Key: YARN-514
> URL: https://issues.apache.org/jira/browse/YARN-514
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Bikas Saha
>Assignee: Zhijie Shen
> Attachments: YARN-514.1.patch
>
>
> Currently, app submission is the only store operation performed synchronously
> because the app must be stored before the request returns with success. This
> makes the RM susceptible to blocking all client threads on slow store
> operations, resulting in RM being perceived as unavailable by clients.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
