[jira] [Moved] (YARN-149) ZK-based High Availability (HA) for ResourceManager (RM)
[
https://issues.apache.org/jira/browse/YARN-149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J moved MAPREDUCE-4345 to YARN-149:
-
Issue Type: New Feature (was: Improvement)
Key: YARN-149 (was: MAPREDUCE-4345)
Project: Hadoop YARN (was: Hadoop Map/Reduce)
ZK-based High Availability (HA) for ResourceManager (RM)
Key: YARN-149
URL: https://issues.apache.org/jira/browse/YARN-149
Project: Hadoop YARN
Issue Type: New Feature
Reporter: Harsh J
Assignee: Bikas Saha
One of the goals presented on MAPREDUCE-279 was to have high availability.
One way that was discussed, per Mahadev/others on
https://issues.apache.org/jira/browse/MAPREDUCE-2648 and other places, was ZK:
{quote}
Am not sure, if you already know about the MR-279 branch (the next version of
MR framework). We've been trying to integrate ZK into the framework from the
beginning. As for now, we are just doing restart with ZK but soon we should
have a HA soln with ZK.
{quote}
There is now MAPREDUCE-4343 that tracks recoverability via ZK. This JIRA is
meant to track HA via ZK.
Currently there isn't a HA solution for RM, via ZK or otherwise.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-127) Move RMAdmin tool to the client package
[ https://issues.apache.org/jira/browse/YARN-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472334#comment-13472334 ] Hudson commented on YARN-127: - Integrated in Hadoop-Hdfs-trunk #1190 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1190/]) YARN-127. Move RMAdmin tool to its correct location - the client module. Contributed by Vinod Kumar Vavilapalli. (Revision 1395781) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1395781 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/bin/yarn * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/RMAdminProtocol.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/RMAdminProtocolPB.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshAdminAclsRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshAdminAclsResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshNodesRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshNodesResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshQueuesRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshQueuesResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshServiceAclsRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshServiceAclsResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshSuperUserGroupsConfigurationRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshSuperUserGroupsConfigurationResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshUserToGroupsMappingsRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshUserToGroupsMappingsResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshAdminAclsRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshAdminAclsResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshNodesRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshNodesResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshQueuesRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshQueuesResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshServiceAclsRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshServiceAclsResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshSuperUserGroupsConfigurationRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshSuperUserGroupsConfigurationResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshUserToGroupsMappingsRequestPBImpl.java *
[jira] [Commented] (YARN-134) ClientToAMSecretManager creates keys without checking for validity of the appID
[ https://issues.apache.org/jira/browse/YARN-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472335#comment-13472335 ] Hudson commented on YARN-134: - Integrated in Hadoop-Hdfs-trunk #1190 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1190/]) YARN-134. Fixes ClientToAMSecretManager creates keys without checking for validity of the appID. (Contributed by Vinod Kumar Vavilapalli) (Revision 1395841) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1395841 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/BaseClientToAMTokenSecretManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientToAMSecretManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientToAMTokenSecretManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientTokenIdentifier.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/ApplicationMasterLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/ClientToAMTokenSecretManagerInRM.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRMWithCustomAMLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientTokens.java ClientToAMSecretManager creates keys without checking for validity of the appID --- Key: YARN-134 URL: https://issues.apache.org/jira/browse/YARN-134 Project: Hadoop YARN Issue Type: Sub-task Affects Versions: 0.23.3 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 2.0.3-alpha Attachments: YARN-134-20120927.txt, YARN-134-20121008.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-33) LocalDirsHandler should validate the configured local and log dirs
[ https://issues.apache.org/jira/browse/YARN-33?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472341#comment-13472341 ] Hudson commented on YARN-33: Integrated in Hadoop-Hdfs-trunk #1190 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1190/]) YARN-33. Change LocalDirsHandlerService to validate the configured local and log dirs. (Contributed by Mayank Bansal) (Revision 1395844) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1395844 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LocalDirsHandlerService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestLocalDirsHandlerService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java LocalDirsHandler should validate the configured local and log dirs -- Key: YARN-33 URL: https://issues.apache.org/jira/browse/YARN-33 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.0.2-alpha Reporter: Mayank Bansal Assignee: Mayank Bansal Fix For: 2.0.3-alpha Attachments: YARN-33-trunk-v1.patch, YARN-33-trunk-v2.patch, YARN-33-trunk-v3.patch, YARN-33-trunk-v4.patch, YARN-33-trunk-v5.patch, YARN-33-trunk-v6.patch, YARN-33-trunk-v7.patch WHen yarn.nodemanager.log-dirs is with file:// URI then startup of node manager creates the directory like file:// under CWD. WHich should not be there. Thanks, Mayank -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-40) Provide support for missing yarn commands
[ https://issues.apache.org/jira/browse/YARN-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472344#comment-13472344 ] Hudson commented on YARN-40: Integrated in Hadoop-Hdfs-trunk #1190 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1190/]) YARN-40. Fixed CHANGES.txt from previous commit. (Revision 1395799) YARN-40. Provided support for missing YARN commands Contributed by Devaraj K and Vinod Kumar Vavilapalli. (Revision 1395793) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1395799 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1395793 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/bin/yarn * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/NodeCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/YarnCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/YarnCommands.apt.vm Provide support for missing yarn commands - Key: YARN-40 URL: https://issues.apache.org/jira/browse/YARN-40 Project: Hadoop YARN Issue Type: Bug Components: client Affects Versions: 2.0.0-alpha Reporter: Devaraj K Assignee: Devaraj K Fix For: 2.0.3-alpha Attachments: MAPREDUCE-4155-1.patch, MAPREDUCE-4155.patch, YARN-40-1.patch, YARN-40-20120917.1.txt, YARN-40-20120917.txt, YARN-40-20120924.txt, YARN-40-20121008.txt, YARN-40.patch 1. status app-id 2. kill app-id (Already issue present with Id : MAPREDUCE-3793) 3. list-apps [all] 4. nodes-report -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-134) ClientToAMSecretManager creates keys without checking for validity of the appID
[ https://issues.apache.org/jira/browse/YARN-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472392#comment-13472392 ] Hudson commented on YARN-134: - Integrated in Hadoop-Mapreduce-trunk #1221 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1221/]) YARN-134. Fixes ClientToAMSecretManager creates keys without checking for validity of the appID. (Contributed by Vinod Kumar Vavilapalli) (Revision 1395841) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1395841 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/BaseClientToAMTokenSecretManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientToAMSecretManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientToAMTokenSecretManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientTokenIdentifier.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/ApplicationMasterLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/security/ClientToAMTokenSecretManagerInRM.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRMWithCustomAMLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientTokens.java ClientToAMSecretManager creates keys without checking for validity of the appID --- Key: YARN-134 URL: https://issues.apache.org/jira/browse/YARN-134 Project: Hadoop YARN Issue Type: Sub-task Affects Versions: 0.23.3 Reporter: Vinod Kumar Vavilapalli Assignee: Vinod Kumar Vavilapalli Fix For: 2.0.3-alpha Attachments: YARN-134-20120927.txt, YARN-134-20121008.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-127) Move RMAdmin tool to the client package
[ https://issues.apache.org/jira/browse/YARN-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472391#comment-13472391 ] Hudson commented on YARN-127: - Integrated in Hadoop-Mapreduce-trunk #1221 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1221/]) YARN-127. Move RMAdmin tool to its correct location - the client module. Contributed by Vinod Kumar Vavilapalli. (Revision 1395781) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1395781 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/bin/yarn * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/RMAdminProtocol.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/RMAdminProtocolPB.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshAdminAclsRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshAdminAclsResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshNodesRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshNodesResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshQueuesRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshQueuesResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshServiceAclsRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshServiceAclsResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshSuperUserGroupsConfigurationRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshSuperUserGroupsConfigurationResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshUserToGroupsMappingsRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RefreshUserToGroupsMappingsResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshAdminAclsRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshAdminAclsResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshNodesRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshNodesResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshQueuesRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshQueuesResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshServiceAclsRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshServiceAclsResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshSuperUserGroupsConfigurationRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshSuperUserGroupsConfigurationResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RefreshUserToGroupsMappingsRequestPBImpl.java *
[jira] [Commented] (YARN-33) LocalDirsHandler should validate the configured local and log dirs
[ https://issues.apache.org/jira/browse/YARN-33?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472398#comment-13472398 ] Hudson commented on YARN-33: Integrated in Hadoop-Mapreduce-trunk #1221 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1221/]) YARN-33. Change LocalDirsHandlerService to validate the configured local and log dirs. (Contributed by Mayank Bansal) (Revision 1395844) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1395844 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LocalDirsHandlerService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestLocalDirsHandlerService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java LocalDirsHandler should validate the configured local and log dirs -- Key: YARN-33 URL: https://issues.apache.org/jira/browse/YARN-33 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.0.2-alpha Reporter: Mayank Bansal Assignee: Mayank Bansal Fix For: 2.0.3-alpha Attachments: YARN-33-trunk-v1.patch, YARN-33-trunk-v2.patch, YARN-33-trunk-v3.patch, YARN-33-trunk-v4.patch, YARN-33-trunk-v5.patch, YARN-33-trunk-v6.patch, YARN-33-trunk-v7.patch WHen yarn.nodemanager.log-dirs is with file:// URI then startup of node manager creates the directory like file:// under CWD. WHich should not be there. Thanks, Mayank -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-140) Add capacity-scheduler-default.xml to provide a default set of configurations for the capacity scheduler.
[
https://issues.apache.org/jira/browse/YARN-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472496#comment-13472496
]
Alejandro Abdelnur commented on YARN-140:
-
Ahmed, overall looks good, there is one last change I'd do, where you add the
check *if (capacity == UNDEFINED queue.equals(root))* it should be *if
(capacity == UNDEFINED || queue.equals(root))*, as for ROOT it should always
be 100.
Add capacity-scheduler-default.xml to provide a default set of configurations
for the capacity scheduler.
-
Key: YARN-140
URL: https://issues.apache.org/jira/browse/YARN-140
Project: Hadoop YARN
Issue Type: Bug
Components: capacityscheduler
Reporter: Ahmed Radwan
Assignee: Ahmed Radwan
Attachments: YARN-140.patch, YARN-140_rev2.patch,
YARN-140_rev3.patch, YARN-140_rev4.patch, YARN-140_rev5_onlyForJenkins.patch,
YARN-140_rev5.patch, YARN-140_rev5_svn_mv.patch
When setting up the capacity scheduler users are faced with problems like:
{code}
FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
starting ResourceManager
java.lang.IllegalArgumentException: Illegal capacity of -1 for queue root
{code}
Which basically arises from missing basic configurations, which in many
cases, there is no need to explicitly provide, and a default configuration
will be sufficient. For example, to address the error above, the user need to
add a capacity of 100 to the root queue.
So, we need to add a capacity-scheduler-default.xml, this will be helpful to
provide the basic set of default configurations required to run the capacity
scheduler. The user can still override existing configurations or provide new
ones in capacity-scheduler.xml. This is similar to *-default.xml vs
*-site.xml for yarn, core, mapred, hdfs, etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-30) TestNMWebServicesApps, TestRMWebServicesApps and TestRMWebServicesNodes fail on jdk7
[ https://issues.apache.org/jira/browse/YARN-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472542#comment-13472542 ] Thomas Graves commented on YARN-30: --- I filed MAPREDUCE-4716 for the mapreduce side. I will upload new patches here shortly. TestNMWebServicesApps, TestRMWebServicesApps and TestRMWebServicesNodes fail on jdk7 Key: YARN-30 URL: https://issues.apache.org/jira/browse/YARN-30 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.0.2-alpha, 0.23.3 Reporter: Thomas Graves Assignee: Thomas Graves Labels: java7 Attachments: YARN-30.1.patch, YARN-30.patch, YARN-30.patch It looks like the string changed from const class to constant. Tests run: 19, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 6.786 sec FAILURE! testNodeAppsStateInvalid(org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesApps) Time elapsed: 0.248 sec FAILURE! java.lang.AssertionError: exception message doesn't match, got: No enum constant org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationState.FOO_STATE expected: No enum const class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationState.FOO_STATE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-30) TestNMWebServicesApps, TestRMWebServicesApps and TestRMWebServicesNodes fail on jdk7
[ https://issues.apache.org/jira/browse/YARN-30?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-30: -- Attachment: YARN-30.patch trunk patch without mapreduce sides as that will be in MAPREDUCE-4716. This is probably the same as Vinod's last patch, but I upmerged to latest and retested to be sure. TestNMWebServicesApps, TestRMWebServicesApps and TestRMWebServicesNodes fail on jdk7 Key: YARN-30 URL: https://issues.apache.org/jira/browse/YARN-30 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.0.2-alpha, 0.23.3 Reporter: Thomas Graves Assignee: Thomas Graves Labels: java7 Attachments: YARN-30.1.patch, YARN-30-branch23.patch, YARN-30.patch, YARN-30.patch, YARN-30.patch It looks like the string changed from const class to constant. Tests run: 19, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 6.786 sec FAILURE! testNodeAppsStateInvalid(org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesApps) Time elapsed: 0.248 sec FAILURE! java.lang.AssertionError: exception message doesn't match, got: No enum constant org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationState.FOO_STATE expected: No enum const class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationState.FOO_STATE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-140) Add capacity-scheduler-default.xml to provide a default set of configurations for the capacity scheduler.
[
https://issues.apache.org/jira/browse/YARN-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472594#comment-13472594
]
Alejandro Abdelnur commented on YARN-140:
-
the thing is ... root.capacity is not a config property, that is why we removed
it from the c-s.xml. it should ALWAYS be 100, I'd use an OR and I'd invert the
order to check first for queue=='root'
Add capacity-scheduler-default.xml to provide a default set of configurations
for the capacity scheduler.
-
Key: YARN-140
URL: https://issues.apache.org/jira/browse/YARN-140
Project: Hadoop YARN
Issue Type: Bug
Components: capacityscheduler
Reporter: Ahmed Radwan
Assignee: Ahmed Radwan
Attachments: YARN-140.patch, YARN-140_rev2.patch,
YARN-140_rev3.patch, YARN-140_rev4.patch, YARN-140_rev5_onlyForJenkins.patch,
YARN-140_rev5.patch, YARN-140_rev5_svn_mv.patch
When setting up the capacity scheduler users are faced with problems like:
{code}
FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
starting ResourceManager
java.lang.IllegalArgumentException: Illegal capacity of -1 for queue root
{code}
Which basically arises from missing basic configurations, which in many
cases, there is no need to explicitly provide, and a default configuration
will be sufficient. For example, to address the error above, the user need to
add a capacity of 100 to the root queue.
So, we need to add a capacity-scheduler-default.xml, this will be helpful to
provide the basic set of default configurations required to run the capacity
scheduler. The user can still override existing configurations or provide new
ones in capacity-scheduler.xml. This is similar to *-default.xml vs
*-site.xml for yarn, core, mapred, hdfs, etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-94) DistributedShell jar should point to Client as the main class by default
[
https://issues.apache.org/jira/browse/YARN-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472633#comment-13472633
]
Vinod Kumar Vavilapalli commented on YARN-94:
-
+1, this looks good. Looks like I had a stale jar the last time around. Kicking
jenkins.
DistributedShell jar should point to Client as the main class by default
Key: YARN-94
URL: https://issues.apache.org/jira/browse/YARN-94
Project: Hadoop YARN
Issue Type: Bug
Components: applications/distributed-shell
Reporter: Vinod Kumar Vavilapalli
Assignee: Hitesh Shah
Attachments: YARN-94.1.patch, YARN-94.2.patch
Today, it says so..
{code}
$ $YARN_HOME/bin/yarn jar
$YARN_HOME/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-$VERSION.jar
RunJar jarFile [mainClass] args...
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-94) DistributedShell jar should point to Client as the main class by default
[
https://issues.apache.org/jira/browse/YARN-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472681#comment-13472681
]
Hudson commented on YARN-94:
Integrated in Hadoop-Common-trunk-Commit #2837 (See
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2837/])
YARN-94. Modify DistributedShell to point to main-class by default, clean
up the help message, and hard-code the AM class. Contributed by Hitesh Shah.
(Revision 1396226)
Result = SUCCESS
vinodkv :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1396226
Files :
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/pom.xml
*
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
*
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
DistributedShell jar should point to Client as the main class by default
Key: YARN-94
URL: https://issues.apache.org/jira/browse/YARN-94
Project: Hadoop YARN
Issue Type: Bug
Components: applications/distributed-shell
Reporter: Vinod Kumar Vavilapalli
Assignee: Hitesh Shah
Attachments: YARN-94.1.patch, YARN-94.2.patch
Today, it says so..
{code}
$ $YARN_HOME/bin/yarn jar
$YARN_HOME/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-$VERSION.jar
RunJar jarFile [mainClass] args...
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-94) DistributedShell jar should point to Client as the main class by default
[
https://issues.apache.org/jira/browse/YARN-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472680#comment-13472680
]
Hudson commented on YARN-94:
Integrated in Hadoop-Hdfs-trunk-Commit #2899 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2899/])
YARN-94. Modify DistributedShell to point to main-class by default, clean
up the help message, and hard-code the AM class. Contributed by Hitesh Shah.
(Revision 1396226)
Result = SUCCESS
vinodkv :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1396226
Files :
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/pom.xml
*
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
*
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
DistributedShell jar should point to Client as the main class by default
Key: YARN-94
URL: https://issues.apache.org/jira/browse/YARN-94
Project: Hadoop YARN
Issue Type: Bug
Components: applications/distributed-shell
Reporter: Vinod Kumar Vavilapalli
Assignee: Hitesh Shah
Attachments: YARN-94.1.patch, YARN-94.2.patch
Today, it says so..
{code}
$ $YARN_HOME/bin/yarn jar
$YARN_HOME/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-$VERSION.jar
RunJar jarFile [mainClass] args...
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-30) TestNMWebServicesApps, TestRMWebServicesApps and TestRMWebServicesNodes fail on jdk7
[ https://issues.apache.org/jira/browse/YARN-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472712#comment-13472712 ] Vinod Kumar Vavilapalli commented on YARN-30: - Tx Thomas. I missed this one. Kicking Jenkins. TestNMWebServicesApps, TestRMWebServicesApps and TestRMWebServicesNodes fail on jdk7 Key: YARN-30 URL: https://issues.apache.org/jira/browse/YARN-30 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.0.2-alpha, 0.23.3 Reporter: Thomas Graves Assignee: Thomas Graves Labels: java7 Attachments: YARN-30.1.patch, YARN-30-branch23.patch, YARN-30.patch, YARN-30.patch, YARN-30.patch It looks like the string changed from const class to constant. Tests run: 19, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 6.786 sec FAILURE! testNodeAppsStateInvalid(org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesApps) Time elapsed: 0.248 sec FAILURE! java.lang.AssertionError: exception message doesn't match, got: No enum constant org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationState.FOO_STATE expected: No enum const class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationState.FOO_STATE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-94) DistributedShell jar should point to Client as the main class by default
[
https://issues.apache.org/jira/browse/YARN-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472717#comment-13472717
]
Hudson commented on YARN-94:
Integrated in Hadoop-Mapreduce-trunk-Commit #2860 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2860/])
YARN-94. Modify DistributedShell to point to main-class by default, clean
up the help message, and hard-code the AM class. Contributed by Hitesh Shah.
(Revision 1396226)
Result = FAILURE
vinodkv :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1396226
Files :
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/pom.xml
*
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
*
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
DistributedShell jar should point to Client as the main class by default
Key: YARN-94
URL: https://issues.apache.org/jira/browse/YARN-94
Project: Hadoop YARN
Issue Type: Bug
Components: applications/distributed-shell
Reporter: Vinod Kumar Vavilapalli
Assignee: Hitesh Shah
Fix For: 2.0.3-alpha
Attachments: YARN-94.1.patch, YARN-94.2.patch
Today, it says so..
{code}
$ $YARN_HOME/bin/yarn jar
$YARN_HOME/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-$VERSION.jar
RunJar jarFile [mainClass] args...
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-30) TestNMWebServicesApps, TestRMWebServicesApps and TestRMWebServicesNodes fail on jdk7
[
https://issues.apache.org/jira/browse/YARN-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472730#comment-13472730
]
Hadoop QA commented on YARN-30:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548427/YARN-30.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-YARN-Build/83//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/83//console
This message is automatically generated.
TestNMWebServicesApps, TestRMWebServicesApps and TestRMWebServicesNodes fail
on jdk7
Key: YARN-30
URL: https://issues.apache.org/jira/browse/YARN-30
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 2.0.2-alpha, 0.23.3
Reporter: Thomas Graves
Assignee: Thomas Graves
Labels: java7
Attachments: YARN-30.1.patch, YARN-30-branch23.patch, YARN-30.patch,
YARN-30.patch, YARN-30.patch
It looks like the string changed from const class to constant.
Tests run: 19, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 6.786 sec
FAILURE!
testNodeAppsStateInvalid(org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesApps)
Time elapsed: 0.248 sec FAILURE!
java.lang.AssertionError: exception message doesn't match, got: No enum
constant
org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationState.FOO_STATE
expected: No enum const class
org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationState.FOO_STATE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-30) TestNMWebServicesApps, TestRMWebServicesApps and TestRMWebServicesNodes fail on jdk7
[ https://issues.apache.org/jira/browse/YARN-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472848#comment-13472848 ] Hudson commented on YARN-30: Integrated in Hadoop-Common-trunk-Commit #2838 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2838/]) YARN-30. Fixed tests verifying web-services to work on JDK7. Contributed by Thomas Graves. (Revision 1396391) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1396391 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/webapp/WebServicesTestUtils.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestNMWebServicesApps.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java TestNMWebServicesApps, TestRMWebServicesApps and TestRMWebServicesNodes fail on jdk7 Key: YARN-30 URL: https://issues.apache.org/jira/browse/YARN-30 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.0.2-alpha, 0.23.3 Reporter: Thomas Graves Assignee: Thomas Graves Labels: java7 Fix For: 2.0.3-alpha, 0.23.5 Attachments: YARN-30.1.patch, YARN-30-branch23.patch, YARN-30.patch, YARN-30.patch, YARN-30.patch It looks like the string changed from const class to constant. Tests run: 19, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 6.786 sec FAILURE! testNodeAppsStateInvalid(org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesApps) Time elapsed: 0.248 sec FAILURE! java.lang.AssertionError: exception message doesn't match, got: No enum constant org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationState.FOO_STATE expected: No enum const class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationState.FOO_STATE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-103) Add a yarn AM - RM client module
[ https://issues.apache.org/jira/browse/YARN-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472863#comment-13472863 ] Siddharth Seth commented on YARN-103: - bq. This client is the barebones implementation and hence has no smarts. The checks can be implemented/may not be necessary in smarter versions of the client. Additionally, like you say, some information may be better exposed at different API's. Why have an additional smarter client on top of this one in YARN ? Individual apps may choose to have their own - MR for RPC specific error handling, etc. We could even considering getting rid of getNumClusterNodes / getAvailableResources since they're available via AllocateResponse. Would like others to take a look at the entire API specification before committing this. bq. I actually want to test against the MiniMRCluster instead of an expected behavior via a mockRM. I want the client test to fail if the RM behavior changes. So yes this is more of a functional test than a unit test. RM behaviour depends upon the underlying scheduler. For a simple case like this, the behaviour is likely to be the same across schedulers - but the test runs against the default - which is the CS right now. We don't differentiate between functional / unit tests at the moment - which makes the test runtimes reasonably high. (MiniClusetr tests take several seconds versus fractions of a second for mocks). In the current functional test - there's a sleep-poll cycle which could get stuck. bq. Per your comments in MAPREDUCE-4671, I have attached another version of the 4th patch that uses a Wrapper class for ResourceRequest instead of not deleting objects. You can compare both and see which one seems more palatable. Based on that MAPREDUCE-4671 can be updated if needed. A custom comparator may be sufficient - o.a.h.y.s.r.Application already uses this. Alternately, using the wrapper only for the 'ask' table may be simpler ? Looks like the patches accidentally include some hdfs changes. Add a yarn AM - RM client module Key: YARN-103 URL: https://issues.apache.org/jira/browse/YARN-103 Project: Hadoop YARN Issue Type: Improvement Reporter: Bikas Saha Assignee: Bikas Saha Attachments: YARN-103.1.patch, YARN-103.2.patch, YARN-103.3.patch, YARN-103.4.patch, YARN-103.4.wrapper.patch Add a basic client wrapper library to the AM RM protocol in order to prevent proliferation of code being duplicated everywhere. Provide helper functions to perform reverse mapping of container requests to RM allocation resource request table format. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-30) TestNMWebServicesApps, TestRMWebServicesApps and TestRMWebServicesNodes fail on jdk7
[ https://issues.apache.org/jira/browse/YARN-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472871#comment-13472871 ] Hudson commented on YARN-30: Integrated in Hadoop-Mapreduce-trunk-Commit #2861 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2861/]) YARN-30. Fixed tests verifying web-services to work on JDK7. Contributed by Thomas Graves. (Revision 1396391) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1396391 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/webapp/WebServicesTestUtils.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestNMWebServicesApps.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesNodes.java TestNMWebServicesApps, TestRMWebServicesApps and TestRMWebServicesNodes fail on jdk7 Key: YARN-30 URL: https://issues.apache.org/jira/browse/YARN-30 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.0.2-alpha, 0.23.3 Reporter: Thomas Graves Assignee: Thomas Graves Labels: java7 Fix For: 2.0.3-alpha, 0.23.5 Attachments: YARN-30.1.patch, YARN-30-branch23.patch, YARN-30.patch, YARN-30.patch, YARN-30.patch It looks like the string changed from const class to constant. Tests run: 19, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 6.786 sec FAILURE! testNodeAppsStateInvalid(org.apache.hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesApps) Time elapsed: 0.248 sec FAILURE! java.lang.AssertionError: exception message doesn't match, got: No enum constant org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationState.FOO_STATE expected: No enum const class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationState.FOO_STATE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Moved] (YARN-150) AppRejectedTransition does not unregister app from master service and scheduler
[ https://issues.apache.org/jira/browse/YARN-150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth moved MAPREDUCE-4436 to YARN-150: Affects Version/s: (was: 3.0.0) (was: 2.0.0-alpha) (was: 0.23.1) 2.0.0-alpha 3.0.0 0.23.3 Key: YARN-150 (was: MAPREDUCE-4436) Project: Hadoop YARN (was: Hadoop Map/Reduce) AppRejectedTransition does not unregister app from master service and scheduler --- Key: YARN-150 URL: https://issues.apache.org/jira/browse/YARN-150 Project: Hadoop YARN Issue Type: Bug Affects Versions: 0.23.3, 3.0.0, 2.0.0-alpha Reporter: Bikas Saha Assignee: Bikas Saha Attachments: MAPREDUCE-4436.1.patch AttemptStartedTransition() adds the app to the ApplicationMasterService and scheduler. when the scheduler rejects the app then AppRejectedTransition() forgets to unregister it from the ApplicationMasterService. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-150) AppRejectedTransition does not unregister app from master service and scheduler
[ https://issues.apache.org/jira/browse/YARN-150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472884#comment-13472884 ] Hudson commented on YARN-150: - Integrated in Hadoop-Hdfs-trunk-Commit #2901 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2901/]) YARN-150. Fixes AppRejectedTransition does not unregister a rejected app-attempt from the ApplicationMasterService (Contributed by Bikas Saha) (Revision 1396429) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1396429 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java AppRejectedTransition does not unregister app from master service and scheduler --- Key: YARN-150 URL: https://issues.apache.org/jira/browse/YARN-150 Project: Hadoop YARN Issue Type: Bug Affects Versions: 0.23.3, 3.0.0, 2.0.0-alpha Reporter: Bikas Saha Assignee: Bikas Saha Fix For: 2.0.3-alpha Attachments: MAPREDUCE-4436.1.patch AttemptStartedTransition() adds the app to the ApplicationMasterService and scheduler. when the scheduler rejects the app then AppRejectedTransition() forgets to unregister it from the ApplicationMasterService. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-150) AppRejectedTransition does not unregister app from master service and scheduler
[ https://issues.apache.org/jira/browse/YARN-150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472886#comment-13472886 ] Hudson commented on YARN-150: - Integrated in Hadoop-Common-trunk-Commit #2839 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2839/]) YARN-150. Fixes AppRejectedTransition does not unregister a rejected app-attempt from the ApplicationMasterService (Contributed by Bikas Saha) (Revision 1396429) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1396429 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java AppRejectedTransition does not unregister app from master service and scheduler --- Key: YARN-150 URL: https://issues.apache.org/jira/browse/YARN-150 Project: Hadoop YARN Issue Type: Bug Affects Versions: 0.23.3, 3.0.0, 2.0.0-alpha Reporter: Bikas Saha Assignee: Bikas Saha Fix For: 2.0.3-alpha Attachments: MAPREDUCE-4436.1.patch AttemptStartedTransition() adds the app to the ApplicationMasterService and scheduler. when the scheduler rejects the app then AppRejectedTransition() forgets to unregister it from the ApplicationMasterService. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-150) AppRejectedTransition does not unregister app from master service and scheduler
[ https://issues.apache.org/jira/browse/YARN-150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472912#comment-13472912 ] Hudson commented on YARN-150: - Integrated in Hadoop-Mapreduce-trunk-Commit #2862 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2862/]) YARN-150. Fixes AppRejectedTransition does not unregister a rejected app-attempt from the ApplicationMasterService (Contributed by Bikas Saha) (Revision 1396429) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1396429 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java AppRejectedTransition does not unregister app from master service and scheduler --- Key: YARN-150 URL: https://issues.apache.org/jira/browse/YARN-150 Project: Hadoop YARN Issue Type: Bug Affects Versions: 0.23.3, 3.0.0, 2.0.0-alpha Reporter: Bikas Saha Assignee: Bikas Saha Fix For: 2.0.3-alpha Attachments: MAPREDUCE-4436.1.patch AttemptStartedTransition() adds the app to the ApplicationMasterService and scheduler. when the scheduler rejects the app then AppRejectedTransition() forgets to unregister it from the ApplicationMasterService. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
