[jira] [Commented] (YARN-1758) MiniYARNCluster broken post YARN-1666
[ https://issues.apache.org/jira/browse/YARN-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13919288#comment-13919288 ] Hudson commented on YARN-1758: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1691 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1691/]) YARN-1758. Fixed ResourceManager to not mandate the presence of site specific configuration files and thus fix failures in downstream tests. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1573695) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/FileSystemBasedConfigurationProvider.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java MiniYARNCluster broken post YARN-1666 - Key: YARN-1758 URL: https://issues.apache.org/jira/browse/YARN-1758 Project: Hadoop YARN Issue Type: Bug Reporter: Hitesh Shah Assignee: Xuan Gong Priority: Blocker Fix For: 2.4.0 Attachments: YARN-1758.1.patch, YARN-1758.2.patch NPE seen when trying to use MiniYARNCluster -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1758) MiniYARNCluster broken post YARN-1666
[ https://issues.apache.org/jira/browse/YARN-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918462#comment-13918462 ] Vinod Kumar Vavilapalli commented on YARN-1758: --- This looks fine enough for me for now. In the interest of progress, let's track YARN-1759 separately. +1. Checking this in now. MiniYARNCluster broken post YARN-1666 - Key: YARN-1758 URL: https://issues.apache.org/jira/browse/YARN-1758 Project: Hadoop YARN Issue Type: Bug Reporter: Hitesh Shah Assignee: Xuan Gong Priority: Blocker Attachments: YARN-1758.1.patch, YARN-1758.2.patch NPE seen when trying to use MiniYARNCluster -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1758) MiniYARNCluster broken post YARN-1666
[ https://issues.apache.org/jira/browse/YARN-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918525#comment-13918525 ] Hudson commented on YARN-1758: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5254 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5254/]) YARN-1758. Fixed ResourceManager to not mandate the presence of site specific configuration files and thus fix failures in downstream tests. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1573695) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/FileSystemBasedConfigurationProvider.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java MiniYARNCluster broken post YARN-1666 - Key: YARN-1758 URL: https://issues.apache.org/jira/browse/YARN-1758 Project: Hadoop YARN Issue Type: Bug Reporter: Hitesh Shah Assignee: Xuan Gong Priority: Blocker Fix For: 2.4.0 Attachments: YARN-1758.1.patch, YARN-1758.2.patch NPE seen when trying to use MiniYARNCluster -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1758) MiniYARNCluster broken post YARN-1666
[ https://issues.apache.org/jira/browse/YARN-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917511#comment-13917511 ] Hitesh Shah commented on YARN-1758: --- [~xgong] Have you given though to YARN-1759 as part of this fix? MiniYARNCluster broken post YARN-1666 - Key: YARN-1758 URL: https://issues.apache.org/jira/browse/YARN-1758 Project: Hadoop YARN Issue Type: Bug Reporter: Hitesh Shah Assignee: Xuan Gong Priority: Blocker Attachments: YARN-1758.1.patch, YARN-1758.2.patch NPE seen when trying to use MiniYARNCluster -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1758) MiniYARNCluster broken post YARN-1666
[ https://issues.apache.org/jira/browse/YARN-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917757#comment-13917757 ] Xuan Gong commented on YARN-1758: - bq. Xuan Gong Have you given though to YARN-1759 as part of this fix? I have some comments in YARN-1759. We can start the discussion there. These two tickets are not much related. MiniYARNCluster broken post YARN-1666 - Key: YARN-1758 URL: https://issues.apache.org/jira/browse/YARN-1758 Project: Hadoop YARN Issue Type: Bug Reporter: Hitesh Shah Assignee: Xuan Gong Priority: Blocker Attachments: YARN-1758.1.patch, YARN-1758.2.patch NPE seen when trying to use MiniYARNCluster -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1758) MiniYARNCluster broken post YARN-1666
[ https://issues.apache.org/jira/browse/YARN-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917291#comment-13917291 ] Xuan Gong commented on YARN-1758: - create the patch based on the latests trunk code MiniYARNCluster broken post YARN-1666 - Key: YARN-1758 URL: https://issues.apache.org/jira/browse/YARN-1758 Project: Hadoop YARN Issue Type: Bug Reporter: Hitesh Shah Assignee: Xuan Gong Priority: Blocker Attachments: YARN-1758.1.patch, YARN-1758.2.patch NPE seen when trying to use MiniYARNCluster -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1758) MiniYARNCluster broken post YARN-1666
[
https://issues.apache.org/jira/browse/YARN-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13917296#comment-13917296
]
Hadoop QA commented on YARN-1758:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12631992/YARN-1758.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3225//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3225//console
This message is automatically generated.
MiniYARNCluster broken post YARN-1666
-
Key: YARN-1758
URL: https://issues.apache.org/jira/browse/YARN-1758
Project: Hadoop YARN
Issue Type: Bug
Reporter: Hitesh Shah
Assignee: Xuan Gong
Priority: Blocker
Attachments: YARN-1758.1.patch, YARN-1758.2.patch
NPE seen when trying to use MiniYARNCluster
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1758) MiniYARNCluster broken post YARN-1666
[ https://issues.apache.org/jira/browse/YARN-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13915440#comment-13915440 ] Xuan Gong commented on YARN-1758: - Upload a patch to fix this issue, which includes: 1. if Configuration file is not exist, instead of throwing exception, it will simply return null 2. RM, Scheduler or other active services will do the null check before calling conf.addResource(). If the inputStream is NULL (meaning the configuration files are not exist), we will not call conf.addResource() to reload the configuration. In this case, we will get NullPointerException 3. create a testcase to verify that RM can start successfully even we do not provide any configuration files. MiniYARNCluster broken post YARN-1666 - Key: YARN-1758 URL: https://issues.apache.org/jira/browse/YARN-1758 Project: Hadoop YARN Issue Type: Bug Reporter: Hitesh Shah Assignee: Xuan Gong Priority: Blocker Attachments: YARN-1758.1.patch NPE seen when trying to use MiniYARNCluster -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1758) MiniYARNCluster broken post YARN-1666
[
https://issues.apache.org/jira/browse/YARN-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13915464#comment-13915464
]
Hadoop QA commented on YARN-1758:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12631668/YARN-1758.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3212//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3212//console
This message is automatically generated.
MiniYARNCluster broken post YARN-1666
-
Key: YARN-1758
URL: https://issues.apache.org/jira/browse/YARN-1758
Project: Hadoop YARN
Issue Type: Bug
Reporter: Hitesh Shah
Assignee: Xuan Gong
Priority: Blocker
Attachments: YARN-1758.1.patch
NPE seen when trying to use MiniYARNCluster
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1758) MiniYARNCluster broken post YARN-1666
[ https://issues.apache.org/jira/browse/YARN-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13911827#comment-13911827 ] Tsuyoshi OZAWA commented on YARN-1758: -- Thank you for reporting, [~hitesh]. I cannot reproduce NPE with a command mvn test under hadoop-yarn-project on local. Could you tell me the case this problem occurs? MiniYARNCluster broken post YARN-1666 - Key: YARN-1758 URL: https://issues.apache.org/jira/browse/YARN-1758 Project: Hadoop YARN Issue Type: Bug Reporter: Hitesh Shah NPE seen when trying to use MiniYARNCluster -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1758) MiniYARNCluster broken post YARN-1666
[ https://issues.apache.org/jira/browse/YARN-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13911863#comment-13911863 ] Hitesh Shah commented on YARN-1758: --- [~ozawa] The problem is due to the loading of yarn-site.xml and other resources in ResourceManager::serviceInit(). It does not show up in yarn tests as the yarn-site.xml is added into yarn-resourcemanager-tests.jar I believe. However, for all downstream projects, they depend on hadoop-yarn-server-tests-tests.jar for MiniYarnCluster which itself does not have the necessary yarn-site, etc. I think the fix might be as simple as moving the required configs from hadoop-yarn-server-resourcemanager/src/test/resources/ to hadoop-yarn-server-tests/src/test/resources/ so that the required confs are bundled in the same tests jar. MiniYARNCluster broken post YARN-1666 - Key: YARN-1758 URL: https://issues.apache.org/jira/browse/YARN-1758 Project: Hadoop YARN Issue Type: Bug Reporter: Hitesh Shah NPE seen when trying to use MiniYARNCluster -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1758) MiniYARNCluster broken post YARN-1666
[ https://issues.apache.org/jira/browse/YARN-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13911902#comment-13911902 ] Xuan Gong commented on YARN-1758: - Do not move the configs out of hadoop-yarn-server-resourcemanager/src/test/resources/. TestRMAdminService does not use miniYarnCluster and need these configs MiniYARNCluster broken post YARN-1666 - Key: YARN-1758 URL: https://issues.apache.org/jira/browse/YARN-1758 Project: Hadoop YARN Issue Type: Bug Reporter: Hitesh Shah Assignee: Xuan Gong NPE seen when trying to use MiniYARNCluster -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1758) MiniYARNCluster broken post YARN-1666
[ https://issues.apache.org/jira/browse/YARN-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13910894#comment-13910894 ] Hitesh Shah commented on YARN-1758: --- Exception: null java.lang.NullPointerException at org.apache.hadoop.conf.Configuration$Resource.init(Configuration.java:182) at org.apache.hadoop.conf.Configuration.addResource(Configuration.java:751) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:193) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:268) at org.apache.hadoop.yarn.server.MiniYARNCluster.access$400(MiniYARNCluster.java:90) at org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceInit(MiniYARNCluster.java:419) MiniYARNCluster broken post YARN-1666 - Key: YARN-1758 URL: https://issues.apache.org/jira/browse/YARN-1758 Project: Hadoop YARN Issue Type: Bug Reporter: Hitesh Shah NPE seen when trying to use MiniYARNCluster -- This message was sent by Atlassian JIRA (v6.1.5#6160)
