[jira] [Commented] (YARN-4330) MiniYARNCluster prints multiple Failed to instantiate default resource calculator warning messages
[
https://issues.apache.org/jira/browse/YARN-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072148#comment-15072148
]
Inigo Goiri commented on YARN-4330:
---
[~ste...@apache.org], is this good to go?
> MiniYARNCluster prints multiple Failed to instantiate default resource
> calculator warning messages
> ---
>
> Key: YARN-4330
> URL: https://issues.apache.org/jira/browse/YARN-4330
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test, yarn
>Affects Versions: 2.8.0
> Environment: OSX, JUnit
>Reporter: Steve Loughran
>Assignee: Varun Saxena
>Priority: Blocker
> Attachments: YARN-4330.01.patch
>
>
> Whenever I try to start a MiniYARNCluster on Branch-2 (commit #0b61cca), I
> see multiple stack traces warning me that a resource calculator plugin could
> not be created
> {code}
> (ResourceCalculatorPlugin.java:getResourceCalculatorPlugin(184)) -
> java.lang.UnsupportedOperationException: Could not determine OS: Failed to
> instantiate default resource calculator.
> java.lang.UnsupportedOperationException: Could not determine OS
> {code}
> This is a minicluster. It doesn't need resource calculation. It certainly
> doesn't need test logs being cluttered with even more stack traces which will
> only generate false alarms about tests failing.
> There needs to be a way to turn this off, and the minicluster should have it
> that way by default.
> Being ruthless and marking as a blocker, because its a fairly major
> regression for anyone testing with the minicluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4330) MiniYARNCluster prints multiple Failed to instantiate default resource calculator warning messages
[
https://issues.apache.org/jira/browse/YARN-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14998326#comment-14998326
]
Varun Saxena commented on YARN-4330:
TestContainerManagerSecurity is failing even without this patch.
Checkstyle related(longer than 80 lines).
For findbugs, there is YARN-4298
> MiniYARNCluster prints multiple Failed to instantiate default resource
> calculator warning messages
> ---
>
> Key: YARN-4330
> URL: https://issues.apache.org/jira/browse/YARN-4330
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test, yarn
>Affects Versions: 2.8.0
> Environment: OSX, JUnit
>Reporter: Steve Loughran
>Assignee: Varun Saxena
>Priority: Blocker
> Attachments: YARN-4330.01.patch
>
>
> Whenever I try to start a MiniYARNCluster on Branch-2 (commit #0b61cca), I
> see multiple stack traces warning me that a resource calculator plugin could
> not be created
> {code}
> (ResourceCalculatorPlugin.java:getResourceCalculatorPlugin(184)) -
> java.lang.UnsupportedOperationException: Could not determine OS: Failed to
> instantiate default resource calculator.
> java.lang.UnsupportedOperationException: Could not determine OS
> {code}
> This is a minicluster. It doesn't need resource calculation. It certainly
> doesn't need test logs being cluttered with even more stack traces which will
> only generate false alarms about tests failing.
> There needs to be a way to turn this off, and the minicluster should have it
> that way by default.
> Being ruthless and marking as a blocker, because its a fairly major
> regression for anyone testing with the minicluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4330) MiniYARNCluster prints multiple Failed to instantiate default resource calculator warning messages
[
https://issues.apache.org/jira/browse/YARN-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994272#comment-14994272
]
Varun Saxena commented on YARN-4330:
Patch does the following :
1. If node resource monitoring interval or container monitoring interval is <=
0, considering this is as disabling monitoring. Interval <=0 doesnt make much
sense anyways. Resource calculator plugin(even the default one) wont be
required if interval is <=0. Have made changes in relevant classes to take care
of this change. Also, I have set this config to 0 in MiniYARNCluster. Dummy
plugin wont be required in this case.
2. In NodeManagerHardwareUtils, we take the memory and CPU from config if
hardware detection is disabled irrespective of whether resource calculator
plugin can be created or not . Moved around the code in the class to check for
the config for disable first and returning value from config if its so. In
MiniYARNCluster have explicitly set it to false. I dont think hardware
detection is required for tests.
3. Catching UnsupportedOperationException and logging it at info. No stack
trace is printed. For other exceptions, stack trace will be printed(keeping it
consistent with previous behavior). Maybe stack trace in case of other
unexpected exceptions may be useful.
> MiniYARNCluster prints multiple Failed to instantiate default resource
> calculator warning messages
> ---
>
> Key: YARN-4330
> URL: https://issues.apache.org/jira/browse/YARN-4330
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test, yarn
>Affects Versions: 2.8.0
> Environment: OSX, JUnit
>Reporter: Steve Loughran
>Assignee: Varun Saxena
>Priority: Blocker
> Attachments: YARN-4330.01.patch
>
>
> Whenever I try to start a MiniYARNCluster on Branch-2 (commit #0b61cca), I
> see multiple stack traces warning me that a resource calculator plugin could
> not be created
> {code}
> (ResourceCalculatorPlugin.java:getResourceCalculatorPlugin(184)) -
> java.lang.UnsupportedOperationException: Could not determine OS: Failed to
> instantiate default resource calculator.
> java.lang.UnsupportedOperationException: Could not determine OS
> {code}
> This is a minicluster. It doesn't need resource calculation. It certainly
> doesn't need test logs being cluttered with even more stack traces which will
> only generate false alarms about tests failing.
> There needs to be a way to turn this off, and the minicluster should have it
> that way by default.
> Being ruthless and marking as a blocker, because its a fairly major
> regression for anyone testing with the minicluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4330) MiniYARNCluster prints multiple Failed to instantiate default resource calculator warning messages
[
https://issues.apache.org/jira/browse/YARN-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14993711#comment-14993711
]
Steve Loughran commented on YARN-4330:
--
+1 for downgrading the stack trace to DEBUG level; anything at INFO/WARN should
include the calculator plugin conf value in case that is the problem.
and another +1 for having a way to turn this off for minicluster tests. Having
a dummy plugin would be more generally useful, and avoid having yet another
config option
> MiniYARNCluster prints multiple Failed to instantiate default resource
> calculator warning messages
> ---
>
> Key: YARN-4330
> URL: https://issues.apache.org/jira/browse/YARN-4330
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test, yarn
>Affects Versions: 2.8.0
> Environment: OSX, JUnit
>Reporter: Steve Loughran
>Assignee: Varun Saxena
>Priority: Blocker
>
> Whenever I try to start a MiniYARNCluster on Branch-2 (commit #0b61cca), I
> see multiple stack traces warning me that a resource calculator plugin could
> not be created
> {code}
> (ResourceCalculatorPlugin.java:getResourceCalculatorPlugin(184)) -
> java.lang.UnsupportedOperationException: Could not determine OS: Failed to
> instantiate default resource calculator.
> java.lang.UnsupportedOperationException: Could not determine OS
> {code}
> This is a minicluster. It doesn't need resource calculation. It certainly
> doesn't need test logs being cluttered with even more stack traces which will
> only generate false alarms about tests failing.
> There needs to be a way to turn this off, and the minicluster should have it
> that way by default.
> Being ruthless and marking as a blocker, because its a fairly major
> regression for anyone testing with the minicluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4330) MiniYARNCluster prints multiple Failed to instantiate default resource calculator warning messages
[
https://issues.apache.org/jira/browse/YARN-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994386#comment-14994386
]
Hadoop QA commented on YARN-4330:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 5s
{color} | {color:blue} docker + precommit patch detected. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} trunk passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} trunk passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
39s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 17s
{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common in
trunk has 3 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} trunk passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 10s
{color} | {color:green} trunk passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 23s
{color} | {color:red} Patch generated 1 new checkstyle issues in
hadoop-yarn-project/hadoop-yarn (total was 29, now 29). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} the patch passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 46s
{color} | {color:green} hadoop-yarn-common in the patch passed with JDK
v1.8.0_60. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 23s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.8.0_60. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 5m 8s {color} |
{color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.8.0_60.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 2s
{color} | {color:green} hadoop-yarn-common in the patch passed with JDK
v1.7.0_79. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 51s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.7.0_79. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 5m 10s {color}
| {color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.7.0_79.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} |
[jira] [Commented] (YARN-4330) MiniYARNCluster prints multiple Failed to instantiate default resource calculator warning messages
[
https://issues.apache.org/jira/browse/YARN-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992473#comment-14992473
]
Varun Saxena commented on YARN-4330:
Its not retrying per say. Its just that we are trying to get memory and CPU
info at multiple places. And at some places(for monitoring) its either trying
to read the calculator plugin class from config and at some just directly the
default one(while trying to detect NM's CPU/memory capability). As for default
resource calculator plugin, Mac is not supported, hence the
UnsupportedOperationException.
While monitoring, if resource calculator plugin class is not configured, trying
to load default calculator plugin(and hence this code path) is the default
behavior. We cant really switch it off. But we need not print the whole stack
trace for UnsupportedOperationException.
For MiniYARNCluster, we can do a few more things.
When we try and load default resource calculator plugin via
NodeManagerHardwareUtils class(to detect NMs' CPU/memory), we can switch off
this behavior via a config. Code can be rearranged so that this error doesnt
show up and we check the config first. And this config can be set to false in
MiniYarnCluster
Also, a dummy plugin implementation can also be included in MiniYarnCluster and
set in config so that it does not try to load default resource calculator
> MiniYARNCluster prints multiple Failed to instantiate default resource
> calculator warning messages
> ---
>
> Key: YARN-4330
> URL: https://issues.apache.org/jira/browse/YARN-4330
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test, yarn
>Affects Versions: 2.8.0
> Environment: OSX, JUnit
>Reporter: Steve Loughran
>Assignee: Varun Saxena
>Priority: Blocker
>
> Whenever I try to start a MiniYARNCluster on Branch-2 (commit #0b61cca), I
> see multiple stack traces warning me that a resource calculator plugin could
> not be created
> {code}
> (ResourceCalculatorPlugin.java:getResourceCalculatorPlugin(184)) -
> java.lang.UnsupportedOperationException: Could not determine OS: Failed to
> instantiate default resource calculator.
> java.lang.UnsupportedOperationException: Could not determine OS
> {code}
> This is a minicluster. It doesn't need resource calculation. It certainly
> doesn't need test logs being cluttered with even more stack traces which will
> only generate false alarms about tests failing.
> There needs to be a way to turn this off, and the minicluster should have it
> that way by default.
> Being ruthless and marking as a blocker, because its a fairly major
> regression for anyone testing with the minicluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4330) MiniYARNCluster prints multiple Failed to instantiate default resource calculator warning messages
[
https://issues.apache.org/jira/browse/YARN-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990047#comment-14990047
]
Steve Loughran commented on YARN-4330:
--
Looks like YARN-3534 triggered this.
Full Stack: note the sheer number of repeated traces
{code}
Projects/slider/slider-core/target/teststandalonerest/teststandalonerest-logDir-nm-0_0
2015-11-04 17:49:31,322 [Thread-2] INFO server.MiniYARNCluster
(MiniYARNCluster.java:serviceInit(540)) - Starting NM: 0
2015-11-04 17:49:31,383 [Thread-2] INFO nodemanager.NodeManager
(NodeManager.java:getNodeHealthScriptRunner(255)) - Node Manager health check
script is not available or doesn't have execute permission, so not starting the
node health script runner.
2015-11-04 17:49:31,469 [Thread-2] WARN util.ResourceCalculatorPlugin
(ResourceCalculatorPlugin.java:getResourceCalculatorPlugin(184)) -
java.lang.UnsupportedOperationException: Could not determine OS: Failed to
instantiate default resource calculator.
java.lang.UnsupportedOperationException: Could not determine OS
at org.apache.hadoop.util.SysInfo.newInstance(SysInfo.java:43)
at
org.apache.hadoop.yarn.util.ResourceCalculatorPlugin.(ResourceCalculatorPlugin.java:41)
at
org.apache.hadoop.yarn.util.ResourceCalculatorPlugin.getResourceCalculatorPlugin(ResourceCalculatorPlugin.java:182)
at
org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl.serviceInit(NodeResourceMonitorImpl.java:73)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:356)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper.serviceInit(MiniYARNCluster.java:541)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at
org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:273)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.service.Service$init.call(Unknown Source)
at
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
at
org.apache.slider.test.YarnMiniClusterTestBase.createMiniCluster(YarnMiniClusterTestBase.groovy:291)
at
org.apache.slider.test.YarnZKMiniClusterTestBase.createMiniCluster(YarnZKMiniClusterTestBase.groovy:110)
at
org.apache.slider.test.YarnZKMiniClusterTestBase.createMiniCluster(YarnZKMiniClusterTestBase.groovy:127)
at
org.apache.slider.agent.rest.TestStandaloneREST.testStandaloneREST(TestStandaloneREST.groovy:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
2015-11-04 17:49:31,472 [Thread-2] INFO nodemanager.NodeResourceMonitorImpl
(NodeResourceMonitorImpl.java:serviceInit(76)) - Using
ResourceCalculatorPlugin : null
2015-11-04 17:49:31,475 [Thread-2] INFO containermanager.ContainerManagerImpl
(ContainerManagerImpl.java:serviceInit(261)) - AMRMProxyService is disabled
2015-11-04 17:49:31,475 [Thread-2] INFO localizer.ResourceLocalizationService
(ResourceLocalizationService.java:validateConf(224)) - per directory file limit
= 8192
2015-11-04 17:49:31,549 [Thread-2] WARN util.ResourceCalculatorPlugin
(ResourceCalculatorPlugin.java:getResourceCalculatorPlugin(184)) -
java.lang.UnsupportedOperationException: Could not determine OS: Failed to
instantiate default resource calculator.
java.lang.UnsupportedOperationException: Could not determine OS
[jira] [Commented] (YARN-4330) MiniYARNCluster prints multiple Failed to instantiate default resource calculator warning messages
[
https://issues.apache.org/jira/browse/YARN-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990051#comment-14990051
]
Steve Loughran commented on YARN-4330:
--
As well as having a way to turn this feature off for miniclusters, the code
trying to instantiate the resource calculator should recognise the falure and
fallback, rather than retry. Retrying isn't going to fix this.
> MiniYARNCluster prints multiple Failed to instantiate default resource
> calculator warning messages
> ---
>
> Key: YARN-4330
> URL: https://issues.apache.org/jira/browse/YARN-4330
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test, yarn
>Affects Versions: 2.8.0
> Environment: OSX, JUnit
>Reporter: Steve Loughran
>Priority: Blocker
>
> Whenever I try to start a MiniYARNCluster on Branch-2 (commit #0b61cca), I
> see multiple stack traces warning me that a resource calculator plugin could
> not be created
> {code}
> (ResourceCalculatorPlugin.java:getResourceCalculatorPlugin(184)) -
> java.lang.UnsupportedOperationException: Could not determine OS: Failed to
> instantiate default resource calculator.
> java.lang.UnsupportedOperationException: Could not determine OS
> {code}
> This is a minicluster. It doesn't need resource calculation. It certainly
> doesn't need test logs being cluttered with even more stack traces which will
> only generate false alarms about tests failing.
> There needs to be a way to turn this off, and the minicluster should have it
> that way by default.
> Being ruthless and marking as a blocker, because its a fairly major
> regression for anyone testing with the minicluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
