[
https://issues.apache.org/jira/browse/YARN-6547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16008995#comment-16008995
]
Wangda Tan commented on YARN-6547:
----------------------------------
[~curino],
I found a several issues once applied the patch, not sure if it is related to
my environment:
Issue #1:
{code}
java.lang.ClassCastException:
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueMetrics
cannot be cast to
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueueMetrics
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueueMetrics.forQueue(FSQueueMetrics.java:223)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueueMetrics.forQueue(FSQueueMetrics.java:191)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1298)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.serviceInit(FairScheduler.java:1384)
at
org.apache.hadoop.yarn.sls.scheduler.SLSFairScheduler.serviceInit(SLSFairScheduler.java:322)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:737)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1165)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:317)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.sls.SLSRunner.startRM(SLSRunner.java:226)
at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:184)
at org.apache.hadoop.yarn.sls.SLSRunner.run(SLSRunner.java:845)
at
org.apache.hadoop.yarn.sls.BaseSLSRunnerTest.runSLS(BaseSLSRunnerTest.java:116)
at
org.apache.hadoop.yarn.sls.TestSLSRunner.testSimulatorRunning(TestSLSRunner.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
Issue #2:
{code}
org.apache.hadoop.yarn.server.resourcemanager.monitor.invariants.InvariantViolationException:
Invariant "AppsSubmitted == 10" is NOT holding, with bindings:
{AppsSubmitted=0}
at
org.apache.hadoop.yarn.server.resourcemanager.monitor.invariants.InvariantsChecker.logOrThrow(InvariantsChecker.java:74)
at
org.apache.hadoop.yarn.server.resourcemanager.monitor.invariants.MetricsInvariantChecker.editSchedule(MetricsInvariantChecker.java:175)
at
org.apache.hadoop.yarn.sls.BaseSLSRunnerTest.shutdownHookInvariantCheck(BaseSLSRunnerTest.java:144)
at
org.apache.hadoop.yarn.sls.BaseSLSRunnerTest.runSLS(BaseSLSRunnerTest.java:132)
at
org.apache.hadoop.yarn.sls.TestSLSRunner.testSimulatorRunning(TestSLSRunner.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
This is RUMEN test, not sure if it is related to YARN-6111
Issue #3:
{code}
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.monitor.invariants.MetricsInvariantChecker.init(MetricsInvariantChecker.java:93)
at
org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor.serviceInit(SchedulingMonitor.java:55)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:737)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1165)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:317)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.sls.SLSRunner.startRM(SLSRunner.java:226)
at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:184)
at org.apache.hadoop.yarn.sls.SLSRunner.run(SLSRunner.java:845)
at
org.apache.hadoop.yarn.sls.BaseSLSRunnerTest.runSLS(BaseSLSRunnerTest.java:116)
at
org.apache.hadoop.yarn.sls.TestSLSRunner.testSimulatorRunning(TestSLSRunner.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
In my local environment, all tests of TestSLSRunner cannot get passed. I'm
running under OSX, could you have chance to look at these problems?
> Enhance SLS-based tests leveraging invariant checker
> ----------------------------------------------------
>
> Key: YARN-6547
> URL: https://issues.apache.org/jira/browse/YARN-6547
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Carlo Curino
> Assignee: Carlo Curino
> Attachments: YARN-6547.v0.patch
>
>
> We can leverage {{InvariantChecker}}s to provide a more thorough validation
> of SLS-based tests. This patch introduces invariants checking during and at
> the end of the run.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
