[ https://issues.apache.org/jira/browse/YARN-9844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16933510#comment-16933510 ]
Jim Brennan commented on YARN-9844: ----------------------------------- Here is a output from running the tests: {noformat} mvn test -DRunCapacitySchedulerPerfTests=true -Dtest=TestCapacitySchedulerPerf [INFO] Scanning for projects... [WARNING] [WARNING] Some problems were encountered while building the effective model for org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.10.0-SNAPSHOT [WARNING] 'dependencyManagement.dependencies.dependency.(groupId:artifactId:type:classifier)' must be unique: com.microsoft.azure:azure-storage:jar -> version 7.0.0 vs 5.4.0 @ org.apache.hadoop:hadoop-project:2.10.0-SNAPSHOT, /Users/jbrennan02/git/apache-hadoop/hadoop-project/pom.xml, line 1175, column 19 [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. [WARNING] [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Apache Hadoop YARN ResourceManager 2.10.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-yarn-server-resourcemanager --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- hadoop-maven-plugins:2.10.0-SNAPSHOT:protoc (compile-protoc) @ hadoop-yarn-server-resourcemanager --- [INFO] Wrote protoc checksums to file /Users/jbrennan02/git/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/hadoop-maven-plugins-protoc-checksums.json [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hadoop-yarn-server-resourcemanager --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hadoop-yarn-server-resourcemanager --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /Users/jbrennan02/git/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/resources [INFO] Copying 2 resources [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hadoop-yarn-server-resourcemanager --- [INFO] Compiling 3 source files to /Users/jbrennan02/git/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/classes [INFO] [INFO] --- hadoop-maven-plugins:2.10.0-SNAPSHOT:test-protoc (compile-test-protoc) @ hadoop-yarn-server-resourcemanager --- [INFO] Wrote protoc checksums to file /Users/jbrennan02/git/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/hadoop-maven-plugins-protoc-checksums.json [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hadoop-yarn-server-resourcemanager --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 11 resources [INFO] Copying 1 resource [INFO] Copying 2 resources [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hadoop-yarn-server-resourcemanager --- [INFO] Compiling 1 source file to /Users/jbrennan02/git/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes [INFO] [INFO] --- maven-jar-plugin:2.5:test-jar (default) @ hadoop-yarn-server-resourcemanager --- [INFO] Building jar: /Users/jbrennan02/git/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/hadoop-yarn-server-resourcemanager-2.10.0-SNAPSHOT-tests.jar [INFO] [INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hadoop-yarn-server-resourcemanager --- [INFO] [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf [ERROR] Tests run: 4, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 42.365 s <<< FAILURE! - in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf [ERROR] testUserLimitThroughputForFiveResources(org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf) Time elapsed: 0.038 s <<< ERROR! java.lang.ArrayIndexOutOfBoundsException: 2 at org.apache.hadoop.yarn.api.records.Resource.getResourceInformation(Resource.java:241) at org.apache.hadoop.yarn.api.records.Resource.setResourceValue(Resource.java:351) at org.apache.hadoop.yarn.util.resource.ResourceUtils.getResourceTypesMinimumAllocation(ResourceUtils.java:534) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.getMinimumAllocation(AbstractYarnScheduler.java:1341) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:322) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:395) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:749) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1146) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:302) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:161) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:140) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:136) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf.testUserLimitThroughputWithNumberOfResourceTypes(TestCapacitySchedulerPerf.java:114) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf.testUserLimitThroughputForFiveResources(TestCapacitySchedulerPerf.java:263) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) [ERROR] testUserLimitThroughputForFourResources(org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf) Time elapsed: 0.031 s <<< ERROR! java.lang.ArrayIndexOutOfBoundsException: 2 at org.apache.hadoop.yarn.api.records.Resource.getResourceInformation(Resource.java:241) at org.apache.hadoop.yarn.api.records.Resource.setResourceValue(Resource.java:351) at org.apache.hadoop.yarn.util.resource.ResourceUtils.getResourceTypesMinimumAllocation(ResourceUtils.java:534) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.getMinimumAllocation(AbstractYarnScheduler.java:1341) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:322) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:395) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:749) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1146) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:302) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:161) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:140) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:136) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf.testUserLimitThroughputWithNumberOfResourceTypes(TestCapacitySchedulerPerf.java:114) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf.testUserLimitThroughputForFourResources(TestCapacitySchedulerPerf.java:258) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) [ERROR] testUserLimitThroughputForThreeResources(org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf) Time elapsed: 0.031 s <<< ERROR! java.lang.ArrayIndexOutOfBoundsException: 2 at org.apache.hadoop.yarn.api.records.Resource.getResourceInformation(Resource.java:241) at org.apache.hadoop.yarn.api.records.Resource.setResourceValue(Resource.java:351) at org.apache.hadoop.yarn.util.resource.ResourceUtils.getResourceTypesMinimumAllocation(ResourceUtils.java:534) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.getMinimumAllocation(AbstractYarnScheduler.java:1341) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:322) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:395) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:749) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1146) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:302) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:161) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:140) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:136) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf.testUserLimitThroughputWithNumberOfResourceTypes(TestCapacitySchedulerPerf.java:114) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf.testUserLimitThroughputForThreeResources(TestCapacitySchedulerPerf.java:253) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) [INFO] [INFO] Results: [INFO] [ERROR] Errors: [ERROR] TestCapacitySchedulerPerf.testUserLimitThroughputForFiveResources:263->testUserLimitThroughputWithNumberOfResourceTypes:114 » ArrayIndexOutOfBounds [ERROR] TestCapacitySchedulerPerf.testUserLimitThroughputForFourResources:258->testUserLimitThroughputWithNumberOfResourceTypes:114 » ArrayIndexOutOfBounds [ERROR] TestCapacitySchedulerPerf.testUserLimitThroughputForThreeResources:253->testUserLimitThroughputWithNumberOfResourceTypes:114 » ArrayIndexOutOfBounds [INFO] [ERROR] Tests run: 4, Failures: 0, Errors: 3, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 52.113 s [INFO] Finished at: 2019-09-19T09:57:09-05:00 [INFO] Final Memory: 49M/626M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.21.0:test (default-test) on project hadoop-yarn-server-resourcemanager: There are test failures. [ERROR] [ERROR] Please refer to /Users/jbrennan02/git/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire-reports for the individual test results. [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureExcep {noformat} cc: [~jhung] > TestCapacitySchedulerPerf test errors in branch-2 > ------------------------------------------------- > > Key: YARN-9844 > URL: https://issues.apache.org/jira/browse/YARN-9844 > Project: Hadoop YARN > Issue Type: Bug > Components: test, yarn > Affects Versions: 2.10.0 > Reporter: Jim Brennan > Priority: Major > > These TestCapacitySchedulerPerf throughput tests are failing in branch-2: > {{[ERROR] > TestCapacitySchedulerPerf.testUserLimitThroughputForFiveResources:263->testUserLimitThroughputWithNumberOfResourceTypes:114 > » ArrayIndexOutOfBounds}} > {{[ERROR] > TestCapacitySchedulerPerf.testUserLimitThroughputForFourResources:258->testUserLimitThroughputWithNumberOfResourceTypes:114 > » ArrayIndexOutOfBounds}} > {{[ERROR] > TestCapacitySchedulerPerf.testUserLimitThroughputForThreeResources:253->testUserLimitThroughputWithNumberOfResourceTypes:114 > » ArrayIndexOutOfBounds}} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org