[
https://issues.apache.org/jira/browse/YARN-482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634676#comment-13634676
]
Alejandro Abdelnur commented on YARN-482:
-----------------------------------------
Karthik, i've run the tests from the patch locally and the JVM is crashing, can
you please check what may be the problem.
I've run:
{code}
$ mvn test
-Dtest=TestFairScheduler,TestComputeFairShares,TestSchedulingPolicy,TestSchedulingModemvn
-X
{code}
Got:
{code}
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Forking command line: /bin/sh -c cd
/Users/tucu/src/apache/hadoop/svn/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
&& /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java
-Xmx1024m -XX:+HeapDumpOnOutOfMemoryError -jar
/Users/tucu/src/apache/hadoop/svn/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefirebooter5486352351438243241.jar
/Users/tucu/src/apache/hadoop/svn/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefire4968832785999611492tmp
/Users/tucu/src/apache/hadoop/svn/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefire_01026711510169742052tmp
Running
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestComputeFairShares
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.302 sec
Forking command line: /bin/sh -c cd
/Users/tucu/src/apache/hadoop/svn/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
&& /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java
-Xmx1024m -XX:+HeapDumpOnOutOfMemoryError -jar
/Users/tucu/src/apache/hadoop/svn/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefirebooter6956429096288379227.jar
/Users/tucu/src/apache/hadoop/svn/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefire3733324656903328591tmp
/Users/tucu/src/apache/hadoop/svn/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefire_11280932507923501382tmp
Running
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
Forking command line: /bin/sh -c cd
/Users/tucu/src/apache/hadoop/svn/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
&& /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java
-Xmx1024m -XX:+HeapDumpOnOutOfMemoryError -jar
/Users/tucu/src/apache/hadoop/svn/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefirebooter4319097526565824667.jar
/Users/tucu/src/apache/hadoop/svn/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefire3968109606310307351tmp
/Users/tucu/src/apache/hadoop/svn/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire/surefire_29016085273868453729tmp
Running
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestSchedulingPolicy
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.272 sec
Results :
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS [0.335s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [0.349s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [0.432s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [0.093s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.094s]
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [1.362s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [0.128s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [0.097s]
[INFO] Apache Hadoop Common .............................. SUCCESS [4.521s]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.066s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [9.140s]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [0.164s]
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [0.345s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.022s]
[INFO] hadoop-yarn ....................................... SUCCESS [0.107s]
[INFO] hadoop-yarn-api ................................... SUCCESS [2.293s]
[INFO] hadoop-yarn-common ................................ SUCCESS [0.675s]
[INFO] hadoop-yarn-server ................................ SUCCESS [0.125s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [0.502s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [0.434s]
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [0.243s]
[INFO] hadoop-yarn-server-resourcemanager ................ FAILURE [4.374s]
[INFO] hadoop-yarn-server-tests .......................... SKIPPED
[INFO] hadoop-yarn-client ................................ SKIPPED
[INFO] hadoop-yarn-applications .......................... SKIPPED
[INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
[INFO] hadoop-mapreduce-client ........................... SKIPPED
[INFO] hadoop-mapreduce-client-core ...................... SKIPPED
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SKIPPED
[INFO] hadoop-yarn-site .................................. SKIPPED
[INFO] hadoop-yarn-project ............................... SKIPPED
[INFO] hadoop-mapreduce-client-common .................... SKIPPED
[INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
[INFO] hadoop-mapreduce-client-app ....................... SKIPPED
[INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
[INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
[INFO] hadoop-mapreduce-client-hs-plugins ................ SKIPPED
[INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
[INFO] hadoop-mapreduce .................................. SKIPPED
[INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
[INFO] Apache Hadoop Distributed Copy .................... SKIPPED
[INFO] Apache Hadoop Archives ............................ SKIPPED
[INFO] Apache Hadoop Rumen ............................... SKIPPED
[INFO] Apache Hadoop Gridmix ............................. SKIPPED
[INFO] Apache Hadoop Data Join ........................... SKIPPED
[INFO] Apache Hadoop Extras .............................. SKIPPED
[INFO] Apache Hadoop Pipes ............................... SKIPPED
[INFO] Apache Hadoop Tools Dist .......................... SKIPPED
[INFO] Apache Hadoop Tools ............................... SKIPPED
[INFO] Apache Hadoop Distribution ........................ SKIPPED
[INFO] Apache Hadoop Client .............................. SKIPPED
[INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 27.073s
[INFO] Finished at: Wed Apr 17 17:28:32 PDT 2013
[INFO] Final Memory: 40M/150M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on
project hadoop-yarn-server-resourcemanager: ExecutionException; nested
exception is java.util.concurrent.ExecutionException:
java.lang.RuntimeException: The forked VM terminated without saying properly
goodbye. VM crash or System.exit called ? -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on
project hadoop-yarn-server-resourcemanager: ExecutionException; nested
exception is java.util.concurrent.ExecutionException:
java.lang.RuntimeException: The forked VM terminated without saying properly
goodbye. VM crash or System.exit called ?
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at
org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
at
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
at
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Caused by: org.apache.maven.plugin.MojoExecutionException: ExecutionException;
nested exception is java.util.concurrent.ExecutionException:
java.lang.RuntimeException: The forked VM terminated without saying properly
goodbye. VM crash or System.exit called ?
at
org.apache.maven.plugin.surefire.SurefirePlugin.assertNoException(SurefirePlugin.java:160)
at
org.apache.maven.plugin.surefire.SurefirePlugin.handleSummary(SurefirePlugin.java:149)
at
org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:650)
at
org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:586)
at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
... 19 more
Caused by: org.apache.maven.surefire.booter.SurefireBooterForkException:
ExecutionException; nested exception is
java.util.concurrent.ExecutionException: java.lang.RuntimeException: The forked
VM terminated without saying properly goodbye. VM crash or System.exit called ?
at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkPerTestSet(ForkStarter.java:192)
at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:121)
at
org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:740)
at
org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAllProviders(AbstractSurefireMojo.java:682)
at
org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:648)
... 22 more
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException:
The forked VM terminated without saying properly goodbye. VM crash or
System.exit called ?
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkPerTestSet(ForkStarter.java:176)
... 26 more
Caused by: java.lang.RuntimeException: The forked VM terminated without saying
properly goodbye. VM crash or System.exit called ?
at
org.apache.maven.plugin.surefire.booterclient.output.ForkClient.close(ForkClient.java:257)
at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:301)
at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.access$100(ForkStarter.java:75)
at
org.apache.maven.plugin.surefire.booterclient.ForkStarter$1.call(ForkStarter.java:163)
at
org.apache.maven.plugin.surefire.booterclient.ForkStarter$1.call(ForkStarter.java:160)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:680)
[ERROR]
{code}
> FS: Extend SchedulingMode to intermediate queues
> ------------------------------------------------
>
> Key: YARN-482
> URL: https://issues.apache.org/jira/browse/YARN-482
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
> Affects Versions: 2.0.3-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: yarn-482.patch, yarn-482.patch, yarn-482.patch,
> yarn-482.patch, yarn-482.patch
>
>
> FS allows setting {{SchedulingMode}} for leaf queues. Extending this to
> non-leaf queues allows using different kinds of fairness: e.g., root can have
> three child queues - fair-mem, drf-cpu-mem, drf-cpu-disk-mem taking different
> number of resources into account. In turn, this allows users to decide on the
> scheduling latency vs sophistication of the scheduling mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
