I've switched YARN log retention on and now see a lot of errors, starting with java.lang.RuntimeException: java.io.IOException: No FileSystem for scheme: hdfs
Do I have a problem with libs or my env vars like HADOOP_CONF? Going one step back, how would a Twill job be properly packaged from Maven? Currently, I'm using maven-assembly-plugin:jar-with-dependencies which builds one big jar with all libs, which ends up in a libs/ directory in the twill-bundled jar. Bernd On Sat, Dec 21, 2013 at 11:36 AM, Bernd Fondermann < [email protected]> wrote: > this is the exit log: > > 2013-12-17 16:06:48,470 INFO > org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Application > application_1387295722390_0002 failed 2 times due to AM Container for > appattempt_1387295722390_0002_000002 exited with exitCode: 1 due to: > Exception from container-launch: > org.apache.hadoop.util.Shell$ExitCodeException: > at org.apache.hadoop.util.Shell.runCommand(Shell.java:464) > at org.apache.hadoop.util.Shell.run(Shell.java:379) > at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589) > at > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79) > at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > > I'll try do more diagnostics... > > Bernd > > > > On Sat, Dec 21, 2013 at 7:14 AM, Terence Yim <[email protected]> wrote: > >> Hi Bernd, >> >> I never see this before. Do you know what's the exit code from the >> YARN log? Currently there is no good way to stage an Twill app besides >> running a unit test with a mini YARN cluster (see YarnTestSuite in the >> Twill source). >> >> Terence >> >> On Wed, Dec 18, 2013 at 6:01 AM, Bernd Fondermann >> <[email protected]> wrote: >> > Hi everyone, >> > >> > I'm trying to run a Twill app on YARN. Attempts fail immediately with >> > org.apache.hadoop.util.Shell$ExitCodeException. >> > >> > The rm logs report the launch cmd line as: >> > java -Djava.io.tmpdir=tmp -Dyarn.appId=$YARN_APP_ID_STR >> > -Dtwill.app=$TWILL_APP_NAME -cp launcher.jar:$HADOOP_CONF_DIR -Xmx362m >> > org.apache.twill.launcher.TwillLauncher appMaster.jar >> > org.apache.twill.internal.appmaster.ApplicationMasterMain false >> > 1><LOG_DIR>/stdout 2><LOG_DIR>/stderr >> > >> > Any idea what the problem could be? >> > Ist there a recommended way to stage a Twill app before running it on a >> > remote cluster? >> > >> > Thanks, >> > >> > Bernd >> > >
