Him strange. What is the Hadoop version I'm your cluster? Sent from my iPhone
> On Dec 24, 2013, at 6:40 AM, Bernd Fondermann <[email protected]> > wrote: > > strange. I had to do two things to solve this: > > 1. I added to core-site.xml the property fs.hdfs.impl = > org.apache.hadoop.hdfs.DistributedFileSystem > I don't really understand why this is needed, but it solved problems with > not being able to use hdfs URLs when starting the local Twill Driver and > also the on-cluster app. > > 2. I excluded snappy 1.0.4.1 from twill-core and added a new dependency on > 1.0.5, as suggested by a sparse post on StackOverflow. > > Bernd > > > > On Mon, Dec 23, 2013 at 8:19 AM, Bernd Fondermann < > [email protected]> wrote: > >> I've switched YARN log retention on and now see a lot of errors, starting >> with java.lang.RuntimeException: java.io.IOException: No FileSystem for >> scheme: hdfs >> >> Do I have a problem with libs or my env vars like HADOOP_CONF? >> >> Going one step back, how would a Twill job be properly packaged from >> Maven? >> >> Currently, I'm using >> maven-assembly-plugin:jar-with-dependencies >> which builds one big jar with all libs, which ends up in a libs/ directory >> in the twill-bundled jar. >> >> Bernd >> >> >> >> On Sat, Dec 21, 2013 at 11:36 AM, Bernd Fondermann < >> [email protected]> wrote: >> >>> this is the exit log: >>> >>> 2013-12-17 16:06:48,470 INFO >>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Application >>> application_1387295722390_0002 failed 2 times due to AM Container for >>> appattempt_1387295722390_0002_000002 exited with exitCode: 1 due to: >>> Exception from container-launch: >>> org.apache.hadoop.util.Shell$ExitCodeException: >>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:464) >>> at org.apache.hadoop.util.Shell.run(Shell.java:379) >>> at >>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589) >>> at >>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) >>> at >>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283) >>> at >>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79) >>> at >>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>> at java.lang.Thread.run(Thread.java:662) >>> >>> I'll try do more diagnostics... >>> >>> Bernd >>> >>> >>> >>>> On Sat, Dec 21, 2013 at 7:14 AM, Terence Yim <[email protected]> wrote: >>>> >>>> Hi Bernd, >>>> >>>> I never see this before. Do you know what's the exit code from the >>>> YARN log? Currently there is no good way to stage an Twill app besides >>>> running a unit test with a mini YARN cluster (see YarnTestSuite in the >>>> Twill source). >>>> >>>> Terence >>>> >>>> On Wed, Dec 18, 2013 at 6:01 AM, Bernd Fondermann >>>> <[email protected]> wrote: >>>>> Hi everyone, >>>>> >>>>> I'm trying to run a Twill app on YARN. Attempts fail immediately with >>>>> org.apache.hadoop.util.Shell$ExitCodeException. >>>>> >>>>> The rm logs report the launch cmd line as: >>>>> java -Djava.io.tmpdir=tmp -Dyarn.appId=$YARN_APP_ID_STR >>>>> -Dtwill.app=$TWILL_APP_NAME -cp launcher.jar:$HADOOP_CONF_DIR -Xmx362m >>>>> org.apache.twill.launcher.TwillLauncher appMaster.jar >>>>> org.apache.twill.internal.appmaster.ApplicationMasterMain false >>>>> 1><LOG_DIR>/stdout 2><LOG_DIR>/stderr >>>>> >>>>> Any idea what the problem could be? >>>>> Ist there a recommended way to stage a Twill app before running it on a >>>>> remote cluster? >>>>> >>>>> Thanks, >>>>> >>>>> Bernd >>
