I'm using a vanilla 2.2.0. Bernd
On Tue, Dec 24, 2013 at 6:44 PM, Terence Yim <[email protected]> wrote: > Him strange. What is the Hadoop version I'm your cluster? > > Sent from my iPhone > > > On Dec 24, 2013, at 6:40 AM, Bernd Fondermann < > [email protected]> wrote: > > > > strange. I had to do two things to solve this: > > > > 1. I added to core-site.xml the property fs.hdfs.impl = > > org.apache.hadoop.hdfs.DistributedFileSystem > > I don't really understand why this is needed, but it solved problems with > > not being able to use hdfs URLs when starting the local Twill Driver and > > also the on-cluster app. > > > > 2. I excluded snappy 1.0.4.1 from twill-core and added a new dependency > on > > 1.0.5, as suggested by a sparse post on StackOverflow. > > > > Bernd > > > > > > > > On Mon, Dec 23, 2013 at 8:19 AM, Bernd Fondermann < > > [email protected]> wrote: > > > >> I've switched YARN log retention on and now see a lot of errors, > starting > >> with java.lang.RuntimeException: java.io.IOException: No FileSystem for > >> scheme: hdfs > >> > >> Do I have a problem with libs or my env vars like HADOOP_CONF? > >> > >> Going one step back, how would a Twill job be properly packaged from > >> Maven? > >> > >> Currently, I'm using > >> maven-assembly-plugin:jar-with-dependencies > >> which builds one big jar with all libs, which ends up in a libs/ > directory > >> in the twill-bundled jar. > >> > >> Bernd > >> > >> > >> > >> On Sat, Dec 21, 2013 at 11:36 AM, Bernd Fondermann < > >> [email protected]> wrote: > >> > >>> this is the exit log: > >>> > >>> 2013-12-17 16:06:48,470 INFO > >>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: > Application > >>> application_1387295722390_0002 failed 2 times due to AM Container for > >>> appattempt_1387295722390_0002_000002 exited with exitCode: 1 due to: > >>> Exception from container-launch: > >>> org.apache.hadoop.util.Shell$ExitCodeException: > >>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:464) > >>> at org.apache.hadoop.util.Shell.run(Shell.java:379) > >>> at > >>> > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589) > >>> at > >>> > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) > >>> at > >>> > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283) > >>> at > >>> > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79) > >>> at > >>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > >>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) > >>> at > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > >>> at > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > >>> at java.lang.Thread.run(Thread.java:662) > >>> > >>> I'll try do more diagnostics... > >>> > >>> Bernd > >>> > >>> > >>> > >>>> On Sat, Dec 21, 2013 at 7:14 AM, Terence Yim <[email protected]> > wrote: > >>>> > >>>> Hi Bernd, > >>>> > >>>> I never see this before. Do you know what's the exit code from the > >>>> YARN log? Currently there is no good way to stage an Twill app besides > >>>> running a unit test with a mini YARN cluster (see YarnTestSuite in the > >>>> Twill source). > >>>> > >>>> Terence > >>>> > >>>> On Wed, Dec 18, 2013 at 6:01 AM, Bernd Fondermann > >>>> <[email protected]> wrote: > >>>>> Hi everyone, > >>>>> > >>>>> I'm trying to run a Twill app on YARN. Attempts fail immediately with > >>>>> org.apache.hadoop.util.Shell$ExitCodeException. > >>>>> > >>>>> The rm logs report the launch cmd line as: > >>>>> java -Djava.io.tmpdir=tmp -Dyarn.appId=$YARN_APP_ID_STR > >>>>> -Dtwill.app=$TWILL_APP_NAME -cp launcher.jar:$HADOOP_CONF_DIR > -Xmx362m > >>>>> org.apache.twill.launcher.TwillLauncher appMaster.jar > >>>>> org.apache.twill.internal.appmaster.ApplicationMasterMain false > >>>>> 1><LOG_DIR>/stdout 2><LOG_DIR>/stderr > >>>>> > >>>>> Any idea what the problem could be? > >>>>> Ist there a recommended way to stage a Twill app before running it > on a > >>>>> remote cluster? > >>>>> > >>>>> Thanks, > >>>>> > >>>>> Bernd > >> >
