Hi Eli, no I think that the help message is clear enough, generally when I read “jar” I always mean absolute path for jar, but this is a my personal misunderstanding.
Thanks, Alessandro Il giorno 16/nov/2014, alle ore 21:39, Eli Reisman <[email protected]> ha scritto: > Sounds like you got it figured out since last time I checked this list, sorry > it was a pain. Feel free to drop a JIRA ticket if we can make help message > etc. for -yj option more clear, there's lots to improve there. > > On Sat, Nov 8, 2014 at 7:26 AM, Alessandro Negro <[email protected]> wrote: > Hi Tripti, > finally I was able to run the test with success. It was an issue of > permission since I was running as ale not as yarn. > Let me say that now I’m able to run graph examples on Yarn 2.5.1. This is the > final result: > > 14/11/08 16:24:00 INFO yarn.GiraphYarnClient: Completed Giraph: > org.apache.giraph.examples.SimpleShortestPathsComputation: SUCCEEDED, total > running time: 0 minutes, 21 seconds. > > > Many thanks for your support, > Alessandro > > Il giorno 06/nov/2014, alle ore 15:16, Tripti Singh <[email protected]> ha > scritto: > >> I don't know if u have access to this node. But if u do, u could check if >> the file is indeed there and u have access to it. >> >> Sent from my iPhone >> >> On 06-Nov-2014, at 6:12 pm, "Alessandro Negro" <[email protected]> wrote: >> >>> You are right it works, but now I receive the following error: >>> >>> SLF4J: Class path contains multiple SLF4J bindings. >>> SLF4J: Found binding in >>> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0009/filecache/10/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>> SLF4J: Found binding in >>> [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >>> explanation. >>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >>> 2014-11-06 13:15:37.120 java[10158:1803] Unable to load realm info from >>> SCDynamicStore >>> Exception in thread "pool-4-thread-1" java.lang.IllegalStateException: >>> Could not configure the containerlaunch context for GiraphYarnTasks. >>> at >>> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:391) >>> at >>> org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicationMaster.java:78) >>> at >>> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.buildContainerLaunchContext(GiraphApplicationMaster.java:522) >>> at >>> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.run(GiraphApplicationMaster.java:479) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>> at java.lang.Thread.run(Thread.java:744) >>> Caused by: java.io.FileNotFoundException: File does not exist: >>> hdfs://hadoop-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_0009/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>> at >>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072) >>> at >>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064) >>> at >>> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) >>> at >>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064) >>> at org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153) >>> at org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77) >>> at >>> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:387) >>> ... 6 more >>> >>> >>> That justify the other error I receive in the task: >>>>> Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask >>> >>> >>> Thanks, >>> >>> Il giorno 06/nov/2014, alle ore 13:07, Tripti Singh <[email protected]> >>> ha scritto: >>> >>>> Why r u adding two jars? Example jar ideally contains core library so >>>> everything should be available with just one example jar included >>>> >>>> Sent from my iPhone >>>> >>>> On 06-Nov-2014, at 4:33 pm, "Alessandro Negro" <[email protected]> wrote: >>>> >>>>> Hi, >>>>> now it seems better, I need to add: >>>>> >>>>> giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>> >>>>> >>>>> Now it seems that after a lot of cycle it fail with this error: >>>>> >>>>> Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask >>>>> >>>>> But in this case the error appear in task-3-stderr.log not in >>>>> gam-stderr.log where there is the following error: >>>>> >>>>> LF4J: Class path contains multiple SLF4J bindings. >>>>> SLF4J: Found binding in >>>>> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0006/filecache/12/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>>>> SLF4J: Found binding in >>>>> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0006/filecache/10/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>>>> SLF4J: Found binding in >>>>> [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >>>>> explanation. >>>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >>>>> 2014-11-06 11:31:25.186 java[8162:1803] Unable to load realm info from >>>>> SCDynamicStore >>>>> Exception in thread "pool-4-thread-1" java.lang.IllegalStateException: >>>>> Could not configure the containerlaunch context for GiraphYarnTasks. >>>>> at >>>>> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:391) >>>>> at >>>>> org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicationMaster.java:78) >>>>> at >>>>> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.buildContainerLaunchContext(GiraphApplicationMaster.java:522) >>>>> at >>>>> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.run(GiraphApplicationMaster.java:479) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>> at java.lang.Thread.run(Thread.java:744) >>>>> Caused by: java.io.FileNotFoundException: File does not exist: >>>>> hdfs://hadoop-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_0006/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>> at >>>>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072) >>>>> at >>>>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064) >>>>> at >>>>> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) >>>>> at >>>>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064) >>>>> at >>>>> org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153) >>>>> at org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77) >>>>> at >>>>> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:387) >>>>> ... 6 more >>>>> >>>>> >>>>> Il giorno 06/nov/2014, alle ore 11:32, Tripti Singh >>>>> <[email protected]> ha scritto: >>>>> >>>>>> Hi, >>>>>> I remember seeing this thread, but since i was already deep into fixing >>>>>> this issue in production, I didn't really wait for the patch, not even >>>>>> the munge symbol as I was not even able to compile the yarn profile( >>>>>> sorry for being so ruthless to giraph :-( ) >>>>>> I did later cross check with the patch and it was quite similar to my >>>>>> change but sure it wouldn't work for all profiles(in fact some profiles >>>>>> may even break due to Hadoop class incompatibility ) I can share those >>>>>> changes with u once I get back to my computer but I don't think that >>>>>> could have caused this problem. >>>>>> Just try the small changed I mentioned in my previous reply and let me >>>>>> know if that helped. >>>>>> >>>>>> Thanks, >>>>>> Tripti >>>>>> Sent from my iPhone >>>>>> >>>>>> On 06-Nov-2014, at 3:52 pm, "Alessandro Negro" <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Tripti, >>>>>>> let me add one info, As you can see from this thread of email: >>>>>>> >>>>>>> http://mail-archives.apache.org/mod_mbox/giraph-user/201410.mbox/%[email protected]%3E >>>>>>> >>>>>>> I had some issue while compiling for Hadoop 2.5.1 could be that into >>>>>>> the process I loose something? >>>>>>> >>>>>>> Thanks again, >>>>>>> Alessandro >>>>>>> >>>>>>> >>>>>>> >>>>>>> Il giorno 06/nov/2014, alle ore 10:47, Alessandro Negro >>>>>>> <[email protected]> ha scritto: >>>>>>> >>>>>>>> Hi Tripti, >>>>>>>> yes I run in this way: >>>>>>>> >>>>>>>> >>>>>>>> $HADOOP_INSTALL/bin/hadoop jar >>>>>>>> /Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>>> -vip /tiny_graph.txt -vof >>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>>> /shortestpaths -w 1 -yj >>>>>>>> /Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>> >>>>>>>> And this is the complete trace: >>>>>>>> >>>>>>>> 2014-11-06 10:06:05.638 java[7778:1903] Unable to load realm info from >>>>>>>> SCDynamicStore >>>>>>>> 14/11/06 10:06:05 WARN util.NativeCodeLoader: Unable to load >>>>>>>> native-hadoop library for your platform... using builtin-java classes >>>>>>>> where applicable >>>>>>>> 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge input format >>>>>>>> specified. Ensure your InputFormat does not require one. >>>>>>>> 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge output format >>>>>>>> specified. Ensure your OutputFormat does not require one. >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Final output path is: >>>>>>>> hdfs://hadoop-master:9000/shortestpaths >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Running Client >>>>>>>> 14/11/06 10:06:06 INFO client.RMProxy: Connecting to ResourceManager >>>>>>>> at /0.0.0.0:8032 >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got node report from ASM >>>>>>>> for, nodeId=10.61.137.28:59677, nodeAddress 10.61.137.28:8042, >>>>>>>> nodeRackName /default-rack, nodeNumContainers 0 >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Defaulting per-task heap >>>>>>>> size to 1024MB. >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Obtained new Application >>>>>>>> ID: application_1415264041937_0005 >>>>>>>> 14/11/06 10:06:06 INFO Configuration.deprecation: mapred.job.id is >>>>>>>> deprecated. Instead, use mapreduce.job.id >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Set the environment for >>>>>>>> the application master >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Environment for AM >>>>>>>> :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*} >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: buildLocalResourceMap >>>>>>>> .... >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Registered file in >>>>>>>> LocalResources :: >>>>>>>> hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_1415264041937_0005/giraph-conf.xml >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS >>>>>>>> :/Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name . >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking . >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name null >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking null >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/* >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /opt/yarn/hadoop-2.5.1/share/myLib/*.jar >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /opt/yarn/hadoop-2.5.1/share/myLib >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /Users/ale/extprj/giraph/myGiraph/*.jar >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /Users/ale/extprj/giraph/myGiraph >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /Users/ale/extprj/giraph/myGiraph/lib/*.jar >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /Users/ale/extprj/giraph/myGiraph/lib >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>> /contrib/capacity-scheduler/*.jar >>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>> /contrib/capacity-scheduler >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: >>>>>>>> ApplicationSumbissionContext for GiraphApplicationMaster launch >>>>>>>> container is populated. >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application >>>>>>>> to ASM >>>>>>>> 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application >>>>>>>> application_1415264041937_0005 >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after >>>>>>>> submission :application_1415264041937_0005 >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: GiraphApplicationMaster >>>>>>>> container request was submitted to ResourceManager for job: Giraph: >>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: >>>>>>>> 0,83 secs >>>>>>>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: >>>>>>>> appattempt_1415264041937_0005_000001, State: ACCEPTED, Containers >>>>>>>> used: 1 >>>>>>>> 14/11/06 10:06:10 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation reports >>>>>>>> FAILED state, diagnostics show: Application >>>>>>>> application_1415264041937_0005 failed 2 times due to AM Container for >>>>>>>> appattempt_1415264041937_0005_000002 exited with exitCode: 1 due to: >>>>>>>> Exception from container-launch: ExitCodeException exitCode=1: >>>>>>>> ExitCodeException exitCode=1: >>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>>> at >>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>>> at >>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) >>>>>>>> at >>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300) >>>>>>>> at >>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>>>> >>>>>>>> >>>>>>>> Container exited with a non-zero exit code 1 >>>>>>>> .Failing this attempt.. Failing the application. >>>>>>>> 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Cleaning up HDFS >>>>>>>> distributed cache directory for Giraph job. >>>>>>>> 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Completed Giraph: >>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation: FAILED, >>>>>>>> total running time: 0 minutes, 3 seconds. >>>>>>>> ale:~/extprj/giraph/myGiraph />yarn logs -applicationId >>>>>>>> application_1415264041937_0005 >>>>>>>> >>>>>>>> As you can see there is this piece: >>>>>>>> >>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS >>>>>>>> :/Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>> >>>>>>>> >>>>>>>> But I continue to receive the error: >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Alessandro >>>>>>>> >>>>>>>> >>>>>>>> Il giorno 06/nov/2014, alle ore 10:25, Tripti Singh >>>>>>>> <[email protected]> ha scritto: >>>>>>>> >>>>>>>>> For GiraphApplicationMaster not found, the simple thing that worked >>>>>>>>> for me was adding a -yj parameter to my command. >>>>>>>>> (hadoop jar {jar-file-path} {path to giraph runner} >>>>>>>>> {computation-class} {bunch of parameters} -yj {jar-file-name}) >>>>>>>>> If u look at job logs, >>>>>>>>> Do u see a log statement defining "LIB JARS"? >>>>>>>>> >>>>>>>>> Sent from my iPhone >>>>>>>>> >>>>>>>>> On 06-Nov-2014, at 2:45 pm, "Alessandro Negro" <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> yea my logs shows that error. I follow this example: >>>>>>>>>> >>>>>>>>>> http://mail-archives.apache.org/mod_mbox/giraph-user/201408.mbox/%[email protected]%3E >>>>>>>>>> >>>>>>>>>> without success. >>>>>>>>>> Then I try to add all jars I can from giraph adding the following >>>>>>>>>> lines into the hadoop-env.sh: >>>>>>>>>> >>>>>>>>>> export >>>>>>>>>> MYLIB=/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar >>>>>>>>>> export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$MYLIB >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> when I run the application now I see this: >>>>>>>>>> >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/myLib/*.jar >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/myLib >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>>>> /Users/ale/extprj/giraph/myGiraph/*.jar >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>>>> /Users/ale/extprj/giraph/myGiraph >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>>>>> /Users/ale/extprj/giraph/myGiraph/lib/*.jar >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>>>>> /Users/ale/extprj/giraph/myGiraph/lib >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> But again the same error. What I think is that there is no problem >>>>>>>>>> in the library but while is starting up. >>>>>>>>>> I saw this piece in the logs: >>>>>>>>>> >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: >>>>>>>>>> ApplicationSumbissionContext for GiraphApplicationMaster launch >>>>>>>>>> container is populated. >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application >>>>>>>>>> to ASM >>>>>>>>>> 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application >>>>>>>>>> application_1415264041937_0005 >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after >>>>>>>>>> submission :application_1415264041937_0005 >>>>>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: >>>>>>>>>> GiraphApplicationMaster container request was submitted to >>>>>>>>>> ResourceManager for job: Giraph: >>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: >>>>>>>>>> 0,83 secs >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> That show that it is able to load GiraphApplicationMaster but fail >>>>>>>>>> during the startup. >>>>>>>>>> Where can I find more logs? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Alessandro >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Il giorno 06/nov/2014, alle ore 06:56, Tripti Singh >>>>>>>>>> <[email protected]> ha scritto: >>>>>>>>>> >>>>>>>>>>> Can u run the following command to find container logs and post it >>>>>>>>>>> here: >>>>>>>>>>> >>>>>>>>>>> yarn logs -applicationId application_1415203231170_0002 >>>>>>>>>>> >>>>>>>>>>> If u find this error: >>>>>>>>>>> Could not find or load main class >>>>>>>>>>> org.apache.giraph.yarn.GiraphApplicationMaster >>>>>>>>>>> It may just be due to non-availablity of the yarn jars >>>>>>>>>>> Thanks, >>>>>>>>>>> Tripti >>>>>>>>>>> Tech Yahoo >>>>>>>>>>> >>>>>>>>>>> From: Jing Fan <[email protected]> >>>>>>>>>>> Reply-To: "[email protected]" <[email protected]> >>>>>>>>>>> Date: Thursday, November 6, 2014 at 1:56 AM >>>>>>>>>>> To: "[email protected]" <[email protected]> >>>>>>>>>>> Subject: Re: Help with Giraph on Yarn >>>>>>>>>>> >>>>>>>>>>> Then do you know to solve this error? >>>>>>>>>>> >>>>>>>>>>> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation reports >>>>>>>>>>> FAILED state, diagnostics show: Application >>>>>>>>>>> application_1415203231170_0002 failed 2 times due to AM Container >>>>>>>>>>> for appattempt_1415203231170_0002_000002 exited with exitCode: 1 >>>>>>>>>>> due to: Exception from container-launch. >>>>>>>>>>> Container id: container_1415203231170_0002_02_000001 >>>>>>>>>>> Exit code: 1 >>>>>>>>>>> Stack trace: ExitCodeException exitCode=1: >>>>>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>>>>>> at >>>>>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>>>>>> at >>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196) >>>>>>>>>>> at >>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) >>>>>>>>>>> at >>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>>>>> at >>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>>>>> at >>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Nov 5, 2014 at 12:10 PM, Alessandro Negro >>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> in the mean time I found the reason why it hangs for long time >>>>>>>>>>>> here: >>>>>>>>>>>> >>>>>>>>>>>> base path checking . >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> It scan all the directory tree starting from current directory, if >>>>>>>>>>>> you run it into your home (and you have a lot of files and >>>>>>>>>>>> directories) it takes ages to complete. >>>>>>>>>>>> >>>>>>>>>>>> I continue to got the same error. >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Alessandro >>>>>>>>>>>> >>>>>>>>>>>> Il giorno 05/nov/2014, alle ore 18:33, Jing Fan >>>>>>>>>>>> <[email protected]> ha scritto: >>>>>>>>>>>> >>>>>>>>>>>>> I run with: >>>>>>>>>>>>> >>>>>>>>>>>>> hadoop jar >>>>>>>>>>>>> giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar >>>>>>>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>>>>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>>>>>>>> -vip /user/jing/tiny_graph.txt -vof >>>>>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>>>>>>>> /user/jing/shortestpaths -w 1 -yj >>>>>>>>>>>>> giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Nov 5, 2014 at 11:25 AM, Alessandro Negro >>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>> This is my complete logs with the command I run: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> ale:~ />$HADOOP_INSTALL/bin/hadoop jar >>>>>>>>>>>>>> /Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>>>>>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>>>>>>>>> -vip /tiny_graph.txt -vof >>>>>>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>>>>>>>>> /shortestpaths -w 1 >>>>>>>>>>>>>> 2014-11-05 13:39:30.235 java[3147:1903] Unable to load realm >>>>>>>>>>>>>> info from SCDynamicStore >>>>>>>>>>>>>> 14/11/05 13:39:30 WARN util.NativeCodeLoader: Unable to load >>>>>>>>>>>>>> native-hadoop library for your platform... using builtin-java >>>>>>>>>>>>>> classes where applicable >>>>>>>>>>>>>> 14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge input >>>>>>>>>>>>>> format specified. Ensure your InputFormat does not require one. >>>>>>>>>>>>>> 14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge output >>>>>>>>>>>>>> format specified. Ensure your OutputFormat does not require one. >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Final output path >>>>>>>>>>>>>> is: hdfs://hadoop-master:9000/shortestpaths >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Running Client >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO client.RMProxy: Connecting to >>>>>>>>>>>>>> ResourceManager at /0.0.0.0:8032 >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Got node report >>>>>>>>>>>>>> from ASM for, nodeId=192.168.0.105:50658, nodeAddress >>>>>>>>>>>>>> 192.168.0.105:8042, nodeRackName /default-rack, >>>>>>>>>>>>>> nodeNumContainers 0 >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Defaulting >>>>>>>>>>>>>> per-task heap size to 1024MB. >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Obtained new >>>>>>>>>>>>>> Application ID: application_1415189069846_0004 >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO Configuration.deprecation: mapred.job.id >>>>>>>>>>>>>> is deprecated. Instead, use mapreduce.job.id >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Set the >>>>>>>>>>>>>> environment for the application master >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Environment for AM >>>>>>>>>>>>>> :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*} >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>> buildLocalResourceMap .... >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: Registered file in >>>>>>>>>>>>>> LocalResources :: >>>>>>>>>>>>>> hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_1415189069846_0004/giraph-conf.xml >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: LIB JARS : >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: Class path name . >>>>>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: base path checking . >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name null >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking null >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/* >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>>>>> /contrib/capacity-scheduler/*.jar >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>>>>> /contrib/capacity-scheduler >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>> ApplicationSumbissionContext for GiraphApplicationMaster launch >>>>>>>>>>>>>> container is populated. >>>>>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.GiraphYarnClient: Submitting >>>>>>>>>>>>>> application to ASM >>>>>>>>>>>>>> 14/11/05 14:42:57 INFO impl.YarnClientImpl: Submitted >>>>>>>>>>>>>> application application_1415189069846_0004 >>>>>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Got new appId >>>>>>>>>>>>>> after submission :application_1415189069846_0004 >>>>>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>> GiraphApplicationMaster container request was submitted to >>>>>>>>>>>>>> ResourceManager for job: Giraph: >>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>> Elapsed: 0,82 secs >>>>>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>> appattempt_1415189069846_0004_000001, State: ACCEPTED, >>>>>>>>>>>>>> Containers used: 1 >>>>>>>>>>>>>> 14/11/05 14:43:01 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>>>>>> reports FAILED state, diagnostics show: Application >>>>>>>>>>>>>> application_1415189069846_0004 failed 2 times due to AM >>>>>>>>>>>>>> Container for appattempt_1415189069846_0004_000002 exited with >>>>>>>>>>>>>> exitCode: 1 due to: Exception from container-launch: >>>>>>>>>>>>>> ExitCodeException exitCode=1: >>>>>>>>>>>>>> ExitCodeException exitCode=1: >>>>>>>>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>>>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>>>>>>>> at >>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Container exited with a non-zero exit code 1 >>>>>>>>>>>>>> .Failing this attempt.. Failing the application. >>>>>>>>>>>>>> 14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Cleaning up HDFS >>>>>>>>>>>>>> distributed cache directory for Giraph job. >>>>>>>>>>>>>> 14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Completed Giraph: >>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation: >>>>>>>>>>>>>> FAILED, total running time: 0 minutes, 3 seconds. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Il giorno 05/nov/2014, alle ore 18:17, Tripti Singh >>>>>>>>>>>>>> <[email protected]> ha scritto: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I faced the similar problem where the workflow just hangs. >>>>>>>>>>>>>>> Can u see in ur logs if there is a task with id 0. >>>>>>>>>>>>>>> I made a local fix and I could give u the patch if u see a >>>>>>>>>>>>>>> similar problem and ur workflow is unable to find the zookeeper >>>>>>>>>>>>>>> manager file and repeats the following message: >>>>>>>>>>>>>>> For task xyz, got file 'null' >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Tripti >>>>>>>>>>>>>>> Sent from my iPhone >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 05-Nov-2014, at 10:40 pm, "Alessandro Negro" >>>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> have you seen the hadoop logs? >>>>>>>>>>>>>>>> Could you write here how are you running the command? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Alessandro >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Il giorno 05/nov/2014, alle ore 18:04, Jing Fan >>>>>>>>>>>>>>>> <[email protected]> ha scritto: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Yeah, I add -yj option. The result is the same. Still error >>>>>>>>>>>>>>>>> here. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Wed, Nov 5, 2014 at 10:52 AM, Alessandro Negro >>>>>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>>>>> Hi Jing, >>>>>>>>>>>>>>>>>> I’m facing the same problem. Do you check the hadoop logs? >>>>>>>>>>>>>>>>>> In my case I got: >>>>>>>>>>>>>>>>>> Could not find or load main class >>>>>>>>>>>>>>>>>> org.apache.giraph.yarn.GiraphApplicationMaster >>>>>>>>>>>>>>>>>> Moreover I don’t know why my application hang for a long >>>>>>>>>>>>>>>>>> time here: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 14/11/05 17:11:17 INFO yarn.YarnUtils: base path checking . >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> And then continue failing with your error into the output >>>>>>>>>>>>>>>>>> and the other into the hadoop logs. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Did you test adding this? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -yj >>>>>>>>>>>>>>>>>> /Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> Alessandro >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Il giorno 05/nov/2014, alle ore 17:32, Jing Fan >>>>>>>>>>>>>>>>>> <[email protected]> ha scritto: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I install the Giraph with Yarn. >>>>>>>>>>>>>>>>>>> But when I run the program with following command: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> hadoop jar >>>>>>>>>>>>>>>>>>> giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar >>>>>>>>>>>>>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>>>>>>>>>>> -vif >>>>>>>>>>>>>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>>>>>>>>>>>>>> -vip /user/j/tiny_graph.txt -vof >>>>>>>>>>>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat >>>>>>>>>>>>>>>>>>> -op /user/j/shortestpaths -w 1 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The program exited with error: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>>>>> Elapsed: 0.83 secs >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000001, State: ACCEPTED, >>>>>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>>>>> Elapsed: 4.84 secs >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000001, State: ACCEPTED, >>>>>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>>>>> Elapsed: 8.85 secs >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000002, State: ACCEPTED, >>>>>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>>>>> Elapsed: 12.86 secs >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000002, State: ACCEPTED, >>>>>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>>>>> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>>>>>>>>>>> reports FAILED state, diagnostics show: Application >>>>>>>>>>>>>>>>>>> application_1415203231170_0002 failed 2 times due to AM >>>>>>>>>>>>>>>>>>> Container for appattempt_1415203231170_0002_000002 exited >>>>>>>>>>>>>>>>>>> with exitCode: 1 due to: Exception from container-launch. >>>>>>>>>>>>>>>>>>> Container id: container_1415203231170_0002_02_000001 >>>>>>>>>>>>>>>>>>> Exit code: 1 >>>>>>>>>>>>>>>>>>> Stack trace: ExitCodeException exitCode=1: >>>>>>>>>>>>>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>>>>>>>>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196) >>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) >>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>>>>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Anyone can help? >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> > >
