You are right it works, but now I receive the following error: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0009/filecache/10/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 2014-11-06 13:15:37.120 java[10158:1803] Unable to load realm info from SCDynamicStore Exception in thread "pool-4-thread-1" java.lang.IllegalStateException: Could not configure the containerlaunch context for GiraphYarnTasks. at org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:391) at org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicationMaster.java:78) at org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.buildContainerLaunchContext(GiraphApplicationMaster.java:522) at org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.run(GiraphApplicationMaster.java:479) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: java.io.FileNotFoundException: File does not exist: hdfs://hadoop-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_0009/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072) at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064) at org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153) at org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77) at org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:387) ... 6 more
That justify the other error I receive in the task: >> Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask Thanks, Il giorno 06/nov/2014, alle ore 13:07, Tripti Singh <[email protected]> ha scritto: > Why r u adding two jars? Example jar ideally contains core library so > everything should be available with just one example jar included > > Sent from my iPhone > > On 06-Nov-2014, at 4:33 pm, "Alessandro Negro" <[email protected]> wrote: > >> Hi, >> now it seems better, I need to add: >> >> giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >> >> >> Now it seems that after a lot of cycle it fail with this error: >> >> Could not find or load main class org.apache.giraph.yarn.GiraphYarnTask >> >> But in this case the error appear in task-3-stderr.log not in >> gam-stderr.log where there is the following error: >> >> LF4J: Class path contains multiple SLF4J bindings. >> SLF4J: Found binding in >> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0006/filecache/12/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] >> SLF4J: Found binding in >> [jar:file:/private/tmp/hadoop-yarn/nm-local-dir/usercache/ale/appcache/application_1415264041937_0006/filecache/10/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] >> SLF4J: Found binding in >> [jar:file:/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] >> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >> explanation. >> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >> 2014-11-06 11:31:25.186 java[8162:1803] Unable to load realm info from >> SCDynamicStore >> Exception in thread "pool-4-thread-1" java.lang.IllegalStateException: Could >> not configure the containerlaunch context for GiraphYarnTasks. >> at >> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:391) >> at >> org.apache.giraph.yarn.GiraphApplicationMaster.access$500(GiraphApplicationMaster.java:78) >> at >> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.buildContainerLaunchContext(GiraphApplicationMaster.java:522) >> at >> org.apache.giraph.yarn.GiraphApplicationMaster$LaunchContainerRunnable.run(GiraphApplicationMaster.java:479) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:744) >> Caused by: java.io.FileNotFoundException: File does not exist: >> hdfs://hadoop-master:9000/user/yarn/giraph_yarn_jar_cache/application_1415264041937_0006/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >> at >> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072) >> at >> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064) >> at >> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) >> at >> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064) >> at org.apache.giraph.yarn.YarnUtils.addFileToResourceMap(YarnUtils.java:153) >> at org.apache.giraph.yarn.YarnUtils.addFsResourcesToMap(YarnUtils.java:77) >> at >> org.apache.giraph.yarn.GiraphApplicationMaster.getTaskResourceMap(GiraphApplicationMaster.java:387) >> ... 6 more >> >> >> Il giorno 06/nov/2014, alle ore 11:32, Tripti Singh <[email protected]> >> ha scritto: >> >>> Hi, >>> I remember seeing this thread, but since i was already deep into fixing >>> this issue in production, I didn't really wait for the patch, not even the >>> munge symbol as I was not even able to compile the yarn profile( sorry for >>> being so ruthless to giraph :-( ) >>> I did later cross check with the patch and it was quite similar to my >>> change but sure it wouldn't work for all profiles(in fact some profiles may >>> even break due to Hadoop class incompatibility ) I can share those changes >>> with u once I get back to my computer but I don't think that could have >>> caused this problem. >>> Just try the small changed I mentioned in my previous reply and let me know >>> if that helped. >>> >>> Thanks, >>> Tripti >>> Sent from my iPhone >>> >>> On 06-Nov-2014, at 3:52 pm, "Alessandro Negro" <[email protected]> wrote: >>> >>>> Hi Tripti, >>>> let me add one info, As you can see from this thread of email: >>>> >>>> http://mail-archives.apache.org/mod_mbox/giraph-user/201410.mbox/%[email protected]%3E >>>> >>>> I had some issue while compiling for Hadoop 2.5.1 could be that into the >>>> process I loose something? >>>> >>>> Thanks again, >>>> Alessandro >>>> >>>> >>>> >>>> Il giorno 06/nov/2014, alle ore 10:47, Alessandro Negro >>>> <[email protected]> ha scritto: >>>> >>>>> Hi Tripti, >>>>> yes I run in this way: >>>>> >>>>> >>>>> $HADOOP_INSTALL/bin/hadoop jar >>>>> /Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>> org.apache.giraph.GiraphRunner >>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>> -vip /tiny_graph.txt -vof >>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>> /shortestpaths -w 1 -yj >>>>> /Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>> >>>>> And this is the complete trace: >>>>> >>>>> 2014-11-06 10:06:05.638 java[7778:1903] Unable to load realm info from >>>>> SCDynamicStore >>>>> 14/11/06 10:06:05 WARN util.NativeCodeLoader: Unable to load >>>>> native-hadoop library for your platform... using builtin-java classes >>>>> where applicable >>>>> 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge input format >>>>> specified. Ensure your InputFormat does not require one. >>>>> 14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge output format >>>>> specified. Ensure your OutputFormat does not require one. >>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Final output path is: >>>>> hdfs://hadoop-master:9000/shortestpaths >>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Running Client >>>>> 14/11/06 10:06:06 INFO client.RMProxy: Connecting to ResourceManager at >>>>> /0.0.0.0:8032 >>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got node report from ASM >>>>> for, nodeId=10.61.137.28:59677, nodeAddress 10.61.137.28:8042, >>>>> nodeRackName /default-rack, nodeNumContainers 0 >>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Defaulting per-task heap >>>>> size to 1024MB. >>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Obtained new Application >>>>> ID: application_1415264041937_0005 >>>>> 14/11/06 10:06:06 INFO Configuration.deprecation: mapred.job.id is >>>>> deprecated. Instead, use mapreduce.job.id >>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Set the environment for the >>>>> application master >>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Environment for AM >>>>> :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*} >>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: buildLocalResourceMap .... >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Registered file in LocalResources >>>>> :: >>>>> hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_1415264041937_0005/giraph-conf.xml >>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS >>>>> :/Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name . >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking . >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name null >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking null >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/* >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/* >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/* >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/* >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/* >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/* >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib/* >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/* >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /opt/yarn/hadoop-2.5.1/share/myLib/*.jar >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /opt/yarn/hadoop-2.5.1/share/myLib >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /Users/ale/extprj/giraph/myGiraph/*.jar >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /Users/ale/extprj/giraph/myGiraph >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /Users/ale/extprj/giraph/myGiraph/lib/*.jar >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /Users/ale/extprj/giraph/myGiraph/lib >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>> /contrib/capacity-scheduler/*.jar >>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>> /contrib/capacity-scheduler >>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: >>>>> ApplicationSumbissionContext for GiraphApplicationMaster launch container >>>>> is populated. >>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to >>>>> ASM >>>>> 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application >>>>> application_1415264041937_0005 >>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after >>>>> submission :application_1415264041937_0005 >>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: GiraphApplicationMaster >>>>> container request was submitted to ResourceManager for job: Giraph: >>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph: >>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0,83 >>>>> secs >>>>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: >>>>> appattempt_1415264041937_0005_000001, State: ACCEPTED, Containers used: 1 >>>>> 14/11/06 10:06:10 ERROR yarn.GiraphYarnClient: Giraph: >>>>> org.apache.giraph.examples.SimpleShortestPathsComputation reports FAILED >>>>> state, diagnostics show: Application application_1415264041937_0005 >>>>> failed 2 times due to AM Container for >>>>> appattempt_1415264041937_0005_000002 exited with exitCode: 1 due to: >>>>> Exception from container-launch: ExitCodeException exitCode=1: >>>>> ExitCodeException exitCode=1: >>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>> at >>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>> at >>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) >>>>> at >>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300) >>>>> at >>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>> at java.lang.Thread.run(Thread.java:744) >>>>> >>>>> >>>>> Container exited with a non-zero exit code 1 >>>>> .Failing this attempt.. Failing the application. >>>>> 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Cleaning up HDFS >>>>> distributed cache directory for Giraph job. >>>>> 14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Completed Giraph: >>>>> org.apache.giraph.examples.SimpleShortestPathsComputation: FAILED, total >>>>> running time: 0 minutes, 3 seconds. >>>>> ale:~/extprj/giraph/myGiraph />yarn logs -applicationId >>>>> application_1415264041937_0005 >>>>> >>>>> As you can see there is this piece: >>>>> >>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS >>>>> :/Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>> >>>>> >>>>> But I continue to receive the error: >>>>> >>>>> Thanks, >>>>> Alessandro >>>>> >>>>> >>>>> Il giorno 06/nov/2014, alle ore 10:25, Tripti Singh >>>>> <[email protected]> ha scritto: >>>>> >>>>>> For GiraphApplicationMaster not found, the simple thing that worked for >>>>>> me was adding a -yj parameter to my command. >>>>>> (hadoop jar {jar-file-path} {path to giraph runner} {computation-class} >>>>>> {bunch of parameters} -yj {jar-file-name}) >>>>>> If u look at job logs, >>>>>> Do u see a log statement defining "LIB JARS"? >>>>>> >>>>>> Sent from my iPhone >>>>>> >>>>>> On 06-Nov-2014, at 2:45 pm, "Alessandro Negro" <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> yea my logs shows that error. I follow this example: >>>>>>> >>>>>>> http://mail-archives.apache.org/mod_mbox/giraph-user/201408.mbox/%[email protected]%3E >>>>>>> >>>>>>> without success. >>>>>>> Then I try to add all jars I can from giraph adding the following lines >>>>>>> into the hadoop-env.sh: >>>>>>> >>>>>>> export >>>>>>> MYLIB=/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar >>>>>>> export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$MYLIB >>>>>>> >>>>>>> >>>>>>> when I run the application now I see this: >>>>>>> >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /opt/yarn/hadoop-2.5.1/share/myLib/*.jar >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /opt/yarn/hadoop-2.5.1/share/myLib >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /Users/ale/extprj/giraph/myGiraph/*.jar >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /Users/ale/extprj/giraph/myGiraph >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name >>>>>>> /Users/ale/extprj/giraph/myGiraph/lib/*.jar >>>>>>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking >>>>>>> /Users/ale/extprj/giraph/myGiraph/lib >>>>>>> >>>>>>> >>>>>>> But again the same error. What I think is that there is no problem in >>>>>>> the library but while is starting up. >>>>>>> I saw this piece in the logs: >>>>>>> >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: >>>>>>> ApplicationSumbissionContext for GiraphApplicationMaster launch >>>>>>> container is populated. >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to >>>>>>> ASM >>>>>>> 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application >>>>>>> application_1415264041937_0005 >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after >>>>>>> submission :application_1415264041937_0005 >>>>>>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: GiraphApplicationMaster >>>>>>> container request was submitted to ResourceManager for job: Giraph: >>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph: >>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: >>>>>>> 0,83 secs >>>>>>> >>>>>>> >>>>>>> That show that it is able to load GiraphApplicationMaster but fail >>>>>>> during the startup. >>>>>>> Where can I find more logs? >>>>>>> >>>>>>> Thanks, >>>>>>> Alessandro >>>>>>> >>>>>>> >>>>>>> Il giorno 06/nov/2014, alle ore 06:56, Tripti Singh >>>>>>> <[email protected]> ha scritto: >>>>>>> >>>>>>>> Can u run the following command to find container logs and post it >>>>>>>> here: >>>>>>>> >>>>>>>> yarn logs -applicationId application_1415203231170_0002 >>>>>>>> >>>>>>>> If u find this error: >>>>>>>> Could not find or load main class >>>>>>>> org.apache.giraph.yarn.GiraphApplicationMaster >>>>>>>> It may just be due to non-availablity of the yarn jars >>>>>>>> Thanks, >>>>>>>> Tripti >>>>>>>> Tech Yahoo >>>>>>>> >>>>>>>> From: Jing Fan <[email protected]> >>>>>>>> Reply-To: "[email protected]" <[email protected]> >>>>>>>> Date: Thursday, November 6, 2014 at 1:56 AM >>>>>>>> To: "[email protected]" <[email protected]> >>>>>>>> Subject: Re: Help with Giraph on Yarn >>>>>>>> >>>>>>>> Then do you know to solve this error? >>>>>>>> >>>>>>>> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation reports >>>>>>>> FAILED state, diagnostics show: Application >>>>>>>> application_1415203231170_0002 failed 2 times due to AM Container for >>>>>>>> appattempt_1415203231170_0002_000002 exited with exitCode: 1 due to: >>>>>>>> Exception from container-launch. >>>>>>>> Container id: container_1415203231170_0002_02_000001 >>>>>>>> Exit code: 1 >>>>>>>> Stack trace: ExitCodeException exitCode=1: >>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>>> at >>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>>> at >>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196) >>>>>>>> at >>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) >>>>>>>> at >>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>> at >>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Nov 5, 2014 at 12:10 PM, Alessandro Negro >>>>>>>> <[email protected]> wrote: >>>>>>>>> Hi, >>>>>>>>> in the mean time I found the reason why it hangs for long time here: >>>>>>>>> >>>>>>>>> base path checking . >>>>>>>>> >>>>>>>>> >>>>>>>>> It scan all the directory tree starting from current directory, if >>>>>>>>> you run it into your home (and you have a lot of files and >>>>>>>>> directories) it takes ages to complete. >>>>>>>>> >>>>>>>>> I continue to got the same error. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Alessandro >>>>>>>>> >>>>>>>>> Il giorno 05/nov/2014, alle ore 18:33, Jing Fan <[email protected]> >>>>>>>>> ha scritto: >>>>>>>>> >>>>>>>>>> I run with: >>>>>>>>>> >>>>>>>>>> hadoop jar >>>>>>>>>> giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar >>>>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>>>>> -vip /user/jing/tiny_graph.txt -vof >>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>>>>> /user/jing/shortestpaths -w 1 -yj >>>>>>>>>> giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar >>>>>>>>>> >>>>>>>>>> On Wed, Nov 5, 2014 at 11:25 AM, Alessandro Negro >>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>> This is my complete logs with the command I run: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ale:~ />$HADOOP_INSTALL/bin/hadoop jar >>>>>>>>>>> /Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>>>>>> -vip /tiny_graph.txt -vof >>>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>>>>>> /shortestpaths -w 1 >>>>>>>>>>> 2014-11-05 13:39:30.235 java[3147:1903] Unable to load realm info >>>>>>>>>>> from SCDynamicStore >>>>>>>>>>> 14/11/05 13:39:30 WARN util.NativeCodeLoader: Unable to load >>>>>>>>>>> native-hadoop library for your platform... using builtin-java >>>>>>>>>>> classes where applicable >>>>>>>>>>> 14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge input >>>>>>>>>>> format specified. Ensure your InputFormat does not require one. >>>>>>>>>>> 14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge output >>>>>>>>>>> format specified. Ensure your OutputFormat does not require one. >>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Final output path is: >>>>>>>>>>> hdfs://hadoop-master:9000/shortestpaths >>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Running Client >>>>>>>>>>> 14/11/05 13:39:31 INFO client.RMProxy: Connecting to >>>>>>>>>>> ResourceManager at /0.0.0.0:8032 >>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Got node report from >>>>>>>>>>> ASM for, nodeId=192.168.0.105:50658, nodeAddress >>>>>>>>>>> 192.168.0.105:8042, nodeRackName /default-rack, nodeNumContainers 0 >>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Defaulting per-task >>>>>>>>>>> heap size to 1024MB. >>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Obtained new >>>>>>>>>>> Application ID: application_1415189069846_0004 >>>>>>>>>>> 14/11/05 13:39:31 INFO Configuration.deprecation: mapred.job.id is >>>>>>>>>>> deprecated. Instead, use mapreduce.job.id >>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Set the environment >>>>>>>>>>> for the application master >>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Environment for AM >>>>>>>>>>> :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*} >>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: buildLocalResourceMap >>>>>>>>>>> .... >>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: Registered file in >>>>>>>>>>> LocalResources :: >>>>>>>>>>> hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_1415189069846_0004/giraph-conf.xml >>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: LIB JARS : >>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: Class path name . >>>>>>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: base path checking . >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name null >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking null >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/* >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/* >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/* >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/* >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/* >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/* >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib/* >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/* >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name >>>>>>>>>>> /contrib/capacity-scheduler/*.jar >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking >>>>>>>>>>> /contrib/capacity-scheduler >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.GiraphYarnClient: >>>>>>>>>>> ApplicationSumbissionContext for GiraphApplicationMaster launch >>>>>>>>>>> container is populated. >>>>>>>>>>> 14/11/05 14:42:56 INFO yarn.GiraphYarnClient: Submitting >>>>>>>>>>> application to ASM >>>>>>>>>>> 14/11/05 14:42:57 INFO impl.YarnClientImpl: Submitted application >>>>>>>>>>> application_1415189069846_0004 >>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Got new appId after >>>>>>>>>>> submission :application_1415189069846_0004 >>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: >>>>>>>>>>> GiraphApplicationMaster container request was submitted to >>>>>>>>>>> ResourceManager for job: Giraph: >>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: >>>>>>>>>>> 0,82 secs >>>>>>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: >>>>>>>>>>> appattempt_1415189069846_0004_000001, State: ACCEPTED, Containers >>>>>>>>>>> used: 1 >>>>>>>>>>> 14/11/05 14:43:01 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation reports >>>>>>>>>>> FAILED state, diagnostics show: Application >>>>>>>>>>> application_1415189069846_0004 failed 2 times due to AM Container >>>>>>>>>>> for appattempt_1415189069846_0004_000002 exited with exitCode: 1 >>>>>>>>>>> due to: Exception from container-launch: ExitCodeException >>>>>>>>>>> exitCode=1: >>>>>>>>>>> ExitCodeException exitCode=1: >>>>>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>>>>>> at >>>>>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>>>>>> at >>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) >>>>>>>>>>> at >>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300) >>>>>>>>>>> at >>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>>>>> at >>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>>>>> at >>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Container exited with a non-zero exit code 1 >>>>>>>>>>> .Failing this attempt.. Failing the application. >>>>>>>>>>> 14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Cleaning up HDFS >>>>>>>>>>> distributed cache directory for Giraph job. >>>>>>>>>>> 14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Completed Giraph: >>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation: FAILED, >>>>>>>>>>> total running time: 0 minutes, 3 seconds. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Il giorno 05/nov/2014, alle ore 18:17, Tripti Singh >>>>>>>>>>> <[email protected]> ha scritto: >>>>>>>>>>> >>>>>>>>>>>> I faced the similar problem where the workflow just hangs. >>>>>>>>>>>> Can u see in ur logs if there is a task with id 0. >>>>>>>>>>>> I made a local fix and I could give u the patch if u see a similar >>>>>>>>>>>> problem and ur workflow is unable to find the zookeeper manager >>>>>>>>>>>> file and repeats the following message: >>>>>>>>>>>> For task xyz, got file 'null' >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Tripti >>>>>>>>>>>> Sent from my iPhone >>>>>>>>>>>> >>>>>>>>>>>> On 05-Nov-2014, at 10:40 pm, "Alessandro Negro" >>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> have you seen the hadoop logs? >>>>>>>>>>>>> Could you write here how are you running the command? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Alessandro >>>>>>>>>>>>> >>>>>>>>>>>>> Il giorno 05/nov/2014, alle ore 18:04, Jing Fan >>>>>>>>>>>>> <[email protected]> ha scritto: >>>>>>>>>>>>> >>>>>>>>>>>>>> Yeah, I add -yj option. The result is the same. Still error here. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Nov 5, 2014 at 10:52 AM, Alessandro Negro >>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>> Hi Jing, >>>>>>>>>>>>>>> I’m facing the same problem. Do you check the hadoop logs? In >>>>>>>>>>>>>>> my case I got: >>>>>>>>>>>>>>> Could not find or load main class >>>>>>>>>>>>>>> org.apache.giraph.yarn.GiraphApplicationMaster >>>>>>>>>>>>>>> Moreover I don’t know why my application hang for a long time >>>>>>>>>>>>>>> here: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 14/11/05 17:11:17 INFO yarn.YarnUtils: base path checking . >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> And then continue failing with your error into the output and >>>>>>>>>>>>>>> the other into the hadoop logs. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Did you test adding this? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -yj >>>>>>>>>>>>>>> /Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Alessandro >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Il giorno 05/nov/2014, alle ore 17:32, Jing Fan >>>>>>>>>>>>>>> <[email protected]> ha scritto: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I install the Giraph with Yarn. >>>>>>>>>>>>>>>> But when I run the program with following command: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> hadoop jar >>>>>>>>>>>>>>>> giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar >>>>>>>>>>>>>>>> org.apache.giraph.GiraphRunner >>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>>>>>>>>>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >>>>>>>>>>>>>>>> -vip /user/j/tiny_graph.txt -vof >>>>>>>>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>>>>>>>>>>>>>>> /user/j/shortestpaths -w 1 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The program exited with error: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>> Elapsed: 0.83 secs >>>>>>>>>>>>>>>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000001, State: ACCEPTED, >>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>> Elapsed: 4.84 secs >>>>>>>>>>>>>>>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000001, State: ACCEPTED, >>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>> Elapsed: 8.85 secs >>>>>>>>>>>>>>>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000002, State: ACCEPTED, >>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, >>>>>>>>>>>>>>>> Elapsed: 12.86 secs >>>>>>>>>>>>>>>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient: >>>>>>>>>>>>>>>> appattempt_1415203231170_0002_000002, State: ACCEPTED, >>>>>>>>>>>>>>>> Containers used: 0 >>>>>>>>>>>>>>>> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph: >>>>>>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation >>>>>>>>>>>>>>>> reports FAILED state, diagnostics show: Application >>>>>>>>>>>>>>>> application_1415203231170_0002 failed 2 times due to AM >>>>>>>>>>>>>>>> Container for appattempt_1415203231170_0002_000002 exited with >>>>>>>>>>>>>>>> exitCode: 1 due to: Exception from container-launch. >>>>>>>>>>>>>>>> Container id: container_1415203231170_0002_02_000001 >>>>>>>>>>>>>>>> Exit code: 1 >>>>>>>>>>>>>>>> Stack trace: ExitCodeException exitCode=1: >>>>>>>>>>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) >>>>>>>>>>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) >>>>>>>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Anyone can help? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>
