Hi Tripti,
yes I run in this way:
$HADOOP_INSTALL/bin/hadoop jar
/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip
/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat
-op /shortestpaths -w 1 -yj
/Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
And this is the complete trace:
2014-11-06 10:06:05.638 java[7778:1903] Unable to load realm info from
SCDynamicStore
14/11/06 10:06:05 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
14/11/06 10:06:06 INFO utils.ConfigurationUtils: No edge output format
specified. Ensure your OutputFormat does not require one.
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Final output path is:
hdfs://hadoop-master:9000/shortestpaths
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Running Client
14/11/06 10:06:06 INFO client.RMProxy: Connecting to ResourceManager at
/0.0.0.0:8032
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got node report from ASM for,
nodeId=10.61.137.28:59677, nodeAddress 10.61.137.28:8042, nodeRackName
/default-rack, nodeNumContainers 0
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Defaulting per-task heap size to
1024MB.
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Obtained new Application ID:
application_1415264041937_0005
14/11/06 10:06:06 INFO Configuration.deprecation: mapred.job.id is deprecated.
Instead, use mapreduce.job.id
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Set the environment for the
application master
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Environment for AM
:{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*}
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: buildLocalResourceMap ....
14/11/06 10:06:06 INFO yarn.YarnUtils: Registered file in LocalResources ::
hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_1415264041937_0005/giraph-conf.xml
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS
:/Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name .
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking .
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name null
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking null
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/etc/hadoop
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/etc/hadoop
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/*
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/common/*
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/common
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/*
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/*
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/*
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/yarn
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib/*
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/*
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/myLib/*.jar
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/myLib
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
/Users/ale/extprj/giraph/myGiraph/*.jar
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
/Users/ale/extprj/giraph/myGiraph
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
/Users/ale/extprj/giraph/myGiraph/lib/*.jar
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
/Users/ale/extprj/giraph/myGiraph/lib
14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
/contrib/capacity-scheduler/*.jar
14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
/contrib/capacity-scheduler
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: ApplicationSumbissionContext for
GiraphApplicationMaster launch container is populated.
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to ASM
14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application
application_1415264041937_0005
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after submission
:application_1415264041937_0005
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: GiraphApplicationMaster container
request was submitted to ResourceManager for job: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation
14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0,83 secs
14/11/06 10:06:07 INFO yarn.GiraphYarnClient:
appattempt_1415264041937_0005_000001, State: ACCEPTED, Containers used: 1
14/11/06 10:06:10 ERROR yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation reports FAILED state,
diagnostics show: Application application_1415264041937_0005 failed 2 times due
to AM Container for appattempt_1415264041937_0005_000002 exited with exitCode:
1 due to: Exception from container-launch: ExitCodeException exitCode=1:
ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.
14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Cleaning up HDFS distributed
cache directory for Giraph job.
14/11/06 10:06:10 INFO yarn.GiraphYarnClient: Completed Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation: FAILED, total
running time: 0 minutes, 3 seconds.
ale:~/extprj/giraph/myGiraph />yarn logs -applicationId
application_1415264041937_0005
As you can see there is this piece:
14/11/06 10:06:06 INFO yarn.GiraphYarnClient: LIB JARS
:/Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar,/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
But I continue to receive the error:
Thanks,
Alessandro
Il giorno 06/nov/2014, alle ore 10:25, Tripti Singh <[email protected]> ha
scritto:
> For GiraphApplicationMaster not found, the simple thing that worked for me
> was adding a -yj parameter to my command.
> (hadoop jar {jar-file-path} {path to giraph runner} {computation-class}
> {bunch of parameters} -yj {jar-file-name})
> If u look at job logs,
> Do u see a log statement defining "LIB JARS"?
>
> Sent from my iPhone
>
> On 06-Nov-2014, at 2:45 pm, "Alessandro Negro" <[email protected]> wrote:
>
>> Hi,
>> yea my logs shows that error. I follow this example:
>>
>> http://mail-archives.apache.org/mod_mbox/giraph-user/201408.mbox/%[email protected]%3E
>>
>> without success.
>> Then I try to add all jars I can from giraph adding the following lines into
>> the hadoop-env.sh:
>>
>> export
>> MYLIB=/opt/yarn/hadoop-2.5.1/share/myLib/*.jar:/Users/ale/extprj/giraph/myGiraph/*.jar:/Users/ale/extprj/giraph/myGiraph/lib/*.jar
>> export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$MYLIB
>>
>>
>> when I run the application now I see this:
>>
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /opt/yarn/hadoop-2.5.1/share/myLib/*.jar
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /opt/yarn/hadoop-2.5.1/share/myLib
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /Users/ale/extprj/giraph/myGiraph/*.jar
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /Users/ale/extprj/giraph/myGiraph
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: Class path name
>> /Users/ale/extprj/giraph/myGiraph/lib/*.jar
>> 14/11/06 10:06:06 INFO yarn.YarnUtils: base path checking
>> /Users/ale/extprj/giraph/myGiraph/lib
>>
>>
>> But again the same error. What I think is that there is no problem in the
>> library but while is starting up.
>> I saw this piece in the logs:
>>
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: ApplicationSumbissionContext
>> for GiraphApplicationMaster launch container is populated.
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Submitting application to ASM
>> 14/11/06 10:06:06 INFO impl.YarnClientImpl: Submitted application
>> application_1415264041937_0005
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: Got new appId after submission
>> :application_1415264041937_0005
>> 14/11/06 10:06:06 INFO yarn.GiraphYarnClient: GiraphApplicationMaster
>> container request was submitted to ResourceManager for job: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation
>> 14/11/06 10:06:07 INFO yarn.GiraphYarnClient: Giraph:
>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0,83 secs
>>
>>
>> That show that it is able to load GiraphApplicationMaster but fail during
>> the startup.
>> Where can I find more logs?
>>
>> Thanks,
>> Alessandro
>>
>>
>> Il giorno 06/nov/2014, alle ore 06:56, Tripti Singh <[email protected]>
>> ha scritto:
>>
>>> Can u run the following command to find container logs and post it here:
>>>
>>> yarn logs -applicationId application_1415203231170_0002
>>>
>>> If u find this error:
>>> Could not find or load main class
>>> org.apache.giraph.yarn.GiraphApplicationMaster
>>> It may just be due to non-availablity of the yarn jars
>>> Thanks,
>>> Tripti
>>> Tech Yahoo
>>>
>>> From: Jing Fan <[email protected]>
>>> Reply-To: "[email protected]" <[email protected]>
>>> Date: Thursday, November 6, 2014 at 1:56 AM
>>> To: "[email protected]" <[email protected]>
>>> Subject: Re: Help with Giraph on Yarn
>>>
>>> Then do you know to solve this error?
>>>
>>> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph:
>>> org.apache.giraph.examples.SimpleShortestPathsComputation reports FAILED
>>> state, diagnostics show: Application application_1415203231170_0002 failed
>>> 2 times due to AM Container for appattempt_1415203231170_0002_000002 exited
>>> with exitCode: 1 due to: Exception from container-launch.
>>> Container id: container_1415203231170_0002_02_000001
>>> Exit code: 1
>>> Stack trace: ExitCodeException exitCode=1:
>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>>> at org.apache.hadoop.util.Shell.run(Shell.java:455)
>>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>>> at
>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196)
>>> at
>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
>>> at
>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>>
>>> On Wed, Nov 5, 2014 at 12:10 PM, Alessandro Negro <[email protected]>
>>> wrote:
>>>> Hi,
>>>> in the mean time I found the reason why it hangs for long time here:
>>>>
>>>> base path checking .
>>>>
>>>>
>>>> It scan all the directory tree starting from current directory, if you run
>>>> it into your home (and you have a lot of files and directories) it takes
>>>> ages to complete.
>>>>
>>>> I continue to got the same error.
>>>>
>>>> Regards,
>>>> Alessandro
>>>>
>>>> Il giorno 05/nov/2014, alle ore 18:33, Jing Fan <[email protected]> ha
>>>> scritto:
>>>>
>>>>> I run with:
>>>>>
>>>>> hadoop jar
>>>>> giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar
>>>>> org.apache.giraph.GiraphRunner
>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>>>>> -vip /user/jing/tiny_graph.txt -vof
>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>>>> /user/jing/shortestpaths -w 1 -yj
>>>>> giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar
>>>>>
>>>>> On Wed, Nov 5, 2014 at 11:25 AM, Alessandro Negro <[email protected]>
>>>>> wrote:
>>>>>> This is my complete logs with the command I run:
>>>>>>
>>>>>>
>>>>>> ale:~ />$HADOOP_INSTALL/bin/hadoop jar
>>>>>> /Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>>>>>> org.apache.giraph.GiraphRunner
>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>>>>>> -vip /tiny_graph.txt -vof
>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>>>>> /shortestpaths -w 1
>>>>>> 2014-11-05 13:39:30.235 java[3147:1903] Unable to load realm info from
>>>>>> SCDynamicStore
>>>>>> 14/11/05 13:39:30 WARN util.NativeCodeLoader: Unable to load
>>>>>> native-hadoop library for your platform... using builtin-java classes
>>>>>> where applicable
>>>>>> 14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge input format
>>>>>> specified. Ensure your InputFormat does not require one.
>>>>>> 14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge output format
>>>>>> specified. Ensure your OutputFormat does not require one.
>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Final output path is:
>>>>>> hdfs://hadoop-master:9000/shortestpaths
>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Running Client
>>>>>> 14/11/05 13:39:31 INFO client.RMProxy: Connecting to ResourceManager at
>>>>>> /0.0.0.0:8032
>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Got node report from ASM
>>>>>> for, nodeId=192.168.0.105:50658, nodeAddress 192.168.0.105:8042,
>>>>>> nodeRackName /default-rack, nodeNumContainers 0
>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Defaulting per-task heap
>>>>>> size to 1024MB.
>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Obtained new Application
>>>>>> ID: application_1415189069846_0004
>>>>>> 14/11/05 13:39:31 INFO Configuration.deprecation: mapred.job.id is
>>>>>> deprecated. Instead, use mapreduce.job.id
>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Set the environment for
>>>>>> the application master
>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Environment for AM
>>>>>> :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*}
>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: buildLocalResourceMap ....
>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: Registered file in LocalResources
>>>>>> ::
>>>>>> hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_1415189069846_0004/giraph-conf.xml
>>>>>> 14/11/05 13:39:31 INFO yarn.GiraphYarnClient: LIB JARS :
>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: Class path name .
>>>>>> 14/11/05 13:39:31 INFO yarn.YarnUtils: base path checking .
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name null
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking null
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>>>>>> /opt/yarn/hadoop-2.5.1/etc/hadoop
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/*
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/lib
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common/*
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/common
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/*
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/*
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/*
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/yarn
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib/*
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/*
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>>>>>> /opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
>>>>>> /contrib/capacity-scheduler/*.jar
>>>>>> 14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
>>>>>> /contrib/capacity-scheduler
>>>>>> 14/11/05 14:42:56 INFO yarn.GiraphYarnClient:
>>>>>> ApplicationSumbissionContext for GiraphApplicationMaster launch
>>>>>> container is populated.
>>>>>> 14/11/05 14:42:56 INFO yarn.GiraphYarnClient: Submitting application to
>>>>>> ASM
>>>>>> 14/11/05 14:42:57 INFO impl.YarnClientImpl: Submitted application
>>>>>> application_1415189069846_0004
>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Got new appId after
>>>>>> submission :application_1415189069846_0004
>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: GiraphApplicationMaster
>>>>>> container request was submitted to ResourceManager for job: Giraph:
>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation
>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Giraph:
>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0,82
>>>>>> secs
>>>>>> 14/11/05 14:42:57 INFO yarn.GiraphYarnClient:
>>>>>> appattempt_1415189069846_0004_000001, State: ACCEPTED, Containers used: 1
>>>>>> 14/11/05 14:43:01 ERROR yarn.GiraphYarnClient: Giraph:
>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation reports FAILED
>>>>>> state, diagnostics show: Application application_1415189069846_0004
>>>>>> failed 2 times due to AM Container for
>>>>>> appattempt_1415189069846_0004_000002 exited with exitCode: 1 due to:
>>>>>> Exception from container-launch: ExitCodeException exitCode=1:
>>>>>> ExitCodeException exitCode=1:
>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455)
>>>>>> at
>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>>>>>> at
>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>>>>>> at
>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
>>>>>> at
>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>>> at java.lang.Thread.run(Thread.java:744)
>>>>>>
>>>>>>
>>>>>> Container exited with a non-zero exit code 1
>>>>>> .Failing this attempt.. Failing the application.
>>>>>> 14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Cleaning up HDFS
>>>>>> distributed cache directory for Giraph job.
>>>>>> 14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Completed Giraph:
>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation: FAILED, total
>>>>>> running time: 0 minutes, 3 seconds.
>>>>>>
>>>>>>
>>>>>> Il giorno 05/nov/2014, alle ore 18:17, Tripti Singh
>>>>>> <[email protected]> ha scritto:
>>>>>>
>>>>>>> I faced the similar problem where the workflow just hangs.
>>>>>>> Can u see in ur logs if there is a task with id 0.
>>>>>>> I made a local fix and I could give u the patch if u see a similar
>>>>>>> problem and ur workflow is unable to find the zookeeper manager file
>>>>>>> and repeats the following message:
>>>>>>> For task xyz, got file 'null'
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Tripti
>>>>>>> Sent from my iPhone
>>>>>>>
>>>>>>> On 05-Nov-2014, at 10:40 pm, "Alessandro Negro" <[email protected]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> have you seen the hadoop logs?
>>>>>>>> Could you write here how are you running the command?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Alessandro
>>>>>>>>
>>>>>>>> Il giorno 05/nov/2014, alle ore 18:04, Jing Fan <[email protected]>
>>>>>>>> ha scritto:
>>>>>>>>
>>>>>>>>> Yeah, I add -yj option. The result is the same. Still error here.
>>>>>>>>>
>>>>>>>>> On Wed, Nov 5, 2014 at 10:52 AM, Alessandro Negro
>>>>>>>>> <[email protected]> wrote:
>>>>>>>>>> Hi Jing,
>>>>>>>>>> I’m facing the same problem. Do you check the hadoop logs? In my
>>>>>>>>>> case I got:
>>>>>>>>>> Could not find or load main class
>>>>>>>>>> org.apache.giraph.yarn.GiraphApplicationMaster
>>>>>>>>>> Moreover I don’t know why my application hang for a long time here:
>>>>>>>>>>
>>>>>>>>>> 14/11/05 17:11:17 INFO yarn.YarnUtils: base path checking .
>>>>>>>>>>
>>>>>>>>>> And then continue failing with your error into the output and the
>>>>>>>>>> other into the hadoop logs.
>>>>>>>>>>
>>>>>>>>>> Did you test adding this?
>>>>>>>>>>
>>>>>>>>>> -yj
>>>>>>>>>> /Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Alessandro
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Il giorno 05/nov/2014, alle ore 17:32, Jing Fan
>>>>>>>>>> <[email protected]> ha scritto:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I install the Giraph with Yarn.
>>>>>>>>>>> But when I run the program with following command:
>>>>>>>>>>>
>>>>>>>>>>> hadoop jar
>>>>>>>>>>> giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar
>>>>>>>>>>> org.apache.giraph.GiraphRunner
>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>>>>>>>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>>>>>>>>>>> -vip /user/j/tiny_graph.txt -vof
>>>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>>>>>>>>>> /user/j/shortestpaths -w 1
>>>>>>>>>>>
>>>>>>>>>>> The program exited with error:
>>>>>>>>>>>
>>>>>>>>>>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient: Giraph:
>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
>>>>>>>>>>> 0.83 secs
>>>>>>>>>>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient:
>>>>>>>>>>> appattempt_1415203231170_0002_000001, State: ACCEPTED, Containers
>>>>>>>>>>> used: 0
>>>>>>>>>>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient: Giraph:
>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
>>>>>>>>>>> 4.84 secs
>>>>>>>>>>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient:
>>>>>>>>>>> appattempt_1415203231170_0002_000001, State: ACCEPTED, Containers
>>>>>>>>>>> used: 0
>>>>>>>>>>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient: Giraph:
>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
>>>>>>>>>>> 8.85 secs
>>>>>>>>>>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient:
>>>>>>>>>>> appattempt_1415203231170_0002_000002, State: ACCEPTED, Containers
>>>>>>>>>>> used: 0
>>>>>>>>>>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient: Giraph:
>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
>>>>>>>>>>> 12.86 secs
>>>>>>>>>>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient:
>>>>>>>>>>> appattempt_1415203231170_0002_000002, State: ACCEPTED, Containers
>>>>>>>>>>> used: 0
>>>>>>>>>>> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph:
>>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation reports
>>>>>>>>>>> FAILED state, diagnostics show: Application
>>>>>>>>>>> application_1415203231170_0002 failed 2 times due to AM Container
>>>>>>>>>>> for appattempt_1415203231170_0002_000002 exited with exitCode: 1
>>>>>>>>>>> due to: Exception from container-launch.
>>>>>>>>>>> Container id: container_1415203231170_0002_02_000001
>>>>>>>>>>> Exit code: 1
>>>>>>>>>>> Stack trace: ExitCodeException exitCode=1:
>>>>>>>>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>>>>>>>>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455)
>>>>>>>>>>> at
>>>>>>>>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>>>>>>>>>>> at
>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196)
>>>>>>>>>>> at
>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
>>>>>>>>>>> at
>>>>>>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>>>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>>>>>>>>> at
>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>>>>>>>> at
>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>>>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Anyone can help?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
