This is my complete logs with the command I run:
ale:~ />$HADOOP_INSTALL/bin/hadoop jar
/Users/ale/extprj/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip
/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat
-op /shortestpaths -w 1
2014-11-05 13:39:30.235 java[3147:1903] Unable to load realm info from
SCDynamicStore
14/11/05 13:39:30 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
14/11/05 13:39:30 INFO utils.ConfigurationUtils: No edge output format
specified. Ensure your OutputFormat does not require one.
14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Final output path is:
hdfs://hadoop-master:9000/shortestpaths
14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Running Client
14/11/05 13:39:31 INFO client.RMProxy: Connecting to ResourceManager at
/0.0.0.0:8032
14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Got node report from ASM for,
nodeId=192.168.0.105:50658, nodeAddress 192.168.0.105:8042, nodeRackName
/default-rack, nodeNumContainers 0
14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Defaulting per-task heap size to
1024MB.
14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Obtained new Application ID:
application_1415189069846_0004
14/11/05 13:39:31 INFO Configuration.deprecation: mapred.job.id is deprecated.
Instead, use mapreduce.job.id
14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Set the environment for the
application master
14/11/05 13:39:31 INFO yarn.GiraphYarnClient: Environment for AM
:{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*}
14/11/05 13:39:31 INFO yarn.GiraphYarnClient: buildLocalResourceMap ....
14/11/05 13:39:31 INFO yarn.YarnUtils: Registered file in LocalResources ::
hdfs://hadoop-master:9000/user/ale/giraph_yarn_jar_cache/application_1415189069846_0004/giraph-conf.xml
14/11/05 13:39:31 INFO yarn.GiraphYarnClient: LIB JARS :
14/11/05 13:39:31 INFO yarn.YarnUtils: Class path name .
14/11/05 13:39:31 INFO yarn.YarnUtils: base path checking .
14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name null
14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking null
14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/etc/hadoop
14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/etc/hadoop
14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib/*
14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/common/lib
14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/common/*
14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/common
14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib/*
14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/lib
14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs/*
14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/hdfs
14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib/*
14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/lib
14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/yarn/*
14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/yarn
14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib/*
14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/lib
14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce/*
14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
/opt/yarn/hadoop-2.5.1/share/hadoop/mapreduce
14/11/05 14:42:56 INFO yarn.YarnUtils: Class path name
/contrib/capacity-scheduler/*.jar
14/11/05 14:42:56 INFO yarn.YarnUtils: base path checking
/contrib/capacity-scheduler
14/11/05 14:42:56 INFO yarn.GiraphYarnClient: ApplicationSumbissionContext for
GiraphApplicationMaster launch container is populated.
14/11/05 14:42:56 INFO yarn.GiraphYarnClient: Submitting application to ASM
14/11/05 14:42:57 INFO impl.YarnClientImpl: Submitted application
application_1415189069846_0004
14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Got new appId after submission
:application_1415189069846_0004
14/11/05 14:42:57 INFO yarn.GiraphYarnClient: GiraphApplicationMaster container
request was submitted to ResourceManager for job: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation
14/11/05 14:42:57 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0,82 secs
14/11/05 14:42:57 INFO yarn.GiraphYarnClient:
appattempt_1415189069846_0004_000001, State: ACCEPTED, Containers used: 1
14/11/05 14:43:01 ERROR yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation reports FAILED state,
diagnostics show: Application application_1415189069846_0004 failed 2 times due
to AM Container for appattempt_1415189069846_0004_000002 exited with exitCode:
1 due to: Exception from container-launch: ExitCodeException exitCode=1:
ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.
14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Cleaning up HDFS distributed
cache directory for Giraph job.
14/11/05 14:43:01 INFO yarn.GiraphYarnClient: Completed Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation: FAILED, total
running time: 0 minutes, 3 seconds.
Il giorno 05/nov/2014, alle ore 18:17, Tripti Singh <[email protected]> ha
scritto:
> I faced the similar problem where the workflow just hangs.
> Can u see in ur logs if there is a task with id 0.
> I made a local fix and I could give u the patch if u see a similar problem
> and ur workflow is unable to find the zookeeper manager file and repeats the
> following message:
> For task xyz, got file 'null'
>
> Thanks,
> Tripti
> Sent from my iPhone
>
> On 05-Nov-2014, at 10:40 pm, "Alessandro Negro" <[email protected]> wrote:
>
>> Hi,
>> have you seen the hadoop logs?
>> Could you write here how are you running the command?
>>
>> Thanks,
>> Alessandro
>>
>> Il giorno 05/nov/2014, alle ore 18:04, Jing Fan <[email protected]> ha
>> scritto:
>>
>>> Yeah, I add -yj option. The result is the same. Still error here.
>>>
>>> On Wed, Nov 5, 2014 at 10:52 AM, Alessandro Negro <[email protected]>
>>> wrote:
>>> Hi Jing,
>>> I’m facing the same problem. Do you check the hadoop logs? In my case I
>>> got:
>>> Could not find or load main class
>>> org.apache.giraph.yarn.GiraphApplicationMaster
>>> Moreover I don’t know why my application hang for a long time here:
>>>
>>> 14/11/05 17:11:17 INFO yarn.YarnUtils: base path checking .
>>>
>>> And then continue failing with your error into the output and the other
>>> into the hadoop logs.
>>>
>>> Did you test adding this?
>>>
>>> -yj
>>> /Users/ale/extprj/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar
>>>
>>> Thanks,
>>> Alessandro
>>>
>>>
>>> Il giorno 05/nov/2014, alle ore 17:32, Jing Fan <[email protected]> ha
>>> scritto:
>>>
>>>> Hi,
>>>>
>>>> I install the Giraph with Yarn.
>>>> But when I run the program with following command:
>>>>
>>>> hadoop jar
>>>> giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.5.0-cdh5.2.0-jar-with-dependencies.jar
>>>> org.apache.giraph.GiraphRunner
>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>>>> -vip /user/j/tiny_graph.txt -vof
>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>>> /user/j/shortestpaths -w 1
>>>>
>>>> The program exited with error:
>>>>
>>>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient: Giraph:
>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 0.83
>>>> secs
>>>> 14/11/05 10:25:14 INFO yarn.GiraphYarnClient:
>>>> appattempt_1415203231170_0002_000001, State: ACCEPTED, Containers used: 0
>>>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient: Giraph:
>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 4.84
>>>> secs
>>>> 14/11/05 10:25:18 INFO yarn.GiraphYarnClient:
>>>> appattempt_1415203231170_0002_000001, State: ACCEPTED, Containers used: 0
>>>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient: Giraph:
>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 8.85
>>>> secs
>>>> 14/11/05 10:25:22 INFO yarn.GiraphYarnClient:
>>>> appattempt_1415203231170_0002_000002, State: ACCEPTED, Containers used: 0
>>>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient: Giraph:
>>>> org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 12.86
>>>> secs
>>>> 14/11/05 10:25:26 INFO yarn.GiraphYarnClient:
>>>> appattempt_1415203231170_0002_000002, State: ACCEPTED, Containers used: 0
>>>> 14/11/05 10:25:29 ERROR yarn.GiraphYarnClient: Giraph:
>>>> org.apache.giraph.examples.SimpleShortestPathsComputation reports FAILED
>>>> state, diagnostics show: Application application_1415203231170_0002 failed
>>>> 2 times due to AM Container for appattempt_1415203231170_0002_000002
>>>> exited with exitCode: 1 due to: Exception from container-launch.
>>>> Container id: container_1415203231170_0002_02_000001
>>>> Exit code: 1
>>>> Stack trace: ExitCodeException exitCode=1:
>>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
>>>> at org.apache.hadoop.util.Shell.run(Shell.java:455)
>>>> at
>>>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
>>>> at
>>>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196)
>>>> at
>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
>>>> at
>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>>
>>>>
>>>> Anyone can help?
>>>>
>>>
>>>
>>
