There's a patch that was never applied because the YARN profile has moved to Hadoop 2.2.x GA compatibility, but it will help you run on Hadoop-2.0.5-alpha if you want. Search it on the JIRA site, it creates build profiles for 2.0.4 and 2.0.5 alpha releases.
I think when you set workers greater than 1, you are confusing the cluster because it thinks you are in LocalRunnerMode. There are configurations you must set in your *-site.xml files for your Hadoop install if you do not wish to run in LocalMode. I'm note sure if Giraph at present understands the difference between local mode with multiple-tasks or actual cluster mode with many tasks. On Mon, Oct 14, 2013 at 6:34 AM, Wei Zhang <[email protected]> wrote: > Hi Matthew, > > It is my understanding that to run LocalJobRunner, you need to change your > Hadoop configuration a bit. > > That is what I did: > (1) in core-site.xml, comment out the fs.default.name property > (2) in mapred-site.xml, comment out the mapred.job.tracker property > > Hope it helps. > > There are probably other smarter ways of doing this, and I would like to > know as well. > > Wei > > > > [image: Inactive hide details for Matthew Laird ---10/13/2013 04:57:50 > PM---I've been trying to work through the Quick Start examples a]Matthew > Laird ---10/13/2013 04:57:50 PM---I've been trying to work through the > Quick Start examples and adapting it to my local installation, > > From: Matthew Laird <[email protected]> > To: [email protected], > Date: 10/13/2013 04:57 PM > Subject: LocalJobRunner & examples > ------------------------------ > > > > I've been trying to work through the Quick Start examples and adapting > it to my local installation, I'm running Hadoop 2.0.5-alpha across 4 > machines. > > I've done the checkout from github > > sudo git clone https://github.com/apache/giraph.git > > Applied the patch to ZooKeeperManager > > http://www.mail-archive.com/[email protected]/msg00945/check.diff > > Fixed the catching of the exception in > giraph-core/src/main/java/org/apache/giraph/job/HaltApplicationUtils.java > > Then building with > > mvn clean install -Phadoop_2.0.3 -DskipTests > > Then when I go to run the examples, I set -w 4 because I am running a > split master setup. Unfortunately this leads to: > > lairdm@x007:/opt/giraph$ $HADOOP_HOME/bin/hadoop jar > $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.0.3-alpha-jar-with-dependencies.jar > > org.apache.giraph.GiraphRunner > org.apache.giraph.examples.SimpleShortestPathsComputation -vif > org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat > -vip /in/tiny_graph.txt -vof > org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op > /user/hduser/output/shortestpaths -w 4 > 13/10/13 13:50:48 WARN util.NativeCodeLoader: Unable to load > native-hadoop library for your platform... using builtin-java classes > where applicable > 13/10/13 13:50:48 INFO utils.ConfigurationUtils: No edge input format > specified. Ensure your InputFormat does not require one. > 13/10/13 13:50:48 INFO utils.ConfigurationUtils: No edge output format > specified. Ensure your OutputFormat does not require one. > 13/10/13 13:50:49 WARN conf.Configuration: mapreduce.job.counters.limit > is deprecated. Instead, use mapreduce.job.counters.max > 13/10/13 13:50:49 WARN conf.Configuration: mapred.job.map.memory.mb is > deprecated. Instead, use mapreduce.map.memory.mb > 13/10/13 13:50:49 WARN conf.Configuration: mapred.job.reduce.memory.mb > is deprecated. Instead, use mapreduce.reduce.memory.mb > 13/10/13 13:50:49 WARN conf.Configuration: > mapred.map.tasks.speculative.execution is deprecated. Instead, use > mapreduce.map.speculative > 13/10/13 13:50:49 WARN conf.Configuration: > mapreduce.user.classpath.first is deprecated. Instead, use > mapreduce.job.user.classpath.first > 13/10/13 13:50:49 WARN conf.Configuration: mapred.map.max.attempts is > deprecated. Instead, use mapreduce.map.maxattempts > 13/10/13 13:50:49 INFO job.GiraphJob: run: Since checkpointing is > disabled (default), do not allow any task retries (setting > mapred.map.max.attempts = 0, old value = 4) > 13/10/13 13:50:49 WARN conf.Configuration: mapred.job.tracker is > deprecated. Instead, use mapreduce.jobtracker.address > Exception in thread "main" java.lang.IllegalArgumentException: > checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have > only one worker since only 1 task at a time! > at > > org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:159) > at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:233) > at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at org.apache.hadoop.util.RunJar.main(RunJar.java:212) > > I'm not sure what parameters I need to give giraph to signal not to use > LocalJobRunner. Does anyone have a suggestion? > > Thanks. > > >
<<graycol.gif>>
