Update: The shortest path example runs now, but only with *1 worker*. Using more than 1 worker gives the same *LocalJobRunner *error:
----------------------------------------- *Command:* hadoop jar giraph-examples-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/kenrick/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/kenrick/output/shortestpaths -w 2 -ca giraph.SplitMasterWorker=false *Error:* Exception in thread "main" java.lang.IllegalArgumentException: checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have only one worker since only 1 task at a time! at org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:162) at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:236) at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) ----------------------------------------- Suggestions/fixes would be much appreciated. Thanks, Kenrick On Mon, Mar 23, 2015 at 1:30 PM, Kenrick Fernandes <[email protected]> wrote: > Hi Steve, > > Running the Shortest Paths example with that option gives me an > *IncompatibleClassChange* error, which > I am still trying to figure out how to fix. However, running the PageRank > benchmark gives an *Unrecognized option* > error (both are below) - is there a specific way to pass the param > depending on whether GiraphRunner is present? > > > ----------------------------------------- > *Command:* > hadoop jar > giraph-examples-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar > org.apache.giraph.GiraphRunner > org.apache.giraph.examples.SimpleShortestPathsComputation > -vif > org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat > -vip /user/kenrick/input/tiny_graph.txt > -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat > -op /user/kenrick/output/shortestpaths -w 1 -ca > giraph.SplitMasterWorker=false > > *Error:* > Exception in thread "main" java.lang.IncompatibleClassChangeError: Found > interface org.apache.hadoop.mapreduce.JobContext, but class was expected > at > org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43) > at > org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562) > at > org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296) > > ----------------------------------------- > *Command:* > hadoop jar giraph-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar > org.apache.giraph.benchmark.PageRankBenchmark -e 1 -s 3 -v -V 50000000 -w > 30 -ca giraph.SplitMasterWorker=false > > *Error:* > Exception in thread "main" > org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -ca > at org.apache.commons.cli.Parser.processOption(Parser.java:363) > at org.apache.commons.cli.Parser.parse(Parser.java:199) > at org.apache.commons.cli.Parser.parse(Parser.java:85) > at org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:71) > > ----------------------------------------- > > Thanks, > Kenrick > > On Mon, Mar 23, 2015 at 9:26 AM, Steven Harenberg <[email protected]> > wrote: > >> Try adding this option: -ca giraph.SplitMasterWorker=false >> >> --Steve >> >> >> On Sun, Mar 22, 2015 at 4:28 PM, Kenrick Fernandes <[email protected] >> > wrote: >> >>> Hi, >>> >>> I am working with Giraph 1.1.0 and a YARN cluster with Hadoop 2.6.0. >>> I build Giraph with >>> mvn -Phadoop_2 -Dhadoop.version=2.6.0 clean package -DskipTests >>> >>> So far, when I run any of the benchmarks or Shortest path examples, I >>> always get the LocalJobRunner error : >>> >>> >>> >>> --------------------------------------------------------------------------------------- >>> Exception in thread "main" java.lang.IllegalArgumentException: >>> checkLocalJobRunnerConfiguration: When using LocalJobRunner, you cannot run >>> in split master / worker mode since there is only 1 task at a time! >>> at >>> org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:168) >>> at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:236) >>> at >>> org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:96) >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >>> at >>> org.apache.giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:158) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:606) >>> at org.apache.hadoop.util.RunJar.run(RunJar.java:221) >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:136) >>> ------------------------------------------------------------------- >>> >>> I have tried some solutions from forums/StackOverflow/lists, but so far >>> nothing has worked. As far as I can tell, Hadoop is configured right (other >>> MR benchmarks run fine). I tried changing the above Giraph code file >>> (making the check function returned what I wanted), but that only starts >>> the Job and gets it running on a single machine - it never uses more than 1 >>> machine. >>> >>> Any help or pointers in the right direction would be much appreciated. >>> >>> Thanks, >>> Kenrick >>> >> >> >
