I tried the following command few months back. I found you are not passing some params.
bin/hadoop --config etc/hadoop jar /export/home/clei/giraph/giraph-1.0.0/giraph-examples/target/giraph-examples-1.?.0-for-hadoop-???-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimplePageRankVertexOpt -c org.apache.giraph.combiner.DoubleSumCombiner -w 35 -yh 20480 -yj giraph-examples-1.?.?-for-hadoop-2.??-jar-with-dependencies.jar -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat -vip /user/me/input -op /user/me/output_35w_20G_80m_1 On Thursday, November 14, 2013 3:22 AM, Rob Vesse <[email protected]> wrote: It's the same command line I showed earlier, just trying to run the example application: mbp-rvesse:hadoop-2.2.0 rvesse$ bin/hadoop jar /Users/rvesse/Documents/Work/Code/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/rvesse/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/rvesse/shortestpaths/2 -w 1 Rob From: Mohammad Islam <[email protected]> Reply-To: <[email protected]> Date: Thursday, 14 November 2013 05:57 To: "[email protected]" <[email protected]> Subject: Re: Giraph and Hadoop 2.2.0 What are you running? PR? >What is the command line? > > > > > >On Wednesday, November 13, 2013 8:30 AM, Rob Vesse <[email protected]> >wrote: > >So I fixed the previous issue I was having which I reported and submitted a >patch as GIRAPH-795 > > >I then went ahead with installing and configuring my external Zookeeper >appropriately and tried running again but now the YARN application fails and I >find the following error in the logs: > > >Exception in thread "main" java.lang.NoClassDefFoundError: >org/apache/giraph/yarn/GiraphApplicationMaster Caused by: java.lang.ClassNotFoundException: org.apache.giraph.yarn.GiraphApplicationMaster at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > > >Anybody have idea what the cause of this one might be or how to fix it? > > >Cheers, > > >Rob > > >From: Rob Vesse <[email protected]> >Reply-To: <[email protected]> >Date: Wednesday, 13 November 2013 15:35 >To: <[email protected]> >Subject: Re: Giraph and Hadoop 2.2.0 > > > >Mohammad >> >> >>This got me a build that at least attempts to start to run but not I see the >>following: >> >> >>mbp-rvesse:hadoop-2.2.0 rvesse$ bin/hadoop jar >>/Users/rvesse/Documents/Work/Code/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar >> org.apache.giraph.GiraphRunner >>org.apache.giraph.examples.SimpleShortestPathsComputation -vif >>org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip >>/user/rvesse/tiny_graph.txt -vof >>org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >>/user/rvesse/shortestpaths/2 -w 1 >>2013-11-13 15:13:29.908 java[17402:1203] Unable to load realm info from >>SCDynamicStore >>13/11/13 15:13:29 WARN util.NativeCodeLoader: Unable to load native-hadoop >>library for your platform... using builtin-java classes where applicable >>13/11/13 15:13:31 INFO utils.ConfigurationUtils: No edge input format >>specified. Ensure your InputFormat does not require one. >>13/11/13 15:13:31 INFO utils.ConfigurationUtils: No edge output format >>specified. Ensure your OutputFormat does not require one. >>13/11/13 15:13:31 INFO yarn.GiraphYarnClient: Final output path is: >>hdfs://localhost/user/rvesse/shortestpaths/2 >>Exception in thread "main" java.lang.NullPointerException >>at >>org.apache.giraph.yarn.GiraphYarnClient.checkJobLocalZooKeeperSupported(GiraphYarnClient.java:460) >>at org.apache.giraph.yarn.GiraphYarnClient.run(GiraphYarnClient.java:116) >>at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:96) >>at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >>at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >>at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126) >>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>at >>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>at >>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>at java.lang.reflect.Method.invoke(Method.java:597) >>at org.apache.hadoop.util.RunJar.main(RunJar.java:212) >> >> >>Which indirectly implies that I have some missing configuration to run Giraph >>on YARN. I don't think that should be a NPE and it looks to be a trivial fix >>so I'll try and fix and send a pull request for that. >> >> >>The error message I should receive if the NPE wasn't happening is the >>following: >> >> >>Giraph on YARN does not currentlysupport Giraph-managed ZK instances: use a >>standalone ZooKeeper. >> >> >>What configuration options do I need to have set to point Giraph at an >>external Zookeeper? >> >> >>Cheers, >> >> >>Rob >> >> >>From: Mohammad Islam <[email protected]> >>Reply-To: <[email protected]> >>Date: Wednesday, 13 November 2013 11:15 >>To: "[email protected]" <[email protected]> >>Subject: Re: Giraph and Hadoop 2.2.0 >> >> >> >>Hi Rob, >>>Did you try with this command "mvn -Phadoop_yarn -Dhadoop.version=2.2.0 >>>clean package -DskipTests"? >>>Regards, >>>Mohammad >>> >>> >>> >>>On Wednesday, November 13, 2013 2:55 AM, Rob Vesse <[email protected]> >>>wrote: >>> >>>Hi all >>> >>> >>>I'm having trouble getting Giraph to build and run against a Hadoop 2.2.0 >>>installation. I tried the hadoop_2.0.3 profile with no luck because I get >>>class cast errors when trying to run the examples. >>> >>> >>>Is there a specific profile I should be building to support Hadoop 2.2.0 or >>>is this not supported yet? >>> >>> >>>Thanks, >>> >>> >>>Rob >>> >>> > >
