I'm sorry, but I don't follow and am a bit confused. Is there an example I could look at?
What I'm trying to achieve is using the giraph-*.jar as a dependency of my project. In my project I would like to define a class similar to org.apache.giraph.examples.SimpleShortestPathsVertex. Then I would like to create a jar from my project and use that with "hadoop jar ..." or "bin/giraph ...". Hope that made sense. Thank you, Shaunak On Thu, Feb 2, 2012 at 1:40 PM, David Garcia <[email protected]>wrote: > Before you run the jar, run mvn clean package from trunk. After that, put > the snapshot (or the with-dependencies) jar on the class path for your > example. It should work after that. > > From: Shaunak Kashyap <[email protected]> > Reply-To: "[email protected]" < > [email protected]> > Date: Wed, 1 Feb 2012 20:44:09 -0600 > To: "[email protected]" <[email protected]> > Subject: Re: Getting started > > Thanks, Jakob. > > I downloaded the source from > https://svn.apache.org/repos/asf/incubator/giraph/trunk, built it using > "mvn compile", then issued this command from the base directory (where > pom.xml lives): > > $ ./bin/giraph ../giraph-example-shortest-path-with-libs.jar > com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex > > And I got back this error: > > Can't find Giraph jar. > > Digging into the contents of ./bin/giraph, it looks like its expecting the > Giraph JAR in lib/ whereas "mvn compile" places it in target/. > > Perhaps I shouldn't be compiling from source but using the "regular Giraph > .tar.gz" you mentioned in your email. I couldn't find it referenced from > anywhere on http://incubator.apache.org/giraph/ (but maybe I haven't > looked hard enough). > > Thanks, > > Shaunak > > On Wed, Feb 1, 2012 at 11:17 AM, Jakob Homan <[email protected]> wrote: > >> We need to update the wiki; it's out of date. Now we have bin/giraph >> to run user jars. Once you have the regular Giraph .tar.gz unpacked >> on your machine and your vertex jar somewhere: >> bin/giraph <your vertex jar> <your vertex class> <other params such as >> directories, formats, etc...> >> >> >> On Wed, Feb 1, 2012 at 11:09 AM, Shaunak Kashyap <[email protected]> >> wrote: >> > Hi, >> > >> > I was able to run the shortest paths example as described >> > here: >> https://cwiki.apache.org/confluence/display/GIRAPH/Shortest+Paths+Example >> . >> > >> > After this I tried to create my own JAR containing a class similar >> > to SimpleShortestPathsVertex and a "lib" directory containing the >> > giraph-*.jar (obtained from the target/ directory after building from >> > source) and other dependent JARs. When I tried to run my JAR as a hadoop >> > job, I get the following errors: >> > >> > $ hadoop jar giraph-example-shortest-path-with-libs.jar >> > com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex >> > shortestPathsInputGraph shortestPathsOutputGraph 0 3 >> > >> > 12/02/01 11:05:01 INFO mapred.JobClient: Running job: >> job_201111161517_1471 >> > 12/02/01 11:05:02 INFO mapred.JobClient: map 0% reduce 0% >> > 12/02/01 11:05:07 INFO mapred.JobClient: Task Id : >> > attempt_201111161517_1471_m_000001_0, Status : FAILED >> > java.lang.IllegalStateException: run: Caught an unrecoverable exception >> > java.lang.RuntimeException: java.lang.ClassNotFoundException: >> > com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex >> > at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:650) >> > at >> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647) >> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) >> > at org.apache.hadoop.mapred.Child$4.run(Child.java:270) >> > at java.security.AccessController.doPrivileged(Native Method) >> > at javax.security.auth.Subject.doAs(Subject.java:396) >> > at >> > >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127) >> > at org.apache.hadoop.mapred.Child.main(Child.java:264) >> > Caused by: java.lang.RuntimeException: java.lang.RuntimeException: >> > java.lang.ClassNotFoundException: >> > com.skyboximaging.janus.giraph.example.SkyboxShortestPathsVertex >> > at >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1028) >> > at >> > org.apache.giraph.graph.BspUtils.getVertexClass(BspUtils.java:299) >> > >> > When I checked, the class does exist in my JAR, in the right path: >> > >> > $ jar tvf giraph-example-shortest-path-with-libs.jar | grep >> > SkyboxShortestPathsVertex >> > 6029 Tue Jan 31 13:22:50 PST 2012 >> > com/skyboximaging/janus/giraph/example/SkyboxShortestPathsVertex.class >> > >> > Maybe this is yet another Hadoop issue (i.e. not specific to Giraph) >> but I'd >> > appreciate any help with it. >> > >> > Thank you, >> > >> > Shaunak >> > >> > On Wed, Dec 14, 2011 at 6:57 AM, Shaunak Kashyap <[email protected] >> > >> > wrote: >> >> >> >> Hi Avery, >> >> >> >> You're right; my error turned out to be a Hadoop issue which I have >> >> worked around now. Thanks for the official documentation link. >> >> >> >> Shaunak >> >> >> >> On Tue, Dec 13, 2011 at 5:07 PM, Avery Ching <[email protected]> >> wrote: >> >> > Hi Shaunak, >> >> > >> >> > The official GIRAPH documentation is located at >> >> > https://cwiki.apache.org/confluence/display/GIRAPH/Index . That >> being >> >> > said, >> >> > your error looks like a Hadoop issue. Are you able to run normal >> Hadoop >> >> > example jobs? >> >> > >> >> > Thanks, >> >> > >> >> > Avery >> >> > >> >> > >> >> > On 12/13/11 9:27 AM, Shaunak Kashyap wrote: >> >> >> >> >> >> Hi, >> >> >> >> >> >> Maybe this page - >> >> >> https://github.com/aching/Giraph/wiki/Quick-Start-Guide - is >> outdated >> >> >> but I tried following the instructions there and am getting the >> >> >> following error: >> >> >> >> >> >> shaunak@hadoop-1:~/Giraph$ hadoop jar >> >> >> target/giraph-0.70-jar-with-dependencies.jar >> >> >> org.apache.giraph.benchmark.PageRankBenchmark -h >> >> >> 11/12/13 09:25:08 FATAL conf.Configuration: error parsing conf file: >> >> >> javax.xml.parsers.ParserConfigurationException: Feature >> >> >> 'http://apache.org/xml/features/xinclude' is not recognized. >> >> >> Exception in thread "main" java.lang.RuntimeException: >> >> >> javax.xml.parsers.ParserConfigurationException: Feature >> >> >> 'http://apache.org/xml/features/xinclude' is not recognized. >> >> >> at >> >> >> >> >> >> >> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1396) >> >> >> at >> >> >> >> >> >> >> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1251) >> >> >> at >> >> >> >> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1192) >> >> >> at >> >> >> org.apache.hadoop.conf.Configuration.get(Configuration.java:415) >> >> >> at org.apache.hadoop.util.RunJar.main(RunJar.java:143) >> >> >> Caused by: javax.xml.parsers.ParserConfigurationException: Feature >> >> >> 'http://apache.org/xml/features/xinclude' is not recognized. >> >> >> at >> >> >> >> >> >> >> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown >> >> >> Source) >> >> >> at >> >> >> >> >> >> >> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1282) >> >> >> ... 4 more >> >> >> >> >> >> What am I missing? >> >> >> >> >> >> Thank you, >> >> >> >> >> >> Shaunak >> >> >> >> >> > >> >> >> >> >> >> >> >> -- >> >> "Now the hardness of this world slowly grinds your dreams away / >> >> Makin' a fool's joke out of the promises we make" --- Bruce >> >> Springsteen, "Blood Brothers" >> > >> > >> > >> > >> > -- >> > "Now the hardness of this world slowly grinds your dreams away / Makin' >> a >> > fool's joke out of the promises we make" --- Bruce Springsteen, "Blood >> > Brothers" >> > > > > -- > "Now the hardness of this world slowly grinds your dreams away / Makin' a > fool's joke out of the promises we make" --- Bruce Springsteen, "Blood > Brothers" > -- "Now the hardness of this world slowly grinds your dreams away / Makin' a fool's joke out of the promises we make" --- Bruce Springsteen, "Blood Brothers"
