Hi all, first, I cannot find this node runner "bin/giraph" mentioned in GIRAPH-64 in the 0.7 trunk and, second, I'm not quite sure about the essence from that ueber-fat-jar discussion. Is there a better/best practice way to run my implementation (vertex, io, of, etc) other then the ueber jar?
What am I missing? Thanks. Christoph --- original msg --- Gianmarco- GIRAPH-64 has been committed, so it should be much easier to treat Giraph as a library and link your vertices against it. Try it out and see if it meets your needs. If not, be sure to open a JIRA so we can improve it. Thanks, Jakob On Thu, Nov 10, 2011 at 7:31 AM, Gianmarco De Francisci Morales <[email protected]> wrote: > Nice! > It looks cleaner than my ant+ivy hack :) > Thanks for sharing, > -- > Gianmarco > > > On Thu, Nov 10, 2011 at 16:18, Claudio Martella <[email protected]> > wrote: >> >> Ok, >> >> quite basically this is my pom.xml: >> >> <project xmlns="http://maven.apache.org/POM/4.0.0" >> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" >> � � � �xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 >> http://maven.apache.org/maven-v4_0_0.xsd"> >> � � � �<modelVersion>4.0.0</modelVersion> >> � � � �<groupId>org.acaro.graffiti</groupId> >> � � � �<artifactId>graffiti</artifactId> >> � � � �<packaging>jar</packaging> >> � � � �<version>1.0-SNAPSHOT</version> >> � � � �<name>graffiti</name> >> � � � �<url>http://maven.apache.org</url> >> � � � �<build> >> � � � � � � � �<plugins> >> � � � � � � � � � � � �<plugin> >> � � � � � � � � � � � � � � � �<groupId>org.apache.maven.plugins</groupId> >> >> �<artifactId>maven-assembly-plugin</artifactId> >> � � � � � � � � � � � � � � � �<version>2.2</version> >> � � � � � � � � � � � � � � � �<configuration> >> >> �<outputDirectory>target</outputDirectory> >> � � � � � � � � � � � � � � � � � � � �<descriptorRefs> >> >> �<descriptorRef>jar-with-dependencies</descriptorRef> >> � � � � � � � � � � � � � � � � � � � �</descriptorRefs> >> � � � � � � � � � � � � � � � �</configuration> >> � � � � � � � � � � � � � � � �<executions> >> � � � � � � � � � � � � � � � � � � � �<execution> >> � � � � � � � � � � � � � � � � � � � � � � � �<id>make-assembly</id> >> � � � � � � � � � � � � � � � � � � � � � � � �<phase>compile</phase> >> � � � � � � � � � � � � � � � � � � � � � � � �<goals> >> � � � � � � � � � � � � � � � � � � � � � � � � � � � �<goal>single</goal> >> � � � � � � � � � � � � � � � � � � � � � � � �</goals> >> � � � � � � � � � � � � � � � � � � � �</execution> >> � � � � � � � � � � � � � � � �</executions> >> � � � � � � � � � � � �</plugin> >> � � � � � � � � � � � �<plugin> >> � � � � � � � � � � � � � � � �<groupId>org.apache.maven.plugins</groupId> >> >> �<artifactId>maven-eclipse-plugin</artifactId> >> � � � � � � � � � � � � � � � �<configuration> >> >> �<downloadSources>true</downloadSources> >> >> �<downloadJavadocs>true</downloadJavadocs> >> � � � � � � � � � � � � � � � �</configuration> >> � � � � � � � � � � � �</plugin> >> � � � � � � � � � � � �<plugin> >> � � � � � � � � � � � � � � � �<groupId>org.apache.maven.plugins</groupId> >> � � � � � � � � � � � � � � � �<artifactId>maven-shade-plugin</artifactId> >> � � � � � � � � � � � � � � � �<version>1.5</version> >> � � � � � � � � � � � � � � � �<executions> >> � � � � � � � � � � � � � � � � � � � �<execution> >> � � � � � � � � � � � � � � � � � � � � � � � �<phase>package</phase> >> � � � � � � � � � � � � � � � � � � � � � � � �<goals> >> � � � � � � � � � � � � � � � � � � � � � � � � � � � �<goal>shade</goal> >> � � � � � � � � � � � � � � � � � � � � � � � �</goals> >> � � � � � � � � � � � � � � � � � � � � � � � �<configuration> >> � � � � � � � � � � � � � � � � � � � � � � � �</configuration> >> � � � � � � � � � � � � � � � � � � � �</execution> >> � � � � � � � � � � � � � � � �</executions> >> � � � � � � � � � � � �</plugin> >> � � � � � � � �</plugins> >> � � � �</build> >> � � � �<dependencies> >> � � � � � � � �<dependency> >> � � � � � � � � � � � �<groupId>junit</groupId> >> � � � � � � � � � � � �<artifactId>junit</artifactId> >> � � � � � � � � � � � �<version>4.8.2</version> >> � � � � � � � � � � � �<scope>test</scope> >> � � � � � � � �</dependency> >> � � � � � � � �<dependency> >> � � � � � � � � � � � �<groupId>ch.qos.logback</groupId> >> � � � � � � � � � � � �<artifactId>logback-classic</artifactId> >> � � � � � � � � � � � �<version>0.9.27</version> >> � � � � � � � �</dependency> >> � � � � � � � �<dependency> >> � � � � � � � � � � � �<groupId>com.google.guava</groupId> >> � � � � � � � � � � � �<artifactId>guava</artifactId> >> � � � � � � � � � � � �<version>r09</version> >> � � � � � � � �</dependency> >> � � � � � � � �<dependency> >> � � � � � � � � � � � �<groupId>org.antlr</groupId> >> � � � � � � � � � � � �<artifactId>antlr</artifactId> >> � � � � � � � � � � � �<version>3.3</version> >> � � � � � � � �</dependency> >> � � � � � � � �<dependency> >> � � � � � � � � � � � �<groupId>org.apache.giraph</groupId> >> � � � � � � � � � � � �<artifactId>giraph</artifactId> >> � � � � � � � � � � � �<version>0.70</version> >> � � � � � � � �</dependency> >> � � � � � � � �<dependency> >> � � � � � � � � � � � �<groupId>org.apache.hadoop</groupId> >> � � � � � � � � � � � �<artifactId>hadoop-core</artifactId> >> � � � � � � � � � � � �<version>0.20.204.0</version> >> � � � � � � � � � � � �<scope>provided</scope> >> � � � � � � � �</dependency> >> � � � �</dependencies> >> </project> >> >> >> You set your project dependecies as usual, set hadoop-core (you don't >> really need it in the job jar) as <provided> and then use the >> maven-shade (http://maven.apache.org/plugins/maven-shade-plugin/). At >> package goal it will upack your dependencies and re-pack them along >> with your project classes in an uberjar. >> >> maven-shade allows you to set excludes in case of conflicts. >> >> Easy, automatic and clean. >> >> >> Hope this helps, >> Claudio >> >> On Wed, Nov 9, 2011 at 12:19 PM, Avery Ching <[email protected]> wrote: >> > Would be great if you can document what you did. =) >> > >> > Thanks, >> > >> > Avery >> > >> > On 11/8/11 3:13 PM, Claudio Martella wrote: >> >> >> >> Sorry guys, may bad. >> >> >> >> Was calling job.waitForCompletion() directly. I've been coding >> >> standard mapreduce whole weekend... >> >> >> >> Anyway I got a solution for clean packaging of your own application >> >> over giraph, and that is exactly using maven-shade-plugin. it will >> >> prepare the uberjar for you. >> >> >> >> On Tue, Nov 8, 2011 at 9:33 PM, Claudio Martella >> >> <[email protected]> �wrote: >> >>> >> >>> Hello list, >> >>> >> >>> I'm actually having troubles as well to get my application running. >> >>> >> >>> I've give a shot to maven-shade plugin which unpacks my dependencies >> >>> and packs them all together with my classes in a new jar. >> >>> >> >>> I attach the hierarchy of the jar so that somebody can maybe spot >> >>> what's missing, because i can't get it working. I get an identity >> >>> map-reduce job with jobconf complaining about no job jar being set. >> >>> >> >>> Any idea? >> >>> >> >>> On Sat, Nov 5, 2011 at 5:09 PM, Avery Ching<[email protected]> �wrote: >> >>>> >> >>>> Hi Gianmarco, >> >>>> >> >>>> You're right, most of us (to my knowledge) have been using Giraph >> >>>> with >> >>>> an >> >>>> uberjar as you've put it. �However, Jakob has been doing some work to >> >>>> make >> >>>> this easier. �See the below issue: >> >>>> >> >>>> https://issues.apache.org/jira/browse/GIRAPH-64 >> >>>> >> >>>> If you can suggest a better approach, please add to the issue or >> >>>> create >> >>>> a >> >>>> new one if appropriate. >> >>>> >> >>>> Thanks, >> >>>> >> >>>> Avery >> >>>> >> >>>> On 11/5/11 4:11 AM, Gianmarco De Francisci Morales wrote: >> >>>>> >> >>>>> Hi community, >> >>>>> >> >>>>> I was wondering what is the current best practice to package an >> >>>>> application in a jar for deployment. >> >>>>> I tried the 'hadoop way' by putting giraph-*.jar in the /lib >> >>>>> directory >> >>>>> of >> >>>>> my jar, and using the -libjars option but none of them worked. It >> >>>>> looks >> >>>>> like >> >>>>> the backend classloader is doing some mess and it doesn't find my >> >>>>> own >> >>>>> classes in the jar. >> >>>>> >> >>>>> I resorted to uncompressing the giraph-*.jar and repackaging my >> >>>>> classes >> >>>>> with it, all at the same level (an uber-fat jar), but even though it >> >>>>> works >> >>>>> it doesn't sound like the right approach. >> >>>>> >> >>>>> Any suggestions? >> >>>>> >> >>>>> Thanks, >> >>>>> -- >> >>>>> Gianmarco >> >>>>> >> >>>>>
