I run the follow command of hadoop: hduser@carmen-HP-Pavilion-Sleekbook-15:/usr/local/hadoop$ hadoop jar /home/carmen/workspace/sisinflab/target/sisinflab-0.0.1-SNAPSHOT.jar org.apache.giraph.GiraphRunner sisinflab.Query2 -vif sisinflab.Vinputformat -vip /user/hduser/labeldimezzato -w 1 -ca ARQ.query="SELECT *"
hduser has /.m2/repository with /com/hp/hpl/jena/ with all jars inside which are the same of dependencies i've specified in my pom... if it could be useful i want to send you the complete error status in output from hadoop : java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@2143db07 at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: java.lang.IllegalStateException: waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@2143db07 at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:193) at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:151) at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:136) at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:99) at org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:233) at org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:722) at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:300) at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91) ... 7 more Caused by: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: com/hp/hpl/jena/shared/PrefixMapping at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:202) at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:312) at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:185) ... 14 more Caused by: java.lang.NoClassDefFoundError: com/hp/hpl/jena/shared/PrefixMapping at com.hp.hpl.jena.query.QueryFactory.create(QueryFactory.java:78) at com.hp.hpl.jena.query.QueryFactory.create(QueryFactory.java:52) at com.hp.hpl.jena.query.QueryFactory.create(QueryFactory.java:40) at sisinflab.Query2.compute(Query2.java:27) at org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:247) at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:168) at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:71) at org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassNotFoundException: com.hp.hpl.jena.shared.PrefixMapping at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... YES after some tests the error is java.lang.NoClassDefFoundError: com/hp/hpl/jena/shared/PrefixMapping again. i can't understand which kind of problem could be because the syntax in my code seems to be corrected but as you could see there are errors in QueryFactory's calls (in particular create) and when i worked with only giraph i hadn't this problem with maven, dependencies,classpath etc... i want to thank you for your help...i'm glade ad i hope these informations could be a way to undestand the root of my problem.. 2014-07-30 10:38 GMT+02:00 Rob Vesse <[email protected]>: > Carmen > > I assume you are just building a normal Maven JAR? > > In which case the JAR produced will contain only your code and will not > contain the dependencies of your code and you are expected to provide them > to Hadoop's class path separately or you'll get these kinds of errors when > trying to run your program > > What I usually do (and I've seen others do) is to use the maven assembly > plugin to build a fat JAR with my code and all the necessary dependencies > and then run this instead. > > For an example maven assembler plugin configuration see > https://svn.apache.org/viewvc/jena/Experimental/hadoop-rdf/hadoop-rdf-stats > /pom.xml?view=markup lines 82-100 > > And the corresponding maven assembler file can be seen at > https://svn.apache.org/viewvc/jena/Experimental/hadoop-rdf/hadoop-rdf-stats > /hadoop-job.xml?view=markup > > Rob > > On 29/07/2014 20:53, "Andy Seaborne" <[email protected]> wrote: > > > > >On 29/07/14 15:01, Carmen Manzulli wrote: > >> Ok, sorry; i'm trying to use query and queryFactory classes of jena in > >>my > >> code in this way: > >> > >> > >> String que=getConf().get("ARQ.query"); //ARQ in a string > >> parameter in input (the query, for example SELECT *) > >> System.out.println("request to process query : "+que); > >> Query query = QueryFactory.create(que); > >> > >> in the third line i got error.... > > > >So is the error is different to your initial report and currently: > > > >java.lang.NoClassDefFoundError: org.apache.xerces.util.XMLChar > > > >what's the stacktrace for that one? > > > > > >The maven output looks OK but > > > >1/ Are that there endorsed jars around? > >2/ What the runtime classpath? > > > > Andy > > > >> > >> i used maven to create my project and i wrote a pom with dependencies; > >>this > >> is the cut of my pom: > >> > >> <dependencies> > >> <dependency> > >> <groupId>org.apache.hadoop</groupId> > >> <artifactId>hadoop-core</artifactId> > >> <version>1.2.1</version> > >> </dependency> > >> <dependency> > >> <groupId>org.apache.giraph</groupId> > >> <artifactId>giraph-core</artifactId> > >> <version>1.1.0</version> > >> </dependency> > >> <dependency> > >> <groupId>org.apache.jena</groupId> > >> <artifactId>jena-arq</artifactId> > >> <version>2.12.0-SNAPSHOT</version> > >> </dependency> > >> </dependencies> > >> > >> > >> this is the dependency tree: > >> > >> [INFO] --- maven-dependency-plugin:2.1:tree (default-cli) @ sisinflab > >>--- > >> [INFO] sisinflab:sisinflab:jar:1.0-SNAPSHOT > >> [INFO] +- org.apache.hadoop:hadoop-core:jar:1.2.1:compile > >> [INFO] | +- commons-cli:commons-cli:jar:1.2:compile > >> [INFO] | +- xmlenc:xmlenc:jar:0.52:compile > >> [INFO] | +- com.sun.jersey:jersey-core:jar:1.8:compile > >> [INFO] | +- com.sun.jersey:jersey-json:jar:1.8:compile > >> [INFO] | | +- org.codehaus.jettison:jettison:jar:1.1:compile > >> [INFO] | | | \- stax:stax-api:jar:1.0.1:compile > >> [INFO] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile > >> [INFO] | | | \- javax.xml.bind:jaxb-api:jar:2.2.2:compile > >> [INFO] | | | +- javax.xml.stream:stax-api:jar:1.0-2:compile > >> [INFO] | | | \- javax.activation:activation:jar:1.1:compile > >> [INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.7.1:compile > >> [INFO] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.7.1:compile > >> [INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.7.1:compile > >> [INFO] | +- com.sun.jersey:jersey-server:jar:1.8:compile > >> [INFO] | | \- asm:asm:jar:3.1:compile > >> [INFO] | +- commons-io:commons-io:jar:2.1:compile > >> [INFO] | +- commons-httpclient:commons-httpclient:jar:3.0.1:compile > >> [INFO] | | +- junit:junit:jar:3.8.1:compile > >> [INFO] | | \- commons-logging:commons-logging:jar:1.0.3:compile > >> [INFO] | +- commons-codec:commons-codec:jar:1.4:compile > >> [INFO] | +- org.apache.commons:commons-math:jar:2.1:compile > >> [INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile > >> [INFO] | | +- > >>commons-collections:commons-collections:jar:3.2.1:compile > >> [INFO] | | +- commons-lang:commons-lang:jar:2.4:compile > >> [INFO] | | +- commons-digester:commons-digester:jar:1.8:compile > >> [INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile > >> [INFO] | | \- > >>commons-beanutils:commons-beanutils-core:jar:1.8.0:compile > >> [INFO] | +- commons-net:commons-net:jar:1.4.1:compile > >> [INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile > >> [INFO] | | \- org.mortbay.jetty:servlet-api:jar:2.5-20081211:compile > >> [INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile > >> [INFO] | +- tomcat:jasper-runtime:jar:5.5.12:compile > >> [INFO] | +- tomcat:jasper-compiler:jar:5.5.12:compile > >> [INFO] | +- org.mortbay.jetty:jsp-api-2.1:jar:6.1.14:compile > >> [INFO] | | \- org.mortbay.jetty:servlet-api-2.5:jar:6.1.14:compile > >> [INFO] | +- org.mortbay.jetty:jsp-2.1:jar:6.1.14:compile > >> [INFO] | | \- ant:ant:jar:1.6.5:compile > >> [INFO] | +- commons-el:commons-el:jar:1.0:compile > >> [INFO] | +- net.java.dev.jets3t:jets3t:jar:0.6.1:compile > >> [INFO] | +- hsqldb:hsqldb:jar:1.8.0.10:compile > >> [INFO] | +- oro:oro:jar:2.0.8:compile > >> [INFO] | +- org.eclipse.jdt:core:jar:3.1.1:compile > >> [INFO] | \- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile > >> [INFO] +- org.apache.giraph:giraph-core:jar:1.1.0:compile > >> [INFO] \- org.apache.jena:jena-arq:jar:2.12.0-SNAPSHOT:compile > >> [INFO] +- org.apache.jena:jena-core:jar:2.12.0-SNAPSHOT:compile > >> [INFO] | +- org.apache.jena:jena-iri:jar:1.1.0-SNAPSHOT:compile > >> [INFO] | \- xerces:xercesImpl:jar:2.11.0:compile > >> [INFO] | \- xml-apis:xml-apis:jar:1.4.01:compile > >> [INFO] +- org.apache.httpcomponents:httpclient:jar:4.2.6:compile > >> [INFO] | \- org.apache.httpcomponents:httpcore:jar:4.2.5:compile > >> [INFO] +- com.github.jsonld-java:jsonld-java:jar:0.5.0:compile > >> [INFO] | +- > >>com.fasterxml.jackson.core:jackson-core:jar:2.3.3:compile > >> [INFO] | \- > >> com.fasterxml.jackson.core:jackson-databind:jar:2.3.3:compile > >> [INFO] | \- > >> com.fasterxml.jackson.core:jackson-annotations:jar:2.3.0:compile > >> [INFO] +- > >>org.apache.httpcomponents:httpclient-cache:jar:4.2.6:compile > >> [INFO] +- org.slf4j:jcl-over-slf4j:jar:1.7.6:compile > >> [INFO] +- org.slf4j:slf4j-api:jar:1.7.6:compile > >> [INFO] +- org.slf4j:slf4j-log4j12:jar:1.7.6:compile > >> [INFO] \- log4j:log4j:jar:1.2.17:compile > >> > >> after i created jar and executed this one using hadoop > >> > >> in a first time the problem was NoClassDefFound com.hp.hpl.jena.query > >> now the output is NoClassDefFound org.apache.xerces.util.XMLChar > > > > > >> > >> > >> > >> > >> 2014-07-29 13:43 GMT+02:00 Rob Vesse <[email protected]>: > >> > >>> Carmen > >>> > >>> It would be helpful if you showed us what you are doing, otherwise we > >>>can > >>> only guess and offer vague suggestions. > >>> > >>> Assuming you are using Maven as previously stated please provide the > >>> <dependencies> section of your POM > >>> > >>> It is also worth running mvn dependency:tree and checking that you > >>>don't > >>> have multiple entries for Jena libraries present. Including the > >>>output of > >>> this would also help us debug your problem > >>> > >>> Rob > >>> > >>> On 29/07/2014 12:08, "Carmen Manzulli" <[email protected]> > >>>wrote: > >>> > >>>> i've proved with jena-arq and jena-core 2.12.0 but these don't fix my > >>>> problem...the output is the same: > >>>> > >>>> NoClassDefFoundError: com/hp/hpl/jena/shared/PrefixMapping > >>>> > >>>> > >>>> can you suggest me a right way to insert jena as dependency keeping in > >>>> mind > >>>> that i need only Query and QueryFactory classes? > >>>> > >>>> > >>>> 2014-07-28 18:21 GMT+02:00 Andy Seaborne <[email protected]>: > >>>> > >>>>> On 28/07/14 16:36, Carmen Manzulli wrote: > >>>>> > >>>>>> ok....i'm using giraph 1.1.0, hadoop 1.2.1 jena arq 2.12.0 and > >>>>>> com.hp.hpl.jena 2.8.8; I've introduced the last one because there > >>>>>>were > >>>>>> class not found exceptions for query and queryfactory....i make use > >>>>>>of > >>>>>> these two classes to implement a simple select and just for the > >>>>>>moment > >>>>>> i'm > >>>>>> using select* as query string to test the code! > >>>>>> > >>>>> > >>>>> jena arq 2.12.0 and jena-core 2.8.8 don't mix. > >>>>> > >>>>> Use a consistent cut of jena. > >>>>> > >>>>> Andy > >>>>> > >>>>> > >>>>> > >>>>>> > >>>>>> 2014-07-28 16:43 GMT+02:00 Andy Seaborne <[email protected]>: > >>>>>> > >>>>>> On 28/07/14 15:00, Carmen Manzulli wrote: > >>>>>>> > >>>>>>> Hi experts, > >>>>>>>> > >>>>>>>> i'm trying to use some jena classes (as Query and QueryFactory) > >>>>>>>>in a > >>>>>>>> project but when i run my code i get: > >>>>>>>> > >>>>>>>> java.lang.NoClassDefFoundError: > >>>>>>>> com/hp/hpl/jena/shared/PrefixMapping$IllegalPrefixException > >>>>>>>> at com.hp.hpl.jena.query.QueryFactory.create( > >>>>>>>> QueryFactory.java:68) > >>>>>>>> at com.hp.hpl.jena.query.QueryFactory.create( > >>>>>>>> QueryFactory.java:40) > >>>>>>>> at com.hp.hpl.jena.query.QueryFactory.create( > >>>>>>>> QueryFactory.java:28) > >>>>>>>> at sisinflab.Query2.compute(Query2.java:27) > >>>>>>>> > >>>>>>>> and also : > >>>>>>>> > >>>>>>>> java.lang.ClassNotFoundException: > >>>>>>>> com.hp.hpl.jena.shared.PrefixMapping$IllegalPrefixException > >>>>>>>> at > >>>>>>>>java.net.URLClassLoader$1.run(URLClassLoader.java:366) > >>>>>>>> at > >>>>>>>>java.net.URLClassLoader$1.run(URLClassLoader.java:355) > >>>>>>>> at java.security.AccessController.doPrivileged(Native > >>>>>>>> Method) > >>>>>>>> at > >>>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:354) > >>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > >>>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass( > >>>>>>>> Launcher.java:308) > >>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > >>>>>>>> ... 12 more > >>>>>>>> > >>>>>>>> who can help me? > >>>>>>>> > >>>>>>>> > >>>>>>> "NoClassDefFoundError" means it was there are compile time (of > >>>>>>>Jena) > >>>>>>> and > >>>>>>> not at runtime (of your code). > >>>>>>> > >>>>>>> You have an incomplete or incompatible set of jars files. > >>>>>>> > >>>>>>> Andy > >>>>>>> > >>>>>>> PS Please could you include things like version numbers (needed for > >>>>>>> line > >>>>>>> numbers to mean anything), the query in question, etc when > >>>>>>>describing > >>>>>>> something. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >>> > >>> > >>> > >>> > >> > > > > > > >
