Hi Chahinez,
This seems like a Giraph problem and not Gora's. I am not sure if Giraph
works with Hadoop 2 out of the box or if there is something trick to be
done. Try writing to their mailing list. The problem is that in Hadoop 1
JobContext is a class and in Hadoop 2 JobContext is an interface, so when
Giraph tries to execute the job gets into this exception.
Now regarding to where the data comes from. The Giraph-Gora integration
provides the means for users that have stored their data through Gora in a
supported data store (HBase, Cassandra, Mongo, etc) to run graph
algorithms. Then you (the user) are responsible to store your data (your
graph) in a data store and tell Gora how it should convert it for Giraph.
So first you write or you have written your graph into a datastore using
Gora, and then you tell Giraph where it is and how it should read it
through the parameters
-Dgiraph.gora.datastore.class=org.apache.gora.hbase.store.HBaseStore
-----------------------------> where your data is
-Dgiraph.gora.key.class=java.lang.String
-----------------------------> the key you used when
inserting data
-Dgiraph.gora.persistent.class=org.apache.giraph.io.gora.generated.GEdge
----------------------------->
what you inserted as edges
-Dgiraph.gora.start.key=0
-----------------------------> Start
reading keys from
-Dgiraph.gora.end.key=10
-----------------------------> Read keys to
-Dgiraph.gora.keys.factory.class=org.apache.giraph.io.gora.utils.KeyFactory
-------------------> How to convert the keys into actual objects
-Dgiraph.gora.output.datastore.class=org.apache.gora.hbase.store.HBaseStore
--------------> where you want to put your results
-Dgiraph.gora.output.key.class=java.lang.String
-----------------------------> The key to be used
-Dgiraph.gora.output.persistent.class=org.apache.giraph.io.gora.generated.GEdgeResult
--> What the resulting objects from the computation are.
Makes sense? I hope it does.
This is the beauty of open source. There will usually be someone willing to
help you going :)
@Lewis
Yeah, this is true. We talked about something similar with Claudio Martella
a while ago, but we didn't get the time to write it down.
Renato M.
2014-11-09 16:56 GMT+01:00 Chahinez BENKOUSSAS <[email protected]
>:
> Hi Renato,
>
>
>
> Thank your. It was the problem of Classpath. I fixed it but now, I have
> another problem.
>
> Sorry for requesting you again, but I'm using Apache projects for the
> first time, so I have some diffeculties to understand the errors.
>
> The error is:
>
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/local/hadoop-2.4.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/giraph/giraph-gora/target/giraph-gora-1.1.0-SNAPSHOT-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.4.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 14/11/09 16:30:24 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 14/11/09 16:30:24 INFO utils.ConfigurationUtils: No vertex input format
> specified. Ensure your InputFormat does not require one.
> 14/11/09 16:30:24 INFO utils.ConfigurationUtils: No vertex output format
> specified. Ensure your OutputFormat does not require one.
> 14/11/09 16:30:24 INFO utils.ConfigurationUtils: No input path for edge
> data was specified. Ensure your InputFormat does not require one.
> 14/11/09 16:30:24 INFO Configuration.deprecation:
> mapreduce.job.counters.limit is deprecated. Instead, use
> mapreduce.job.counters.max
> 14/11/09 16:30:24 INFO Configuration.deprecation: mapred.job.map.memory.mb
> is deprecated. Instead, use mapreduce.map.memory.mb
> 14/11/09 16:30:24 INFO Configuration.deprecation:
> mapred.job.reduce.memory.mb is deprecated. Instead, use
> mapreduce.reduce.memory.mb
> 14/11/09 16:30:24 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 14/11/09 16:30:24 INFO Configuration.deprecation:
> mapreduce.user.classpath.first is deprecated. Instead, use
> mapreduce.job.user.classpath.first
> 14/11/09 16:30:24 INFO Configuration.deprecation: mapred.map.max.attempts
> is deprecated. Instead, use mapreduce.map.maxattempts
> 14/11/09 16:30:24 INFO job.GiraphJob: run: Since checkpointing is disabled
> (default), do not allow any task retries (setting mapred.map.max.attempts =
> 0, old value = 4)
> 14/11/09 16:30:24 INFO Configuration.deprecation: mapred.job.tracker is
> deprecated. Instead, use mapreduce.jobtracker.address
> 14/11/09 16:30:24 INFO client.RMProxy: Connecting to ResourceManager at
> localhost/127.0.0.1:8050
> *Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
> interface org.apache.hadoop.mapreduce.JobContext, but class was expected*
> at
> org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:458)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:343)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:260)
> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>
> In the command I call Giraph, Gora and HBase but I can't see where we give
> the data (graph)??
>
> Chahinez.
>
>
>
> Le 2014-11-09 16:17, Renato Marroquín Mogrovejo a écrit :
>
> Hi Chahinez,
>
> Try putting
> *giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.4.0-jar-with-dependencies.jar *on
> your classpath.
> The jar might have the class but that doesn't mean the jar is on your
> classpath.
>
>
> Renato M.
>
> 2014-11-09 15:52 GMT+01:00 Chahinez BENKOUSSAS <
> [email protected]>:
>
>> Hi,
>>
>> I'm using the 1.1.0 version for Giraph, the 0.3 for Gora, 2.4.0 for
>> Hadoop and 0.94.8 for HBase.
>>
>> I've changed the options for the Input and Output format and the line
>> command became:
>>
>> hadoop/bin/hadoop jar $GIRAPH_EXAMPLES_JAR org.apache.giraph.GiraphRunner
>> -files
>> /usr/local/giraph/giraph-gora/conf/gora.properties,/usr/local/giraph/giraph-gora/conf/gora-hbase-mapping.xml,/usr/local/hbase/conf/hbase-site.xml
>>
>>
>> -Dio.serializations=org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.JavaSerialization
>>
>> -Dgiraph.gora.datastore.class=org.apache.gora.hbase.store.HBaseStore
>> -Dgiraph.gora.key.class=java.lang.String
>>
>> -Dgiraph.gora.persistent.class=org.apache.giraph.io.gora.generated.GEdge
>> -Dgiraph.gora.start.key=0
>> -Dgiraph.gora.end.key=10
>>
>> -Dgiraph.gora.keys.factory.class=org.apache.giraph.io.gora.utils.KeyFactory
>>
>> -Dgiraph.gora.output.datastore.class=org.apache.gora.hbase.store.HBaseStore
>>
>> -Dgiraph.gora.output.key.class=java.lang.String
>>
>> -Dgiraph.gora.output.persistent.class=org.apache.giraph.io.gora.generated.GEdgeResult
>>
>> -libjars $GIRAPH_GORA_JAR,$GORA_HBASE_JAR,$HBASE_JAR
>> org.apache.giraph.examples.SimpleShortestPathsComputation
>> -eif org.apache.giraph.io.gora.GoraGEdgeEdgeInputFormat
>> -eof org.apache.giraph.io.gora.GoraGEdgeEdgeOutputFormat
>> -w 1
>>
>> And now I have this error :/
>>
>>
>> SLF4J: Class path contains multiple SLF4J bindings.
>> SLF4J: Found binding in
>> [jar:file:/usr/local/hadoop-2.4.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in
>> [jar:file:/usr/local/giraph/giraph-gora/target/giraph-gora-1.1.0-SNAPSHOT-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> explanation.
>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> 14/11/09 13:36:55 WARN util.NativeCodeLoader: Unable to load
>> native-hadoop library for your platform... using builtin-java classes where
>> applicable
>> 14/11/09 13:36:55 INFO utils.ConfigurationUtils: No vertex input format
>> specified. Ensure your InputFormat does not require one.
>> 14/11/09 13:36:55 INFO utils.ConfigurationUtils: No vertex output format
>> specified. Ensure your OutputFormat does not require one.
>> 14/11/09 13:36:55 INFO utils.ConfigurationUtils: No input path for edge
>> data was specified. Ensure your InputFormat does not require one.
>> *Exception in thread "main" java.lang.ClassNotFoundException:
>> org.apache.giraph.examples.SimpleShortestPathsComputation*
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:190)
>> at
>> org.apache.giraph.utils.ConfigurationUtils.handleComputationClass(ConfigurationUtils.java:485)
>> at
>> org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:468)
>> at
>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:209)
>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>>
>>
>> It can't find the class *SimpleShortestPathsComputation* !!!
>> I've checked the
>> *"giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.4.0-jar-with-dependencies.jar*"
>> and the class exists, so I don't know where is the problem!!
>>
>> Any help PLEASE ....
>> Chahinez
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Le 2014-11-09 13:06, Renato Marroquín Mogrovejo a écrit :
>>
>> Hi Chahinez,
>>
>> There is a problem in your configuration parameters.
>>
>> -vif org.apache.giraph.io.gora.GoraGEdgeEdgeInputFormat
>> -eof org.apache.giraph.io.gora.GoraGEdgeEdgeOutputFormat
>>
>> The option <vif> means <Vertex input format> and as you can see you are
>> using GoraGEdgeInputformat which as you can see is an EDGE input format. A
>> graph can be represented by denoting the set of vertices with their edges
>> or the set of edges that join vertices. That is why Giraph gives you the
>> option to use either of them. So if you want to use the vertex input format
>> you should use org.apache.giraph.io.gora.GoraGVertexVertexInputFormat or
>> a class that extends from org.apache.giraph.io.gora.GoraVertexInput
>> Format.
>>
>>
>> Renato M.
>>
>>
>> 2014-11-07 19:34 GMT+01:00 Alfonso Nishikawa <[email protected]
>> >:
>>
>>> Hi, Chahinez,
>>>
>>> I am not a Giraph user, and I really don't know anything about, but this
>>> error:
>>>
>>> java.lang.RuntimeException: class org.apache.giraph.io.gora.Gora
>>> GEdgeEdgeInputFormat not org.apache.giraph.io.VertexInputFormat
>>>
>>> maybe can mean that it expects a VertexInputFormat and finds a
>>> GoraGEdgeEdgeInputFormat.
>>> Taking a look at the tutorial you talk about, tells something about two
>>> classes that can be used (as "Vertex" and as "Edges").
>>> I suggest to try with the class GoraGVertexVertexInputFormat instead of
>>> GoraGEdgeEdgeInputFormat.
>>> I comment this as a 1000% ignorant non-user of Giraph.
>>>
>>> Hope it luckily works! ;)
>>>
>>> Regards,
>>>
>>> Alfonso
>>>
>>>
>>> 2014-11-07 17:42 GMT+01:00 Lewis John Mcgibbney <
>>> [email protected]>:
>>>
>>>> Hi Chahinez,
>>>> Which versions of Giraph and therefore Gora are you using?
>>>> Renato recently upgraded Gora in Giraph to Gora v0.5 [0]. This will
>>>> ship with Giraph 1.1.0.
>>>> I'm going to work through the example with Giraph master this coming
>>>> lunchtime and will feed my results back to this thread.
>>>> It would be great if you could work with Giraph master as well as that
>>>> we are both on the same page.
>>>> Thanks
>>>>
>>>> [0] https://issues.apache.org/jira/browse/GIRAPH-946
>>>>
>>>> On Fri, Nov 7, 2014 at 7:35 AM, Chahinez BENKOUSSAS <
>>>> [email protected]> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I followed this tuorial to run Giraph with Gora (and also Hbase)
>>>>> http://giraph.apache.org/gora.html
>>>>>
>>>>> At the end of the tutorial, there is a command line that enable to
>>>>> calculate the ShortestPath.
>>>>>
>>>>> hadoop/bin/hadoop jar $GIRAPH_EXAMPLES_JAR
>>>>> org.apache.giraph.GiraphRunner
>>>>> -files /usr/local/giraph/giraph-gora/conf/gora.properties,/usr/
>>>>> local/giraph/giraph-gora/conf/gora-hbase-mapping.xml,/usr/
>>>>> local/hbase/conf/hbase-site.xml
>>>>> -Dio.serializations=org.apache.hadoop.io.serializer.
>>>>> WritableSerialization,org.apache.hadoop.io.serializer.
>>>>> JavaSerialization
>>>>> -Dgiraph.gora.datastore.class=org.apache.gora.hbase.store.HBaseStore
>>>>> -Dgiraph.gora.key.class=java.lang.String
>>>>> -Dgiraph.gora.persistent.class=org.apache.giraph.io.
>>>>> gora.generated.GEdge
>>>>> -Dgiraph.gora.start.key=0
>>>>> -Dgiraph.gora.end.key=10
>>>>> -Dgiraph.gora.keys.factory.class=org.apache.giraph.io.
>>>>> gora.utils.KeyFactory
>>>>> -Dgiraph.gora.output.datastore.class=org.apache.
>>>>> gora.hbase.store.HBaseStore
>>>>> -Dgiraph.gora.output.key.class=java.lang.String
>>>>> -Dgiraph.gora.output.persistent.class=org.apache.
>>>>> giraph.io.gora.generated.GEdgeResult
>>>>> -libjars $GIRAPH_GORA_JAR,$GORA_HBASE_JAR,$HBASE_JAR
>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation
>>>>> -vif org.apache.giraph.io.gora.GoraGEdgeEdgeInputFormat
>>>>> -eof org.apache.giraph.io.gora.GoraGEdgeEdgeOutputFormat
>>>>> -w 1
>>>>>
>>>>>
>>>>> I have the following error:
>>>>>
>>>>> SLF4J: Class path contains multiple SLF4J bindings.
>>>>> SLF4J: Found binding in [jar:file:/usr/local/hadoop-2.
>>>>> 4.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/
>>>>> slf4j/impl/StaticLoggerBinder.class]
>>>>> SLF4J: Found binding in [jar:file:/usr/local/giraph/
>>>>> giraph-gora/target/giraph-gora-1.1.0-SNAPSHOT-jar-with-
>>>>> dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>>>> explanation.
>>>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>>>> 14/11/07 14:45:02 WARN util.NativeCodeLoader: Unable to load
>>>>> native-hadoop library for your platform... using builtin-java classes
>>>>> where
>>>>> applicable
>>>>> Exception in thread "main" java.lang.RuntimeException: class
>>>>> org.apache.giraph.io.gora.GoraGEdgeEdgeInputFormat not
>>>>> org.apache.giraph.io.VertexInputFormat
>>>>> at org.apache.hadoop.conf.Configuration.setClass(
>>>>> Configuration.java:1969)
>>>>> at org.apache.giraph.conf.ClassConfOption.set(
>>>>> ClassConfOption.java:182)
>>>>> at org.apache.giraph.conf.GiraphConfiguration.
>>>>> setVertexInputFormatClass(GiraphConfiguration.java:250)
>>>>> at org.apache.giraph.utils.ConfigurationUtils.
>>>>> populateGiraphConfiguration(ConfigurationUtils.java:315)
>>>>> at org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>>>> ConfigurationUtils.java:209)
>>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(
>>>>> NativeMethodAccessorImpl.java:57)
>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>>> DelegatingMethodAccessorImpl.java:43)
>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>>>>>
>>>>>
>>>>> I can't understand how can hadoop get the input (graph or other data)
>>>>> for this example !!?? and why did I got this error ???
>>>>>
>>>>>
>>>>> Any help will be greatly appreciated !!
>>>>> Thank you.
>>>>>
>>>>> Chahinez
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Lewis*
>>>>
>>>
>
> --
> Mlle Chahinez BENKOUSSAS
> Doctorante en Informatique,
> Université d'Aix Marseille,
> LSIS Campus Saint Jérôme /
> Centre de l'édition électronique Ouverte CLéO
>
>
