Hello,
I've been having some problems getting Giraph to work with a table
that is stored in HBase. I've created my own classes that extend
HBaseVertexInputFormat and HBaseVertexReader (respectively) and
implemented nextVertex() and getVertex() accordingly, but I still
can't get it to work. Here is what I typed into the command line after
specifying the GiraphRunner class:
org.apache.giraph.examples.SimplePageRankComputation -vif
HBasePractice.LongDoubleFloatHBaseVertexInputFormat -of
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /hbaseOut
-w 2 -mc
org.apache.giraph.examples.SimplePageRankComputation\$SimplePageRankMasterCompute
This is the error that I'm getting from the logs:
2013-07-31 17:20:28,880 ERROR org.apache.giraph.master.MasterThread:
masterThread: Master algorithm failed with NullPointerException
java.lang.NullPointerException
at
org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94)
at
org.apache.giraph.io.hbase.HBaseVertexInputFormat.getSplits(HBaseVertexInputFormat.java:169)
at
org.apache.giraph.io.internal.WrappedVertexInputFormat.getSplits(WrappedVertexInputFormat.java:72)
at
org.apache.giraph.master.BspServiceMaster.generateInputSplits(BspServiceMaster.java:315)
at
org.apache.giraph.master.BspServiceMaster.createInputSplits(BspServiceMaster.java:627)
at
org.apache.giraph.master.BspServiceMaster.createVertexInputSplits(BspServiceMaster.java:694)
at org.apache.giraph.master.MasterThread.run(MasterThread.java:100)
2013-07-31 17:20:28,881 FATAL org.apache.giraph.graph.GraphMapper:
uncaughtException: OverrideExceptionHandler on thread
org.apache.giraph.master.MasterThread, msg =
java.lang.NullPointerException, exiting...
java.lang.IllegalStateException: java.lang.NullPointerException
at org.apache.giraph.master.MasterThread.run(MasterThread.java:185)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94)
at
org.apache.giraph.io.hbase.HBaseVertexInputFormat.getSplits(HBaseVertexInputFormat.java:169)
at
org.apache.giraph.io.internal.WrappedVertexInputFormat.getSplits(WrappedVertexInputFormat.java:72)
at
org.apache.giraph.master.BspServiceMaster.generateInputSplits(BspServiceMaster.java:315)
at
org.apache.giraph.master.BspServiceMaster.createInputSplits(BspServiceMaster.java:627)
at
org.apache.giraph.master.BspServiceMaster.createVertexInputSplits(BspServiceMaster.java:694)
at org.apache.giraph.master.MasterThread.run(MasterThread.java:100)
I don't know how/why a NullPointerException is being thrown. Do I
need to specify the -vip option? I didn't think so, because in my code
for my class that extends HBaseVertexInputFormat I do the following:
private ImmutableClassesGiraphConfiguration<LongWritable,
DoubleWritable, FloatWritable> conf;
@Override
public void setConf(ImmutableClassesGiraphConfiguration<LongWritable,
DoubleWritable, FloatWritable> conf) {
conf.set(TableInputFormat.INPUT_TABLE, "edges");
this.conf = conf;
}
Help would be much appreciated. I actually was a bit hesitant to post
this at first; I was determined to figure it out on my own, but then I
noticed that there have been virtually no questions asked regarding
HBase + Giraph on this mailing list, so I figured that posting this
might help someone out with similar problems later on in the future.
Thanks,
--
Kyle Orlando
Computer Engineering Major
University of Maryland
