Hi Avery, and thanks for your reply.
On 21 Mar 2012, at 06:35, Avery Ching wrote:
> Benjamin, my guess is that your jar might not have all the ZooKeeper
> dependencies. Can you look at the log for the process that was supposed to
> start ZooKeeper? I'm thinking it didn't start...
I checked the logs and there is nothing which clearly looks like a ZooKeeper
error, or any other kind of information which might provide
a clue to why ZooKeeper is not starting. (It least there is nothing which I
can see when I grep -i zookeeper logs/*.log )
In addition, I checked the dependencies (Eclipse allows viewing them like a
All of the dependencies of zookeeper are part of the jar (in the /lib dir).
So that should not be the reason for zookeeper to fail in starting.
In addition, today I tried using bin/giraph (and GiraphRunner ) to run my job.
This also was ultimately unsuccessful.
As you can see from the patch which I submitted, I found a very annoying bug in
However, after fixing that, the hadoop class loader complained the following:
Exception in thread "main" java.lang.NoClassDefFoundError:
That is very puzzling to me, as the
giraph-0.2-SNAPSHOT-jar-with-dependencies.jar is provided two times.
First as the "jar" argument, and secondly as part of the "-libjars" argument.
Right now I am freshly out of ideas as to what could be going wrong.
So I checked with my supervisor, and it is okay to put my code on github.
It is available here:
A few questions which come to my mind as a sort of checklist:
* are my assembly instructions in pom.xml and in hadoop-job.xml correct ?
* is my invocation of the job as described in README correct ?
* is the construction of the GiraphJob in LoadRDFAdjacencyCSVAndHalt correct ?
Feel free to run the SimpleRDFVertexTest class, there is a small data file so
that you can see if it works for you.
I really hope somebody can explain to me why my giraph job is failing in such a