It shouldn't be, your code looks very similar to the unittests (i.e. So, you're trying to run your test with the local hadoop (similar to the unittests)? Or are you using an actual hadoop setup?


On 4/10/12 11:41 PM, Paolo Castagna wrote:
I am using hadoop-core-1.0.1.jar ... could that be a problem?


Paolo Castagna wrote:
Hi Avery,
nope, no luck.

I have changed all my log.debug(...) into Same behavior.

I have a [1] file in my classpath and it has:
I also tried to change that to:
No luck.

My Giraph job has:
GiraphJob job = new GiraphJob(getConf(), getClass().getName());

But, if I run in debug with a breakpoint in the TurtleVertexInputFormat.class
constructor, it is never instanciated. How can it be?

So perhaps the problem is not the logging, it is the fact that
my GiraphJob is not using TurtleVertexInputFormat.class and
TurtleVertexOutputFormat.class, but I don't see what I am doing
wrong. :-/



Avery Ching wrote:
I think the issue might be that Hadoop only logs INFO and above messages
by default.  Can you retry with INFO level logging?


On 4/10/12 12:17 PM, Paolo Castagna wrote:
I am still learning Giraph, so, please, be patient with me and forgive my
trivial questions.

As a simple initial use case, I want to compute the shortest paths
from a single
source in a social graph in RDF format using the FOAF [1] vocabulary.
This example also will hopefully inform GIRAPH-170 [2] and related
issues, such
as: GIRAPH-141 [3].

Here is an example in Turtle [4] format of a tiny graph using FOAF:
@prefix :<>   .
@prefix foaf:<>   .

      a           foaf:Person ;
      foaf:name   "Alice" ;
      foaf:mbox<>   ;
      foaf:knows  :bob ;
      foaf:knows  :charlie ;
      foaf:knows  :snoopy ;

      foaf:name   "Bob" ;
      foaf:knows  :charlie ;

      foaf:name   "Charlie" ;
      foaf:knows  :alice ;
This is nice, human friendly (RDF without angle brackets!), but not
splittable to be processed with MapReduce (or Giraph).

Here is the same graph in N-Triples [5] format:
<>   .
<>   <>   "Alice" .
<>   <>
<>   .
<>   <>
<>   .
<>   <>
<>   .
<>   <>
<>   .
<>   <>
"Charlie" .
<>   <>
<>   .
<>   <>   "Bob" .
<>   <>
<>   .
This is more verbose and ugly, but splittable.

The graph I am interested in is the graph represented by foaf:knows
relationships/links between people (please, note --knows-->
relationship here
has a direction, this isn't symmetric as in centralized social networking
websites such as Facebook or LinkedIn. Alice can claim to know Bob,
without Bob
knowing it and/or it might even be a false claim):

alice --knows-->   bob
alice --knows-->   charlie
alice --knows-->   snoopy
bob --knows-->   charlie
charlie --knows-->   alice

As a first step, I wrote a MapReduce job [6] to transform the RDF
graph above in
a sort of adjacency list using Turtle syntax, here is the output
(three lines):
<>   <>
<>;<>   "Alice";
<>; .<>
<>   <>.

<>   <>   "Bob";
<>   <>; .
<>   <>

<>   <>
<>   <>; .
<>   <>
<>   <>.
This is legal Turtle, but it is also splittable. Each line has all the
statements (i.e. egdes) for a person (there are also incoming edges).

I wrote a TurtleVertexReader [7] which extends
Text, NodeWritable, Text>   and a TurtleVertexInputFormat [8] which
TextVertexInputFormat<NodeWritable, Text, NodeWritable, Text>.
I wrote (copying from the example SimpleShortestPathsVertex) a
FoafShortestPathsVertex [9] which extends EdgeListVertex<NodeWritable,
IntWritable, NodeWritable, IntWritable>   and I am running it locally
using these
arguments: -Dgiraph.maxWorkers=1 -Dgiraph.SplitMasterWorker=false
-DoverwriteOutput=true src/test/resources/data3.ttl target/foaf 1

TurtleVertexReader, TurtleVertexInputFormat and
FoafShortestPathsVertex are
still work in progress and I am sure there are plenty of stupid errors.
However, I do not understand why when I run FoafShortestPathsVertex
with the
DEBUG level, I see debug statements from FoafShortestPathsVertex:
19:34:44 DEBUG FoafShortestPathsVertex   :: main({-Dgiraph.maxWorkers=1,
-Dgiraph.SplitMasterWorker=false, -DoverwriteOutput=true,
src/test/resources/data3.ttl, target/foaf,, 1})
19:34:44 DEBUG FoafShortestPathsVertex   :: getConf() -->   null
19:34:44 DEBUG FoafShortestPathsVertex   :: setConf(Configuration:
core-default.xml, core-site.xml)
19:34:44 DEBUG FoafShortestPathsVertex   ::
target/foaf,, 1})
19:34:44 DEBUG FoafShortestPathsVertex   :: getConf() -->   Configuration:
core-default.xml, core-site.xml
19:34:44 DEBUG FoafShortestPathsVertex   :: getConf() -->   Configuration:
core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml,

But, I do not see anything else, no log statement from
TurtleVertexReader or
TurtleVertexInputFormat. Why? What am I doing wrong?
Is it because I am running it locally?






Reply via email to