Paolo Castagna wrote:
> Avery Ching wrote:
>> It shouldn't be, your code looks very similar to the unittests (i.e.
>> TestManualCheckpoint.java). So, you're trying to run your test with the
>> local hadoop (similar to the unittests)? Or are you using an actual
>> hadoop setup?
>
> Hi Avery,
> here is a few more details on what I am trying to do, in order to run my
> Giraph
> jobs on a local Hadoop running (for testing and debugging stuff locally):
>
> GiraphJob job = new GiraphJob("shortest paths");
> Configuration conf = job.getConfiguration();
> conf.setBoolean(GiraphJob.SPLIT_MASTER_WORKER, false);
> conf.setBoolean(GiraphJob.LOCAL_TEST_MODE, true);
> // conf.set(GiraphJob.ZOOKEEPER_JAR,
> "file://target/dependency/zookeeper-3.3.3.jar");
> job.setWorkerConfiguration(1, 1, 100.0f);
> job.setVertexClass(SimpleShortestPathsVertex.class);
> job.setVertexInputFormatClass(SimpleShortestPathsVertexInputFormat.class);
> job.setVertexOutputFormatClass(SimpleShortestPathsVertexOutputFormat.class);
> FileInputFormat.addInputPath(job.getInternalJob(), new
> Path("src/main/resources/giraph1.txt"));
> Path outputPath = new Path("target/giraph1");
> FileSystem hdfs = FileSystem.get(conf);
> hdfs.delete(outputPath, true);
> FileOutputFormat.setOutputPath(job.getInternalJob(), outputPath);
> job.run(true);
>
> Am I doing something wrong/stupid here?
> Am I missing something important? (probably! but I do not see what I am
> missing)
This is a better way:
Iterable<String> results = InternalVertexRunner.run(
SimpleShortestPathsVertex.class,
SimpleShortestPathsVertex.SimpleShortestPathsVertexInputFormat.class,
SimpleShortestPathsVertex.SimpleShortestPathsVertexOutputFormat.class,
params, graph);
... which starts a local ZooKeeper properly.
However, I still have a question: when I run it in a unit test everything is
fine. When I run it on a Java main method, it hangs towards the end.
Paolo
