yes i'll check again Sent from my iPhone
On Feb 28, 2013, at 5:18 PM, Thomas Jungblut <[email protected]> wrote: > Can you verify an observation for me please? > > 2 files are created from fastgen, part-00000 and part-00001, both ~2.2kb > sized. > In the below partition directory, there is only a single 5.56kb file. > > Is it intended for the partitioner to write a single file if you configured > two? > It even reads it as a two files, strange huh? > > 2013/2/28 Thomas Jungblut <[email protected]> > >> Will have a look into it. >> >> gen fastgen 100 10 /tmp/randomgraph 1 >> pagerank /tmp/randomgraph /tmp/pageout >> >> did work for me the last time I profiled, maybe the partitioning doesn't >> partition correctly with the input or something else. >> >> >> 2013/2/28 Edward J. Yoon <[email protected]> >> >> Fastgen input seems not work for graph examples. >>> >>> edward@edward-virtualBox:~/workspace/hama-trunk$ bin/hama jar >>> examples/target/hama-examples-0.7.0-SNAPSHOT.jar gen fastgen 100 10 >>> /tmp/randomgraph 2 >>> 13/02/28 10:32:02 WARN util.NativeCodeLoader: Unable to load >>> native-hadoop library for your platform... using builtin-java classes >>> where applicable >>> 13/02/28 10:32:03 INFO bsp.BSPJobClient: Running job: job_localrunner_0001 >>> 13/02/28 10:32:03 INFO bsp.LocalBSPRunner: Setting up a new barrier for 2 >>> tasks! >>> 13/02/28 10:32:06 INFO bsp.BSPJobClient: Current supersteps number: 0 >>> 13/02/28 10:32:06 INFO bsp.BSPJobClient: The total number of supersteps: 0 >>> 13/02/28 10:32:06 INFO bsp.BSPJobClient: Counters: 3 >>> 13/02/28 10:32:06 INFO bsp.BSPJobClient: >>> org.apache.hama.bsp.JobInProgress$JobCounter >>> 13/02/28 10:32:06 INFO bsp.BSPJobClient: SUPERSTEPS=0 >>> 13/02/28 10:32:06 INFO bsp.BSPJobClient: LAUNCHED_TASKS=2 >>> 13/02/28 10:32:06 INFO bsp.BSPJobClient: >>> org.apache.hama.bsp.BSPPeerImpl$PeerCounter >>> 13/02/28 10:32:06 INFO bsp.BSPJobClient: TASK_OUTPUT_RECORDS=100 >>> Job Finished in 3.212 seconds >>> edward@edward-virtualBox:~/workspace/hama-trunk$ bin/hama jar >>> examples/target/hama-examples-0.7.0-SNAPSHOT >>> hama-examples-0.7.0-SNAPSHOT-javadoc.jar >>> hama-examples-0.7.0-SNAPSHOT.jar >>> edward@edward-virtualBox:~/workspace/hama-trunk$ bin/hama jar >>> examples/target/hama-examples-0.7.0-SNAPSHOT.jar pagerank >>> /tmp/randomgraph /tmp/pageour >>> 13/02/28 10:32:29 WARN util.NativeCodeLoader: Unable to load >>> native-hadoop library for your platform... using builtin-java classes >>> where applicable >>> 13/02/28 10:32:29 INFO bsp.FileInputFormat: Total input paths to process >>> : 2 >>> 13/02/28 10:32:29 INFO bsp.FileInputFormat: Total input paths to process >>> : 2 >>> 13/02/28 10:32:30 INFO bsp.BSPJobClient: Running job: job_localrunner_0001 >>> 13/02/28 10:32:30 INFO bsp.LocalBSPRunner: Setting up a new barrier for 2 >>> tasks! >>> 13/02/28 10:32:33 INFO bsp.BSPJobClient: Current supersteps number: 1 >>> 13/02/28 10:32:33 INFO bsp.BSPJobClient: The total number of supersteps: 1 >>> 13/02/28 10:32:33 INFO bsp.BSPJobClient: Counters: 6 >>> 13/02/28 10:32:33 INFO bsp.BSPJobClient: >>> org.apache.hama.bsp.JobInProgress$JobCounter >>> 13/02/28 10:32:33 INFO bsp.BSPJobClient: SUPERSTEPS=1 >>> 13/02/28 10:32:33 INFO bsp.BSPJobClient: LAUNCHED_TASKS=2 >>> 13/02/28 10:32:33 INFO bsp.BSPJobClient: >>> org.apache.hama.bsp.BSPPeerImpl$PeerCounter >>> 13/02/28 10:32:33 INFO bsp.BSPJobClient: SUPERSTEP_SUM=4 >>> 13/02/28 10:32:33 INFO bsp.BSPJobClient: IO_BYTES_READ=4332 >>> 13/02/28 10:32:33 INFO bsp.BSPJobClient: TIME_IN_SYNC_MS=14 >>> 13/02/28 10:32:33 INFO bsp.BSPJobClient: TASK_INPUT_RECORDS=100 >>> 13/02/28 10:32:33 INFO bsp.FileInputFormat: Total input paths to process >>> : 2 >>> 13/02/28 10:32:33 INFO bsp.BSPJobClient: Running job: job_localrunner_0001 >>> 13/02/28 10:32:33 INFO bsp.LocalBSPRunner: Setting up a new barrier for 2 >>> tasks! >>> 13/02/28 10:32:33 INFO graph.GraphJobRunner: 50 vertices are loaded into >>> local:1 >>> 13/02/28 10:32:33 INFO graph.GraphJobRunner: 50 vertices are loaded into >>> local:0 >>> 13/02/28 10:32:33 ERROR bsp.LocalBSPRunner: Exception during BSP >>> execution! >>> java.lang.IllegalArgumentException: Messages must never be behind the >>> vertex in ID! Current Message ID: 1 vs. 50 >>> at >>> org.apache.hama.graph.GraphJobRunner.iterate(GraphJobRunner.java:279) >>> at >>> org.apache.hama.graph.GraphJobRunner.doSuperstep(GraphJobRunner.java:225) >>> at >>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:129) >>> at >>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:256) >>> at >>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286) >>> at >>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211) >>> at >>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) >>> at java.util.concurrent.FutureTask.run(FutureTask.java:166) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >>> at >>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) >>> at java.util.concurrent.FutureTask.run(FutureTask.java:166) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) >>> at java.lang.Thread.run(Thread.java:722) >>> >>> >>> -- >>> Best Regards, Edward J. Yoon >>> @eddieyoon >> >>
