something is wrong when *Reducer* group the Vector by *BlockID*. The bug is introduced by me, so I will fix it now and attach a patch.
On Mon, Dec 1, 2008 at 9:17 AM, Edward J. Yoon <[EMAIL PROTECTED]>wrote: > Samuel, can you guess why it only appears on multi-node? :) > > On Sun, Nov 30, 2008 at 12:20 PM, Edward J. Yoon <[EMAIL PROTECTED]> > wrote: > > I filed this issue - HAMA-116 > > > > Seems like bug related with BlockingMapRed. > > > > On Sun, Nov 30, 2008 at 12:02 PM, Edward J. Yoon <[EMAIL PROTECTED]> > wrote: > >> In large scale, Hama may able to beat the scalapack or other math > >> libraries :) BTW, current block multiplication seems have a lot of > >> bug. > >> > >> ---- > >> 08/11/30 11:52:15 INFO mapred.JobClient: map 95% reduce 0% > >> 08/11/30 11:52:17 INFO mapred.JobClient: map 96% reduce 0% > >> 08/11/30 11:52:19 INFO mapred.JobClient: map 98% reduce 0% > >> 08/11/30 11:52:20 INFO mapred.JobClient: map 100% reduce 0% > >> 08/11/30 11:52:39 INFO mapred.JobClient: map 100% reduce 1% > >> 08/11/30 11:52:46 INFO mapred.JobClient: map 100% reduce 0% > >> 08/11/30 11:52:46 INFO mapred.JobClient: Task Id : > attempt_200811271544_0006_r_0 > >> 00000_0, Status : FAILED > >> java.lang.ArrayIndexOutOfBoundsException: 10 > >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61) > >> at > org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking > >> MapRed.java:146) > >> at > org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking > >> MapRed.java:127) > >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318) > >> at > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207 > >> ) > >> > >> 08/11/30 11:52:48 INFO mapred.JobClient: Task Id : > attempt_200811271544_0006_r_0 > >> 00002_0, Status : FAILED > >> java.lang.ArrayIndexOutOfBoundsException: 10 > >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61) > >> at > org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking > >> MapRed.java:146) > >> at > org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking > >> MapRed.java:127) > >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318) > >> at > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207 > >> ) > >> > >> 08/11/30 11:52:50 INFO mapred.JobClient: Task Id : > attempt_200811271544_0006_r_0 > >> 00001_0, Status : FAILED > >> java.lang.ArrayIndexOutOfBoundsException: 10 > >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61) > >> at > org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking > >> MapRed.java:146) > >> at > org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking > >> MapRed.java:127) > >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318) > >> at > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207 > >> ) > >> > >> 08/11/30 11:54:02 INFO mapred.JobClient: map 100% reduce 1% > >> 08/11/30 11:54:22 INFO mapred.JobClient: map 100% reduce 0% > >> 08/11/30 11:54:22 INFO mapred.JobClient: Task Id : > attempt_200811271544_0006_r_0 > >> 00003_0, Status : FAILED > >> java.lang.ArrayIndexOutOfBoundsException: 10 > >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61) > >> at > org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking > >> MapRed.java:146) > >> at > org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking > >> MapRed.java:127) > >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318) > >> at > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207 > >> ) > >> > >> 08/11/30 11:54:42 INFO mapred.JobClient: map 100% reduce 1% > >> 08/11/30 11:54:51 INFO mapred.JobClient: map 100% reduce 0% > >> 08/11/30 11:54:51 INFO mapred.JobClient: Task Id : > attempt_200811271544_0006_r_0 > >> 00000_1, Status : FAILED > >> java.lang.ArrayIndexOutOfBoundsException: 10 > >> at org.apache.hama.SubMatrix.set(SubMatrix.java:61) > >> at > org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking > >> MapRed.java:146) > >> at > org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking > >> MapRed.java:127) > >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318) > >> at > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207 > >> ) > >> > >> -- > >> Best Regards, Edward J. Yoon @ NHN, corp. > >> [EMAIL PROTECTED] > >> http://blog.udanax.org > >> > > > > > > > > -- > > Best Regards, Edward J. Yoon @ NHN, corp. > > [EMAIL PROTECTED] > > http://blog.udanax.org > > > > > > -- > Best Regards, Edward J. Yoon @ NHN, corp. > [EMAIL PROTECTED] > http://blog.udanax.org >
