[
https://issues.apache.org/jira/browse/HAMA-116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12652224#action_12652224
]
Samuel Guo commented on HAMA-116:
---------------------------------
> BTW, There is too many read/write operation.
Yes, it is a big problem when the size of the matrix is huge. Because the whole
matrix will be transfer to reduce stage, it is not efficient.
We may need a new InputFormat to read the matrix. As we know the number of the
block, we can split the table carefully so that a block's sub-vectors are in
the same *Matrix-Split*. Then we can use a simple *Combiner* to do local
*blocking*. No data is needed to be transfered to *Reducer*.
> ArrayIndexOutOfBoundsException at BlockingMapRed
> ------------------------------------------------
>
> Key: HAMA-116
> URL: https://issues.apache.org/jira/browse/HAMA-116
> Project: Hama
> Issue Type: Bug
> Components: mapred
> Reporter: Edward J. Yoon
> Assignee: Samuel Guo
> Fix For: 0.1.0
>
> Attachments: HAMA-116.patch
>
>
> [d8g053:/root/hama-trunk]# bin/hama examples mult -m 100 -r 100 1000 1000
> ----
> 08/11/30 11:56:07 INFO mapred.JobClient: map 100% reduce 1%
> 08/11/30 11:56:26 INFO mapred.JobClient: map 100% reduce 0%
> 08/11/30 11:56:26 INFO mapred.JobClient: Task Id :
> attempt_200811271544_0006_r_0
> 00001_1, Status : FAILED
> java.lang.ArrayIndexOutOfBoundsException: 10
> at org.apache.hama.SubMatrix.set(SubMatrix.java:61)
> at
> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
> MapRed.java:146)
> at
> org.apache.hama.mapred.BlockingMapRed$BlockingReducer.reduce(Blocking
> MapRed.java:127)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207
> )
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
