Hama Trunk doesn't work for large matrices multiplication with
mapred.task.timeout and scanner.timeout exception. I tried 1,000,000 *
1,000,000 matrix multiplication on 100 node. (Rests are good)
To reduce read operation of duplicated block, I thought as describe
below. But, each map processing seems too large.
----
// c[i][k] += a[i][j] * b[j][k];
map() {
SubMatrix a = value.get();
for (RowResult row : scan) {
collect : c[i][k] = a * b[j][k];
}
}
reduce() {
c[i][k] += c[i][k];
}
----
Should we increase {mapred.task.timeout and scanner.timeout}?
or any good idea?
--
Best Regards, Edward J. Yoon @ NHN, corp.
[email protected]
http://blog.udanax.org
