[
https://issues.apache.org/jira/browse/HAMA-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12646245#action_12646245
]
udanax edited comment on HAMA-103 at 11/10/08 4:30 AM:
---------------------------------------------------------------
Hmm...
After patch apply, Performance was improved on same condition.
I submit to hudson.
----
[d8g053:/root/hama-trunk]# bin/hama examples mult -m 60 -r 2 300 300
08/11/10 21:21:01 INFO hama.AbstractMatrix: Initializing the matrix storage.
08/11/10 21:21:05 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randhgpxz
08/11/10 21:21:06 INFO hama.AbstractMatrix: Create the 300 * 300 random matrix
: DenseMatrix_randhgpxz
08/11/10 21:21:06 INFO hama.AbstractMatrix: Initializing the matrix storage.
08/11/10 21:21:12 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randbdpyj
08/11/10 21:21:14 INFO hama.AbstractMatrix: Create the 300 * 300 random matrix
: DenseMatrix_randbdpyj
08/11/10 21:21:14 INFO hama.AbstractMatrix: Initializing the matrix storage.
08/11/10 21:21:20 INFO hama.AbstractMatrix: Create Matrix DenseMatrix_randeqkrg
08/11/10 21:21:20 WARN mapred.JobClient: Use GenericOptionsParser for parsing
the arguments. Applications should implement Tool for the same.
08/11/10 21:21:20 WARN mapred.JobClient: Use genericOptions for the option
-libjars
08/11/10 21:21:20 WARN mapred.JobClient: No job jar file set. User classes may
not be found. See JobConf(Class) or JobConf#setJar(String).
08/11/10 21:21:20 INFO mapred.JobClient: Running job: job_200811101756_0003
08/11/10 21:21:21 INFO mapred.JobClient: map 0% reduce 0%
08/11/10 21:21:26 INFO mapred.JobClient: map 1% reduce 0%
08/11/10 21:22:18 INFO mapred.JobClient: map 5% reduce 0%
08/11/10 21:22:19 INFO mapred.JobClient: map 6% reduce 0%
08/11/10 21:22:25 INFO mapred.JobClient: map 8% reduce 0%
08/11/10 21:22:27 INFO mapred.JobClient: map 10% reduce 0%
08/11/10 21:22:28 INFO mapred.JobClient: map 10% reduce 1%
08/11/10 21:22:33 INFO mapred.JobClient: map 11% reduce 2%
08/11/10 21:22:37 INFO mapred.JobClient: map 13% reduce 2%
08/11/10 21:22:40 INFO mapred.JobClient: map 15% reduce 3%
08/11/10 21:22:43 INFO mapred.JobClient: map 15% reduce 4%
08/11/10 21:22:45 INFO mapred.JobClient: map 16% reduce 4%
08/11/10 21:22:47 INFO mapred.JobClient: map 18% reduce 4%
08/11/10 21:22:50 INFO mapred.JobClient: map 18% reduce 5%
08/11/10 21:22:53 INFO mapred.JobClient: map 20% reduce 5%
08/11/10 21:22:54 INFO mapred.JobClient: map 21% reduce 5%
08/11/10 21:23:01 INFO mapred.JobClient: map 25% reduce 5%
08/11/10 21:23:06 INFO mapred.JobClient: map 25% reduce 6%
08/11/10 21:23:08 INFO mapred.JobClient: map 26% reduce 7%
08/11/10 21:23:09 INFO mapred.JobClient: map 28% reduce 7%
08/11/10 21:23:10 INFO mapred.JobClient: map 31% reduce 7%
08/11/10 21:23:17 INFO mapred.JobClient: map 35% reduce 8%
08/11/10 21:23:18 INFO mapred.JobClient: map 36% reduce 8%
08/11/10 21:23:22 INFO mapred.JobClient: map 36% reduce 10%
08/11/10 21:23:23 INFO mapred.JobClient: map 36% reduce 11%
08/11/10 21:23:24 INFO mapred.JobClient: map 38% reduce 11%
08/11/10 21:23:25 INFO mapred.JobClient: map 40% reduce 11%
08/11/10 21:23:26 INFO mapred.JobClient: map 41% reduce 11%
08/11/10 21:23:30 INFO mapred.JobClient: map 41% reduce 12%
08/11/10 21:23:31 INFO mapred.JobClient: map 43% reduce 12%
08/11/10 21:23:33 INFO mapred.JobClient: map 46% reduce 12%
08/11/10 21:23:35 INFO mapred.JobClient: map 46% reduce 13%
08/11/10 21:23:39 INFO mapred.JobClient: map 48% reduce 13%
08/11/10 21:23:40 INFO mapred.JobClient: map 50% reduce 13%
08/11/10 21:23:41 INFO mapred.JobClient: map 51% reduce 13%
08/11/10 21:23:43 INFO mapred.JobClient: map 51% reduce 14%
08/11/10 21:23:44 INFO mapred.JobClient: map 51% reduce 15%
08/11/10 21:23:46 INFO mapred.JobClient: map 53% reduce 15%
08/11/10 21:23:48 INFO mapred.JobClient: map 56% reduce 15%
08/11/10 21:23:53 INFO mapred.JobClient: map 56% reduce 16%
08/11/10 21:23:54 INFO mapred.JobClient: map 58% reduce 16%
08/11/10 21:23:55 INFO mapred.JobClient: map 60% reduce 17%
08/11/10 21:23:56 INFO mapred.JobClient: map 61% reduce 18%
08/11/10 21:24:01 INFO mapred.JobClient: map 63% reduce 18%
08/11/10 21:24:03 INFO mapred.JobClient: map 66% reduce 20%
08/11/10 21:24:09 INFO mapred.JobClient: map 68% reduce 20%
08/11/10 21:24:10 INFO mapred.JobClient: map 69% reduce 20%
08/11/10 21:24:11 INFO mapred.JobClient: map 71% reduce 20%
08/11/10 21:24:16 INFO mapred.JobClient: map 73% reduce 20%
08/11/10 21:24:17 INFO mapred.JobClient: map 74% reduce 20%
08/11/10 21:24:18 INFO mapred.JobClient: map 76% reduce 20%
08/11/10 21:24:19 INFO mapred.JobClient: map 76% reduce 22%
08/11/10 21:24:21 INFO mapred.JobClient: map 76% reduce 23%
08/11/10 21:24:23 INFO mapred.JobClient: map 76% reduce 24%
08/11/10 21:24:24 INFO mapred.JobClient: map 79% reduce 24%
08/11/10 21:24:26 INFO mapred.JobClient: map 81% reduce 25%
08/11/10 21:24:31 INFO mapred.JobClient: map 83% reduce 25%
08/11/10 21:24:32 INFO mapred.JobClient: map 84% reduce 25%
08/11/10 21:24:33 INFO mapred.JobClient: map 86% reduce 25%
08/11/10 21:24:34 INFO mapred.JobClient: map 86% reduce 26%
08/11/10 21:24:37 INFO mapred.JobClient: map 86% reduce 27%
08/11/10 21:24:38 INFO mapred.JobClient: map 88% reduce 28%
08/11/10 21:24:39 INFO mapred.JobClient: map 89% reduce 28%
08/11/10 21:24:41 INFO mapred.JobClient: map 91% reduce 28%
08/11/10 21:24:45 INFO mapred.JobClient: map 93% reduce 28%
08/11/10 21:24:47 INFO mapred.JobClient: map 94% reduce 29%
08/11/10 21:24:49 INFO mapred.JobClient: map 96% reduce 29%
08/11/10 21:24:54 INFO mapred.JobClient: map 96% reduce 31%
08/11/10 21:24:58 INFO mapred.JobClient: map 98% reduce 31%
08/11/10 21:24:59 INFO mapred.JobClient: map 98% reduce 32%
08/11/10 21:25:49 INFO mapred.JobClient: map 100% reduce 32%
08/11/10 21:25:57 INFO mapred.JobClient: map 100% reduce 66%
08/11/10 21:26:01 INFO mapred.JobClient: Job complete: job_200811101756_0003
08/11/10 21:26:01 INFO mapred.JobClient: Counters: 13
08/11/10 21:26:01 INFO mapred.JobClient: File Systems
08/11/10 21:26:01 INFO mapred.JobClient: Local bytes read=8251746
08/11/10 21:26:01 INFO mapred.JobClient: Local bytes written=16507848
08/11/10 21:26:01 INFO mapred.JobClient: Job Counters
08/11/10 21:26:01 INFO mapred.JobClient: Launched reduce tasks=3
08/11/10 21:26:01 INFO mapred.JobClient: Launched map tasks=62
08/11/10 21:26:01 INFO mapred.JobClient: Map-Reduce Framework
08/11/10 21:26:01 INFO mapred.JobClient: Reduce input groups=300
08/11/10 21:26:01 INFO mapred.JobClient: Combine output records=0
08/11/10 21:26:01 INFO mapred.JobClient: Map input records=835
08/11/10 21:26:01 INFO mapred.JobClient: Reduce output records=300
08/11/10 21:26:01 INFO mapred.JobClient: Map output bytes=8184414
08/11/10 21:26:01 INFO mapred.JobClient: Map input bytes=0
08/11/10 21:26:01 INFO mapred.JobClient: Combine input records=0
08/11/10 21:26:01 INFO mapred.JobClient: Map output records=835
08/11/10 21:26:01 INFO mapred.JobClient: Reduce input records=300
was (Author: udanax):
Hmm...
After patch apply, Performance was improved on same condition.
I submit to hudson.
----
[d8g053:/root/hama-trunk]# bin/hama examples mult -m 10 -r 1 200 200
....
08/11/10 21:21:21 INFO mapred.JobClient: map 0% reduce 0%
08/11/10 21:21:26 INFO mapred.JobClient: map 1% reduce 0%
08/11/10 21:22:18 INFO mapred.JobClient: map 5% reduce 0%
08/11/10 21:22:19 INFO mapred.JobClient: map 6% reduce 0%
08/11/10 21:22:25 INFO mapred.JobClient: map 8% reduce 0%
08/11/10 21:22:27 INFO mapred.JobClient: map 10% reduce 0%
08/11/10 21:22:28 INFO mapred.JobClient: map 10% reduce 1%
08/11/10 21:22:33 INFO mapred.JobClient: map 11% reduce 2%
08/11/10 21:22:37 INFO mapred.JobClient: map 13% reduce 2%
08/11/10 21:22:40 INFO mapred.JobClient: map 15% reduce 3%
08/11/10 21:22:43 INFO mapred.JobClient: map 15% reduce 4%
08/11/10 21:22:45 INFO mapred.JobClient: map 16% reduce 4%
08/11/10 21:22:47 INFO mapred.JobClient: map 18% reduce 4%
08/11/10 21:22:50 INFO mapred.JobClient: map 18% reduce 5%
08/11/10 21:22:53 INFO mapred.JobClient: map 20% reduce 5%
08/11/10 21:22:54 INFO mapred.JobClient: map 21% reduce 5%
08/11/10 21:23:01 INFO mapred.JobClient: map 25% reduce 5%
08/11/10 21:23:06 INFO mapred.JobClient: map 25% reduce 6%
08/11/10 21:23:08 INFO mapred.JobClient: map 26% reduce 7%
08/11/10 21:23:09 INFO mapred.JobClient: map 28% reduce 7%
08/11/10 21:23:10 INFO mapred.JobClient: map 31% reduce 7%
08/11/10 21:23:17 INFO mapred.JobClient: map 35% reduce 8%
08/11/10 21:23:18 INFO mapred.JobClient: map 36% reduce 8%
08/11/10 21:23:22 INFO mapred.JobClient: map 36% reduce 10%
08/11/10 21:23:23 INFO mapred.JobClient: map 36% reduce 11%
08/11/10 21:23:24 INFO mapred.JobClient: map 38% reduce 11%
08/11/10 21:23:25 INFO mapred.JobClient: map 40% reduce 11%
08/11/10 21:23:26 INFO mapred.JobClient: map 41% reduce 11%
08/11/10 21:23:30 INFO mapred.JobClient: map 41% reduce 12%
08/11/10 21:23:31 INFO mapred.JobClient: map 43% reduce 12%
08/11/10 21:23:33 INFO mapred.JobClient: map 46% reduce 12%
08/11/10 21:23:35 INFO mapred.JobClient: map 46% reduce 13%
08/11/10 21:23:39 INFO mapred.JobClient: map 48% reduce 13%
08/11/10 21:23:40 INFO mapred.JobClient: map 50% reduce 13%
08/11/10 21:23:41 INFO mapred.JobClient: map 51% reduce 13%
08/11/10 21:23:43 INFO mapred.JobClient: map 51% reduce 14%
08/11/10 21:23:44 INFO mapred.JobClient: map 51% reduce 15%
08/11/10 21:23:46 INFO mapred.JobClient: map 53% reduce 15%
08/11/10 21:23:48 INFO mapred.JobClient: map 56% reduce 15%
08/11/10 21:23:53 INFO mapred.JobClient: map 56% reduce 16%
08/11/10 21:23:54 INFO mapred.JobClient: map 58% reduce 16%
08/11/10 21:23:55 INFO mapred.JobClient: map 60% reduce 17%
08/11/10 21:23:56 INFO mapred.JobClient: map 61% reduce 18%
08/11/10 21:24:01 INFO mapred.JobClient: map 63% reduce 18%
08/11/10 21:24:03 INFO mapred.JobClient: map 66% reduce 20%
08/11/10 21:24:09 INFO mapred.JobClient: map 68% reduce 20%
08/11/10 21:24:10 INFO mapred.JobClient: map 69% reduce 20%
08/11/10 21:24:11 INFO mapred.JobClient: map 71% reduce 20%
08/11/10 21:24:16 INFO mapred.JobClient: map 73% reduce 20%
08/11/10 21:24:17 INFO mapred.JobClient: map 74% reduce 20%
08/11/10 21:24:18 INFO mapred.JobClient: map 76% reduce 20%
08/11/10 21:24:19 INFO mapred.JobClient: map 76% reduce 22%
08/11/10 21:24:21 INFO mapred.JobClient: map 76% reduce 23%
08/11/10 21:24:23 INFO mapred.JobClient: map 76% reduce 24%
08/11/10 21:24:24 INFO mapred.JobClient: map 79% reduce 24%
08/11/10 21:24:26 INFO mapred.JobClient: map 81% reduce 25%
08/11/10 21:24:31 INFO mapred.JobClient: map 83% reduce 25%
08/11/10 21:24:32 INFO mapred.JobClient: map 84% reduce 25%
08/11/10 21:24:33 INFO mapred.JobClient: map 86% reduce 25%
08/11/10 21:24:34 INFO mapred.JobClient: map 86% reduce 26%
08/11/10 21:24:37 INFO mapred.JobClient: map 86% reduce 27%
08/11/10 21:24:38 INFO mapred.JobClient: map 88% reduce 28%
08/11/10 21:24:39 INFO mapred.JobClient: map 89% reduce 28%
08/11/10 21:24:41 INFO mapred.JobClient: map 91% reduce 28%
08/11/10 21:24:45 INFO mapred.JobClient: map 93% reduce 28%
08/11/10 21:24:47 INFO mapred.JobClient: map 94% reduce 29%
08/11/10 21:24:49 INFO mapred.JobClient: map 96% reduce 29%
08/11/10 21:24:54 INFO mapred.JobClient: map 96% reduce 31%
08/11/10 21:24:58 INFO mapred.JobClient: map 98% reduce 31%
08/11/10 21:24:59 INFO mapred.JobClient: map 98% reduce 32%
08/11/10 21:25:49 INFO mapred.JobClient: map 100% reduce 32%
08/11/10 21:25:57 INFO mapred.JobClient: map 100% reduce 66%
08/11/10 21:26:01 INFO mapred.JobClient: Job complete: job_200811101756_0003
08/11/10 21:26:01 INFO mapred.JobClient: Counters: 13
08/11/10 21:26:01 INFO mapred.JobClient: File Systems
08/11/10 21:26:01 INFO mapred.JobClient: Local bytes read=8251746
08/11/10 21:26:01 INFO mapred.JobClient: Local bytes written=16507848
08/11/10 21:26:01 INFO mapred.JobClient: Job Counters
08/11/10 21:26:01 INFO mapred.JobClient: Launched reduce tasks=3
08/11/10 21:26:01 INFO mapred.JobClient: Launched map tasks=62
08/11/10 21:26:01 INFO mapred.JobClient: Map-Reduce Framework
08/11/10 21:26:01 INFO mapred.JobClient: Reduce input groups=300
08/11/10 21:26:01 INFO mapred.JobClient: Combine output records=0
08/11/10 21:26:01 INFO mapred.JobClient: Map input records=835
08/11/10 21:26:01 INFO mapred.JobClient: Reduce output records=300
08/11/10 21:26:01 INFO mapred.JobClient: Map output bytes=8184414
08/11/10 21:26:01 INFO mapred.JobClient: Map input bytes=0
08/11/10 21:26:01 INFO mapred.JobClient: Combine input records=0
08/11/10 21:26:01 INFO mapred.JobClient: Map output records=835
08/11/10 21:26:01 INFO mapred.JobClient: Reduce input records=300
> Reduce an rows of intermediate
> ------------------------------
>
> Key: HAMA-103
> URL: https://issues.apache.org/jira/browse/HAMA-103
> Project: Hama
> Issue Type: Improvement
> Reporter: Edward J. Yoon
> Assignee: Edward J. Yoon
> Fix For: 0.1.0
>
> Attachments: HAMA-103.patch
>
>
> Current multiplication map emit the n^2 rows to intermediate files. So, It
> have a lot of overhead to read/write them.
> Summation also can be moved to map phase.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
