Hi Edward, I only tested pagerank. Another problem is “OutOfMem”. The test environment is the same as I mentioned in last mail. For hama, I tested with 0.4M vertex with 100 edges using 20 task, it shows “OutOfMem”. And the local file size is about 260M. But for giraph, I tested with 10M vertex with 10 edges using 19 task, it can run well. And the local file size is about 830M.
So do you encounter the same problem and how to avoid it?? I think the test data is not very large. ---------------------------------------------------------------------------------------------------------------------------------------------------------- Wenhua XU Research Dept. of Software Co., Ltd [cid:[email protected]] To enrich life through communication Office: +86-755-28970151 Fax: +86-755-28975077 ---------------------------------------------------------------------------------------------------------------------------------------------------------- This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! ---------------------------------------------------------------------------------------------------------------------------------------------------------- 发件人: Edward J. Yoon [mailto:[email protected]] 发送时间: 2013年12月13日 19:29 收件人: Xuwenhua 抄送: [email protected]; [email protected] 主题: Re: 答复: About hama performance > According to your message, I think this issue shall be high priority, right?? > So which version and when will it be fixed and released?? Yes, it's on my todo list. I plan to fix before releasing version 0.7. BTW, have you compared the performance of SSSP or K-Means? You'll see another result. I'm CC'ing [email protected]<mailto:[email protected]> and [email protected]<mailto:[email protected]> lists. On Fri, Dec 13, 2013 at 7:26 PM, Xuwenhua <[email protected]<mailto:[email protected]>> wrote: Hi Edward, Thank you for your information. You mean I re-send this email in hama user@ list? Or you copy it to list directly?? For I do not subscribe the list now, so if need I re-send, I will send it next week. If you can directly copy it, and you can do it. According to your message, I think this issue shall be high priority, right?? So which version and when will it be fixed and released?? ---------------------------------------------------------------------------------------------------------------------------------------------------------- Wenhua XU Research Dept. of Software Co., Ltd [cid:[email protected]] To enrich life through communication Office: +86-755-28970151<tel:%2B86-755-28970151> Fax: +86-755-28975077<tel:%2B86-755-28975077> ---------------------------------------------------------------------------------------------------------------------------------------------------------- This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! ---------------------------------------------------------------------------------------------------------------------------------------------------------- 发件人: Edward J. Yoon [mailto:[email protected]<mailto:[email protected]>] 发送时间: 2013年12月13日 17:47 收件人: Xuwenhua 主题: Re: About hama performance Hi Xuwenhua, > From the test report, we can see that hama cost twice time. I guess you did correctly. Hama's Graph package is still under heavy development. Currently Hama uses additional supersteps to broadcast aggregators, and sends messages twice (https://issues.apache.org/jira/browse/HAMA-734). It means that the performance will be improved by at least 50%. BTW, can we move this to Hama user@ list? On Fri, Dec 13, 2013 at 5:08 PM, Xuwenhua <[email protected]<mailto:[email protected]>> wrote: Dear Edward, This is Wenhua From Huawei. We are interested in Hama for it BSP feature. And those days I study some hama material and do some performance tests compared with Giraph. I used 3 servers with total core 20 as follows, and 1G for “bsp.child.java.opts” CPU(8 core) * 1, 8G*6 master/name node，slave/data node CPU(8 core）* 1，4G*6 slave/data node CPU(2 core) * 2，2G*4 slave/data node And the input format is like “vertex neighbour1 neighbour2”. I tested from 0.1m vertex with 10 edges to 1m vertex with 10 edges. Giraph using 19 task and Hama using 20 task, so that each core can deal with one task. I test the pagerank, I force it to run 30 iterations, if(this.getSuperstepCount() >= 30) { voteToHalt(); return; } And the test result is as follows. [cid:[email protected]] From the test report, we can see that hama cost twice time. So my query is that is there any problem in my experiments?? And do you have some hama performance information compared with Giraph?? Because I only see some hama’s own performance in hama website. Looking forward to receiving your comments. Best Regards ---------------------------------------------------------------------------------------------------------------------------------------------------------- Wenhua XU Research Dept. of Software Co., Ltd [cid:[email protected]] To enrich life through communication Office: +86-755-28970151<tel:%2B86-755-28970151> Fax: +86-755-28975077<tel:%2B86-755-28975077> ---------------------------------------------------------------------------------------------------------------------------------------------------------- This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! ---------------------------------------------------------------------------------------------------------------------------------------------------------- -- Best Regards, Edward J. Yoon @eddieyoon -- Best Regards, Edward J. Yoon @eddieyoon
