oh, the HDFS block size is 128Mb, not 64Mb, so the 73Mb graph will not be split-ed on the HDFS.
2012/9/14 庄克琛 <[email protected]> > em... I have try your configure advise and restart the hama. > I use the Google web graph( > http://wiki.apache.org/hama/WriteHamaGraphFile ), > Nodes: 875713 Edges: 5105039, which is about 73Mb, upload to a small HDFS > cluster(block size is 64Mb), test the PageRank in ( > http://wiki.apache.org/hama/WriteHamaGraphFile ), got the result as: > ################ > function@624-PC:~/hadoop-1.0.3/hama-0.6.0$ hama jar hama-6-P* > input-google ouput-google > 12/09/14 14:27:50 INFO bsp.FileInputFormat: Total input paths to process : > 1 > 12/09/14 14:27:50 INFO bsp.FileInputFormat: Total # of splits: 3 > 12/09/14 14:27:50 INFO bsp.BSPJobClient: Running job: job_201008141420_0004 > 12/09/14 14:27:53 INFO bsp.BSPJobClient: Current supersteps number: 0 > Java HotSpot(TM) Server VM warning: Attempt to allocate stack guard pages > failed. > ################### > > Last time the supersteps could be 1 or 2, then the same result. > the task attempt****.err files are empty. > Is the graph too large? > I test on a small graph, get the right Rank results > > > 2012/9/14 Edward J. Yoon <[email protected]> > > I've added multi-step partitioning method to save memory[1]. >> >> Please try to configure below property to hama-site.xml. >> >> <property> >> <name>hama.graph.multi.step.partitioning.interval</name> >> <value>10000000</value> >> </property> >> >> 1. https://issues.apache.org/jira/browse/HAMA-599 >> >> On Fri, Sep 14, 2012 at 3:13 PM, 庄克琛 <[email protected]> wrote: >> > HI, Actually I use this ( >> > >> https://builds.apache.org/job/Hama-Nightly/672/artifact/.repository/org/apache/hama/hama-dist/0.6.0-SNAPSHOT/ >> > ) >> > to test again, I mean use this 0.6.0SNAPSHOT version replace everything, >> > got the same out of memory results. I just don't know what cause the >> out of >> > memory fails, only some small graph computing can be finished. Is this >> > version finished the " >> > [HAMA-596<https://issues.apache.org/jira/browse/HAMA-596>]:Optimize >> > memory usage of graph job" ? >> > Thanks >> > >> > 2012/9/14 Thomas Jungblut <[email protected]> >> > >> >> Hey, what jar did you exactly replace? >> >> Am 14.09.2012 07:49 schrieb "庄克琛" <[email protected]>: >> >> >> >> > hi, every one: >> >> > I use the hama-0.5.0 with the hadoop-1.0.3, try to do some large >> graphs >> >> > analysis. >> >> > When I test the PageRank examples, as the ( >> >> > http://wiki.apache.org/hama/WriteHamaGraphFile) shows, I download >> the >> >> > graph >> >> > data, and run the PageRank job on a small distributed cluser, I can >> only >> >> > get the out of memory failed, with Superstep 0,1,2 works well, then >> get >> >> the >> >> > memory out fail.(Each computer have 2G memory) But when I test some >> small >> >> > graph, everything went well. >> >> > Also I try the trunk version( >> >> > https://builds.apache.org/job/Hama-Nightly/672/changes#detail3), >> replace >> >> > my >> >> > hama-0.5.0 with the hama-0.6.0-snapshot, only get the same results. >> >> > Anyone got better ideas? >> >> > >> >> > Thanks! >> >> > >> >> > -- >> >> > >> >> > *Zhuang Kechen >> >> > * >> >> > >> >> >> > >> > >> > >> > -- >> > >> > *Zhuang Kechen* >> > >> > School of Computer Science & Technology >> > >> > ** >> > Nanjing University of Science & Technology >> > >> > Lab.623, School of Computer Sci. & Tech. >> > >> > No.200, Xiaolingwei Street >> > >> > Nanjing, Jiangsu, 210094 >> > >> > P.R. China >> > >> > Tel: 025-84315982** >> > >> > Email: [email protected] >> >> >> >> -- >> Best Regards, Edward J. Yoon >> @eddieyoon >> > > > > -- > > *Zhuang Kechen > * > > > -- *Zhuang Kechen* School of Computer Science & Technology ** Nanjing University of Science & Technology Lab.623, School of Computer Sci. & Tech. No.200, Xiaolingwei Street Nanjing, Jiangsu, 210094 P.R. China Tel: 025-84315982** Email: [email protected]
