Solved by set HAMA_HEAPSIZE to 4 GB. On Tue, Nov 20, 2012 at 8:37 PM, Edward J. Yoon <[email protected]> wrote: > Default of hdfs block size is 64 MB but these days many people usually > use 128 MB or 256 MB. If we can reduce memory use, we must do. > > On Tue, Nov 20, 2012 at 4:25 PM, Thomas Jungblut > <[email protected]> wrote: >> Try 32m blocks ;-) >> We have to recheck if the pre job partitioner works, I missed checking it. >> Am 20.11.2012 08:18 schrieb "Edward" <[email protected]>: >> >>> Okay. It looks like a memory issue. I'll check tomorrow. >>> >>> Sent from my iPhone >>> >>> On Nov 20, 2012, at 3:55 PM, Thomas Jungblut <[email protected]> >>> wrote: >>> >>> > Like in mr you should steer this number by the block size. You can't >>> > determine the splits correctly without iterating them. >>> > Am 20.11.2012 07:42 schrieb "Thomas Jungblut" <[email protected] >>> >: >>> > >>> >> The splits are never deterministic when we are not splitting across >>> block >>> >> boundaries. >>> >> Am 20.11.2012 07:39 schrieb "Edward J. Yoon" <[email protected]>: >>> >> >>> >>> changed to use just block nums? >>> >>> >>> >>> Sadly graph examples on 1 ~ 2 GB random graph are not works well. My >>> >>> settings are: >>> >>> >>> >>> HDFS block size: 256 mb >>> >>> bsp.tasks.maximum: 3 >>> >>> bsp.child.java.opts: -Xmx15524m >>> >>> >>> >>> On Tue, Nov 20, 2012 at 2:56 AM, Tommaso Teofili >>> >>> <[email protected]> wrote: >>> >>>> great job, thanks! >>> >>>> Tommaso >>> >>>> >>> >>>> >>> >>>> 2012/11/19 Thomas Jungblut <[email protected]> >>> >>>> >>> >>>>> done >>> >>>>> https://issues.apache.org/jira/browse/HAMA-678 >>> >>>>> >>> >>>>> >>> >>>>> 2012/11/18 Edward J. Yoon <[email protected]> >>> >>>>> >>> >>>>>> We have to fix fileinput splitter. >>> >>>>>> >>> >>>>>> On Thu, Nov 15, 2012 at 4:46 PM, Thomas Jungblut >>> >>>>>> <[email protected]> wrote: >>> >>>>>>> +1. >>> >>>>>>> >>> >>>>>>> 2012/11/15 Tommaso Teofili <[email protected]> >>> >>>>>>> >>> >>>>>>>> +1 >>> >>>>>>>> >>> >>>>>>>> Tommaso >>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> 2012/11/15 Edward J. Yoon <[email protected]> >>> >>>>>>>> >>> >>>>>>>>> Hi all, >>> >>>>>>>>> >>> >>>>>>>>> Here's new RC for Hama 0.6. >>> >>>>>>>>> >>> >>>>>>>>> Artifacts: >>> >>>>>>>>> http://people.apache.org/~edwardyoon/dist/0.6-RC5/ >>> >>>>>>>>> >>> >>>>>>>>> SVN Tag: >>> >>>>>>>>> http://svn.apache.org/repos/asf/hama/tags/0.6-RC5/ >>> >>>>>>>>> >>> >>>>>>>>> Please vote for this release. >>> >>>>>>>>> >>> >>>>>>>>> I'm +1. >>> >>>>>>>>> >>> >>>>>>>>> -- >>> >>>>>>>>> Best Regards, Edward J. Yoon >>> >>>>>>>>> @eddieyoon >>> >>>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> -- >>> >>>>>> Best Regards, Edward J. Yoon >>> >>>>>> @eddieyoon >>> >>> >>> >>> >>> >>> >>> >>> -- >>> >>> Best Regards, Edward J. Yoon >>> >>> @eddieyoon >>> >> >>> > > > > -- > Best Regards, Edward J. Yoon > @eddieyoon
-- Best Regards, Edward J. Yoon @eddieyoon
