Great :) I made a tutorial back then: http://codingwiththomas.blogspot.de/2011/04/profiling-apache-hama-bsps.html
2012/8/30 Edward J. Yoon <[email protected]> > Yes, I'll try it. > > On Thu, Aug 30, 2012 at 10:38 PM, Thomas Jungblut > <[email protected]> wrote: > > Still has some overhead, the wiki dataset I benched were just 1gb. > > Wonder where our hotspots are, did you get your yourkit license to > profile? > > > > 2012/8/30 Edward J. Yoon <[email protected]> > > > >> My settings and envs were like this: > >> > >> * child.java.opts = -Xmx1024m > >> * tasks.num per node = 5 > >> * total tasks = 85 > >> * network env = infiniband 40G > >> * physical nodes = 17 > >> * hadoop-cdh-0.20 > >> * zookeeper-cdh > >> > >> Finished in 127 secs. > >> > >> On Thu, Aug 30, 2012 at 10:27 PM, Thomas Jungblut > >> <[email protected]> wrote: > >> > Yeah, lots of ram ;)) > >> > > >> > 2012/8/30 Edward J. Yoon <[email protected]> > >> > > >> >> > It runs locally in under 120s on my computer > >> >> > >> >> Haha, single machine? > >> >> > >> >> On Thu, Aug 30, 2012 at 10:25 PM, Thomas Jungblut > >> >> <[email protected]> wrote: > >> >> > It runs locally in under 120s on my computer so I assume that this > is > >> >> > correct. > >> >> > > >> >> > 2012/8/30 Edward J. Yoon <[email protected]> > >> >> > > >> >> >> Just two minutes were needed to process wikipedia dataset. Is this > >> >> normal? > >> >> >> > >> >> >> On Tue, Jul 10, 2012 at 8:17 PM, Thomas Jungblut > >> >> >> <[email protected]> wrote: > >> >> >> > http://snap.stanford.edu/data/ > >> >> >> > > >> >> >> > 2012/7/10 Edward J. Yoon <[email protected]> > >> >> >> > > >> >> >> >> Just wondering, do you know more large and downloadable data > set? > >> >> >> >> > >> >> >> >> On Tue, Jul 10, 2012 at 8:01 PM, Edward J. Yoon < > >> >> [email protected]> > >> >> >> >> wrote: > >> >> >> >> > Oh... thanks. now I can guess what it is. > >> >> >> >> > > >> >> >> >> > On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut > >> >> >> >> > <[email protected]> wrote: > >> >> >> >> >> Yes, the example graph is not complete thus resulting in > NPEs. > >> >> >> >> >> > >> >> >> >> >> 2012/7/10 Edward J. Yoon <[email protected]> > >> >> >> >> >> > >> >> >> >> >>> Nope, should I set to true? > >> >> >> >> >>> > >> >> >> >> >>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut > >> >> >> >> >>> <[email protected]> wrote: > >> >> >> >> >>> > Set "hama.graph.repair" to true in conf? > >> >> >> >> >>> > > >> >> >> >> >>> > 2012/7/10 Edward J. Yoon <[email protected]> > >> >> >> >> >>> > > >> >> >> >> >>> >> I installed Hama 0.5 on 4 racks (4 x 18 nodes) w/ 3 > tasks > >> per > >> >> >> each > >> >> >> >> >>> >> node (bsp.child.java.opts = Xmx3512m), tried to run > >> PageRank > >> >> on > >> >> >> >> >>> >> Wikipedia dataset introduced at > >> >> >> >> >>> >> http://wiki.apache.org/hama/WriteHamaGraphFile > >> >> >> >> >>> >> > >> >> >> >> >>> >> Doesn't work, the cause of job fail error is different > >> every > >> >> >> time. > >> >> >> >> >>> >> NullPointerException at loadVertices(), ERROR > bsp.BSPTask: > >> >> >> Started > >> >> >> >> >>> >> pinging to groom, ..., etc. > >> >> >> >> >>> >> > >> >> >> >> >>> >> Has anyone tried this? > >> >> >> >> >>> >> > >> >> >> >> >>> >> -- > >> >> >> >> >>> >> Best Regards, Edward J. Yoon > >> >> >> >> >>> >> @eddieyoon > >> >> >> >> >>> >> > >> >> >> >> >>> > >> >> >> >> >>> > >> >> >> >> >>> > >> >> >> >> >>> -- > >> >> >> >> >>> Best Regards, Edward J. Yoon > >> >> >> >> >>> @eddieyoon > >> >> >> >> >>> > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > >> >> >> >> > -- > >> >> >> >> > Best Regards, Edward J. Yoon > >> >> >> >> > @eddieyoon > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> -- > >> >> >> >> Best Regards, Edward J. Yoon > >> >> >> >> @eddieyoon > >> >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> -- > >> >> >> Best Regards, Edward J. Yoon > >> >> >> @eddieyoon > >> >> >> > >> >> > >> >> > >> >> > >> >> -- > >> >> Best Regards, Edward J. Yoon > >> >> @eddieyoon > >> >> > >> > >> > >> > >> -- > >> Best Regards, Edward J. Yoon > >> @eddieyoon > >> > > > > -- > Best Regards, Edward J. Yoon > @eddieyoon >
