> I remember we had some benchmarks on executing some jobs on clusters, maybe > I'm wrong.
I’m ashamed to say that MR-based generator programs (for SSSP, PageRank, KMeans examples) was used to avoid partitioning/memory issues[1]. 1. https://issues.apache.org/jira/browse/HAMA-531 On Thu, Mar 28, 2013 at 7:30 PM, Tommaso Teofili <[email protected]> wrote: > 2013/3/28 Edward J. Yoon <[email protected]> > >> > Can you elaborate some more on this? Why do you think that's needed? Is >> > that related to MRQL? >> >> It's not related to MRQL. The problem is that every Hama jobs, which >> requires partitioning, doesn't work for only few ten gigabytes. As we >> already know, input partitioning is very important for distributed >> matrix or graph computations. >> >> Moreover, we didn't provide any random generator for examples. (I >> guess) newbies' behavior (before considering adopt Hama) pattern is: >> >> 1. Download, a latest Hama release. >> 2. Setup, a Hama Cluster. >> 3. Evaluation, using examples. >> >> Except Pi and Bench examples, they need to study/do a lot of things. >> > > ok great Edward, now it's much clearer :-) > thanks for explaining it further. > > >> >> > I'm not sure I understand sorry :) what do you mean here exactly? >> >> I mean, Hama is a large scale BSP computing platform but didn't tested >> on large cluster yet. >> > > I remember we had some benchmarks on executing some jobs on clusters, maybe > I'm wrong. > > Tommaso > > >> >> On Thu, Mar 28, 2013 at 5:30 PM, Tommaso Teofili >> <[email protected]> wrote: >> > 2013/3/28 Edward J. Yoon <[email protected]> >> > >> >> > Sure, can we ensure the new downstream application MRQL is fine with >> the >> >> > changes in 0.6.1? >> >> >> >> Maybe, at least, I can test large input with this :) >> >> >> >> I can understand your concerns but, a proof of concept (on semi-large >> >> cluster) is needed (ASAP). >> > >> > >> > Can you elaborate some more on this? Why do you think that's needed? Is >> > that related to MRQL? >> > >> > >> >> IMO, there is the difference between >> >> (somewhat) proven status and (work in progress) conceptual status. >> >> >> > >> > I'm not sure I understand sorry :) what do you mean here exactly? >> > >> > Thanks, >> > Tommaso >> > >> > >> >> >> >> On Thu, Mar 28, 2013 at 4:14 PM, Suraj Menon <[email protected]> >> >> wrote: >> >> > Sure, can we ensure the new downstream application MRQL is fine with >> the >> >> > changes in 0.6.1? >> >> > >> >> > -Suraj >> >> > >> >> > On Thu, Mar 28, 2013 at 2:57 AM, Tommaso Teofili >> >> > <[email protected]>wrote: >> >> > >> >> >> sounds good to me >> >> >> Tommaso >> >> >> >> >> >> >> >> >> 2013/3/28 Edward J. Yoon <[email protected]> >> >> >> >> >> >> > Hi all, >> >> >> > >> >> >> > I plan on cutting a release this week for Hama 0.6.1. We fixed >> input >> >> >> > splitter and made few improvements. >> >> >> > >> >> >> > Thanks. >> >> >> > -- >> >> >> > Best Regards, Edward J. Yoon >> >> >> > @eddieyoon >> >> >> > >> >> >> >> >> >> >> >> >> >> >> -- >> >> Best Regards, Edward J. Yoon >> >> @eddieyoon >> >> >> >> >> >> -- >> Best Regards, Edward J. Yoon >> @eddieyoon >> -- Best Regards, Edward J. Yoon @eddieyoon
