> As mentioned above, I'd like to show only some example of how you implement your own matrix/graph applications using BSP.
I'm not focusing on which one is better between BSP and M/R. I misunderstood your intention due to the above your mention. It is because we will need not only the BSP framework but also other programs to help your to develop matrix algebra/graph processing. Best regards, -- Hyunsik Choi Database & Information Systems Group, Korea Univ. http://diveintodata.org On Tue, May 11, 2010 at 11:19 AM, Edward J. Yoon <[email protected]>wrote: > In this mailing > ( > http://markmail.org/search/?q=hama-dev+discuss#query:hama-dev%20discuss+page:1+mid:amlvccbptom3yro3+state:results > ) > and others, we discussed about: > > 1) We can't implement all of matrix algebra. 2) Even if we do that, > the code complexity based on M/R model is too high. As we learned, the > M/R model isn't fit well for the complex numerical/relational data > processing required for many communications between each others. 3) > Admittedly, BSP is flexible than M/R. > > Therefore, We've aligned our goal: to provide easy *matrix/graph > computing* programming environment on the distributed system (hadoop), > such as M/R framework. (NOT library) So, we won't develop only the BSP > framework if there is other solutions. > > On Tue, May 11, 2010 at 10:29 AM, Hyunsik Choi <[email protected]> > wrote: > > According to you, the main objective of hama appears to be developing > only > > the BSP framework. Is it right? > > > > -- > > Hyunsik Choi > > Database & Information Systems Group, Korea Univ. > > http://diveintodata.org > > > > > > On Mon, May 10, 2010 at 8:34 PM, Edward J. Yoon <[email protected] > >wrote: > > > >> > In the distant future, the graph package would use both MR and BSP. > But, > >> > >> Where will M/R use for? > >> > >> I'd like to cut M/R out of HAMA project and leave only the pure BSP & > >> Graph computing framework as simple as possible. Then, the rest are > >> user's matrix/graph applications. > >> > >> And, I'd like to consider about building an contributor/developer > >> ECOsystem. > >> > >> > now i'll try to develop the graph package with only BSP. How about the > >> > matrix package? > >> > >> As mentioned above, I'd like to show only some example of how you > >> implement your own matrix/graph applications using BSP. > >> > >> I think the block of hadoop families should be added on diagram only > >> to help understand relations between hadoop and hama, > >> > >> On Mon, May 10, 2010 at 7:40 PM, Hyunsik Choi <[email protected]> > >> wrote: > >> > In the distant future, the graph package would use both MR and BSP. > But, > >> > now i'll try to develop the graph package with only BSP. How about the > >> > matrix package? > >> > > >> > -- > >> > Hyunsik Choi > >> > Database & Information Systems Group, Korea Univ. > >> > http://diveintodata.org > >> > > >> > > >> > On Mon, May 10, 2010 at 7:17 PM, Edward J. Yoon < > [email protected] > >> >wrote: > >> > > >> >> > zookeeper should be enclosed in BSP since zookeeper is not > disclosed > >> to > >> >> > external. It is only used inside the BSP. > >> >> > >> >> Right you are. And also, the top of M/R and BSP should be clarified, > >> >> but currently something is odd overall. > >> >> > >> >> On Mon, May 10, 2010 at 6:24 PM, Hyunsik Choi < > [email protected]> > >> >> wrote: > >> >> > Good job! But, the matrix package is omitted. Besides, I think that > >> >> > zookeeper should be enclosed in BSP since zookeeper is not > disclosed > >> to > >> >> > external. It is only used inside the BSP. > >> >> > > >> >> > Best regards, > >> >> > -- > >> >> > Hyunsik Choi > >> >> > Database & Information Systems Group, Korea Univ. > >> >> > http://diveintodata.org > >> >> > > >> >> > > >> >> > On Mon, May 10, 2010 at 3:01 PM, Apache Wiki <[email protected] > > > >> >> wrote: > >> >> > > >> >> >> Dear Wiki user, > >> >> >> > >> >> >> You have subscribed to a wiki page or wiki category on "Hama Wiki" > >> for > >> >> >> change notification. > >> >> >> > >> >> >> The "Architecture" page has been changed by Edward J. Yoon. > >> >> >> > http://wiki.apache.org/hama/Architecture?action=diff&rev1=93&rev2=94 > >> >> >> > >> >> >> -------------------------------------------------- > >> >> >> > >> >> >> > >> >> >> This effort is still a "work in progress". Please feel free to > add > >> >> >> comments. > >> >> >> ---- > >> >> >> + > >> >> >> + == The overall architecture of HAMA == > >> >> >> + {{{ > >> >> >> + +-----------------------------------------------+ > >> >> >> + | HAMA APIs | > >> >> >> + +-----------------------------------------------+ > >> >> >> + -------------------------------------------------------- > >> >> >> + +---------------------------+ > >> >> >> + | Graph Computing Framework | > >> >> >> + +---------------------------+ > Computing > >> >> Engine > >> >> >> + +-----------+ +---------------------------------+ > >> >> >> + | MapReduce | | BSP (Bulk Synchronous Parallel) | > >> >> >> + +-----------+ +---------------------------------+ > >> >> >> + -------------------------------------------------------- > >> >> >> + +-----------------------------------------------+ > >> >> >> + | Zookeeper | > >> Distributed > >> >> >> Locking Service > >> >> >> + +-----------------------------------------------+ > >> >> >> + -------------------------------------------------------- > >> >> >> + +-----------------------------------------------+ > >> >> >> + | | > >> >> >> + | +-----------------------------+ | > >> >> >> + | | HBase : Distributed Storage | | > >> >> >> + | +-----------------------------+ | > >> Distributed > >> >> >> Storage Systems > >> >> >> + | +---------------------------------+ | > >> >> >> + | | HDFS : Distributed File System | | > >> >> >> + | +---------------------------------+ | > >> >> >> + +-----------------------------------------------+ > >> >> >> + }}} > >> >> >> + > >> >> >> == BSP == > >> >> >> > >> >> >> The BSP package is a implementation of BSP over Hadoop > RPC(sockets). > >> >> >> > >> >> > > >> >> > >> >> > >> >> > >> >> -- > >> >> Best Regards, Edward J. Yoon > >> >> [email protected] > >> >> http://blog.udanax.org > >> >> > >> > > >> > >> > >> > >> -- > >> Best Regards, Edward J. Yoon > >> [email protected] > >> http://blog.udanax.org > >> > > > > > > -- > Best Regards, Edward J. Yoon > [email protected] > http://blog.udanax.org >
