I correct typos as follows: I'm not focusing on which one is better between BSP and M/R. I misunderstood your intention due to the above your mention. It is because we will need not only the BSP framework but also other programs to help *users* to develop matrix algebra/graph processing *applications*.
-- Hyunsik Choi Database & Information Systems Group, Korea Univ. http://diveintodata.org On Tue, May 11, 2010 at 12:55 PM, Hyunsik Choi <[email protected]>wrote: > > As mentioned above, I'd like to show only some example of how you > implement your own matrix/graph applications using BSP. > > I'm not focusing on which one is better between BSP and M/R. I > misunderstood your intention due to the above your mention. It is because we > will need not only the BSP framework but also other programs to help your to > develop matrix algebra/graph processing. > > Best regards, > -- > Hyunsik Choi > Database & Information Systems Group, Korea Univ. > http://diveintodata.org > > > On Tue, May 11, 2010 at 11:19 AM, Edward J. Yoon <[email protected]>wrote: > >> In this mailing >> ( >> http://markmail.org/search/?q=hama-dev+discuss#query:hama-dev%20discuss+page:1+mid:amlvccbptom3yro3+state:results >> ) >> and others, we discussed about: >> >> 1) We can't implement all of matrix algebra. 2) Even if we do that, >> the code complexity based on M/R model is too high. As we learned, the >> M/R model isn't fit well for the complex numerical/relational data >> processing required for many communications between each others. 3) >> Admittedly, BSP is flexible than M/R. >> >> Therefore, We've aligned our goal: to provide easy *matrix/graph >> computing* programming environment on the distributed system (hadoop), >> such as M/R framework. (NOT library) So, we won't develop only the BSP >> framework if there is other solutions. >> >> On Tue, May 11, 2010 at 10:29 AM, Hyunsik Choi <[email protected]> >> wrote: >> > According to you, the main objective of hama appears to be developing >> only >> > the BSP framework. Is it right? >> > >> > -- >> > Hyunsik Choi >> > Database & Information Systems Group, Korea Univ. >> > http://diveintodata.org >> > >> > >> > On Mon, May 10, 2010 at 8:34 PM, Edward J. Yoon <[email protected] >> >wrote: >> > >> >> > In the distant future, the graph package would use both MR and BSP. >> But, >> >> >> >> Where will M/R use for? >> >> >> >> I'd like to cut M/R out of HAMA project and leave only the pure BSP & >> >> Graph computing framework as simple as possible. Then, the rest are >> >> user's matrix/graph applications. >> >> >> >> And, I'd like to consider about building an contributor/developer >> >> ECOsystem. >> >> >> >> > now i'll try to develop the graph package with only BSP. How about >> the >> >> > matrix package? >> >> >> >> As mentioned above, I'd like to show only some example of how you >> >> implement your own matrix/graph applications using BSP. >> >> >> >> I think the block of hadoop families should be added on diagram only >> >> to help understand relations between hadoop and hama, >> >> >> >> On Mon, May 10, 2010 at 7:40 PM, Hyunsik Choi <[email protected]> >> >> wrote: >> >> > In the distant future, the graph package would use both MR and BSP. >> But, >> >> > now i'll try to develop the graph package with only BSP. How about >> the >> >> > matrix package? >> >> > >> >> > -- >> >> > Hyunsik Choi >> >> > Database & Information Systems Group, Korea Univ. >> >> > http://diveintodata.org >> >> > >> >> > >> >> > On Mon, May 10, 2010 at 7:17 PM, Edward J. Yoon < >> [email protected] >> >> >wrote: >> >> > >> >> >> > zookeeper should be enclosed in BSP since zookeeper is not >> disclosed >> >> to >> >> >> > external. It is only used inside the BSP. >> >> >> >> >> >> Right you are. And also, the top of M/R and BSP should be clarified, >> >> >> but currently something is odd overall. >> >> >> >> >> >> On Mon, May 10, 2010 at 6:24 PM, Hyunsik Choi < >> [email protected]> >> >> >> wrote: >> >> >> > Good job! But, the matrix package is omitted. Besides, I think >> that >> >> >> > zookeeper should be enclosed in BSP since zookeeper is not >> disclosed >> >> to >> >> >> > external. It is only used inside the BSP. >> >> >> > >> >> >> > Best regards, >> >> >> > -- >> >> >> > Hyunsik Choi >> >> >> > Database & Information Systems Group, Korea Univ. >> >> >> > http://diveintodata.org >> >> >> > >> >> >> > >> >> >> > On Mon, May 10, 2010 at 3:01 PM, Apache Wiki < >> [email protected]> >> >> >> wrote: >> >> >> > >> >> >> >> Dear Wiki user, >> >> >> >> >> >> >> >> You have subscribed to a wiki page or wiki category on "Hama >> Wiki" >> >> for >> >> >> >> change notification. >> >> >> >> >> >> >> >> The "Architecture" page has been changed by Edward J. Yoon. >> >> >> >> >> http://wiki.apache.org/hama/Architecture?action=diff&rev1=93&rev2=94 >> >> >> >> >> >> >> >> -------------------------------------------------- >> >> >> >> >> >> >> >> >> >> >> >> This effort is still a "work in progress". Please feel free to >> add >> >> >> >> comments. >> >> >> >> ---- >> >> >> >> + >> >> >> >> + == The overall architecture of HAMA == >> >> >> >> + {{{ >> >> >> >> + +-----------------------------------------------+ >> >> >> >> + | HAMA APIs | >> >> >> >> + +-----------------------------------------------+ >> >> >> >> + -------------------------------------------------------- >> >> >> >> + +---------------------------+ >> >> >> >> + | Graph Computing Framework | >> >> >> >> + +---------------------------+ >> Computing >> >> >> Engine >> >> >> >> + +-----------+ +---------------------------------+ >> >> >> >> + | MapReduce | | BSP (Bulk Synchronous Parallel) | >> >> >> >> + +-----------+ +---------------------------------+ >> >> >> >> + -------------------------------------------------------- >> >> >> >> + +-----------------------------------------------+ >> >> >> >> + | Zookeeper | >> >> Distributed >> >> >> >> Locking Service >> >> >> >> + +-----------------------------------------------+ >> >> >> >> + -------------------------------------------------------- >> >> >> >> + +-----------------------------------------------+ >> >> >> >> + | | >> >> >> >> + | +-----------------------------+ | >> >> >> >> + | | HBase : Distributed Storage | | >> >> >> >> + | +-----------------------------+ | >> >> Distributed >> >> >> >> Storage Systems >> >> >> >> + | +---------------------------------+ | >> >> >> >> + | | HDFS : Distributed File System | | >> >> >> >> + | +---------------------------------+ | >> >> >> >> + +-----------------------------------------------+ >> >> >> >> + }}} >> >> >> >> + >> >> >> >> == BSP == >> >> >> >> >> >> >> >> The BSP package is a implementation of BSP over Hadoop >> RPC(sockets). >> >> >> >> >> >> >> > >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Best Regards, Edward J. Yoon >> >> >> [email protected] >> >> >> http://blog.udanax.org >> >> >> >> >> > >> >> >> >> >> >> >> >> -- >> >> Best Regards, Edward J. Yoon >> >> [email protected] >> >> http://blog.udanax.org >> >> >> > >> >> >> >> -- >> Best Regards, Edward J. Yoon >> [email protected] >> http://blog.udanax.org >> > >
