In this mailing (http://markmail.org/search/?q=hama-dev+discuss#query:hama-dev%20discuss+page:1+mid:amlvccbptom3yro3+state:results) and others, we discussed about:
1) We can't implement all of matrix algebra. 2) Even if we do that, the code complexity based on M/R model is too high. As we learned, the M/R model isn't fit well for the complex numerical/relational data processing required for many communications between each others. 3) Admittedly, BSP is flexible than M/R. Therefore, We've aligned our goal: to provide easy *matrix/graph computing* programming environment on the distributed system (hadoop), such as M/R framework. (NOT library) So, we won't develop only the BSP framework if there is other solutions. On Tue, May 11, 2010 at 10:29 AM, Hyunsik Choi <[email protected]> wrote: > According to you, the main objective of hama appears to be developing only > the BSP framework. Is it right? > > -- > Hyunsik Choi > Database & Information Systems Group, Korea Univ. > http://diveintodata.org > > > On Mon, May 10, 2010 at 8:34 PM, Edward J. Yoon <[email protected]>wrote: > >> > In the distant future, the graph package would use both MR and BSP. But, >> >> Where will M/R use for? >> >> I'd like to cut M/R out of HAMA project and leave only the pure BSP & >> Graph computing framework as simple as possible. Then, the rest are >> user's matrix/graph applications. >> >> And, I'd like to consider about building an contributor/developer >> ECOsystem. >> >> > now i'll try to develop the graph package with only BSP. How about the >> > matrix package? >> >> As mentioned above, I'd like to show only some example of how you >> implement your own matrix/graph applications using BSP. >> >> I think the block of hadoop families should be added on diagram only >> to help understand relations between hadoop and hama, >> >> On Mon, May 10, 2010 at 7:40 PM, Hyunsik Choi <[email protected]> >> wrote: >> > In the distant future, the graph package would use both MR and BSP. But, >> > now i'll try to develop the graph package with only BSP. How about the >> > matrix package? >> > >> > -- >> > Hyunsik Choi >> > Database & Information Systems Group, Korea Univ. >> > http://diveintodata.org >> > >> > >> > On Mon, May 10, 2010 at 7:17 PM, Edward J. Yoon <[email protected] >> >wrote: >> > >> >> > zookeeper should be enclosed in BSP since zookeeper is not disclosed >> to >> >> > external. It is only used inside the BSP. >> >> >> >> Right you are. And also, the top of M/R and BSP should be clarified, >> >> but currently something is odd overall. >> >> >> >> On Mon, May 10, 2010 at 6:24 PM, Hyunsik Choi <[email protected]> >> >> wrote: >> >> > Good job! But, the matrix package is omitted. Besides, I think that >> >> > zookeeper should be enclosed in BSP since zookeeper is not disclosed >> to >> >> > external. It is only used inside the BSP. >> >> > >> >> > Best regards, >> >> > -- >> >> > Hyunsik Choi >> >> > Database & Information Systems Group, Korea Univ. >> >> > http://diveintodata.org >> >> > >> >> > >> >> > On Mon, May 10, 2010 at 3:01 PM, Apache Wiki <[email protected]> >> >> wrote: >> >> > >> >> >> Dear Wiki user, >> >> >> >> >> >> You have subscribed to a wiki page or wiki category on "Hama Wiki" >> for >> >> >> change notification. >> >> >> >> >> >> The "Architecture" page has been changed by Edward J. Yoon. >> >> >> http://wiki.apache.org/hama/Architecture?action=diff&rev1=93&rev2=94 >> >> >> >> >> >> -------------------------------------------------- >> >> >> >> >> >> >> >> >> This effort is still a "work in progress". Please feel free to add >> >> >> comments. >> >> >> ---- >> >> >> + >> >> >> + == The overall architecture of HAMA == >> >> >> + {{{ >> >> >> + +-----------------------------------------------+ >> >> >> + | HAMA APIs | >> >> >> + +-----------------------------------------------+ >> >> >> + -------------------------------------------------------- >> >> >> + +---------------------------+ >> >> >> + | Graph Computing Framework | >> >> >> + +---------------------------+ Computing >> >> Engine >> >> >> + +-----------+ +---------------------------------+ >> >> >> + | MapReduce | | BSP (Bulk Synchronous Parallel) | >> >> >> + +-----------+ +---------------------------------+ >> >> >> + -------------------------------------------------------- >> >> >> + +-----------------------------------------------+ >> >> >> + | Zookeeper | >> Distributed >> >> >> Locking Service >> >> >> + +-----------------------------------------------+ >> >> >> + -------------------------------------------------------- >> >> >> + +-----------------------------------------------+ >> >> >> + | | >> >> >> + | +-----------------------------+ | >> >> >> + | | HBase : Distributed Storage | | >> >> >> + | +-----------------------------+ | >> Distributed >> >> >> Storage Systems >> >> >> + | +---------------------------------+ | >> >> >> + | | HDFS : Distributed File System | | >> >> >> + | +---------------------------------+ | >> >> >> + +-----------------------------------------------+ >> >> >> + }}} >> >> >> + >> >> >> == BSP == >> >> >> >> >> >> The BSP package is a implementation of BSP over Hadoop RPC(sockets). >> >> >> >> >> > >> >> >> >> >> >> >> >> -- >> >> Best Regards, Edward J. Yoon >> >> [email protected] >> >> http://blog.udanax.org >> >> >> > >> >> >> >> -- >> Best Regards, Edward J. Yoon >> [email protected] >> http://blog.udanax.org >> > -- Best Regards, Edward J. Yoon [email protected] http://blog.udanax.org
