> I would like to suggest that we solve the messaging scalability issue. WDYT?
The Spilling Queue seems works fine on my cluster. So, if we finish the below tasks, I think, Hama will be very "Competitive" in terms of the performance and scalability of the both (pure) BSP and Graph computing engine. HAMA-734 Hama Message Manager should be able to delegate the ownership of internal message queue on request for future superstep. HAMA-723 Implement sorting in Spilling queue. HAMA-816 Add the getMsgIterators method for efficient message looping. HAMA-783 Efficient InMemory Storage for Vertices. If no objection, i would like to arrange the JIRA tasks for 0.7.0, based on this. WDYT? On Tue, Sep 10, 2013 at 12:11 AM, Chia-Hung Lin <[email protected]> wrote: > That looks fine to me. In addition to this, if that task is > accomplished, are we planning to release a new version (e.g. a minor > version plus 1)? Just to check as it seems that we have demands on > frequent releases so that users who need some specific patches can use > it earlier. > > > > > On 3 September 2013 09:45, Edward J. Yoon <[email protected]> wrote: >> According to Suraj's dependency diagram, asynchronous messaging is >> most important and highest priority for us at the moment. How about we >> focus on this one? (Of course, some committers can dedicated on doing >> GPU, ML algorithms, or Interface Refactoring issues, regardless of >> *core* roadmap). >> >> If we agree with this plan, I think we can separate the async >> messaging into smaller sub-tasks: >> >> * Decision of whether we will use existing open source, or not. >> * Design the asynchronous messaging interface (maybe (spilling) >> message queue also should be considered together?). >> * Implementation of asynchronous messaging functions, such as send or flush. >> * Evaluation and adopt asynchronous messaging as a default messenger system. >> >> WDYT? >> >> >> On Mon, Sep 2, 2013 at 11:31 PM, Chia-Hung Lin <[email protected]> wrote: >>> +1 >>> >>> BTW, are we going to prioritize tasks in roadmap? >>> >>> >>> >>> >>> On 28 August 2013 14:17, Tommaso Teofili <[email protected]> wrote: >>>> sure, it looks reasonable to me. >>>> Tommaso >>>> >>>> >>>> >>>> 2013/8/28 Edward J. Yoon <[email protected]> >>>> >>>>> Hi all, >>>>> >>>>> After we release the 0.6.3 (HDFS 2.0 version), we have to work for 0.7.0 >>>>> version now. >>>>> >>>>> I would like to suggest that we solve the messaging scalability issue. >>>>> WDYT? >>>>> >>>>> ... >>>>> >>>>> And, according to my experiments, BSP framework shows very nice >>>>> performance >>>>> (I tested also GraphLab and Spark). Only Graph job is slow. So, I'll >>>>> mainly >>>>> work on improving the performance of GraphJobRunner. >>>>> >>>>> -- >>>>> Best Regards, Edward J. Yoon >>>>> @eddieyoon >>>>> >> >> >> >> -- >> Best Regards, Edward J. Yoon >> @eddieyoon -- Best Regards, Edward J. Yoon @eddieyoon
