According to Suraj's dependency diagram, asynchronous messaging is most important and highest priority for us at the moment. How about we focus on this one? (Of course, some committers can dedicated on doing GPU, ML algorithms, or Interface Refactoring issues, regardless of *core* roadmap).
If we agree with this plan, I think we can separate the async messaging into smaller sub-tasks: * Decision of whether we will use existing open source, or not. * Design the asynchronous messaging interface (maybe (spilling) message queue also should be considered together?). * Implementation of asynchronous messaging functions, such as send or flush. * Evaluation and adopt asynchronous messaging as a default messenger system. WDYT? On Mon, Sep 2, 2013 at 11:31 PM, Chia-Hung Lin <[email protected]> wrote: > +1 > > BTW, are we going to prioritize tasks in roadmap? > > > > > On 28 August 2013 14:17, Tommaso Teofili <[email protected]> wrote: >> sure, it looks reasonable to me. >> Tommaso >> >> >> >> 2013/8/28 Edward J. Yoon <[email protected]> >> >>> Hi all, >>> >>> After we release the 0.6.3 (HDFS 2.0 version), we have to work for 0.7.0 >>> version now. >>> >>> I would like to suggest that we solve the messaging scalability issue. >>> WDYT? >>> >>> ... >>> >>> And, according to my experiments, BSP framework shows very nice performance >>> (I tested also GraphLab and Spark). Only Graph job is slow. So, I'll mainly >>> work on improving the performance of GraphJobRunner. >>> >>> -- >>> Best Regards, Edward J. Yoon >>> @eddieyoon >>> -- Best Regards, Edward J. Yoon @eddieyoon
