>> - Data repartitioning >> - Hadoop 2.0 compatibility P.S., My summer vacation will start from Aug 11th, so I need more time.
On Mon, Jul 9, 2012 at 10:49 PM, Tommaso Teofili <[email protected]> wrote: > 2012/7/9 Edward J. Yoon <[email protected]> > >> Then, let's try to release 0.6 by end of August or September 2012. I >> heard/saw some CDH4 plans/activities, we should not hesitate. :) >> >> Our TODO list: >> >> >> - Fault tolerance >> >> - Data repartitioning >> >> - Hadoop 2.0 compatibility >> >> - Hama documentation book in html >> >> - Bug fixes if any reported and few code improvements. >> > adding the streaming API. >> > > +1 for this roadmap. > I'd like to put some machine learning algorithm in there but I'm not sure > about the timing for it. > Regards, > Tommaso > > >> >> I'll work on yarn module and (large input data) re-partitioning issue. >> >> If you need my help such as test on large cluster, Just let me know. >> >> On Mon, Jul 9, 2012 at 9:37 PM, Thomas Jungblut >> <[email protected]> wrote: >> > Suraj's plan would also be my favorite. >> > We also could split two faster releases from fault tolerance and Hadoop >> 2.0. >> > >> > If we have sorted out the fault tolerance I would like to spend my time >> on >> > adding the streaming API. >> > >> > 2012/7/9 Suraj Menon <[email protected]> >> > >> >> Sorry, I was looking at more time (~6 months) for the next release. This >> >> implies that are we are looking for a release in August end. The desired >> >> feature set: >> >> >> >> - Fault tolerance >> >> - Data repartitioning >> >> - Hadoop 2.0 compatibility >> >> - Hama documentation book in html >> >> - Bug fixes if any reported and few code improvements. >> >> >> >> I have to work on my Jenkins permissions. The last time I tried, it did >> not >> >> work for me. >> >> >> >> -Suraj >> >> >> >> On Mon, Jul 9, 2012 at 6:02 AM, Edward J. Yoon <[email protected] >> >> >wrote: >> >> >> >> > We only need to discuss about 0.6 release roadmap. Any committer may >> >> > create a release candidate. Please feel free to create your own RC and >> >> open >> >> > the vote in future. >> >> > >> >> > Personally, I would like to release along with support for Hadoop 2.0, >> >> > ASAP. Hope I can finish by this end of month or next month. >> >> > >> >> > And IMO, the current high-priority tasks are Input re-partitioning and >> >> > exception handling. >> >> > >> >> > Thanks, >> >> > Ed >> >> > >> >> > >> >> > On 7/9/2012 5:15 PM, Suraj Menon wrote: >> >> > >> >> >> Hello, >> >> >> >> >> >> Fault tolerance patch should be in this week. >> >> >> >> >> >> The other features I want to throw for consideration are written >> below. >> >> >> Please feel free to prioritize for 0.6: >> >> >> >> >> >> BSPPeer API Improvements >> >> >> - Broadcast API >> >> >> We have to experiment a broadcast API that does not use the >> Messaging >> >> >> queue. Few strategies for the purpose could be brainstormed. >> >> >> - HAMA-546 >> >> >> >> >> >> Task Scheduler Improvements >> >> >> - Task Allocation as feature of BSP job instead of system (Coming >> with >> >> the >> >> >> Fault tolerance patch) Today the task allocation responsibility is >> >> wrongly >> >> >> implemented in TaskInProgress class. With this feature we can have >> >> >> pluggable task allocation strategy with data-local allocation as >> default >> >> >> strategy. >> >> >> - To support real time jobs, we would need to implement resource >> local >> >> >> scheduler, where resource here is more than the file input split. >> Based >> >> on >> >> >> the nature of job, it could be nodes with bigger pipe for >> communication, >> >> >> nature of the machine(GPU), Nodes with SSDs, etc. >> >> >> >> >> >> Job Submission Improvements >> >> >> - A Job Submission API, that gives users the flexibility to define >> the >> >> >> nature of messaging queue and other attributes per superstep >> >> >> >> >> >> Improve Performance And Scalability >> >> >> - Benchmark on more nodes >> >> >> - Try Asynchronous communication >> >> >> - HAMA-593 >> >> >> - Spilling message queue and External Sorting Queue. >> >> >> >> >> >> Apart from this, there could be more graph algorithms to be >> implemented, >> >> >> ml >> >> >> algorithms to be implemented and Mikalai's work integrated in the >> >> release. >> >> >> When are we planning to have next release? >> >> >> >> >> >> I can take a shot at release management, but I would definitely need >> >> help >> >> >> for the first time. >> >> >> >> >> >> Thanks, >> >> >> Suraj >> >> >> >> >> >> On Mon, Jul 9, 2012 at 3:22 AM, Thomas Jungblut >> >> >> <[email protected]>**wrote: >> >> >> >> >> >> So now we released 0.5.0 what are our plans for 0.6.0? >> >> >>> And who is going to do the release managing? >> >> >>> >> >> >>> >> >> > -- >> >> > Best Regards, Edward J. Yoon >> >> > @eddieyoon >> >> > >> >> > >> >> >> >> >> >> -- >> Best Regards, Edward J. Yoon >> @eddieyoon >> -- Best Regards, Edward J. Yoon @eddieyoon
