Another heads up.
Things are going well, we have done scale testing (350 nodes) and
continue to do more.
We think we are nearly ready to merge MR-279 into trunk (yaay!).
A proposal for plan forward:
1. I'd like to get MAPREDUCE-2400 committed to both trunk and MR-279.
This is a very important patch by Tom which allows to submit jobs to
both JT and the new RM - thanks Tom!
2. Check-in the 'yarn' subfolder which is just the ResourceManager and
NodeManager i.e. compute fabric to trunk.
3. Rework the src tree in trunk to look like the MR-279 branch to
ensure no changes to trunk are lost - this will ensure separation
between the 'classic' framework (JT/TT) and the MR runtime (map/reduce/
sort/shuffle etc.)
4. Re-play the 4-5 patches needed to the MR runtime from the MR-279
branch.
5. Change build scripts to reflect the new reality.
In terms of timelines, I'm hoping we can get this done in the coming
2-3 weeks.
Thoughts?
thanks,
Arun
