Just an update. Some good news. I have a patch for MR-279 to start working with trunk/common and trunk/hdfs (some minor fixes). I am facing some issues with secuirty which should be sorted out by EOD today. We should be ready to start the process of merging soon.
thanks mahadev On Jul 12, 2011, at 11:52 PM, Arun C Murthy wrote: > On Jun 10, 2011, at 11:44 PM, Arun C Murthy wrote: > >> 1. I'd like to get MAPREDUCE-2400 committed to both trunk and MR-279. >> This is a very important patch by Tom which allows to submit jobs to >> both JT and the new RM - thanks Tom! >> 2. Check-in the 'yarn' subfolder which is just the ResourceManager and >> NodeManager i.e. compute fabric to trunk. >> 3. Rework the src tree in trunk to look like the MR-279 branch to >> ensure no changes to trunk are lost - this will ensure separation >> between the 'classic' framework (JT/TT) and the MR runtime (map/reduce/ >> sort/shuffle etc.) >> 4. Re-play the 4-5 patches needed to the MR runtime from the MR-279 >> branch. >> 5. Change build scripts to reflect the new reality. > > Hello, > > Another update... > > I've been working on 'step 4' outlined above. > > Currently the MR 'runtime' in trunk is nearly identical to the MR-279 > (including MAPREDUCE-2400, thanks Tom!) but for the following 4 jiras for > which I'm requesting some help from the named folks to port to trunk: > > $ git show 99ab6d89acc2fc722489c317be796c15c3975284 > Vinod - MAPREDUCE-2681. Added env vars (stderr/stdout) for hadoop pipes > > $ git show 39a446c12a60041a2740d1a1b99e362a7ea1c8a5 > Robert Evans - MAPREDUCE-2531. Fixed jobcontrol to downgrade JobID > > $ git show 5ece56360977444ca5799dc06c581e215ff3e10b > Robert Evans - MAPREDUCE-2539. Fixed NPE in getMapTaskReports in JobClient > > $ git show 6bddb76f515c3fc178078b688b8d3198b496dba3 > Sid_S - MR-2365 in-lieu > > We also plan to port HADOOP-6929 to MR-279 (Mahadev). > > At the end of this we should have both trunk and MR-279 in sync for the MR > runtime. Hopefully we can get to this by end of this week (optimistically). > > ---- > > Once we achieve this we are done with step 1 and step 4, I plan to go with > the rest of the steps outlined above. > > Thus, in an ideal scenario I should be able to merge MR-279 into trunk by end > of next week! > > We have already have been fortunate to receive several contributions from > people not sitting in my aisle and we'd love more! *smile* > > I'd love to have more feedback and review bandwidth. Please take a look now - > I'm hoping we can get this into trunk soon! > > thanks, > Arun
