HI all, For those working on MR-279. I just committed a patch to MR-279 to make Next gen MR work with trunk/common and trunk/hdfs. With the commit Next gen MR next would no longer work with the common/hdfs in MR-279 branch.
We are now all set to merge Next gen MR to trunk! thanks mahadev On Wed, Jul 27, 2011 at 9:46 AM, Mahadev Konar <maha...@hortonworks.com> wrote: > Just an update. > > Some good news. I have a patch for MR-279 to start working with trunk/common > and trunk/hdfs (some minor fixes). I am facing some issues with secuirty > which should be sorted out by EOD today. > We should be ready to start the process of merging soon. > > thanks > mahadev > > On Jul 12, 2011, at 11:52 PM, Arun C Murthy wrote: > >> On Jun 10, 2011, at 11:44 PM, Arun C Murthy wrote: >> >>> 1. I'd like to get MAPREDUCE-2400 committed to both trunk and MR-279. >>> This is a very important patch by Tom which allows to submit jobs to >>> both JT and the new RM - thanks Tom! >>> 2. Check-in the 'yarn' subfolder which is just the ResourceManager and >>> NodeManager i.e. compute fabric to trunk. >>> 3. Rework the src tree in trunk to look like the MR-279 branch to >>> ensure no changes to trunk are lost - this will ensure separation >>> between the 'classic' framework (JT/TT) and the MR runtime (map/reduce/ >>> sort/shuffle etc.) >>> 4. Re-play the 4-5 patches needed to the MR runtime from the MR-279 >>> branch. >>> 5. Change build scripts to reflect the new reality. >> >> Hello, >> >> Another update... >> >> I've been working on 'step 4' outlined above. >> >> Currently the MR 'runtime' in trunk is nearly identical to the MR-279 >> (including MAPREDUCE-2400, thanks Tom!) but for the following 4 jiras for >> which I'm requesting some help from the named folks to port to trunk: >> >> $ git show 99ab6d89acc2fc722489c317be796c15c3975284 >> Vinod - MAPREDUCE-2681. Added env vars (stderr/stdout) for hadoop pipes >> >> $ git show 39a446c12a60041a2740d1a1b99e362a7ea1c8a5 >> Robert Evans - MAPREDUCE-2531. Fixed jobcontrol to downgrade JobID >> >> $ git show 5ece56360977444ca5799dc06c581e215ff3e10b >> Robert Evans - MAPREDUCE-2539. Fixed NPE in getMapTaskReports in JobClient >> >> $ git show 6bddb76f515c3fc178078b688b8d3198b496dba3 >> Sid_S - MR-2365 in-lieu >> >> We also plan to port HADOOP-6929 to MR-279 (Mahadev). >> >> At the end of this we should have both trunk and MR-279 in sync for the MR >> runtime. Hopefully we can get to this by end of this week (optimistically). >> >> ---- >> >> Once we achieve this we are done with step 1 and step 4, I plan to go with >> the rest of the steps outlined above. >> >> Thus, in an ideal scenario I should be able to merge MR-279 into trunk by >> end of next week! >> >> We have already have been fortunate to receive several contributions from >> people not sitting in my aisle and we'd love more! *smile* >> >> I'd love to have more feedback and review bandwidth. Please take a look now >> - I'm hoping we can get this into trunk soon! >> >> thanks, >> Arun > >
