Hi all, As I've reported recently [1], work on the MAPREDUCE-2841 branch has progressed well and the development team working on it feels that it is ready to be merged into trunk.
For those not familiar with the JIRA (it's a bit lengthy to read from start to finish!) the goal of this work is to build a native implementation of the map-side sort code. The native implementation's primary advantage is its speed: for example, terasort is 30% faster on a wall-clock basis and 60% faster on a resource consumption basis. For clusters which make heavy use of MapReduce, this is a substantial improvement to their efficiency. Users may enable the feature by switching a single configuration flag, and it will fall back to the original implementation in cases where the native code doesn't support the configured features/types. The new work is entirely pluggable and off-by-default to mitigate risk. The merge patch itself does not modify even a single line of existing code: all necessary plug-points have already been committed to trunk for some time. Though we do not yet have a full +1 precommit Jenkins run on the JIRA, there are only a few small nits to fix before merge, so I figured that we could start the vote in parallel. Of course we will not merge until it has a positive precommit run. Though this branch is a new contribution to the Apache repository, it represents work done over several years by a large community of developers including the following: Binglin Chang Yang Dong Sean Zhong Manu Zhang Zhongliang Zhu Vincent Wang Yan Dong Cheng Lian Xusen Yin Fangqin Dai Jiang Weihua Gansha Wu Avik Dey The vote will run for 7 days, ending Friday 9/12 EOD PST. I'll start the voting with my own +1. -Todd [1] http://search-hadoop.com/m/09oay13EwlV/native+task+progress&subj=Native+task+branch+progress
