On 07/26/2011 06:52 PM, Mohamed Riadh Trad wrote:
Dear All,
Is it possible to set up a task with multiple reducers and merge reducers
outputs into one single file?
Bests,
Trad Mohamed Riadh, M.Sc, Ing.
Not within the map-reduce job, but you can merge it after the job is
done. At my previous company we used FileUtil.copyMerge() to do this,
and it worked quite well.
See:
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/fs/FileUtil.html#copyMerge%28org.apache.hadoop.fs.FileSystem,%20org.apache.hadoop.fs.Path,%20org.apache.hadoop.fs.FileSystem,%20org.apache.hadoop.fs.Path,%20boolean,%20org.apache.hadoop.conf.Configuration,%20java.lang.String%29
DR
