Re: Merging many output files from reducer

[Owen O'Malley]

On Jul 8, 2009, at 3:13 PM, Pankil Doshi wrote:
Can anyone guide me to merge my output files from reducer to single  
file in
HDFS.

The usual approach is to leave them as separate files. Often the need  
to merge them into a single file is removed by using a total sort  
order. Basically, that ensures that all of the keys in reduce-0 are  
less than the keys in reduce-1, etc. There is a library that helps  
doing that named org.apache.hadoop.mapred.lib.TotalOrderPartitioner.

-- Owen