pi song wrote:
Dear hadoop mailling-list,
Is there a way to control output filename generation? A sample use case is
when I want 2 MapReduce jobs to output to the same directory.
I think you need to write your own output format (see
http://tinyurl.com/4aszgk). Look at OutputFormat.getRecordWriter(). The
parameter *name* is what determines the output filename. One easy way
would be to append the job-name to this *name* in
OutputFormat.getRecordWriter().
Something like
public RecordWriter<WritableComparable, Writable>
getRecordWriter(FileSystem ignored, JobConf job, String name,
Progressable progress)
throws IOException {
name = name + "_" + job.getJobName();
//rest of the code .. taken from Hadoop-0.16.3
}
Amar
Pi
