Unless some transformation can additionally be applied in the Mapper phases for your text input, this operation could be done without MapReduce itself; since your requirement of a single file output will incur unnecessary intermediate-phase costs.
On Sat, Feb 26, 2011 at 7:22 AM, Mapred Learn <[email protected]> wrote: > Hi guys, > If I have a text file of 10 GB and I want to convert it to sequence file > using map-reduce and make filesplits of 1 GB each so that 10 mappers work in > parallel on it and convert it to Sequence file output. > Can I combine these 10 mapper outputs into 1 sequence file of 10 GB size in > reduce stage ? Is it possible ? If yes, would it be a very slow operation ? > > -thanks > JJ -- Harsh J www.harshj.com
