And when the data of the map-intermediate files is compressed, it's still an IFile?
On Mon, Feb 14, 2011 at 4:44 PM, Harsh J <qwertyman...@gmail.com> wrote: > Hello, > > On Mon, Feb 14, 2011 at 8:51 PM, Pedro Costa <psdc1...@gmail.com> wrote: >> Hi, >> >> 1 - The map output files are always of the type SequenceFileFormat? > > If you mean the Map-intermediate files, then no - they're IFiles. > Otherwise, if your OutputFormat is set to a SequenceFileOutputFormat, > then yes these type of files would be created. > > Map-Reduce intermediate files are of the IFile format. It's not part > of the public API, but you may read its implementation in > src/java/org/apache/hadoop/mapred/IFile.java. > > SequenceFiles are almost similar, but are built for better K-V file > operations such as skipping over keys, etc. which is not essentially > required in case of partitioned-and-sorted-data-containing IFiles. > > -- > Harsh J > www.harshj.com > -- Pedro
