Doug Cutting wrote:
Would one of the SequenceFile#merge() methods suffice?
http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/io/SequenceFile.Sorter.html#merge(java.util.List,%20org.apache.hadoop.fs.Path)
Hmm ... the idea was to avoid the cost of additional I/O, and read the
parts directly as they are. If I understand it correctly, the
Sorter.merge() needs to rewrite the files in order to merge them, which
means a lot of I/O.
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
