SequenceFile and hadoop sync

2010-07-12 Thread Ted Yu
We are using cdh3b2 which contains append/sync functionality. Our code uses SequenceFile.BlockCompressWriter.append() to write to sequence file. Since FSDataOutputStream out doesn't have public visibility (in SequenceFile.Writer), I am wondering how we can utilize the new sync functionality. Your

Re: SequenceFile and hadoop sync

2010-07-12 Thread Todd Lipcon
Hi Ted, You'll find that SequenceFile.Writer has a syncFs() method in the append branch. This syncs the underlying FSDataOutputStream. -Todd On Mon, Jul 12, 2010 at 3:29 PM, Ted Yu yuzhih...@gmail.com wrote: We are using cdh3b2 which contains append/sync functionality. Our code uses