Starting in 0.17 release, an application can invoke DFSOutputStream.fsync() to persist block locations for a file even before the file is closed.
thanks, dhruba On Tue, May 6, 2008 at 8:11 AM, Cagdas Gerede <[EMAIL PROTECTED]> wrote: > If you are writing 10 blocks for a file and let's say in 10th block namenode > fails, all previous 9 blocks are lost because you were not able to close the > file and therefore namenode did not persist the information about 9 blocks > to the fsimage file. > > How would you solve this problem in the application? Why does the hdfs > client make namenode persist every block once the block is written and > instead waits until closing of the file? Then, don't you need to keep a copy > of all the blocks in your application until you close the file successfully > to prevent data loss. Does it make sense to have this semantics with the > assumption of very large files with multiple blocks? > > Thanks for your response, > > -- > ------------ > Best Regards, Cagdas Evren Gerede > Home Page: http://cagdasgerede.info >
