Dear dhruba, Thanks you for your kind explanation. Unfortunately, I failed to locate the code for use of 'targets' as you described. I was only able to see "targets" field be used by dfs.FSEditLog.logOpenFile(). When LeaseMonitor finds a expired lease, it seems to get datanode locations just from FSNamesystem.blockMaps rather than from the 'targets' field. I guess I am missing something. Could you shed some more light on this?
I appreciate your help again. Sangmin On Fri, Aug 1, 2008 at 1:15 AM, Dhruba Borthakur <[EMAIL PROTECTED]> wrote: > The "targets" field is used to record the block location(s of the last > allocated block fro a file. It is important to remember this at block > allocation time itself so that data written to the block can be > recovered if the client dies before it finished writing to the entire > block. > > Take the case when the client started writing to the 3 datanodes in > the pipeline for that block. Now, let's assume that the client died > aftre writing 100 bytes to the first two replicas whereas the third > replica has only 50 bytes in the block. The namenode holds a lease for > this client. If the client dies prematurely, the namenode experiences > a timeout for the client and triggers lease recovery for the file(s) > that were being written by the client. The namenode has to ensure that > all the three replicas are consistent with one another, otherwise > readers of the block might see inconsistent data depending on which > replica they access. Thus, the namenode has to remember which > datanode(s) possibly has replicas of that block. > > hope this help, > dhruba > > On Thu, Jul 31, 2008 at 12:27 PM, Sangmin Lee <[EMAIL PROTECTED]> > wrote: > > To clarify my question, what is the need for storing the locations of > last > > block without being reported by datanodes? > > > > On Thu, Jul 31, 2008 at 3:18 PM, Sangmin Lee <[EMAIL PROTECTED]> > wrote: > > > >> Hi folks, > >> > >> I wonder why hadoop.dfs.INodeFileUnderConstruction Class has "targets" > >> field. > >> Can anyone explain this to me? > >> any help would be appreciated. > >> > >> -sangmin > >> > > >
