someone kindly give some pointers on this!! On Mon, May 2, 2011 at 12:46 PM, Matthew John <[email protected]>wrote:
> Any documentations on how the different daemons do the write/read on HDFS > and Local File System (direct), I mean the different protocols used in the > interactions. I basically wanted to figure out how intricate the coupling > between the Storage (HDFS + Local) and other processes in the Hadoop > infrastructure is. > > > > On Mon, May 2, 2011 at 12:26 PM, Ted Dunning <[email protected]>wrote: > >> Yes. There is quite a bit of need for the local file system in clustered >> mode. >> >> For one think, all of the shuffle intermediate files are on local disk. >> For >> another, the distributed cache is actually stored on local disk. >> >> HFDS is a frail vessel that cannot cope with all the needs. >> >> On Sun, May 1, 2011 at 11:48 PM, Matthew John <[email protected] >> >wrote: >> >> > ... >> > 2) Does the Hadoop system utilize the local storage directly for any >> > purpose >> > (without going through the HDFS) in clustered mode? >> > >> > >> > >
