Any documentations on how the different daemons do the write/read on HDFS and Local File System (direct), I mean the different protocols used in the interactions. I basically wanted to figure out how intricate the coupling between the Storage (HDFS + Local) and other processes in the Hadoop infrastructure is.
On Mon, May 2, 2011 at 12:26 PM, Ted Dunning <[email protected]> wrote: > Yes. There is quite a bit of need for the local file system in clustered > mode. > > For one think, all of the shuffle intermediate files are on local disk. > For > another, the distributed cache is actually stored on local disk. > > HFDS is a frail vessel that cannot cope with all the needs. > > On Sun, May 1, 2011 at 11:48 PM, Matthew John <[email protected] > >wrote: > > > ... > > 2) Does the Hadoop system utilize the local storage directly for any > > purpose > > (without going through the HDFS) in clustered mode? > > > > >
