Yes. There is quite a bit of need for the local file system in clustered mode.
For one think, all of the shuffle intermediate files are on local disk. For another, the distributed cache is actually stored on local disk. HFDS is a frail vessel that cannot cope with all the needs. On Sun, May 1, 2011 at 11:48 PM, Matthew John <[email protected]>wrote: > ... > 2) Does the Hadoop system utilize the local storage directly for any > purpose > (without going through the HDFS) in clustered mode? > >
