Dear Harsh, Thank you for the very clear explanation. Very well noted now :)
> Hi, > > Inline explanation: > > On Tue, Nov 6, 2012 at 1:41 PM, Elaine Gan <[email protected]> wrote: > > Hi, > > > > I'm a bit confused on the local disk and the hdfs disk. > > Through my admin page, i see information like the following. > > > > Configured Capacity : 254.05 GB > > Configured Capacity = Sum(Configured DN Capacity) > = 64*4 = 256. > > > DFS Used : 207.09 GB > > HDFS files are cumulatively using 207 GB of data. This includes > replicated blocks (default replication factor is 3). > > > Non DFS Used : 16.87 GB > > This is Sum(Configured DN Space) that lies outside the HDFS files. > HDFS runs utilities such as "du"/"df" on disks and can pick up space > that is being utilized by non HDFS, local files such as log files, > etc. lying in the same disk as the one used for DNs. > > > DFS Remaining : 30.09 GB > > Your DFS has only 30 GB more to spare, which with a replication factor > of 3, is less than 10 GB of true file length space. > > > DFS Used% : 81.52 % > > DFS Remaining% : 11.84 % > > These should be self-explainatory. > > > I only made configuration on the heap memory and not the disk capacity. > > Where does "Configured Capacity : 254.05 GB " this comes from? > > I have 4 nodes, each with 64GB of disk (local). > > Will HDFS use it fully ? > > Yes it would, unless you configure the "dfs.datanode.du.reserved" > config param at each DN to a space value in bytes that must be left > free on all configured volumes. > > > I still need some place for local files. > > > > Thank you. > > > > Hope this helps! > > -- > Harsh J
