Hello there,

My Web interface DFS Home announce me that I have 55.52 GB of non DFS used data on my 2 nodes cluster.
I would like to know what are those data about.

Live Datanodes : 2

Node    Last
Contact         Admin State     Configured
Capacity (GB)   Used
(GB)    Non DFS
Used (GB)       Remaining
(GB)    Used
(%)     Used
(%)     Remaining
(%)     Blocks  Failed Volumes
hadoop001 <http://hadoop001.back.adencf.local:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F> 2 In Service 336.41 157.72 27.87 150.82 46.88

        

        44.83   6012    0
hadoop002 <http://hadoop002.back.adencf.local:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F> 2 In Service 336.41 157.72 27.65 151.04 46.88

        

        44.9    6162    0

------------------------------------------------------------------------

root@hadoop001:/etc/hadoop/conf# df -hP /var/ /tmp
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg00-var  337G  169G  151G  53% /var
/dev/mapper/vg00-tmp  473M   13M  436M   3% /tmp
root@hadoop001:/etc/hadoop/conf# du -sh /var/*
4.0K    /var/agentx
5.4M    /var/backups
643M    /var/cache
158G    /var/lib
4.0K    /var/local
0    /var/lock
9.8G    /var/log
16K    /var/lost+found
4.0K    /var/mail
4.0K    /var/opt
124K    /var/run
676K    /var/spool
60K    /var/tmp
21M    /var/zookeeper
root@hadoop001:/etc/hadoop/conf# du -sh /var/lib/hadoop-0.20/datanode/hdfs/dfs/data/
159G    /var/lib/hadoop-0.20/datanode/hdfs/dfs/data/

root@hadoop002:/etc/hadoop/conf# df -hP /var/ /tmp
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg00-var  337G  169G  151G  53% /var
/dev/mapper/vg00-tmp  473M   11M  438M   3% /tmp
root@hadoop002:/etc/hadoop/conf# du -sh /var/*
4.0K    /var/agentx
5.3M    /var/backups
102M    /var/cache
158G    /var/lib
4.0K    /var/local
0    /var/lock
11G    /var/log
16K    /var/lost+found
4.0K    /var/mail
4.0K    /var/opt
104K    /var/run
652K    /var/spool
32K    /var/tmp
21M    /var/zookeeper
root@hadoop002:/etc/hadoop/conf# du -sh /var/lib/hadoop-0.20/datanode/hdfs/dfs/data/
159G    /var/lib/hadoop-0.20/datanode/hdfs/dfs/data/

All the things in /var/lib/hadoop-0.20/datanode/hdfs/dfs/data/ was create by the cluster datanodes.

My question is where are hidden those 27Go of "non dfs data" on each node (I can find 15Go at max on each which are mainly in /var/logs).

Thanks for your helps.

Regards,

--
Damien

Reply via email to