Hi Jeff, Did you run the file-upload command on the master node itself? The DFS client attempts to store one replica of the data on the node on which the DFSClient is running.
To get a uniform distribution, it would be good if you upload your data from multiple nodes in your cluster. Thanks, dhruba -----Original Message----- From: Jeff Eastman [mailto:[EMAIL PROTECTED] Sent: Thursday, December 20, 2007 7:15 AM To: hadoop-user@lucene.apache.org Subject: DFS Block Allocation I've brought up a small cluster and uploaded some large files. The master node is cu027 and it seems to be getting an unfair percentage of the blocks allocated to it, especially compared to cu171 which has the same size disk. Can somebody shed some light on the reasons for this? Jeff Node Last Contact Admin State Size (GB) Used (%) Blocks cu009 <http://cu009.cubit.sp.collab.net:50075/browseDirectory.jsp?namenodeInfo Port=50070&dir=%2F> 0 In Service 69.22 75.07 81 cu027 <http://cu027.cubit.sp.collab.net:50075/browseDirectory.jsp?namenodeInfo Port=50070&dir=%2F> 1 In Service 291.24 59.74 333 cu028 <http://cu028.cubit.sp.collab.net:50075/browseDirectory.jsp?namenodeInfo Port=50070&dir=%2F> 1 In Service 71.19 35.34 76 cu034 <http://cu034.cubit.sp.collab.net:50075/browseDirectory.jsp?namenodeInfo Port=50070&dir=%2F> 1 In Service 71.19 17.79 81 cu035 <http://cu035.cubit.sp.collab.net:50075/browseDirectory.jsp?namenodeInfo Port=50070&dir=%2F> 0 In Service 71.19 18.49 64 cu045 <http://cu045.cubit.sp.collab.net:50075/browseDirectory.jsp?namenodeInfo Port=50070&dir=%2F> 1 In Service 71.19 16.68 70 cu050 <http://cu050.cubit.sp.collab.net:50075/browseDirectory.jsp?namenodeInfo Port=50070&dir=%2F> 2 In Service 71.19 31.99 69 cu062 <http://cu062.cubit.sp.collab.net:50075/browseDirectory.jsp?namenodeInfo Port=50070&dir=%2F> 2 In Service 71.19 39.64 72 cu063 <http://cu063.cubit.sp.collab.net:50075/browseDirectory.jsp?namenodeInfo Port=50070&dir=%2F> 2 In Service 71.19 25.03 77 cu171 <http://cu171.cubit.sp.collab.net:50075/browseDirectory.jsp?namenodeInfo Port=50070&dir=%2F> 2 In Service 291.24 13.05 79
