Hi all, I'm planning a rather non-standard HDFS cluster. The machines will be doing more than just DFS, and each machine will have varying local storage utilization outside of DFS. If I use the "dfs.datanode.du.reserved" property and reserve 10 GB, Does that mean DFS will use (total disk size - 10 GB) or that it will always leave 10 GB free? Basically, is the disk usage outside DFS (OS + other data) taken in to account?
As usage outside of DFS grows I'd like DFS to back off the disk, and migrate blocks to other nodes. If this isn't the current behavior, I could create a script to look at disk usage every few hours and modify the reserved property dynamically. If the property is changed on a single datanode and it is restarted, will the datanode then start moving blocks away? My other option is to just set the reserved amount very high on every node, but that will lead to a lot of wasted space as many nodes won't have a very large storage demand outside of DFS. Any comments or suggestions would be welcomed. Thanks, -- Landy Bible Simulation and Computer Specialist School of Nursing - Collins College of Business The University of Tulsa
