Hey Stefan, It's always fun when seemingly trivial problems turn out to be nontrivial. As for the solution: if I recall correctly (someone from Facebook please hop in here), we just jacked up dfs.datanode.du.reserved to a sizable amount, like 2 GB or something.
Regards, Jeff On Tue, Oct 28, 2008 at 11:31 PM, Stefan Will <[EMAIL PROTECTED]> wrote: > Hi Jeff, > > Yeah, it looks like I'm running into the issues described in the bug. I'm > running 0.18.1 on CentOS 5 by the way. Measuring available disk space > appears to be harder than I thought ... and here I was under the impression > the percentage in df was a pretty clear indicator of how full the disk is > ;-) > > How did you guys solve/work around this ? > > -- Stefan > > >> From: Jeff Hammerbacher <[EMAIL PROTECTED]> >> Reply-To: <[email protected]> >> Date: Mon, 27 Oct 2008 12:40:08 -0700 >> To: <[email protected]> >> Subject: Re: Datanode not detecting full disk >> >> Hey Stefan, >> >> We used to have trouble with this issue at Facebook. What version are >> you running? You might get more information on this ticket: >> https://issues.apache.org/jira/browse/HADOOP-2991. >> >> Regards, >> Jeff >> >> On Mon, Oct 27, 2008 at 10:00 AM, Stefan Will <[EMAIL PROTECTED]> wrote: >>> Each of my datanodes has a system and a data partition, with dfs.data.dir >>> pointed to the data partition. The data partition just filled up to 100% on >>> all of my nodes (as evident via df), but the NameNode web ui still shows >>> them only 88-94% full (interestingly, the numbers differ even though the >>> machines are configured identically). I thought the datanodes used df to >>> determine free space ? How is the storage utilization determined ? >>> >>> -- Stefan >>> > > >
