On Sun, Dec 11, 2011 at 10:25 AM, Donald Allen <donaldcal...@gmail.com> wrote: > I should have sent this to the mailing list as well as David. Google > has fixed something that wasn't broke -- gmail. They've introduced a > new UI that I haven't gotten used to yet ... > > > ---------- Forwarded message ---------- > From: Donald Allen <donaldcal...@gmail.com> > Date: Sun, Dec 11, 2011 at 10:23 AM > Subject: Re: Lost file-system story > To: David Holland <dholland-t...@netbsd.org> > > > On Sun, Dec 11, 2011 at 8:57 AM, David Holland <dholland-t...@netbsd.org> > wrote: >> On Tue, Dec 06, 2011 at 11:58:25AM -0500, Thor Lancelot Simon wrote: >> > With the filesystem mounted async *nothing* pushes out most >> > metadata updates, >> >> If this is really true, it's a bug and should be fixed. > > It may very well be true. I just did the following: > > I brought up my test laptop, running 5.1 GENERIC, with /home mounted > async,noatime. I created a new file in my home directory. I should > note that when I ZZ'ed out of vi, the disk light flashed momentarily, > and I could hear the disk doing something. I did an ls -lt | head and > the new file was there. I waited just under a minute (to let syncs > happen; this is longer than any of the sysctl vfs.sync.delays, which I > assume are in seconds; the man page doesn't say) and then I pulled the > plug (no battery in the machine). On restart, I got no fsck errors, > but the new file was not in my home directory. I then repeated this > test, waiting a little over a minute this time. Same result, the new > file was gone (this time I got fsck errors). Then I did the test a > third time, but this time I did a sync before pulling the plug. On > restart, I still got some fsck errors that were fixed, but the new > file was present. > > This does suggest that the meta-data is not being written, at least > within a minute or so of creating a new file. > > One thing I think we have not discussed much or at all in this thread > is that the filesystem surviving a crash and how much data you lose > when it does survive are separate issues. The experiments I did > yesterday demonstrate that a NetBSD ffs async-mounted filesystem, > together with its fsck, *can* survive a crash in bad circumstances -- > lots of write activity at the time of the crash. We don't know what > the probability of survival is, just that it's > 0. > > What I did yesterday also does not address loss of data. If Simon is > correct and NetBSD is just not writing metadata until sync is > explicitly called, then you could have a system up for days or weeks > and lose as many as all of the files created in an async filesystem > since the last re-boot. We don't know definitively what it's doing > yet, but I think I've demonstrated that it's not writing meta-data > within one minute windows. I will do some more playing with this, > waiting longer and will report what I find. We also know from this > morning's tests that a user-called sync does get the meta-data > written, reducing the amount of data lost in a crash that the > filesystem survives. So those who advocated periodically calling sync > in a loop (Christos first suggested this to me in a private email) are > correct -- it's necessary if you are going to use async mounting.
I repeated the test without the sync, but waited 15 minutes after creating the new file before killing the power. When the system came up, I got fsck errors that were fixed, and the new file I created 15 minutes before pulling the plug was not present. Whether this is intentional or a bug, I agree with David Holland -- it's wrong and should be fixed. /Don > > More later ... > > /Don > > >> >> -- >> David A. Holland >> dholl...@netbsd.org
