Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-19 Thread Pavel Machek
Hi! > > I guess I should try to measure it. (Linux already does writeback > > caching, with 2GB of memory. I wonder how important disks's 2MB of > > cache can be). > > It serves essentially the same purpose as the 'async' option in /etc/exports > (i.e. we declare it "done" when the other end of

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-19 Thread Pavel Machek
Hi! I guess I should try to measure it. (Linux already does writeback caching, with 2GB of memory. I wonder how important disks's 2MB of cache can be). It serves essentially the same purpose as the 'async' option in /etc/exports (i.e. we declare it done when the other end of the wire

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-18 Thread Bryan Henderson
I just had a talk with a colleague, John Palmer, who worked on disk drive design for about 5 years in the '90s and he gave me a very confident, credible explanation of some of the things we've been wondering about disk drive power loss in this thread, complete with demonstrations of various

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-18 Thread Jeff Garzik
Ric Wheeler wrote: Theodore Tso wrote: On Thu, Jan 17, 2008 at 04:31:48PM -0800, Bryan Henderson wrote: But I heard some years ago from a disk drive engineer that that is a myth just like the rotational energy thing. I added that to the discussion, but admitted that I haven't actually seen a

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-18 Thread Bryan Henderson
"H. Peter Anvin" <[EMAIL PROTECTED]> wrote on 01/18/2008 07:08:30 AM: > Bryan Henderson wrote: > > > > We weren't actually talking about writing out the cache. While that was > > part of an earlier thread which ultimately conceded that disk drives most > > probably do not use the spinning

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-18 Thread Ric Wheeler
Theodore Tso wrote: On Thu, Jan 17, 2008 at 04:31:48PM -0800, Bryan Henderson wrote: But I heard some years ago from a disk drive engineer that that is a myth just like the rotational energy thing. I added that to the discussion, but admitted that I haven't actually seen a disk drive write a

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-18 Thread H. Peter Anvin
Bryan Henderson wrote: We weren't actually talking about writing out the cache. While that was part of an earlier thread which ultimately conceded that disk drives most probably do not use the spinning disk energy to write out the cache, the claim was then made that the drive at least

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-18 Thread Theodore Tso
On Thu, Jan 17, 2008 at 04:31:48PM -0800, Bryan Henderson wrote: > But I heard some years ago from a disk drive engineer that that is a myth > just like the rotational energy thing. I added that to the discussion, > but admitted that I haven't actually seen a disk drive write a partial >

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-18 Thread H. Peter Anvin
Bryan Henderson wrote: We weren't actually talking about writing out the cache. While that was part of an earlier thread which ultimately conceded that disk drives most probably do not use the spinning disk energy to write out the cache, the claim was then made that the drive at least

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-18 Thread Theodore Tso
On Thu, Jan 17, 2008 at 04:31:48PM -0800, Bryan Henderson wrote: But I heard some years ago from a disk drive engineer that that is a myth just like the rotational energy thing. I added that to the discussion, but admitted that I haven't actually seen a disk drive write a partial sector.

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-18 Thread Bryan Henderson
H. Peter Anvin [EMAIL PROTECTED] wrote on 01/18/2008 07:08:30 AM: Bryan Henderson wrote: We weren't actually talking about writing out the cache. While that was part of an earlier thread which ultimately conceded that disk drives most probably do not use the spinning disk energy to

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-18 Thread Bryan Henderson
I just had a talk with a colleague, John Palmer, who worked on disk drive design for about 5 years in the '90s and he gave me a very confident, credible explanation of some of the things we've been wondering about disk drive power loss in this thread, complete with demonstrations of various

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-18 Thread Jeff Garzik
Ric Wheeler wrote: Theodore Tso wrote: On Thu, Jan 17, 2008 at 04:31:48PM -0800, Bryan Henderson wrote: But I heard some years ago from a disk drive engineer that that is a myth just like the rotational energy thing. I added that to the discussion, but admitted that I haven't actually seen a

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-18 Thread Ric Wheeler
Theodore Tso wrote: On Thu, Jan 17, 2008 at 04:31:48PM -0800, Bryan Henderson wrote: But I heard some years ago from a disk drive engineer that that is a myth just like the rotational energy thing. I added that to the discussion, but admitted that I haven't actually seen a disk drive write a

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Bryan Henderson
Ric Wheeler <[EMAIL PROTECTED]> wrote on 01/17/2008 03:18:05 PM: > Theodore Tso wrote: > > On Wed, Jan 16, 2008 at 09:02:50PM -0500, Daniel Phillips wrote: > >> Have you observed that in the wild? A former engineer of a disk drive > >> company suggests to me that the capacitors on the board

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Ric Wheeler
Theodore Tso wrote: On Wed, Jan 16, 2008 at 09:02:50PM -0500, Daniel Phillips wrote: Have you observed that in the wild? A former engineer of a disk drive company suggests to me that the capacitors on the board provide enough power to complete the last sector, even to park the head. Even if

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Alan Cox
> interrupt which caused the Irix to run around frantically shutting > down DMA's for a controlled shutdown. Of course, PC-class hardware > has none of this. My source for this was Jim Mostek, one of the PC class hardware has a power good signal which drops just before the rest. -- To

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Daniel Phillips
On Jan 17, 2008 7:29 AM, Szabolcs Szakacsits <[EMAIL PROTECTED]> wrote: > Similarly to ZFS, Windows Server 2008 also has self-healing NTFS: I guess that is enough votes to justify going ahead and trying an implementation of the reverse mapping ideas I posted. But of course more votes for this is

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Theodore Tso
On Wed, Jan 16, 2008 at 09:02:50PM -0500, Daniel Phillips wrote: > > Have you observed that in the wild? A former engineer of a disk drive > company suggests to me that the capacitors on the board provide enough > power to complete the last sector, even to park the head. > The problem isn't

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Bryan Henderson
"Daniel Phillips" <[EMAIL PROTECTED]> wrote on 01/16/2008 06:02:50 PM: > On Jan 16, 2008 2:06 PM, Bryan Henderson <[EMAIL PROTECTED]> wrote: > > >The "disk motor as a generator" tale may not be purely folklore. When > > >an IDE drive is not in writeback mode, something special needs to done > >

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Pavel Machek
On Tue 2008-01-15 20:36:16, Chris Mason wrote: > On Tue, 15 Jan 2008 20:24:27 -0500 > "Daniel Phillips" <[EMAIL PROTECTED]> wrote: > > > On Jan 15, 2008 7:15 PM, Alan Cox <[EMAIL PROTECTED]> wrote: > > > > Writeback cache on disk in iteself is not bad, it only gets bad > > > > if the disk is not

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Szabolcs Szakacsits
On Tue, 15 Jan 2008, Daniel Phillips wrote: > Along with this effort, could you let me know if the world actually > cares about online fsck? Now we know how to do it I think, but is it > worth the effort. Most users seem to care deeply about "things just work". Here is why ntfs-3g also took

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Szabolcs Szakacsits
On Tue, 15 Jan 2008, Daniel Phillips wrote: Along with this effort, could you let me know if the world actually cares about online fsck? Now we know how to do it I think, but is it worth the effort. Most users seem to care deeply about things just work. Here is why ntfs-3g also took the

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Pavel Machek
On Tue 2008-01-15 20:36:16, Chris Mason wrote: On Tue, 15 Jan 2008 20:24:27 -0500 Daniel Phillips [EMAIL PROTECTED] wrote: On Jan 15, 2008 7:15 PM, Alan Cox [EMAIL PROTECTED] wrote: Writeback cache on disk in iteself is not bad, it only gets bad if the disk is not engineered to save

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Bryan Henderson
Daniel Phillips [EMAIL PROTECTED] wrote on 01/16/2008 06:02:50 PM: On Jan 16, 2008 2:06 PM, Bryan Henderson [EMAIL PROTECTED] wrote: The disk motor as a generator tale may not be purely folklore. When an IDE drive is not in writeback mode, something special needs to done to ensure the

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Daniel Phillips
On Jan 17, 2008 7:29 AM, Szabolcs Szakacsits [EMAIL PROTECTED] wrote: Similarly to ZFS, Windows Server 2008 also has self-healing NTFS: I guess that is enough votes to justify going ahead and trying an implementation of the reverse mapping ideas I posted. But of course more votes for this is

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Theodore Tso
On Wed, Jan 16, 2008 at 09:02:50PM -0500, Daniel Phillips wrote: Have you observed that in the wild? A former engineer of a disk drive company suggests to me that the capacitors on the board provide enough power to complete the last sector, even to park the head. The problem isn't with

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Bryan Henderson
Ric Wheeler [EMAIL PROTECTED] wrote on 01/17/2008 03:18:05 PM: Theodore Tso wrote: On Wed, Jan 16, 2008 at 09:02:50PM -0500, Daniel Phillips wrote: Have you observed that in the wild? A former engineer of a disk drive company suggests to me that the capacitors on the board provide

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Ric Wheeler
Theodore Tso wrote: On Wed, Jan 16, 2008 at 09:02:50PM -0500, Daniel Phillips wrote: Have you observed that in the wild? A former engineer of a disk drive company suggests to me that the capacitors on the board provide enough power to complete the last sector, even to park the head. Even if

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-17 Thread Alan Cox
interrupt which caused the Irix to run around frantically shutting down DMA's for a controlled shutdown. Of course, PC-class hardware has none of this. My source for this was Jim Mostek, one of the PC class hardware has a power good signal which drops just before the rest. -- To unsubscribe

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Andreas Dilger
On Jan 15, 2008 22:05 -0500, Rik van Riel wrote: > With a filesystem that is compartmentalized and checksums metadata, > I believe that an online fsck is absolutely worth having. > > Instead of the filesystem resorting to mounting the whole volume > read-only on certain errors, part of the

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Daniel Phillips
On Jan 16, 2008 2:06 PM, Bryan Henderson <[EMAIL PROTECTED]> wrote: > >The "disk motor as a generator" tale may not be purely folklore. When > >an IDE drive is not in writeback mode, something special needs to done > >to ensure the last write to media is not a scribble. > > No it doesn't. The

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Eric Sandeen
Alan Cox wrote: >> Writeback cache on disk in iteself is not bad, it only gets bad if the >> disk is not engineered to save all its dirty cache on power loss, >> using the disk motor as a generator or alternatively a small battery. >> It would be awfully nice to know which brands fail here, if

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Valerie Henson
On Jan 16, 2008 3:49 AM, Pavel Machek <[EMAIL PROTECTED]> wrote: > > ext3's "lets fsck on every 20 mounts" is good idea, but it can be > annoying when developing. Having option to fsck while filesystem is > online takes that annoyance away. I'm sure everyone on cc: knows this, but for the record

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Alan Cox
> And I think there's a problem with drives that, upon sensing the > unreadable sector, assign an alternate even though the sector is fine, and > you eventually run out of spares. You are assuming drives can't tell the difference between stray data loss and sectors that can't be recovered by

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Bryan Henderson
>The "disk motor as a generator" tale may not be purely folklore. When >an IDE drive is not in writeback mode, something special needs to done >to ensure the last write to media is not a scribble. No it doesn't. The last write _is_ a scribble. Systems that make atomic updates to disk drives

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Christoph Hellwig
On Wed, Jan 16, 2008 at 08:43:25AM +1100, David Chinner wrote: > ext3 is not the only filesystem that will have trouble due to > volatile write caches. We see problems often enough with XFS > due to volatile write caches that it's in our FAQ: In fact it will hit every filesystem. A write-back

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Valdis . Kletnieks
On Wed, 16 Jan 2008 12:51:44 +0100, Pavel Machek said: > I guess I should try to measure it. (Linux already does writeback > caching, with 2GB of memory. I wonder how important disks's 2MB of > cache can be). It serves essentially the same purpose as the 'async' option in /etc/exports (i.e. we

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Pavel Machek
On Tue 2008-01-15 18:44:26, Daniel Phillips wrote: > On Jan 15, 2008 6:07 PM, Pavel Machek <[EMAIL PROTECTED]> wrote: > > I had write cache enabled on my main computer. Oops. I guess that > > means we do need better documentation. > > Writeback cache on disk in iteself is not bad, it only gets

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Pavel Machek
Hi! > Along with this effort, could you let me know if the world actually > cares about online fsck? I'm not the world's spokeperson (yet ;-). > Now we know how to do it I think, but is it > worth the effort. ext3's "lets fsck on every 20 mounts" is good idea, but it can be annoying when

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Pavel Machek
Hi! Along with this effort, could you let me know if the world actually cares about online fsck? I'm not the world's spokeperson (yet ;-). Now we know how to do it I think, but is it worth the effort. ext3's lets fsck on every 20 mounts is good idea, but it can be annoying when

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Pavel Machek
On Tue 2008-01-15 18:44:26, Daniel Phillips wrote: On Jan 15, 2008 6:07 PM, Pavel Machek [EMAIL PROTECTED] wrote: I had write cache enabled on my main computer. Oops. I guess that means we do need better documentation. Writeback cache on disk in iteself is not bad, it only gets bad if the

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Valdis . Kletnieks
On Wed, 16 Jan 2008 12:51:44 +0100, Pavel Machek said: I guess I should try to measure it. (Linux already does writeback caching, with 2GB of memory. I wonder how important disks's 2MB of cache can be). It serves essentially the same purpose as the 'async' option in /etc/exports (i.e. we

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Christoph Hellwig
On Wed, Jan 16, 2008 at 08:43:25AM +1100, David Chinner wrote: ext3 is not the only filesystem that will have trouble due to volatile write caches. We see problems often enough with XFS due to volatile write caches that it's in our FAQ: In fact it will hit every filesystem. A write-back cache

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Bryan Henderson
The disk motor as a generator tale may not be purely folklore. When an IDE drive is not in writeback mode, something special needs to done to ensure the last write to media is not a scribble. No it doesn't. The last write _is_ a scribble. Systems that make atomic updates to disk drives use a

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Alan Cox
And I think there's a problem with drives that, upon sensing the unreadable sector, assign an alternate even though the sector is fine, and you eventually run out of spares. You are assuming drives can't tell the difference between stray data loss and sectors that can't be recovered by

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Valerie Henson
On Jan 16, 2008 3:49 AM, Pavel Machek [EMAIL PROTECTED] wrote: ext3's lets fsck on every 20 mounts is good idea, but it can be annoying when developing. Having option to fsck while filesystem is online takes that annoyance away. I'm sure everyone on cc: knows this, but for the record you can

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Eric Sandeen
Alan Cox wrote: Writeback cache on disk in iteself is not bad, it only gets bad if the disk is not engineered to save all its dirty cache on power loss, using the disk motor as a generator or alternatively a small battery. It would be awfully nice to know which brands fail here, if any,

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Daniel Phillips
On Jan 16, 2008 2:06 PM, Bryan Henderson [EMAIL PROTECTED] wrote: The disk motor as a generator tale may not be purely folklore. When an IDE drive is not in writeback mode, something special needs to done to ensure the last write to media is not a scribble. No it doesn't. The last write

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-16 Thread Andreas Dilger
On Jan 15, 2008 22:05 -0500, Rik van Riel wrote: With a filesystem that is compartmentalized and checksums metadata, I believe that an online fsck is absolutely worth having. Instead of the filesystem resorting to mounting the whole volume read-only on certain errors, part of the filesystem

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Rik van Riel
On Tue, 15 Jan 2008 20:44:38 -0500 "Daniel Phillips" <[EMAIL PROTECTED]> wrote: > Along with this effort, could you let me know if the world actually > cares about online fsck? Now we know how to do it I think, but is it > worth the effort. With a filesystem that is compartmentalized and

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Daniel Phillips
Hi Pavel, Along with this effort, could you let me know if the world actually cares about online fsck? Now we know how to do it I think, but is it worth the effort. Regards, Daniel -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Chris Mason
On Tue, 15 Jan 2008 20:24:27 -0500 "Daniel Phillips" <[EMAIL PROTECTED]> wrote: > On Jan 15, 2008 7:15 PM, Alan Cox <[EMAIL PROTECTED]> wrote: > > > Writeback cache on disk in iteself is not bad, it only gets bad > > > if the disk is not engineered to save all its dirty cache on > > > power loss,

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Daniel Phillips
On Jan 15, 2008 7:15 PM, Alan Cox <[EMAIL PROTECTED]> wrote: > > Writeback cache on disk in iteself is not bad, it only gets bad if the > > disk is not engineered to save all its dirty cache on power loss, > > using the disk motor as a generator or alternatively a small battery. > > It would be

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Alan Cox
> Writeback cache on disk in iteself is not bad, it only gets bad if the > disk is not engineered to save all its dirty cache on power loss, > using the disk motor as a generator or alternatively a small battery. > It would be awfully nice to know which brands fail here, if any, > because

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Daniel Phillips
On Jan 15, 2008 6:07 PM, Pavel Machek <[EMAIL PROTECTED]> wrote: > I had write cache enabled on my main computer. Oops. I guess that > means we do need better documentation. Writeback cache on disk in iteself is not bad, it only gets bad if the disk is not engineered to save all its dirty cache

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Pavel Machek
Hi! > > > > What are ext3 expectations of disk (is there doc somewhere)? For > > > > example... if disk does not lie, but powerfail during write damages > > > > the sector -- is ext3 still going to work properly? > > > > > > Nope. However the few disks that did this rapidly got firmware updates

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread David Chinner
On Tue, Jan 15, 2008 at 09:16:53PM +0100, Pavel Machek wrote: > Hi! > > > > What are ext3 expectations of disk (is there doc somewhere)? For > > > example... if disk does not lie, but powerfail during write damages > > > the sector -- is ext3 still going to work properly? > > > > Nope. However

[Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Pavel Machek
Hi! > > What are ext3 expectations of disk (is there doc somewhere)? For > > example... if disk does not lie, but powerfail during write damages > > the sector -- is ext3 still going to work properly? > > Nope. However the few disks that did this rapidly got firmware updates > because there are

[Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Pavel Machek
Hi! What are ext3 expectations of disk (is there doc somewhere)? For example... if disk does not lie, but powerfail during write damages the sector -- is ext3 still going to work properly? Nope. However the few disks that did this rapidly got firmware updates because there are other

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread David Chinner
On Tue, Jan 15, 2008 at 09:16:53PM +0100, Pavel Machek wrote: Hi! What are ext3 expectations of disk (is there doc somewhere)? For example... if disk does not lie, but powerfail during write damages the sector -- is ext3 still going to work properly? Nope. However the few disks

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Pavel Machek
Hi! What are ext3 expectations of disk (is there doc somewhere)? For example... if disk does not lie, but powerfail during write damages the sector -- is ext3 still going to work properly? Nope. However the few disks that did this rapidly got firmware updates because there

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Daniel Phillips
On Jan 15, 2008 6:07 PM, Pavel Machek [EMAIL PROTECTED] wrote: I had write cache enabled on my main computer. Oops. I guess that means we do need better documentation. Writeback cache on disk in iteself is not bad, it only gets bad if the disk is not engineered to save all its dirty cache on

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Alan Cox
Writeback cache on disk in iteself is not bad, it only gets bad if the disk is not engineered to save all its dirty cache on power loss, using the disk motor as a generator or alternatively a small battery. It would be awfully nice to know which brands fail here, if any, because writeback

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Daniel Phillips
On Jan 15, 2008 7:15 PM, Alan Cox [EMAIL PROTECTED] wrote: Writeback cache on disk in iteself is not bad, it only gets bad if the disk is not engineered to save all its dirty cache on power loss, using the disk motor as a generator or alternatively a small battery. It would be awfully nice

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Chris Mason
On Tue, 15 Jan 2008 20:24:27 -0500 Daniel Phillips [EMAIL PROTECTED] wrote: On Jan 15, 2008 7:15 PM, Alan Cox [EMAIL PROTECTED] wrote: Writeback cache on disk in iteself is not bad, it only gets bad if the disk is not engineered to save all its dirty cache on power loss, using the disk

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Daniel Phillips
Hi Pavel, Along with this effort, could you let me know if the world actually cares about online fsck? Now we know how to do it I think, but is it worth the effort. Regards, Daniel -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL

Re: [Patch] document ext3 requirements (was Re: [RFD] Incremental fsck)

2008-01-15 Thread Rik van Riel
On Tue, 15 Jan 2008 20:44:38 -0500 Daniel Phillips [EMAIL PROTECTED] wrote: Along with this effort, could you let me know if the world actually cares about online fsck? Now we know how to do it I think, but is it worth the effort. With a filesystem that is compartmentalized and checksums

Re: [RFD] Incremental fsck

2008-01-14 Thread Ric Wheeler
Pavel Machek wrote: On Sat 2008-01-12 09:51:40, Theodore Tso wrote: On Wed, Jan 09, 2008 at 02:52:14PM +0300, Al Boldi wrote: Ok, but let's look at this a bit more opportunistic / optimistic. Even after a black-out shutdown, the corruption is pretty minimal, using ext3fs at least. After a

Re: [RFD] Incremental fsck

2008-01-14 Thread Ric Wheeler
Pavel Machek wrote: On Sat 2008-01-12 09:51:40, Theodore Tso wrote: On Wed, Jan 09, 2008 at 02:52:14PM +0300, Al Boldi wrote: Ok, but let's look at this a bit more opportunistic / optimistic. Even after a black-out shutdown, the corruption is pretty minimal, using ext3fs at least. After a

Re: [RFD] Incremental fsck

2008-01-13 Thread Daniel Phillips
Hi Ted, On Saturday 12 January 2008 06:51, Theodore Tso wrote: > What is very hard to check is whether or not the link count on the > inode is correct. Suppose the link count is 1, but there are > actually two directory entries pointing at it. Now when someone > unlinks the file through one of

Re: [RFD] Incremental fsck

2008-01-13 Thread Alan Cox
> What are ext3 expectations of disk (is there doc somewhere)? For > example... if disk does not lie, but powerfail during write damages > the sector -- is ext3 still going to work properly? Nope. However the few disks that did this rapidly got firmware updates because there are other OS's that

Re: [RFD] Incremental fsck

2008-01-13 Thread Pavel Machek
On Sat 2008-01-12 09:51:40, Theodore Tso wrote: > On Wed, Jan 09, 2008 at 02:52:14PM +0300, Al Boldi wrote: > > > > Ok, but let's look at this a bit more opportunistic / optimistic. > > > > Even after a black-out shutdown, the corruption is pretty minimal, using > > ext3fs at least. > > > >

Re: [RFD] Incremental fsck

2008-01-13 Thread Al Boldi
Theodore Tso wrote: > On Wed, Jan 09, 2008 at 02:52:14PM +0300, Al Boldi wrote: > > Ok, but let's look at this a bit more opportunistic / optimistic. > > > > Even after a black-out shutdown, the corruption is pretty minimal, using > > ext3fs at least. > > After a unclean shutdown, assuming you

Re: [RFD] Incremental fsck

2008-01-13 Thread Al Boldi
Theodore Tso wrote: On Wed, Jan 09, 2008 at 02:52:14PM +0300, Al Boldi wrote: Ok, but let's look at this a bit more opportunistic / optimistic. Even after a black-out shutdown, the corruption is pretty minimal, using ext3fs at least. After a unclean shutdown, assuming you have decent

Re: [RFD] Incremental fsck

2008-01-13 Thread Pavel Machek
On Sat 2008-01-12 09:51:40, Theodore Tso wrote: On Wed, Jan 09, 2008 at 02:52:14PM +0300, Al Boldi wrote: Ok, but let's look at this a bit more opportunistic / optimistic. Even after a black-out shutdown, the corruption is pretty minimal, using ext3fs at least. After a unclean

Re: [RFD] Incremental fsck

2008-01-13 Thread Alan Cox
What are ext3 expectations of disk (is there doc somewhere)? For example... if disk does not lie, but powerfail during write damages the sector -- is ext3 still going to work properly? Nope. However the few disks that did this rapidly got firmware updates because there are other OS's that

Re: [RFD] Incremental fsck

2008-01-13 Thread Daniel Phillips
Hi Ted, On Saturday 12 January 2008 06:51, Theodore Tso wrote: What is very hard to check is whether or not the link count on the inode is correct. Suppose the link count is 1, but there are actually two directory entries pointing at it. Now when someone unlinks the file through one of the

Re: [RFD] Incremental fsck

2008-01-12 Thread Daniel Phillips
On Wednesday 09 January 2008 01:16, Andreas Dilger wrote: > While an _incremental_ fsck isn't so easy for existing filesystem > types, what is pretty easy to automate is making a read-only snapshot > of a filesystem via LVM/DM and then running e2fsck against that. The > kernel and filesystem have

Re: [RFD] Incremental fsck

2008-01-12 Thread Theodore Tso
On Wed, Jan 09, 2008 at 02:52:14PM +0300, Al Boldi wrote: > > Ok, but let's look at this a bit more opportunistic / optimistic. > > Even after a black-out shutdown, the corruption is pretty minimal, using > ext3fs at least. > After a unclean shutdown, assuming you have decent hardware that

Re: [RFD] Incremental fsck

2008-01-12 Thread Al Boldi
Bodo Eggert wrote: > Al Boldi <[EMAIL PROTECTED]> wrote: > > Even after a black-out shutdown, the corruption is pretty minimal, using > > ext3fs at least. So let's take advantage of this fact and do an > > optimistic fsck, to assure integrity per-dir, and assume no external > > corruption. Then

Re: [RFD] Incremental fsck

2008-01-12 Thread Al Boldi
Bodo Eggert wrote: Al Boldi [EMAIL PROTECTED] wrote: Even after a black-out shutdown, the corruption is pretty minimal, using ext3fs at least. So let's take advantage of this fact and do an optimistic fsck, to assure integrity per-dir, and assume no external corruption. Then we release

Re: [RFD] Incremental fsck

2008-01-11 Thread Bodo Eggert
Al Boldi <[EMAIL PROTECTED]> wrote: > Even after a black-out shutdown, the corruption is pretty minimal, using > ext3fs at least. So let's take advantage of this fact and do an optimistic > fsck, to assure integrity per-dir, and assume no external corruption. Then > we release this checked dir

Re: [RFD] Incremental fsck

2008-01-11 Thread Bodo Eggert
Al Boldi [EMAIL PROTECTED] wrote: Even after a black-out shutdown, the corruption is pretty minimal, using ext3fs at least. So let's take advantage of this fact and do an optimistic fsck, to assure integrity per-dir, and assume no external corruption. Then we release this checked dir to the

Re: [RFD] Incremental fsck

2008-01-10 Thread Al Boldi
Rik van Riel wrote: > Al Boldi <[EMAIL PROTECTED]> wrote: > > Ok, but let's look at this a bit more opportunistic / optimistic. > > You can't play fast and loose with data integrity. Correct, but you have to be realistic... > Besides, if we looked at things optimistically, we would conclude >

Re: [RFD] Incremental fsck

2008-01-10 Thread Al Boldi
Rik van Riel wrote: Al Boldi [EMAIL PROTECTED] wrote: Ok, but let's look at this a bit more opportunistic / optimistic. You can't play fast and loose with data integrity. Correct, but you have to be realistic... Besides, if we looked at things optimistically, we would conclude that no

Re: [RFD] Incremental fsck

2008-01-09 Thread Rik van Riel
On Wed, 9 Jan 2008 14:52:14 +0300 Al Boldi <[EMAIL PROTECTED]> wrote: > Ok, but let's look at this a bit more opportunistic / optimistic. You can't play fast and loose with data integrity. Besides, if we looked at things optimistically, we would conclude that no fsck will be needed, ever :) >

Re: [RFD] Incremental fsck

2008-01-09 Thread Al Boldi
Valerie Henson wrote: > On Jan 8, 2008 8:40 PM, Al Boldi <[EMAIL PROTECTED]> wrote: > > Rik van Riel wrote: > > > Al Boldi <[EMAIL PROTECTED]> wrote: > > > > Has there been some thought about an incremental fsck? > > > > > > > > You know, somehow fencing a sub-dir to do an online fsck? > > > > > >

Re: [RFD] Incremental fsck

2008-01-09 Thread Andreas Dilger
Andi Kleen wrote: >> Theodore Tso <[EMAIL PROTECTED]> writes: >> > Now, there are good reasons for doing periodic checks every N mounts >> > and after M months. And it has to do with PC class hardware. (Ted's >> > aphorism: "PC class hardware is cr*p"). >> >> If these reasons are good ones (some

Re: [RFD] Incremental fsck

2008-01-09 Thread Valdis . Kletnieks
On Wed, 09 Jan 2008 07:40:12 +0300, Al Boldi said: > But why wouldn't it be possible to do this on the current fs infrastructure, > using just a smart fsck, working incrementally on some sub-dir? If you have /home/usera, /home/userb, and /home/userc, the vast majority of fs screw-ups can't be

Re: [RFD] Incremental fsck

2008-01-09 Thread Valdis . Kletnieks
On Wed, 09 Jan 2008 07:40:12 +0300, Al Boldi said: But why wouldn't it be possible to do this on the current fs infrastructure, using just a smart fsck, working incrementally on some sub-dir? If you have /home/usera, /home/userb, and /home/userc, the vast majority of fs screw-ups can't be

Re: [RFD] Incremental fsck

2008-01-09 Thread Andreas Dilger
Andi Kleen wrote: Theodore Tso [EMAIL PROTECTED] writes: Now, there are good reasons for doing periodic checks every N mounts and after M months. And it has to do with PC class hardware. (Ted's aphorism: PC class hardware is cr*p). If these reasons are good ones (some skepticism here)

Re: [RFD] Incremental fsck

2008-01-09 Thread Al Boldi
Valerie Henson wrote: On Jan 8, 2008 8:40 PM, Al Boldi [EMAIL PROTECTED] wrote: Rik van Riel wrote: Al Boldi [EMAIL PROTECTED] wrote: Has there been some thought about an incremental fsck? You know, somehow fencing a sub-dir to do an online fsck? Search for chunkfs

Re: [RFD] Incremental fsck

2008-01-09 Thread Rik van Riel
On Wed, 9 Jan 2008 14:52:14 +0300 Al Boldi [EMAIL PROTECTED] wrote: Ok, but let's look at this a bit more opportunistic / optimistic. You can't play fast and loose with data integrity. Besides, if we looked at things optimistically, we would conclude that no fsck will be needed, ever :)

Re: [RFD] Incremental fsck

2008-01-08 Thread Valerie Henson
On Jan 8, 2008 8:40 PM, Al Boldi <[EMAIL PROTECTED]> wrote: > Rik van Riel wrote: > > Al Boldi <[EMAIL PROTECTED]> wrote: > > > Has there been some thought about an incremental fsck? > > > > > > You know, somehow fencing a sub-dir to do an online fsck? > > > > Search for "chunkfs" > > Sure, and

Re: [RFD] Incremental fsck

2008-01-08 Thread Al Boldi
Rik van Riel wrote: > Al Boldi <[EMAIL PROTECTED]> wrote: > > Has there been some thought about an incremental fsck? > > > > You know, somehow fencing a sub-dir to do an online fsck? > > Search for "chunkfs" Sure, and there is TileFS too. But why wouldn't it be possible to do this on the current

Re: [RFD] Incremental fsck

2008-01-08 Thread Alan
> Andi Kleen wrote: >> Theodore Tso <[EMAIL PROTECTED]> writes: >> > Now, there are good reasons for doing periodic checks every N mounts >> > and after M months. And it has to do with PC class hardware. (Ted's >> > aphorism: "PC class hardware is cr*p"). >> >> If these reasons are good ones

Re: [RFD] Incremental fsck

2008-01-08 Thread Rik van Riel
On Wed, 9 Jan 2008 00:22:55 +0300 Al Boldi <[EMAIL PROTECTED]> wrote: > Has there been some thought about an incremental fsck? > > You know, somehow fencing a sub-dir to do an online fsck? Search for "chunkfs" -- All rights reversed. -- To unsubscribe from this list: send the line

[RFD] Incremental fsck

2008-01-08 Thread Al Boldi
Andi Kleen wrote: > Theodore Tso <[EMAIL PROTECTED]> writes: > > Now, there are good reasons for doing periodic checks every N mounts > > and after M months. And it has to do with PC class hardware. (Ted's > > aphorism: "PC class hardware is cr*p"). > > If these reasons are good ones (some

[RFD] Incremental fsck

2008-01-08 Thread Al Boldi
Andi Kleen wrote: Theodore Tso [EMAIL PROTECTED] writes: Now, there are good reasons for doing periodic checks every N mounts and after M months. And it has to do with PC class hardware. (Ted's aphorism: PC class hardware is cr*p). If these reasons are good ones (some skepticism here)

  1   2   >