[RFC] VFS: mnotify (was: [PATCH 00/23] per device dirty throttling -v8)

2007-08-05 Thread Al Boldi
Jakob Oestergaard wrote: > Why on earth would you cripple the kernel defaults for ext3 (which is a > fine FS for boot/root filesystems), when the *fundamental* problem you > really want to solve lie much deeper in the implementation of the > filesystem? Noatime doesn't solve the problem, it just

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread David Chinner
On Sat, Aug 04, 2007 at 09:16:35PM +0200, Florian Weimer wrote: > * Andrew Morton: > > > The easy preventive is to mount with data=writeback. Maybe that should > > have been the default. > > The documentation I could find suggests that this may lead to a > security weakness (old data in blocks

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread David Chinner
On Sun, Aug 05, 2007 at 06:42:30AM -0400, Jeff Garzik wrote: > Jakob Oestergaard wrote: > >Oh dear. > > > >Why not just make ext3 fsync() a no-op while you're at it? > > > >Distros can turn it back on if it's needed... > > > >Of course I'm not serious, but like atime, fsync() is something one > >

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Theodore Tso
On Sat, Aug 04, 2007 at 09:16:35PM +0200, Florian Weimer wrote: > * Andrew Morton: > > > The easy preventive is to mount with data=writeback. Maybe that should > > have been the default. > > The documentation I could find suggests that this may lead to a > security weakness (old data in blocks

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Andi Kleen
Brice Figureau <[EMAIL PROTECTED]> writes: > > 2) I _still_ don't get the "performances" of 2.6.17, but since that's the > better combination I could get, I think there is IMHO progress in the right > direction (to be compared to no progress since 2.6.18, that's better :-)). If you could

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Christoph Hellwig
On Sun, Aug 05, 2007 at 09:57:02AM +0200, Florian Weimer wrote: > For instance, some editors don't perform fsync-then-rename, but simply > truncate the file when saving (because they want to preserve hard > links). With XFS, this tends to cause null bytes on crashes. Since > ext3 has got a much

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Christoph Hellwig
On Sun, Aug 05, 2007 at 02:26:53AM +0200, Andi Kleen wrote: > I always thought the right solution would be to just sync atime only > very very lazily. This means if a inode is only dirty because of an > atime update put it on a "only write out when there is nothing to do > or the memory is really

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Andrew Morton
On Sun, 5 Aug 2007 22:21:12 +0200 Jörn Engel <[EMAIL PROTECTED]> wrote: > On Sun, 5 August 2007 20:37:14 +0200, Jörn Engel wrote: > > > > Guess I should throw in a kernel compile test as well, just to get a > > feel for the performance. > > Three runs each of noatime, relatime and atime, both

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Christoph Hellwig
On Sat, Aug 04, 2007 at 09:42:59PM +0200, J??rn Engel wrote: > On Sat, 4 August 2007 21:26:15 +0200, J??rn Engel wrote: > > > > Given the choice between only "atime" and "noatime" I'd agree with you. > > Heck, I use it myself. But "relatime" seems to combine the best of both > > worlds. It

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Christoph Hellwig
On Sun, Aug 05, 2007 at 11:01:18AM -0700, Arjan van de Ven wrote: > > on the journalling side this would be one transaction (not 5 milion) > and... since inodes are grouped on disk, you can even get some better > coalescing this way... > > Wonder if we could do inode-grouping smartly; eg if we

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jörn Engel
On Sun, 5 August 2007 20:37:14 +0200, Jörn Engel wrote: > > Guess I should throw in a kernel compile test as well, just to get a > feel for the performance. Three runs each of noatime, relatime and atime, both with cold caches and with warm caches. Scripts below. Run on a Thinkpad T40, 1.5GHz,

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox <[EMAIL PROTECTED]> wrote: > > also add the CONFIG_DEFAULT_RELATIME kernel option, which makes > > "norelatime" the default for all mounts without an extra kernel boot > > option. > > Should be a mount option. it is already a mount option too. > > + relatime[FS] default

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Alan Cox
> change relatime updates to be performed once per day. This makes > relatime a compatible solution for HSM, mailer-notification and > tmpwatch applications too. Sweet > > also add the CONFIG_DEFAULT_RELATIME kernel option, which makes > "norelatime" the default for all mounts without an extra

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Alan Cox
On Sun, 5 Aug 2007 20:08:26 +0200 Ingo Molnar <[EMAIL PROTECTED]> wrote: > > * Alan Cox <[EMAIL PROTECTED]> wrote: > > > And you honestly think that putting it in Kconfig as well as allowing > > users to screw up horribly and creating incompatible defaults you > > So far you've not offered

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread david
On Sun, 5 Aug 2007, Diego Calleja wrote: El Sun, 5 Aug 2007 09:13:20 +0200, Ingo Molnar <[EMAIL PROTECTED]> escribió: Measurements show that noatime helps 20-30% on regular desktop workloads, easily 50% for kernel builds and much more than that (in excess of 100%) for file-read-intense

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Linus Torvalds <[EMAIL PROTECTED]> wrote: > On Sun, 5 Aug 2007, Ingo Molnar wrote: > > > > you mean tmpwatch? The trivial change below fixes this. And with that > > we've come to the end of an extremely short list of atime dependencies. > > You wouldn't even need these kinds of games. > >

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread adi
On Sun, Aug 05, 2007 at 02:44:08PM -0400, Dave Jones wrote: > It still fails miserably for me. > > If I hit 'C' and '?' I get a list of my mail folders, with some of them > marked 'N' if they have new mail. Without atime, those N's never show > up and every mbox looks like it has no new mail.

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Dave Jones
On Sun, Aug 05, 2007 at 09:21:41AM +0200, Ingo Molnar wrote: > * Alan Cox <[EMAIL PROTECTED]> wrote: > > > With a Red Hat on if we can move from /dev/hda to /dev/sda in FC7 then > > we can move from atime to noatime by default on FC8 with appropriate > > release note warnings and having a

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jörn Engel
On Sun, 5 August 2007 11:02:33 -0700, Arjan van de Ven wrote: > > but does it work with relatime ? Like a greased penguin. I had to reboot with my ugly patch posted earlier in the patch to actually test it, though. Relatime suffers from a distribution problem, nothing else. Guess I should

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Jeff Garzik <[EMAIL PROTECTED]> wrote: > > yeah, i didnt mean to say that it is _always_ a big issue, but "only > > a small number of files are read" is a very, very small minority of > > even the database server world. > > OTOH, consider a popular Linux task, web serving. atime results in

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Arjan van de Ven
> > In addition, big server boxes are usually not reading a huge *number* > of files per second. The place where you see this as a problem is (a) > compilation, thanks to huge /usr/include hierarchies (and here things > have gotten worse over time as include files have gotten much more >

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox <[EMAIL PROTECTED]> wrote: > And you honestly think that putting it in Kconfig as well as allowing > users to screw up horribly and creating incompatible defaults you So far you've not offered one realistic scenario of "screw up horribly". People have been using noatime for a long

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Arjan van de Ven
On Sun, 2007-08-05 at 16:17 +0200, Jörn Engel wrote: > On Sun, 5 August 2007 10:53:54 +0200, Willy Tarreau wrote: > > On Sun, Aug 05, 2007 at 09:21:41AM +0200, Ingo Molnar wrote: > > > > > > btw., Mutt does not go boom, i use it myself. It works just fine and > > > notices new mails even on a

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Arjan van de Ven
On Sat, 2007-08-04 at 17:48 -0400, Theodore Tso wrote: > On Sat, Aug 04, 2007 at 01:13:19PM -0700, Arjan van de Ven wrote: > > there is another trick possible (more involved though, Al will have to > > jump in on that one I suspect): Have 2 types of "dirty inode" states; > > one is the current

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jeff Garzik
Ingo Molnar wrote: * Theodore Tso <[EMAIL PROTECTED]> wrote: If you are always reading from the same small set of files (i.e., a database workload), then those inodes only get updated every 5 seconds (the traditional/default metadata update sync time, as well as the default ext3 journal

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Theodore Tso <[EMAIL PROTECTED]> wrote: > If you are always reading from the same small set of files (i.e., a > database workload), then those inodes only get updated every 5 seconds > (the traditional/default metadata update sync time, as well as the > default ext3 journal update time),

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Brice Figureau
Hi, Ingo Molnar elte.hu> writes: > * Linus Torvalds linux-foundation.org> wrote: > > On Fri, 3 Aug 2007, Peter Zijlstra wrote: > > > > > > These patches aim to improve balance_dirty_pages() and directly address > > > three issues: > > > 1) inter device starvation > > > 2) stacked device

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Linus Torvalds
On Sun, 5 Aug 2007, Ingo Molnar wrote: > > you mean tmpwatch? The trivial change below fixes this. And with that > we've come to the end of an extremely short list of atime dependencies. You wouldn't even need these kinds of games. What we could do is to make "relatime" updates a bit smarter.

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Theodore Tso
On Sun, Aug 05, 2007 at 01:49:26AM +0100, Alan Cox wrote: > HSM is the usual one, and to a large extent probably why Unix originally > had atime. Basically migrating less used files away so as to keep the > system disks tidy. > > Its not something usally found on desktop boxes so it doesn't in

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Theodore Tso
On Sun, Aug 05, 2007 at 02:26:53AM +0200, Andi Kleen wrote: > I always thought the right solution would be to just sync atime only > very very lazily. This means if a inode is only dirty because of an > atime update put it on a "only write out when there is nothing to do > or the memory is really

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jörn Engel
On Sun, 5 August 2007 10:53:54 +0200, Willy Tarreau wrote: > On Sun, Aug 05, 2007 at 09:21:41AM +0200, Ingo Molnar wrote: > > > > btw., Mutt does not go boom, i use it myself. It works just fine and > > notices new mails even on a noatime,nodiratime filesystem. > > IIRC, atime is used by

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jakob Oestergaard
On Sun, Aug 05, 2007 at 02:46:48PM +0200, Ingo Molnar wrote: > > * Jakob Oestergaard <[EMAIL PROTECTED]> wrote: > > > > If you can show massive amounts of users that will actually be > > > negatively impacted, please present hard evidence. > > > > > > Otherwise all this is useless hot air. > >

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Alan Cox
> it's default off of course. A distro can turn it on or off. ... > i've periodically pushed for a noatime distro kernel for like ... 5-10 > years and last time this argument came up [i brought it up 6 months ago] > most of the distro kernel developer actually recommended using noatime, > but it

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Willy Tarreau
On Sun, Aug 05, 2007 at 02:58:47PM +0200, Ingo Molnar wrote: > > * Alan Cox <[EMAIL PROTECTED]> wrote: > > > > The only remotely valid compatibility argument would be Mutt - but even > > > that handles it just fine. (we broke way more software via noexec) > > > > And went through a sensible

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Diego Calleja
El Sun, 5 Aug 2007 09:13:20 +0200, Ingo Molnar <[EMAIL PROTECTED]> escribió: > Measurements show that noatime helps 20-30% on regular desktop > workloads, easily 50% for kernel builds and much more than that (in > excess of 100%) for file-read-intense workloads. We cannot just walk And as

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox <[EMAIL PROTECTED]> wrote: > > The only remotely valid compatibility argument would be Mutt - but even > > that handles it just fine. (we broke way more software via noexec) > > And went through a sensible process of resolving it. > > And its not just mutt. HSM stuff stops working

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox <[EMAIL PROTECTED]> wrote: > > > we can move from atime to noatime by default on FC8 with > > > appropriate release note warnings and having a couple of betas to > > > find out what other than mutt goes boom. > > > > btw., Mutt does not go boom, i use it myself. It works just fine

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox <[EMAIL PROTECTED]> wrote: > > you try to put the blame into distribution makers' shoes but in > > reality, had the kernel stepped forward with a neat .config option > > sooner (combined with a neat boot option as well to turn it off), > > we'd have had noatime systems 10 years

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Alan Cox
> > we can move from atime to noatime by default on FC8 with appropriate > > release note warnings and having a couple of betas to find out what > > other than mutt goes boom. > > btw., Mutt does not go boom, i use it myself. It works just fine and > notices new mails even on a

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Jakob Oestergaard <[EMAIL PROTECTED]> wrote: > > If you can show massive amounts of users that will actually be > > negatively impacted, please present hard evidence. > > > > Otherwise all this is useless hot air. > > Peace Jeff :) > > In another mail, I gave an example with tmpreaper

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Alan Cox
> The only remotely valid compatibility argument would be Mutt - but even > that handles it just fine. (we broke way more software via noexec) And went through a sensible process of resolving it. And its not just mutt. HSM stuff stops working which is a big deal as stuff clogs up. The /tmp/

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Alan Cox
> you try to put the blame into distribution makers' shoes but in reality, > had the kernel stepped forward with a neat .config option sooner > (combined with a neat boot option as well to turn it off), we'd have had > noatime systems 10 years ago. A new entry into relnotes and done. It's

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jakob Oestergaard
On Sun, Aug 05, 2007 at 06:42:30AM -0400, Jeff Garzik wrote: ... > If you can show massive amounts of users that will actually be > negatively impacted, please present hard evidence. > > Otherwise all this is useless hot air. Peace Jeff :) In another mail, I gave an example with tmpreaper

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jakob Oestergaard
On Sat, Aug 04, 2007 at 02:08:40PM -0400, Jeff Garzik wrote: > Linus Torvalds wrote: > >The "relatime" thing that David mentioned might well be very useful, but > >it's probably even less used than "noatime" is. And sadly, I don't really > >see that changing (unless we were to actually change

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jeff Garzik
Jakob Oestergaard wrote: Oh dear. Why not just make ext3 fsync() a no-op while you're at it? Distros can turn it back on if it's needed... Of course I'm not serious, but like atime, fsync() is something one No, they are nothing alike, and you are just making yourself look silly if you

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jakob Oestergaard
On Sun, Aug 05, 2007 at 09:28:05AM +0200, Ingo Molnar wrote: > > * Alan Cox <[EMAIL PROTECTED]> wrote: > > > > Can you give examples of backup solutions that rely on atime being > > > updated? I can understand backup tools using mtime/ctime for > > > incremental backups (like tar + Amanda,

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jeff Garzik
Ingo Molnar wrote: Distro makers did not dare to do this sooner because some kernel developers came forward with these mostly bogus arguments ... The impact of atime is far better understood by the kernel community, so it is the responsibility of _us_ to signal such things towards

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Willy Tarreau
On Sun, Aug 05, 2007 at 09:21:41AM +0200, Ingo Molnar wrote: > > * Alan Cox <[EMAIL PROTECTED]> wrote: > > > With a Red Hat on if we can move from /dev/hda to /dev/sda in FC7 then > > we can move from atime to noatime by default on FC8 with appropriate > > release note warnings and having a

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Florian Weimer
* Andrew Morton: >> XFS overwrites that data with zeros upon reboot, which tends to >> irritate users when it happens. > > yup. > >> >From this point of view, data=ordered doesn't seem too bad. > > If your computer is used by multiple users who don't trust each other, > sure. That covers, what?

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Andrew Morton <[EMAIL PROTECTED]> wrote: > On Sun, 5 Aug 2007 09:21:41 +0200 Ingo Molnar <[EMAIL PROTECTED]> wrote: > > > even on a noatime,nodiratime filesystem > > noatime is a superset of nodiratime, btw. heh, indeed. I've been using this trick for 10 years on my desktops so it's an

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox <[EMAIL PROTECTED]> wrote: > > it's also perhaps the most stupid Unix design idea of all times. > > Unix is really nice and well done, but think about this a bit: > > Think about the user for a moment instead. > > Do things right. The job of the kernel is not to "correct" for >

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Andrew Morton
On Sun, 5 Aug 2007 09:21:41 +0200 Ingo Molnar <[EMAIL PROTECTED]> wrote: > even on a noatime,nodiratime filesystem noatime is a superset of nodiratime, btw. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox <[EMAIL PROTECTED]> wrote: > > Can you give examples of backup solutions that rely on atime being > > updated? I can understand backup tools using mtime/ctime for > > incremental backups (like tar + Amanda, etc), but I'm having trouble > > figuring out why someone would want to

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox <[EMAIL PROTECTED]> wrote: > With a Red Hat on if we can move from /dev/hda to /dev/sda in FC7 then > we can move from atime to noatime by default on FC8 with appropriate > release note warnings and having a couple of betas to find out what > other than mutt goes boom. btw., Mutt

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox <[EMAIL PROTECTED]> wrote: > > Linux has always been a "POSIX unless its stupid" type of system. > > For the upstream kernel, we should do the right thing -- noatime by > > default -- but allow distros and people that care about rigid > > compliance to easily change the default. >

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox <[EMAIL PROTECTED]> wrote: > > > People just need to know about the performance differences - very > > > few realise its more than a fraction of a percent. I'm sure Gentoo > > > will use relatime the moment anyone knows its > 5% 8) > > > > noatime,nodiratime gave 50% of wall-clock

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Andrew Morton
On Sat, 04 Aug 2007 21:16:35 +0200 Florian Weimer <[EMAIL PROTECTED]> wrote: > * Andrew Morton: > > > The easy preventive is to mount with data=writeback. Maybe that should > > have been the default. > > The documentation I could find suggests that this may lead to a > security weakness (old

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Theodore Tso
On Sun, Aug 05, 2007 at 01:49:26AM +0100, Alan Cox wrote: HSM is the usual one, and to a large extent probably why Unix originally had atime. Basically migrating less used files away so as to keep the system disks tidy. Its not something usally found on desktop boxes so it doesn't in anyway

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Theodore Tso
On Sun, Aug 05, 2007 at 02:26:53AM +0200, Andi Kleen wrote: I always thought the right solution would be to just sync atime only very very lazily. This means if a inode is only dirty because of an atime update put it on a only write out when there is nothing to do or the memory is really

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Linus Torvalds
On Sun, 5 Aug 2007, Ingo Molnar wrote: you mean tmpwatch? The trivial change below fixes this. And with that we've come to the end of an extremely short list of atime dependencies. You wouldn't even need these kinds of games. What we could do is to make relatime updates a bit smarter. A

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Brice Figureau
Hi, Ingo Molnar mingo at elte.hu writes: * Linus Torvalds torvalds at linux-foundation.org wrote: On Fri, 3 Aug 2007, Peter Zijlstra wrote: These patches aim to improve balance_dirty_pages() and directly address three issues: 1) inter device starvation 2) stacked device

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Theodore Tso [EMAIL PROTECTED] wrote: If you are always reading from the same small set of files (i.e., a database workload), then those inodes only get updated every 5 seconds (the traditional/default metadata update sync time, as well as the default ext3 journal update time), it's no

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jeff Garzik
Ingo Molnar wrote: * Theodore Tso [EMAIL PROTECTED] wrote: If you are always reading from the same small set of files (i.e., a database workload), then those inodes only get updated every 5 seconds (the traditional/default metadata update sync time, as well as the default ext3 journal update

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Arjan van de Ven
On Sat, 2007-08-04 at 17:48 -0400, Theodore Tso wrote: On Sat, Aug 04, 2007 at 01:13:19PM -0700, Arjan van de Ven wrote: there is another trick possible (more involved though, Al will have to jump in on that one I suspect): Have 2 types of dirty inode states; one is the current dirty state

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Arjan van de Ven
On Sun, 2007-08-05 at 16:17 +0200, Jörn Engel wrote: On Sun, 5 August 2007 10:53:54 +0200, Willy Tarreau wrote: On Sun, Aug 05, 2007 at 09:21:41AM +0200, Ingo Molnar wrote: btw., Mutt does not go boom, i use it myself. It works just fine and notices new mails even on a

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox [EMAIL PROTECTED] wrote: And you honestly think that putting it in Kconfig as well as allowing users to screw up horribly and creating incompatible defaults you So far you've not offered one realistic scenario of screw up horribly. People have been using noatime for a long time

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Jeff Garzik [EMAIL PROTECTED] wrote: yeah, i didnt mean to say that it is _always_ a big issue, but only a small number of files are read is a very, very small minority of even the database server world. OTOH, consider a popular Linux task, web serving. atime results in a lot of

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Arjan van de Ven
In addition, big server boxes are usually not reading a huge *number* of files per second. The place where you see this as a problem is (a) compilation, thanks to huge /usr/include hierarchies (and here things have gotten worse over time as include files have gotten much more complex than

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jörn Engel
On Sun, 5 August 2007 11:02:33 -0700, Arjan van de Ven wrote: but does it work with relatime ? Like a greased penguin. I had to reboot with my ugly patch posted earlier in the patch to actually test it, though. Relatime suffers from a distribution problem, nothing else. Guess I should throw

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Dave Jones
On Sun, Aug 05, 2007 at 09:21:41AM +0200, Ingo Molnar wrote: * Alan Cox [EMAIL PROTECTED] wrote: With a Red Hat on if we can move from /dev/hda to /dev/sda in FC7 then we can move from atime to noatime by default on FC8 with appropriate release note warnings and having a couple of

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread adi
On Sun, Aug 05, 2007 at 02:44:08PM -0400, Dave Jones wrote: It still fails miserably for me. If I hit 'C' and '?' I get a list of my mail folders, with some of them marked 'N' if they have new mail. Without atime, those N's never show up and every mbox looks like it has no new mail. This

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Linus Torvalds [EMAIL PROTECTED] wrote: On Sun, 5 Aug 2007, Ingo Molnar wrote: you mean tmpwatch? The trivial change below fixes this. And with that we've come to the end of an extremely short list of atime dependencies. You wouldn't even need these kinds of games. What we could

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread david
On Sun, 5 Aug 2007, Diego Calleja wrote: El Sun, 5 Aug 2007 09:13:20 +0200, Ingo Molnar [EMAIL PROTECTED] escribió: Measurements show that noatime helps 20-30% on regular desktop workloads, easily 50% for kernel builds and much more than that (in excess of 100%) for file-read-intense

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Alan Cox
On Sun, 5 Aug 2007 20:08:26 +0200 Ingo Molnar [EMAIL PROTECTED] wrote: * Alan Cox [EMAIL PROTECTED] wrote: And you honestly think that putting it in Kconfig as well as allowing users to screw up horribly and creating incompatible defaults you So far you've not offered one realistic

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Alan Cox
change relatime updates to be performed once per day. This makes relatime a compatible solution for HSM, mailer-notification and tmpwatch applications too. Sweet also add the CONFIG_DEFAULT_RELATIME kernel option, which makes norelatime the default for all mounts without an extra kernel

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox [EMAIL PROTECTED] wrote: also add the CONFIG_DEFAULT_RELATIME kernel option, which makes norelatime the default for all mounts without an extra kernel boot option. Should be a mount option. it is already a mount option too. + relatime[FS] default to enabled

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jörn Engel
On Sun, 5 August 2007 20:37:14 +0200, Jörn Engel wrote: Guess I should throw in a kernel compile test as well, just to get a feel for the performance. Three runs each of noatime, relatime and atime, both with cold caches and with warm caches. Scripts below. Run on a Thinkpad T40, 1.5GHz,

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Christoph Hellwig
On Sun, Aug 05, 2007 at 11:01:18AM -0700, Arjan van de Ven wrote: on the journalling side this would be one transaction (not 5 milion) and... since inodes are grouped on disk, you can even get some better coalescing this way... Wonder if we could do inode-grouping smartly; eg if we HAVE

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Christoph Hellwig
On Sat, Aug 04, 2007 at 09:42:59PM +0200, J??rn Engel wrote: On Sat, 4 August 2007 21:26:15 +0200, J??rn Engel wrote: Given the choice between only atime and noatime I'd agree with you. Heck, I use it myself. But relatime seems to combine the best of both worlds. It currently just

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Andrew Morton
On Sun, 5 Aug 2007 22:21:12 +0200 Jörn Engel [EMAIL PROTECTED] wrote: On Sun, 5 August 2007 20:37:14 +0200, Jörn Engel wrote: Guess I should throw in a kernel compile test as well, just to get a feel for the performance. Three runs each of noatime, relatime and atime, both with cold

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Christoph Hellwig
On Sun, Aug 05, 2007 at 02:26:53AM +0200, Andi Kleen wrote: I always thought the right solution would be to just sync atime only very very lazily. This means if a inode is only dirty because of an atime update put it on a only write out when there is nothing to do or the memory is really

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Christoph Hellwig
On Sun, Aug 05, 2007 at 09:57:02AM +0200, Florian Weimer wrote: For instance, some editors don't perform fsync-then-rename, but simply truncate the file when saving (because they want to preserve hard links). With XFS, this tends to cause null bytes on crashes. Since ext3 has got a much

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Andi Kleen
Brice Figureau [EMAIL PROTECTED] writes: 2) I _still_ don't get the performances of 2.6.17, but since that's the better combination I could get, I think there is IMHO progress in the right direction (to be compared to no progress since 2.6.18, that's better :-)). If you could characterize

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Theodore Tso
On Sat, Aug 04, 2007 at 09:16:35PM +0200, Florian Weimer wrote: * Andrew Morton: The easy preventive is to mount with data=writeback. Maybe that should have been the default. The documentation I could find suggests that this may lead to a security weakness (old data in blocks of a file

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread David Chinner
On Sun, Aug 05, 2007 at 06:42:30AM -0400, Jeff Garzik wrote: Jakob Oestergaard wrote: Oh dear. Why not just make ext3 fsync() a no-op while you're at it? Distros can turn it back on if it's needed... Of course I'm not serious, but like atime, fsync() is something one No, they are

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread David Chinner
On Sat, Aug 04, 2007 at 09:16:35PM +0200, Florian Weimer wrote: * Andrew Morton: The easy preventive is to mount with data=writeback. Maybe that should have been the default. The documentation I could find suggests that this may lead to a security weakness (old data in blocks of a file

[RFC] VFS: mnotify (was: [PATCH 00/23] per device dirty throttling -v8)

2007-08-05 Thread Al Boldi
Jakob Oestergaard wrote: Why on earth would you cripple the kernel defaults for ext3 (which is a fine FS for boot/root filesystems), when the *fundamental* problem you really want to solve lie much deeper in the implementation of the filesystem? Noatime doesn't solve the problem, it just

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Andrew Morton
On Sat, 04 Aug 2007 21:16:35 +0200 Florian Weimer [EMAIL PROTECTED] wrote: * Andrew Morton: The easy preventive is to mount with data=writeback. Maybe that should have been the default. The documentation I could find suggests that this may lead to a security weakness (old data in

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox [EMAIL PROTECTED] wrote: People just need to know about the performance differences - very few realise its more than a fraction of a percent. I'm sure Gentoo will use relatime the moment anyone knows its 5% 8) noatime,nodiratime gave 50% of wall-clock kernel rpm build

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox [EMAIL PROTECTED] wrote: Linux has always been a POSIX unless its stupid type of system. For the upstream kernel, we should do the right thing -- noatime by default -- but allow distros and people that care about rigid compliance to easily change the default. Linux has

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox [EMAIL PROTECTED] wrote: With a Red Hat on if we can move from /dev/hda to /dev/sda in FC7 then we can move from atime to noatime by default on FC8 with appropriate release note warnings and having a couple of betas to find out what other than mutt goes boom. btw., Mutt does

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox [EMAIL PROTECTED] wrote: Can you give examples of backup solutions that rely on atime being updated? I can understand backup tools using mtime/ctime for incremental backups (like tar + Amanda, etc), but I'm having trouble figuring out why someone would want to use atime

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Andrew Morton
On Sun, 5 Aug 2007 09:21:41 +0200 Ingo Molnar [EMAIL PROTECTED] wrote: even on a noatime,nodiratime filesystem noatime is a superset of nodiratime, btw. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Alan Cox [EMAIL PROTECTED] wrote: it's also perhaps the most stupid Unix design idea of all times. Unix is really nice and well done, but think about this a bit: Think about the user for a moment instead. Do things right. The job of the kernel is not to correct for distribution

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Ingo Molnar
* Andrew Morton [EMAIL PROTECTED] wrote: On Sun, 5 Aug 2007 09:21:41 +0200 Ingo Molnar [EMAIL PROTECTED] wrote: even on a noatime,nodiratime filesystem noatime is a superset of nodiratime, btw. heh, indeed. I've been using this trick for 10 years on my desktops so it's an ancient

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Florian Weimer
* Andrew Morton: XFS overwrites that data with zeros upon reboot, which tends to irritate users when it happens. yup. From this point of view, data=ordered doesn't seem too bad. If your computer is used by multiple users who don't trust each other, sure. That covers, what? About 2% of

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Willy Tarreau
On Sun, Aug 05, 2007 at 09:21:41AM +0200, Ingo Molnar wrote: * Alan Cox [EMAIL PROTECTED] wrote: With a Red Hat on if we can move from /dev/hda to /dev/sda in FC7 then we can move from atime to noatime by default on FC8 with appropriate release note warnings and having a couple of

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jeff Garzik
Ingo Molnar wrote: Distro makers did not dare to do this sooner because some kernel developers came forward with these mostly bogus arguments ... The impact of atime is far better understood by the kernel community, so it is the responsibility of _us_ to signal such things towards

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jakob Oestergaard
On Sun, Aug 05, 2007 at 09:28:05AM +0200, Ingo Molnar wrote: * Alan Cox [EMAIL PROTECTED] wrote: Can you give examples of backup solutions that rely on atime being updated? I can understand backup tools using mtime/ctime for incremental backups (like tar + Amanda, etc), but I'm

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-05 Thread Jeff Garzik
Jakob Oestergaard wrote: Oh dear. Why not just make ext3 fsync() a no-op while you're at it? Distros can turn it back on if it's needed... Of course I'm not serious, but like atime, fsync() is something one No, they are nothing alike, and you are just making yourself look silly if you

<    1   2   3   4   >