Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-16 Thread Dave Jones
On Sun, Aug 05, 2007 at 11:00:29AM -0400, Theodore Tso wrote: > P.S. Yet alternative is to specify noatime on an individual > file/directory basis. We've had this capability for a *long* time, > and if a distro were to set noatime for all files in certain > hierarchies (i.e., /usr/include)

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-16 Thread Helge Hafting
Andi Kleen wrote: I always thought the right solution would be to just sync atime only very very lazily. This means if a inode is only dirty because of an atime update put it on a "only write out when there is nothing to do or the memory is really needed" list. Seems like a good idea.

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-16 Thread Helge Hafting
Andi Kleen wrote: I always thought the right solution would be to just sync atime only very very lazily. This means if a inode is only dirty because of an atime update put it on a only write out when there is nothing to do or the memory is really needed list. Seems like a good idea. atimes

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-16 Thread Dave Jones
On Sun, Aug 05, 2007 at 11:00:29AM -0400, Theodore Tso wrote: P.S. Yet alternative is to specify noatime on an individual file/directory basis. We've had this capability for a *long* time, and if a distro were to set noatime for all files in certain hierarchies (i.e., /usr/include) and

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-14 Thread Helge Hafting
Christoph Hellwig wrote: Umm, no f**king way. atime selection is 100% policy and belongs into userspace. Add to that the problem that we can't actually re-enable atimes because of the way the vfs-level mount flags API is designed. Instead of doing such a fugly kernel patch just talk to the

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-14 Thread Brice Figureau
On Tue, 2007-08-14 at 04:25 +0200, Andi Kleen wrote: > On Tue, Aug 14, 2007 at 11:44:56AM +1000, Stewart Smith wrote: > > > Since the database fits in RAM, the only kind of access Mysql is doing > > > is writing to the innodb log, the mysql binlog and finally to the innodb > > > database files. >

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-14 Thread Brice Figureau
On Tue, 2007-08-14 at 04:25 +0200, Andi Kleen wrote: On Tue, Aug 14, 2007 at 11:44:56AM +1000, Stewart Smith wrote: Since the database fits in RAM, the only kind of access Mysql is doing is writing to the innodb log, the mysql binlog and finally to the innodb database files. There are

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-14 Thread Helge Hafting
Christoph Hellwig wrote: Umm, no f**king way. atime selection is 100% policy and belongs into userspace. Add to that the problem that we can't actually re-enable atimes because of the way the vfs-level mount flags API is designed. Instead of doing such a fugly kernel patch just talk to the

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-13 Thread Andi Kleen
On Tue, Aug 14, 2007 at 11:44:56AM +1000, Stewart Smith wrote: > > Since the database fits in RAM, the only kind of access Mysql is doing > > is writing to the innodb log, the mysql binlog and finally to the innodb > > database files. > > There are certainly a whole lot of fsync'ing happening. >

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-13 Thread Stewart Smith
On Mon, 2007-08-06 at 10:40 +0200, Brice Figureau wrote: > Mysql accesses its database files in O_DIRECT mode. binlog is written using buffered IO. for InnoDB, binlog is synced first, then innodb log. on restart (in 5.0) these are synced back up so you don't get inconsistencies. and from a

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-13 Thread Stewart Smith
On Mon, 2007-08-06 at 10:40 +0200, Brice Figureau wrote: Mysql accesses its database files in O_DIRECT mode. binlog is written using buffered IO. for InnoDB, binlog is synced first, then innodb log. on restart (in 5.0) these are synced back up so you don't get inconsistencies. and from a quick

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-13 Thread Andi Kleen
On Tue, Aug 14, 2007 at 11:44:56AM +1000, Stewart Smith wrote: Since the database fits in RAM, the only kind of access Mysql is doing is writing to the innodb log, the mysql binlog and finally to the innodb database files. There are certainly a whole lot of fsync'ing happening. yes.

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-11 Thread Valerie Henson
On Wed, Aug 08, 2007 at 05:54:57PM -0700, Martin Bligh wrote: > Andrew Morton wrote: > >On Wed, 08 Aug 2007 14:10:15 -0700 > >"Martin J. Bligh" <[EMAIL PROTECTED]> wrote: > > > >>Why isn't this easily fixable by just adding an additional dirty > >>flag that says atime has changed? Then we only

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-11 Thread Valerie Henson
On Wed, Aug 08, 2007 at 05:54:57PM -0700, Martin Bligh wrote: Andrew Morton wrote: On Wed, 08 Aug 2007 14:10:15 -0700 Martin J. Bligh [EMAIL PROTECTED] wrote: Why isn't this easily fixable by just adding an additional dirty flag that says atime has changed? Then we only cause a write when

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-10 Thread Valdis . Kletnieks
On Fri, 10 Aug 2007 00:04:45 EDT, Bill Davidsen said: > > I never imagined that itwas the 20%+ hit that is being described, and > > with so little impact, or I would have switched to it across the board > > years ago. > > > To get that magnitude you need slow disk with very fast CPU. It helps

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-10 Thread pointman
Updating the manual page mount(8) with an expanded description of atime/noatime and adding nodirtime and data= seems much more reasonable than hacking the kernel because you want others to run their systems the way you think they should. Almost every web search of "linux fast disk" (or related

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-10 Thread pointman
Updating the manual page mount(8) with an expanded description of atime/noatime and adding nodirtime and data=option seems much more reasonable than hacking the kernel because you want others to run their systems the way you think they should. Almost every web search of linux fast disk (or

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-10 Thread Valdis . Kletnieks
On Fri, 10 Aug 2007 00:04:45 EDT, Bill Davidsen said: I never imagined that itwas the 20%+ hit that is being described, and with so little impact, or I would have switched to it across the board years ago. To get that magnitude you need slow disk with very fast CPU. It helps most of

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-09 Thread Bill Davidsen
Andi Kleen wrote: richard kennedy <[EMAIL PROTECTED]> writes: This is on a standard desktop machine so there are lots of other processes running on it, and although there is a degree of variability in the numbers,they are very repeatable and your patch always out performs the stock mm2. looks

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-09 Thread Bill Davidsen
[EMAIL PROTECTED] wrote: On Sun, 5 Aug 2007, Diego Calleja wrote: El Sun, 5 Aug 2007 09:13:20 +0200, Ingo Molnar <[EMAIL PROTECTED]> escribió: Measurements show that noatime helps 20-30% on regular desktop workloads, easily 50% for kernel builds and much more than that (in excess of 100%)

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-09 Thread Bill Davidsen
Andrew Morton wrote: On Wed, 08 Aug 2007 14:10:15 -0700 "Martin J. Bligh" <[EMAIL PROTECTED]> wrote: Why isn't this easily fixable by just adding an additional dirty flag that says atime has changed? Then we only cause a write when we remove the inode from the inode cache, if only atime is

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-09 Thread Diego Calleja
El Thu, 09 Aug 2007 11:02:38 -0400, Chuck Ebbert <[EMAIL PROTECTED]> escribió: > NT maintains atimes by default, at least up to XP. You have to edit the > registry to turn them off, and it is a single global switch -- not per > mountpoint like Unix. > > And it makes a huge difference there, too.

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-09 Thread Chuck Ebbert
On 08/09/2007 02:25 AM, Lionel Elie Mamane wrote: > >> yeah, it's really ugly. But otherwise i've got no real complaint >> about ext3 - with the obligatory qualification that >> "noatime,nodiratime" in /etc/fstab is a must. This speeds up things >> very visibly (...). So for most file workloads

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-09 Thread Lionel Elie Mamane
On Sat, Aug 04, 2007 at 06:37:33PM +0200, Ingo Molnar wrote: > * Linus Torvalds <[EMAIL PROTECTED]> wrote: >> The fact is, ext3 *sucks* at fsync. I hate hate hate it. It's >> totally unusable, imnsho. > yeah, it's really ugly. But otherwise i've got no real complaint > about ext3 - with the

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-09 Thread Lionel Elie Mamane
On Sat, Aug 04, 2007 at 06:37:33PM +0200, Ingo Molnar wrote: * Linus Torvalds [EMAIL PROTECTED] wrote: The fact is, ext3 *sucks* at fsync. I hate hate hate it. It's totally unusable, imnsho. yeah, it's really ugly. But otherwise i've got no real complaint about ext3 - with the obligatory

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-09 Thread Chuck Ebbert
On 08/09/2007 02:25 AM, Lionel Elie Mamane wrote: yeah, it's really ugly. But otherwise i've got no real complaint about ext3 - with the obligatory qualification that noatime,nodiratime in /etc/fstab is a must. This speeds up things very visibly (...). So for most file workloads we give

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-09 Thread Diego Calleja
El Thu, 09 Aug 2007 11:02:38 -0400, Chuck Ebbert [EMAIL PROTECTED] escribió: NT maintains atimes by default, at least up to XP. You have to edit the registry to turn them off, and it is a single global switch -- not per mountpoint like Unix. And it makes a huge difference there, too. In

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-09 Thread Bill Davidsen
Andrew Morton wrote: On Wed, 08 Aug 2007 14:10:15 -0700 Martin J. Bligh [EMAIL PROTECTED] wrote: Why isn't this easily fixable by just adding an additional dirty flag that says atime has changed? Then we only cause a write when we remove the inode from the inode cache, if only atime is

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-09 Thread Bill Davidsen
Andi Kleen wrote: richard kennedy [EMAIL PROTECTED] writes: This is on a standard desktop machine so there are lots of other processes running on it, and although there is a degree of variability in the numbers,they are very repeatable and your patch always out performs the stock mm2. looks

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-09 Thread Bill Davidsen
[EMAIL PROTECTED] wrote: On Sun, 5 Aug 2007, Diego Calleja wrote: El Sun, 5 Aug 2007 09:13:20 +0200, Ingo Molnar [EMAIL PROTECTED] escribió: Measurements show that noatime helps 20-30% on regular desktop workloads, easily 50% for kernel builds and much more than that (in excess of 100%) for

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread david
On Sat, 4 Aug 2007, Ray Lee wrote: On 8/4/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: On Sat, 4 Aug 2007, Ingo Molnar wrote: At least on a surface level, your report has some similarities to http://lkml.org/lkml/2007/5/21/84 . In that message, John Miller mentions several things he

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Andi Kleen
Greg Trounson <[EMAIL PROTECTED]> writes: > mount [fs] -o remount,noatime,nodiratime nodiratime is implied in noatime. > I get a compile time of 1m23.368s, a mere 6% improvement. 6% is nothing to sneeze at. A lot of optimizations would kill for less -Andi - To unsubscribe from this list: send

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread david
On Thu, 9 Aug 2007, Greg Trounson wrote: Measurements show that noatime helps 20-30% on regular desktop workloads, easily 50% for kernel builds and much more than that (in excess of 100%) for file-read-intense workloads. We cannot just walk past such a _huge_ performance impact so easily

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Greg Trounson
Ingo Molnar wrote: * Alan Cox <[EMAIL PROTECTED]> wrote: People just need to know about the performance differences - very few realise its more than a fraction of a percent. I'm sure Gentoo will use relatime the moment anyone knows its > 5% 8) noatime,nodiratime gave 50% of wall-clock kernel

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Martin Bligh
Andrew Morton wrote: On Wed, 08 Aug 2007 14:10:15 -0700 "Martin J. Bligh" <[EMAIL PROTECTED]> wrote: Why isn't this easily fixable by just adding an additional dirty flag that says atime has changed? Then we only cause a write when we remove the inode from the inode cache, if only atime is

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Alan Cox
On Wed, 08 Aug 2007 15:39:52 -0400 Jeff Garzik <[EMAIL PROTECTED]> wrote: > Bill Davidsen wrote: > > Being standards compliant is not an argument it's a design goal, a > > requirement. Standards compliance is like pregant, you are or you're > > Linux history says different. There was always

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Andrew Morton
On Wed, 08 Aug 2007 14:10:15 -0700 "Martin J. Bligh" <[EMAIL PROTECTED]> wrote: > Why isn't this easily fixable by just adding an additional dirty > flag that says atime has changed? Then we only cause a write > when we remove the inode from the inode cache, if only atime > is updated. I think

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Martin J. Bligh
Christoph Hellwig wrote: On Sat, Aug 04, 2007 at 09:42:59PM +0200, J??rn Engel wrote: On Sat, 4 August 2007 21:26:15 +0200, J??rn Engel wrote: Given the choice between only "atime" and "noatime" I'd agree with you. Heck, I use it myself. But "relatime" seems to combine the best of

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Bill Davidsen
Jeff Garzik wrote: Bill Davidsen wrote: Being standards compliant is not an argument it's a design goal, a requirement. Standards compliance is like pregant, you are or you're Linux history says different. There was always the "final 1%" of compliance that required silliness we really did

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Jeff Garzik
Bill Davidsen wrote: Being standards compliant is not an argument it's a design goal, a requirement. Standards compliance is like pregant, you are or you're Linux history says different. There was always the "final 1%" of compliance that required silliness we really did not want to bother

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Bill Davidsen
Ingo Molnar wrote: || ...For me, I would say 50% is not enough to describe the _visible_ || benefits... Not talking any specific number but past 10sec-1min+ || lagging in X is history, it's gone and I really don't miss it that || much... :-) Cannot reproduce even a second long delay anymore

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Bill Davidsen
Alan Cox wrote: However, relatime has the POSIX behavior without the overhead. Therefore No. relatime has approximately SuS behaviour. Its not the same as "correct" behaviour. Actually correct, but in terms of what can or does break, relatime seems a lot closer than noatime, I can't

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Andi Kleen
richard kennedy <[EMAIL PROTECTED]> writes: > > This is on a standard desktop machine so there are lots of other > processes running on it, and although there is a degree of variability > in the numbers,they are very repeatable and your patch always out > performs the stock mm2. > looks good to

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread richard kennedy
On Fri, 2007-08-03 at 14:37 +0200, Peter Zijlstra wrote: > Per device dirty throttling patches > > These patches aim to improve balance_dirty_pages() and directly address three > issues: > 1) inter device starvation > 2) stacked device deadlocks > 3) inter process starvation Hi Peter, I've

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Karel Zak
On Sat, Aug 04, 2007 at 07:17:24PM +0200, Ingo Molnar wrote: > > * Diego Calleja <[EMAIL PROTECTED]> wrote: > > > El Sat, 4 Aug 2007 18:37:33 +0200, Ingo Molnar <[EMAIL PROTECTED]> escribió: > > > > > thousands of applications. So for most file workloads we give > > > Windows a 20%-30%

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Karel Zak
On Sat, Aug 04, 2007 at 07:17:24PM +0200, Ingo Molnar wrote: * Diego Calleja [EMAIL PROTECTED] wrote: El Sat, 4 Aug 2007 18:37:33 +0200, Ingo Molnar [EMAIL PROTECTED] escribió: thousands of applications. So for most file workloads we give Windows a 20%-30% performance edge, for

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread richard kennedy
On Fri, 2007-08-03 at 14:37 +0200, Peter Zijlstra wrote: Per device dirty throttling patches These patches aim to improve balance_dirty_pages() and directly address three issues: 1) inter device starvation 2) stacked device deadlocks 3) inter process starvation snip Hi Peter, I've

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Andi Kleen
richard kennedy [EMAIL PROTECTED] writes: This is on a standard desktop machine so there are lots of other processes running on it, and although there is a degree of variability in the numbers,they are very repeatable and your patch always out performs the stock mm2. looks good to me iirc

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Bill Davidsen
Alan Cox wrote: However, relatime has the POSIX behavior without the overhead. Therefore No. relatime has approximately SuS behaviour. Its not the same as correct behaviour. Actually correct, but in terms of what can or does break, relatime seems a lot closer than noatime, I can't

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Bill Davidsen
Ingo Molnar wrote: || ...For me, I would say 50% is not enough to describe the _visible_ || benefits... Not talking any specific number but past 10sec-1min+ || lagging in X is history, it's gone and I really don't miss it that || much... :-) Cannot reproduce even a second long delay anymore

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Jeff Garzik
Bill Davidsen wrote: Being standards compliant is not an argument it's a design goal, a requirement. Standards compliance is like pregant, you are or you're Linux history says different. There was always the final 1% of compliance that required silliness we really did not want to bother

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Bill Davidsen
Jeff Garzik wrote: Bill Davidsen wrote: Being standards compliant is not an argument it's a design goal, a requirement. Standards compliance is like pregant, you are or you're Linux history says different. There was always the final 1% of compliance that required silliness we really did

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Martin J. Bligh
Christoph Hellwig wrote: On Sat, Aug 04, 2007 at 09:42:59PM +0200, J??rn Engel wrote: On Sat, 4 August 2007 21:26:15 +0200, J??rn Engel wrote: Given the choice between only atime and noatime I'd agree with you. Heck, I use it myself. But relatime seems to combine the best of both

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Andrew Morton
On Wed, 08 Aug 2007 14:10:15 -0700 Martin J. Bligh [EMAIL PROTECTED] wrote: Why isn't this easily fixable by just adding an additional dirty flag that says atime has changed? Then we only cause a write when we remove the inode from the inode cache, if only atime is updated. I think that

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Alan Cox
On Wed, 08 Aug 2007 15:39:52 -0400 Jeff Garzik [EMAIL PROTECTED] wrote: Bill Davidsen wrote: Being standards compliant is not an argument it's a design goal, a requirement. Standards compliance is like pregant, you are or you're Linux history says different. There was always the final

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Martin Bligh
Andrew Morton wrote: On Wed, 08 Aug 2007 14:10:15 -0700 Martin J. Bligh [EMAIL PROTECTED] wrote: Why isn't this easily fixable by just adding an additional dirty flag that says atime has changed? Then we only cause a write when we remove the inode from the inode cache, if only atime is

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Greg Trounson
Ingo Molnar wrote: * Alan Cox [EMAIL PROTECTED] wrote: People just need to know about the performance differences - very few realise its more than a fraction of a percent. I'm sure Gentoo will use relatime the moment anyone knows its 5% 8) noatime,nodiratime gave 50% of wall-clock kernel rpm

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread david
On Thu, 9 Aug 2007, Greg Trounson wrote: Measurements show that noatime helps 20-30% on regular desktop workloads, easily 50% for kernel builds and much more than that (in excess of 100%) for file-read-intense workloads. We cannot just walk past such a _huge_ performance impact so easily

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread Andi Kleen
Greg Trounson [EMAIL PROTECTED] writes: mount [fs] -o remount,noatime,nodiratime nodiratime is implied in noatime. I get a compile time of 1m23.368s, a mere 6% improvement. 6% is nothing to sneeze at. A lot of optimizations would kill for less -Andi - To unsubscribe from this list: send the

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-08 Thread david
On Sat, 4 Aug 2007, Ray Lee wrote: On 8/4/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: On Sat, 4 Aug 2007, Ingo Molnar wrote: At least on a surface level, your report has some similarities to http://lkml.org/lkml/2007/5/21/84 . In that message, John Miller mentions several things he tried

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-07 Thread Bill Davidsen
Claudio Martins wrote: On Saturday 04 August 2007, Alan Cox wrote: Linux has never been a "suprise your kernel interfaces all just changed today" kernel, nor a "gosh you upgraded and didn't notice your backups broke" kernel. Can you give examples of backup solutions that rely on atime being

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-07 Thread Alan Cox
> However, relatime has the POSIX behavior without the overhead. Therefore No. relatime has approximately SuS behaviour. Its not the same as "correct" behaviour. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-07 Thread Bill Davidsen
Alan Cox wrote: i cannot over-emphasise how much of a deal it is in practice. Atime updates are by far the biggest IO performance deficiency that Linux has today. Getting rid of atime updates would give us more everyday Linux performance than all the pagecache speedups of the past 10 years,

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-07 Thread Bill Davidsen
Jeff Garzik wrote: Alan Cox wrote: In some setups it will and in others it won't. Nor is it the only application that has this requirement. Ext3 currently is a standards compliant file system. Turn off atime and its very non standards compliant, turn to relatime and its not standards compliant

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-07 Thread Ingo Molnar
* Chuck Ebbert <[EMAIL PROTECTED]> wrote: > > Ingo's latest 'not quite noatime' seems to cure mutt/tmpwatch so it > > might finally make sense to do so. > > Do we report max(ctime, mtime) as the atime by default when noatime is > set or do we still need that to be done? noatime is unchanged

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-07 Thread Ingo Molnar
* Chuck Ebbert [EMAIL PROTECTED] wrote: Ingo's latest 'not quite noatime' seems to cure mutt/tmpwatch so it might finally make sense to do so. Do we report max(ctime, mtime) as the atime by default when noatime is set or do we still need that to be done? noatime is unchanged by my

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-07 Thread Bill Davidsen
Jeff Garzik wrote: Alan Cox wrote: In some setups it will and in others it won't. Nor is it the only application that has this requirement. Ext3 currently is a standards compliant file system. Turn off atime and its very non standards compliant, turn to relatime and its not standards compliant

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-07 Thread Bill Davidsen
Alan Cox wrote: i cannot over-emphasise how much of a deal it is in practice. Atime updates are by far the biggest IO performance deficiency that Linux has today. Getting rid of atime updates would give us more everyday Linux performance than all the pagecache speedups of the past 10 years,

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-07 Thread Alan Cox
However, relatime has the POSIX behavior without the overhead. Therefore No. relatime has approximately SuS behaviour. Its not the same as correct behaviour. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-07 Thread Bill Davidsen
Claudio Martins wrote: On Saturday 04 August 2007, Alan Cox wrote: Linux has never been a suprise your kernel interfaces all just changed today kernel, nor a gosh you upgraded and didn't notice your backups broke kernel. Can you give examples of backup solutions that rely on atime being

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Miklos Szeredi
> Per device dirty throttling patches Andrew, may I inquire about your plans with this? > These patches aim to improve balance_dirty_pages() and directly address three > issues: > 1) inter device starvation > 2) stacked device deadlocks This one interests me most, due to various real life,

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Chuck Ebbert
On 08/06/2007 03:37 PM, Alan Cox wrote: >> We already tried that here. The response: "If noatime is so great, why >> isn't it the default in the kernel?" > > Ok so we have a pile of people @redhat.com sitting on linux-kernel > complaining about Red Hat distributions not taking it up. Guys - can >

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Alan Cox
> We already tried that here. The response: "If noatime is so great, why > isn't it the default in the kernel?" Ok so we have a pile of people @redhat.com sitting on linux-kernel complaining about Red Hat distributions not taking it up. Guys - can we just fix it internally please like sensible

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Jeff Garzik
Chuck Ebbert wrote: On 08/05/2007 04:36 PM, Christoph Hellwig wrote: Umm, no f**king way. atime selection is 100% policy and belongs into userspace. Add to that the problem that we can't actually re-enable atimes because of the way the vfs-level mount flags API is designed. Instead of doing

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Chuck Ebbert
On 08/05/2007 04:36 PM, Christoph Hellwig wrote: > > Umm, no f**king way. atime selection is 100% policy and belongs into > userspace. Add to that the problem that we can't actually re-enable > atimes because of the way the vfs-level mount flags API is designed. > Instead of doing such a fugly

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Ingo Molnar
* Dave Jones <[EMAIL PROTECTED]> wrote: > > does it work with the "atime on steroids" patch below? (no need to > > configure anything, just apply the patch and go.) > > people have reported that relatime does work, but my util-linux isn't > new enough to support it, so I've never got it to

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Dave Jones
On Mon, Aug 06, 2007 at 08:39:09AM +0200, Ingo Molnar wrote: > > * Dave Jones <[EMAIL PROTECTED]> wrote: > > > > btw., Mutt does not go boom, i use it myself. It works just fine > > > and notices new mails even on a noatime,nodiratime filesystem. > > > > It still fails miserably for

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Chris Mason
On Sun, 5 Aug 2007 11:00:29 -0400 Theodore Tso <[EMAIL PROTECTED]> wrote: > On Sun, Aug 05, 2007 at 02:26:53AM +0200, Andi Kleen wrote: > > I always thought the right solution would be to just sync atime only > > very very lazily. This means if a inode is only dirty because of an > > atime update

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Willy Tarreau
On Mon, Aug 06, 2007 at 08:57:12AM +0200, Ingo Molnar wrote: > > * Willy Tarreau <[EMAIL PROTECTED]> wrote: > > > In your example above, maybe it's the opposite, users know they can > > keep a file in /tmp one more week by simply cat'ing it. > > sure - and i'm not arguing that noatime should

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Andi Kleen
On Sun, Aug 05, 2007 at 09:41:12PM +0100, Christoph Hellwig wrote: > On Sun, Aug 05, 2007 at 02:26:53AM +0200, Andi Kleen wrote: > > I always thought the right solution would be to just sync atime only > > very very lazily. This means if a inode is only dirty because of an > > atime update put it

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Brice Figureau
Hi Andi, On Mon, 2007-08-06 at 00:17 +0200, Andi Kleen wrote: > Brice Figureau <[EMAIL PROTECTED]> writes: > > > > 2) I _still_ don't get the "performances" of 2.6.17, but since that's the > > better combination I could get, I think there is IMHO progress in the right > > direction (to be

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Ingo Molnar
* Diego Calleja <[EMAIL PROTECTED]> wrote: > > Measurements show that noatime helps 20-30% on regular desktop > > workloads, easily 50% for kernel builds and much more than that (in > > excess of 100%) for file-read-intense workloads. We cannot just walk > > And as everybody knows in servers

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Ingo Molnar
* Willy Tarreau <[EMAIL PROTECTED]> wrote: > In your example above, maybe it's the opposite, users know they can > keep a file in /tmp one more week by simply cat'ing it. sure - and i'm not arguing that noatime should the kernel-wide default. In every single patch i sent it was a .config

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Ingo Molnar
* [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > i've been a linux sysadmin for 10 years, and have known about noatime > for at least 7 years, but I always thought of it in the catagory of > 'use it only on your performance critical machines where you are > trying to extract every ounce of

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Ingo Molnar
* Dave Jones <[EMAIL PROTECTED]> wrote: > > btw., Mutt does not go boom, i use it myself. It works just fine > > and notices new mails even on a noatime,nodiratime filesystem. > > It still fails miserably for me. > > If I hit 'C' and '?' I get a list of my mail folders, with some of >

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Ingo Molnar
* Dave Jones [EMAIL PROTECTED] wrote: btw., Mutt does not go boom, i use it myself. It works just fine and notices new mails even on a noatime,nodiratime filesystem. It still fails miserably for me. If I hit 'C' and '?' I get a list of my mail folders, with some of them marked

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Ingo Molnar
* [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: i've been a linux sysadmin for 10 years, and have known about noatime for at least 7 years, but I always thought of it in the catagory of 'use it only on your performance critical machines where you are trying to extract every ounce of

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Ingo Molnar
* Willy Tarreau [EMAIL PROTECTED] wrote: In your example above, maybe it's the opposite, users know they can keep a file in /tmp one more week by simply cat'ing it. sure - and i'm not arguing that noatime should the kernel-wide default. In every single patch i sent it was a .config option

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Ingo Molnar
* Diego Calleja [EMAIL PROTECTED] wrote: Measurements show that noatime helps 20-30% on regular desktop workloads, easily 50% for kernel builds and much more than that (in excess of 100%) for file-read-intense workloads. We cannot just walk And as everybody knows in servers is a

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Brice Figureau
Hi Andi, On Mon, 2007-08-06 at 00:17 +0200, Andi Kleen wrote: Brice Figureau [EMAIL PROTECTED] writes: 2) I _still_ don't get the performances of 2.6.17, but since that's the better combination I could get, I think there is IMHO progress in the right direction (to be compared to no

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Andi Kleen
On Sun, Aug 05, 2007 at 09:41:12PM +0100, Christoph Hellwig wrote: On Sun, Aug 05, 2007 at 02:26:53AM +0200, Andi Kleen wrote: I always thought the right solution would be to just sync atime only very very lazily. This means if a inode is only dirty because of an atime update put it on a

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Willy Tarreau
On Mon, Aug 06, 2007 at 08:57:12AM +0200, Ingo Molnar wrote: * Willy Tarreau [EMAIL PROTECTED] wrote: In your example above, maybe it's the opposite, users know they can keep a file in /tmp one more week by simply cat'ing it. sure - and i'm not arguing that noatime should the

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Chris Mason
On Sun, 5 Aug 2007 11:00:29 -0400 Theodore Tso [EMAIL PROTECTED] wrote: On Sun, Aug 05, 2007 at 02:26:53AM +0200, Andi Kleen wrote: I always thought the right solution would be to just sync atime only very very lazily. This means if a inode is only dirty because of an atime update put it

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Dave Jones
On Mon, Aug 06, 2007 at 08:39:09AM +0200, Ingo Molnar wrote: * Dave Jones [EMAIL PROTECTED] wrote: btw., Mutt does not go boom, i use it myself. It works just fine and notices new mails even on a noatime,nodiratime filesystem. It still fails miserably for me. If I

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Ingo Molnar
* Dave Jones [EMAIL PROTECTED] wrote: does it work with the atime on steroids patch below? (no need to configure anything, just apply the patch and go.) people have reported that relatime does work, but my util-linux isn't new enough to support it, so I've never got it to work. I'll

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Chuck Ebbert
On 08/05/2007 04:36 PM, Christoph Hellwig wrote: Umm, no f**king way. atime selection is 100% policy and belongs into userspace. Add to that the problem that we can't actually re-enable atimes because of the way the vfs-level mount flags API is designed. Instead of doing such a fugly

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Jeff Garzik
Chuck Ebbert wrote: On 08/05/2007 04:36 PM, Christoph Hellwig wrote: Umm, no f**king way. atime selection is 100% policy and belongs into userspace. Add to that the problem that we can't actually re-enable atimes because of the way the vfs-level mount flags API is designed. Instead of doing

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Alan Cox
We already tried that here. The response: If noatime is so great, why isn't it the default in the kernel? Ok so we have a pile of people @redhat.com sitting on linux-kernel complaining about Red Hat distributions not taking it up. Guys - can we just fix it internally please like sensible folk ?

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Chuck Ebbert
On 08/06/2007 03:37 PM, Alan Cox wrote: We already tried that here. The response: If noatime is so great, why isn't it the default in the kernel? Ok so we have a pile of people @redhat.com sitting on linux-kernel complaining about Red Hat distributions not taking it up. Guys - can we just

Re: [PATCH 00/23] per device dirty throttling -v8

2007-08-06 Thread Miklos Szeredi
Per device dirty throttling patches Andrew, may I inquire about your plans with this? These patches aim to improve balance_dirty_pages() and directly address three issues: 1) inter device starvation 2) stacked device deadlocks This one interests me most, due to various real life,

  1   2   3   4   >