Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-29 Thread Eric St-Laurent
On Wed, 2007-25-07 at 17:09 +1000, Nick Piggin wrote: > Eric St-Laurent wrote: > > I test this on my main system, so patches with basic testing and > > reasonable stability are preferred. I just want to avoid data corruption > > bugs. FYI, I used to run the -rt tree most of the time. > > OK here

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-29 Thread Eric St-Laurent
On Wed, 2007-25-07 at 17:09 +1000, Nick Piggin wrote: Eric St-Laurent wrote: I test this on my main system, so patches with basic testing and reasonable stability are preferred. I just want to avoid data corruption bugs. FYI, I used to run the -rt tree most of the time. OK here is one

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-25 Thread Rik van Riel
Eric St-Laurent wrote: On Wed, 2007-25-07 at 17:09 +1000, Nick Piggin wrote: A new list could be a possibility. One problem with adding lists is just trying to work out how to balance scanning rates between them, another problem is CPU overhead of moving pages from one to another... Disk

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-25 Thread Rik van Riel
Nick Piggin wrote: Eric St-Laurent wrote: On Wed, 2007-25-07 at 15:19 +1000, Nick Piggin wrote: What *I* think is supposed to happen is that newly read in pages get put on the inactive list, and unless they get accessed againbefore being reclaimed, they are allowed to fall off the end of the

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-25 Thread Rik van Riel
Nick Piggin wrote: What *I* think is supposed to happen is that newly read in pages get put on the inactive list, and unless they get accessed againbefore being reclaimed, they are allowed to fall off the end of the list without disturbing active data too much. I think there is a missing piece

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-25 Thread Eric St-Laurent
On Wed, 2007-25-07 at 17:09 +1000, Nick Piggin wrote: > > A new list could be a possibility. One problem with adding lists is just > trying to work out how to balance scanning rates between them, another > problem is CPU overhead of moving pages from one to another... Disk sizes seem to

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-25 Thread Nick Piggin
Eric St-Laurent wrote: On Wed, 2007-25-07 at 15:19 +1000, Nick Piggin wrote: What *I* think is supposed to happen is that newly read in pages get put on the inactive list, and unless they get accessed againbefore being reclaimed, they are allowed to fall off the end of the list without

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-25 Thread Eric St-Laurent
On Wed, 2007-25-07 at 15:19 +1000, Nick Piggin wrote: > What *I* think is supposed to happen is that newly read in pages get > put on the inactive list, and unless they get accessed againbefore > being reclaimed, they are allowed to fall off the end of the list > without disturbing active data

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-25 Thread Eric St-Laurent
On Wed, 2007-25-07 at 15:19 +1000, Nick Piggin wrote: What *I* think is supposed to happen is that newly read in pages get put on the inactive list, and unless they get accessed againbefore being reclaimed, they are allowed to fall off the end of the list without disturbing active data too

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-25 Thread Nick Piggin
Eric St-Laurent wrote: On Wed, 2007-25-07 at 15:19 +1000, Nick Piggin wrote: What *I* think is supposed to happen is that newly read in pages get put on the inactive list, and unless they get accessed againbefore being reclaimed, they are allowed to fall off the end of the list without

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-25 Thread Eric St-Laurent
On Wed, 2007-25-07 at 17:09 +1000, Nick Piggin wrote: A new list could be a possibility. One problem with adding lists is just trying to work out how to balance scanning rates between them, another problem is CPU overhead of moving pages from one to another... Disk sizes seem to increase

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-25 Thread Rik van Riel
Nick Piggin wrote: What *I* think is supposed to happen is that newly read in pages get put on the inactive list, and unless they get accessed againbefore being reclaimed, they are allowed to fall off the end of the list without disturbing active data too much. I think there is a missing piece

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-25 Thread Rik van Riel
Nick Piggin wrote: Eric St-Laurent wrote: On Wed, 2007-25-07 at 15:19 +1000, Nick Piggin wrote: What *I* think is supposed to happen is that newly read in pages get put on the inactive list, and unless they get accessed againbefore being reclaimed, they are allowed to fall off the end of the

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-25 Thread Rik van Riel
Eric St-Laurent wrote: On Wed, 2007-25-07 at 17:09 +1000, Nick Piggin wrote: A new list could be a possibility. One problem with adding lists is just trying to work out how to balance scanning rates between them, another problem is CPU overhead of moving pages from one to another... Disk

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-24 Thread Nick Piggin
Eric St-Laurent wrote: On Mon, 2007-23-07 at 19:00 +1000, Nick Piggin wrote: I don't like this kind of conditional information going from something like readahead into page reclaim. Unless it is for readahead _specific_ data such as "I got these all wrong, so you can reclaim them" (which this

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-24 Thread Eric St-Laurent
On Mon, 2007-23-07 at 19:00 +1000, Nick Piggin wrote: > I don't like this kind of conditional information going from something > like readahead into page reclaim. Unless it is for readahead _specific_ > data such as "I got these all wrong, so you can reclaim them" (which > this isn't). > > But I

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-24 Thread Andreas Dilger
On Jul 23, 2007 18:17 -0700, Andrew Morton wrote: > hm, yes, there is a risk that the code was accidentally correct. Or the > code has only ever dealt with power-of-2 inputs, in which case it happens > to work either way. > > David(s) and ext4-people: could we please have a close review of

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-24 Thread Andreas Dilger
On Jul 23, 2007 18:17 -0700, Andrew Morton wrote: hm, yes, there is a risk that the code was accidentally correct. Or the code has only ever dealt with power-of-2 inputs, in which case it happens to work either way. David(s) and ext4-people: could we please have a close review of these

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-24 Thread Eric St-Laurent
On Mon, 2007-23-07 at 19:00 +1000, Nick Piggin wrote: I don't like this kind of conditional information going from something like readahead into page reclaim. Unless it is for readahead _specific_ data such as I got these all wrong, so you can reclaim them (which this isn't). But I don't

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-24 Thread Nick Piggin
Eric St-Laurent wrote: On Mon, 2007-23-07 at 19:00 +1000, Nick Piggin wrote: I don't like this kind of conditional information going from something like readahead into page reclaim. Unless it is for readahead _specific_ data such as I got these all wrong, so you can reclaim them (which this

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-23 Thread Nick Piggin
Fengguang Wu wrote: On Mon, Jul 23, 2007 at 12:40:09PM -0700, Andrew Morton wrote: This is all fun stuff, but how do we find out that changes like this are good ones, apart from shipping it and seeing who gets hurt 12 months later? One thing I can imagine now is that the first pages may

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-23 Thread Andrew Morton
On Tue, 24 Jul 2007 08:47:28 +0800 Fengguang Wu <[EMAIL PROTECTED]> wrote: > Subject: convert ill defined log2() to ilog2() > > It's *wrong* to have > #define log2(n) ffz(~(n)) > It should be *reversed*: > #define log2(n) flz(~(n)) > or >

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-23 Thread Fengguang Wu
On Mon, Jul 23, 2007 at 12:40:09PM -0700, Andrew Morton wrote: > On Mon, 23 Jul 2007 22:24:57 +0800 > Fengguang Wu <[EMAIL PROTECTED]> wrote: > > > On Mon, Jul 23, 2007 at 07:00:59PM +1000, Nick Piggin wrote: > > > Rusty Russell wrote: > > > >On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote:

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-23 Thread Andrew Morton
On Mon, 23 Jul 2007 22:24:57 +0800 Fengguang Wu <[EMAIL PROTECTED]> wrote: > On Mon, Jul 23, 2007 at 07:00:59PM +1000, Nick Piggin wrote: > > Rusty Russell wrote: > > >On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: > > > > >>So I opt for it being made tunable, safe, and turned off by

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-23 Thread Fengguang Wu
On Mon, Jul 23, 2007 at 07:00:59PM +1000, Nick Piggin wrote: > Rusty Russell wrote: > >On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: > > >>So I opt for it being made tunable, safe, and turned off by default. > > I hate tunables :) Unless we have workload A that gets a reasonable >

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-23 Thread Nick Piggin
Rusty Russell wrote: On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: So I opt for it being made tunable, safe, and turned off by default. I hate tunables :) Unless we have workload A that gets a reasonable benefit from something and workload B that gets a significant regression, and

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-23 Thread Nick Piggin
Rusty Russell wrote: On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: So I opt for it being made tunable, safe, and turned off by default. I hate tunables :) Unless we have workload A that gets a reasonable benefit from something and workload B that gets a significant regression, and

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-23 Thread Fengguang Wu
On Mon, Jul 23, 2007 at 07:00:59PM +1000, Nick Piggin wrote: Rusty Russell wrote: On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: So I opt for it being made tunable, safe, and turned off by default. I hate tunables :) Unless we have workload A that gets a reasonable benefit from

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-23 Thread Andrew Morton
On Mon, 23 Jul 2007 22:24:57 +0800 Fengguang Wu [EMAIL PROTECTED] wrote: On Mon, Jul 23, 2007 at 07:00:59PM +1000, Nick Piggin wrote: Rusty Russell wrote: On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: So I opt for it being made tunable, safe, and turned off by default. I

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-23 Thread Fengguang Wu
On Mon, Jul 23, 2007 at 12:40:09PM -0700, Andrew Morton wrote: On Mon, 23 Jul 2007 22:24:57 +0800 Fengguang Wu [EMAIL PROTECTED] wrote: On Mon, Jul 23, 2007 at 07:00:59PM +1000, Nick Piggin wrote: Rusty Russell wrote: On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: So I

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-23 Thread Andrew Morton
On Tue, 24 Jul 2007 08:47:28 +0800 Fengguang Wu [EMAIL PROTECTED] wrote: Subject: convert ill defined log2() to ilog2() It's *wrong* to have #define log2(n) ffz(~(n)) It should be *reversed*: #define log2(n) flz(~(n)) or

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-23 Thread Nick Piggin
Fengguang Wu wrote: On Mon, Jul 23, 2007 at 12:40:09PM -0700, Andrew Morton wrote: This is all fun stuff, but how do we find out that changes like this are good ones, apart from shipping it and seeing who gets hurt 12 months later? One thing I can imagine now is that the first pages may

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-22 Thread Al Boldi
Fengguang Wu wrote: > On Sun, Jul 22, 2007 at 10:24:37AM +0200, Peter Zijlstra wrote: > > On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: > > > - It will avoid large-file-reads-thrashing-my-desktop problem, > > > so most desktop users should like it. But sure there will be counter > > >

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-22 Thread Peter Zijlstra
On Sun, 2007-07-22 at 18:33 +1000, Rusty Russell wrote: > On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: > > - It will avoid large-file-reads-thrashing-my-desktop problem, > > so most desktop users should like it. But sure there will be counter > > cases when a user want to keep the

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-22 Thread Rusty Russell
On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: > - It will avoid large-file-reads-thrashing-my-desktop problem, > so most desktop users should like it. But sure there will be counter > cases when a user want to keep the data cached. > - File servers may hurt from it. Imagine a mp3/png

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-22 Thread Fengguang Wu
On Sun, Jul 22, 2007 at 10:24:37AM +0200, Peter Zijlstra wrote: > On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: > > > - It will avoid large-file-reads-thrashing-my-desktop problem, > > so most desktop users should like it. But sure there will be counter > > cases when a user want to

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-22 Thread Peter Zijlstra
On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: > - It will avoid large-file-reads-thrashing-my-desktop problem, > so most desktop users should like it. But sure there will be counter > cases when a user want to keep the data cached. > - File servers may hurt from it. Imagine a mp3/png

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-22 Thread Fengguang Wu
On Sat, Jul 21, 2007 at 10:44:28PM -0400, Dave Jones wrote: > On Sun, Jul 22, 2007 at 10:39:23AM +0800, Fengguang Wu wrote: > > > It makes sense to raise it beyond 128K. 1M default readahead > > absolutely makes sense for sequential workloads. For the desktop, > > this increases boot speed

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-22 Thread Fengguang Wu
On Sat, Jul 21, 2007 at 10:44:28PM -0400, Dave Jones wrote: On Sun, Jul 22, 2007 at 10:39:23AM +0800, Fengguang Wu wrote: It makes sense to raise it beyond 128K. 1M default readahead absolutely makes sense for sequential workloads. For the desktop, this increases boot speed and

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-22 Thread Peter Zijlstra
On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: - It will avoid large-file-reads-thrashing-my-desktop problem, so most desktop users should like it. But sure there will be counter cases when a user want to keep the data cached. - File servers may hurt from it. Imagine a mp3/png

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-22 Thread Fengguang Wu
On Sun, Jul 22, 2007 at 10:24:37AM +0200, Peter Zijlstra wrote: On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: - It will avoid large-file-reads-thrashing-my-desktop problem, so most desktop users should like it. But sure there will be counter cases when a user want to keep the

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-22 Thread Rusty Russell
On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: - It will avoid large-file-reads-thrashing-my-desktop problem, so most desktop users should like it. But sure there will be counter cases when a user want to keep the data cached. - File servers may hurt from it. Imagine a mp3/png file

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-22 Thread Peter Zijlstra
On Sun, 2007-07-22 at 18:33 +1000, Rusty Russell wrote: On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: - It will avoid large-file-reads-thrashing-my-desktop problem, so most desktop users should like it. But sure there will be counter cases when a user want to keep the data

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-22 Thread Al Boldi
Fengguang Wu wrote: On Sun, Jul 22, 2007 at 10:24:37AM +0200, Peter Zijlstra wrote: On Sun, 2007-07-22 at 16:10 +0800, Fengguang Wu wrote: - It will avoid large-file-reads-thrashing-my-desktop problem, so most desktop users should like it. But sure there will be counter cases when

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-21 Thread Dave Jones
On Sun, Jul 22, 2007 at 10:39:23AM +0800, Fengguang Wu wrote: > It makes sense to raise it beyond 128K. 1M default readahead > absolutely makes sense for sequential workloads. For the desktop, > this increases boot speed and readahead misses, both due to more > aggressive mmap read-around.

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-21 Thread Fengguang Wu
On Sat, Jul 21, 2007 at 11:00:05PM +0200, Peter Zijlstra wrote: > The various readahead bits I have lying about. > > Wu, would you agree with asking Andrew to stick these behind your latest > readahead series? They are generally good feature to have. Thank you! - default readahead size It

[PATCH 0/3] readahead drop behind and size adjustment

2007-07-21 Thread Peter Zijlstra
The various readahead bits I have lying about. Wu, would you agree with asking Andrew to stick these behind your latest readahead series? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at

[PATCH 0/3] readahead drop behind and size adjustment

2007-07-21 Thread Peter Zijlstra
The various readahead bits I have lying about. Wu, would you agree with asking Andrew to stick these behind your latest readahead series? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-21 Thread Fengguang Wu
On Sat, Jul 21, 2007 at 11:00:05PM +0200, Peter Zijlstra wrote: The various readahead bits I have lying about. Wu, would you agree with asking Andrew to stick these behind your latest readahead series? They are generally good feature to have. Thank you! - default readahead size It makes

Re: [PATCH 0/3] readahead drop behind and size adjustment

2007-07-21 Thread Dave Jones
On Sun, Jul 22, 2007 at 10:39:23AM +0800, Fengguang Wu wrote: It makes sense to raise it beyond 128K. 1M default readahead absolutely makes sense for sequential workloads. For the desktop, this increases boot speed and readahead misses, both due to more aggressive mmap read-around.