Re: [PATCH] mm: swap prefetch improvements

2007-05-23 Thread Ash Milsted
On Wed, 23 May 2007 08:50:01 +1000 Con Kolivas <[EMAIL PROTECTED]> wrote: > On Wednesday 23 May 2007 06:42, Ash Milsted wrote: > > Hi. I just did some video encoding on my desktop and I was noticing > > (for the first time in a while) that running apps had to hit swap quite > > a lot when I

Re: [PATCH] mm: swap prefetch improvements

2007-05-23 Thread Ash Milsted
On Wed, 23 May 2007 08:50:01 +1000 Con Kolivas [EMAIL PROTECTED] wrote: On Wednesday 23 May 2007 06:42, Ash Milsted wrote: Hi. I just did some video encoding on my desktop and I was noticing (for the first time in a while) that running apps had to hit swap quite a lot when I switched to

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Wednesday 23 May 2007 06:42, Ash Milsted wrote: > Hi. I just did some video encoding on my desktop and I was noticing > (for the first time in a while) that running apps had to hit swap quite > a lot when I switched to them (the encoding was going at full blast for > most of the day, and most

Re: [ck] Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ash Milsted
On Tue, 22 May 2007 20:37:54 +1000 Con Kolivas <[EMAIL PROTECTED]> wrote: > On Tuesday 22 May 2007 20:25, Ingo Molnar wrote: > > * Con Kolivas <[EMAIL PROTECTED]> wrote: > > > > > there was nothing else running on the system - so i suspect the > > > > > swapin activity flagged 'itself' as some

Re: [ck] Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Michael Chang <[EMAIL PROTECTED]> wrote: > > It clearly should not consider 'itself' as IO activity. This > > suggests some bug in the 'detect activity' mechanism, agreed? I'm > > wondering whether you are seeing the same problem, or is all > > swap-prefetch IO on your system continuous

Re: [ck] Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Michael Chang
On 5/22/07, Ingo Molnar <[EMAIL PROTECTED]> wrote: * Con Kolivas <[EMAIL PROTECTED]> wrote: > On Tuesday 22 May 2007 20:25, Ingo Molnar wrote: > > * Con Kolivas <[EMAIL PROTECTED]> wrote: > > > > > there was nothing else running on the system - so i suspect the > > > > > swapin activity

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Con Kolivas <[EMAIL PROTECTED]> wrote: > > hm, what do you call 'journal writeout' here that would be happening > > on my system? > > Not really sure what you have in terms of fs, but here even with > nothing going on, ext3 writes to disk every 5 seconds with kjournald. i have ext3, but

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Tuesday 22 May 2007 20:57, Ingo Molnar wrote: > * Con Kolivas <[EMAIL PROTECTED]> wrote: > > On Tuesday 22 May 2007 20:46, Ingo Molnar wrote: > > > It clearly should not consider 'itself' as IO activity. This > > > suggests some bug in the 'detect activity' mechanism, agreed? I'm > > >

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Con Kolivas <[EMAIL PROTECTED]> wrote: > On Tuesday 22 May 2007 20:46, Ingo Molnar wrote: > > It clearly should not consider 'itself' as IO activity. This > > suggests some bug in the 'detect activity' mechanism, agreed? I'm > > wondering whether you are seeing the same problem, or is all >

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Tuesday 22 May 2007 20:46, Ingo Molnar wrote: > It clearly should not consider 'itself' as IO activity. This suggests > some bug in the 'detect activity' mechanism, agreed? I'm wondering > whether you are seeing the same problem, or is all swap-prefetch IO on > your system continuous until it's

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Con Kolivas <[EMAIL PROTECTED]> wrote: > On Tuesday 22 May 2007 20:25, Ingo Molnar wrote: > > * Con Kolivas <[EMAIL PROTECTED]> wrote: > > > > > there was nothing else running on the system - so i suspect the > > > > > swapin activity flagged 'itself' as some 'other' activity and > > > > >

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Tuesday 22 May 2007 20:25, Ingo Molnar wrote: > * Con Kolivas <[EMAIL PROTECTED]> wrote: > > > > there was nothing else running on the system - so i suspect the > > > > swapin activity flagged 'itself' as some 'other' activity and > > > > stopped? The swapins happened in 4 bursts, separated by

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Con Kolivas <[EMAIL PROTECTED]> wrote: > > > there was nothing else running on the system - so i suspect the > > > swapin activity flagged 'itself' as some 'other' activity and > > > stopped? The swapins happened in 4 bursts, separated by 5 seconds > > > total idleness. > > > > I've noted

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Tuesday 22 May 2007 20:15, Antonino Ingargiola wrote: > 2007/5/21, Ingo Molnar <[EMAIL PROTECTED]>: > > * Con Kolivas <[EMAIL PROTECTED]> wrote: > > > > A suggestion for improvement: right now swap-prefetch does a small > > > > bit of swapin every 5 seconds and stays idle inbetween. Could this

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Antonino Ingargiola
2007/5/21, Ingo Molnar <[EMAIL PROTECTED]>: * Con Kolivas <[EMAIL PROTECTED]> wrote: > > A suggestion for improvement: right now swap-prefetch does a small > > bit of swapin every 5 seconds and stays idle inbetween. Could this > > perhaps be made more agressive (optionally perhaps), if the

Re: [ck] Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Michael Chang
On 5/22/07, Ingo Molnar [EMAIL PROTECTED] wrote: * Con Kolivas [EMAIL PROTECTED] wrote: On Tuesday 22 May 2007 20:25, Ingo Molnar wrote: * Con Kolivas [EMAIL PROTECTED] wrote: there was nothing else running on the system - so i suspect the swapin activity flagged 'itself' as some

Re: [ck] Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Michael Chang [EMAIL PROTECTED] wrote: It clearly should not consider 'itself' as IO activity. This suggests some bug in the 'detect activity' mechanism, agreed? I'm wondering whether you are seeing the same problem, or is all swap-prefetch IO on your system continuous until it's

Re: [ck] Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ash Milsted
On Tue, 22 May 2007 20:37:54 +1000 Con Kolivas [EMAIL PROTECTED] wrote: On Tuesday 22 May 2007 20:25, Ingo Molnar wrote: * Con Kolivas [EMAIL PROTECTED] wrote: there was nothing else running on the system - so i suspect the swapin activity flagged 'itself' as some 'other' activity

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Wednesday 23 May 2007 06:42, Ash Milsted wrote: Hi. I just did some video encoding on my desktop and I was noticing (for the first time in a while) that running apps had to hit swap quite a lot when I switched to them (the encoding was going at full blast for most of the day, and most of

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Antonino Ingargiola
2007/5/21, Ingo Molnar [EMAIL PROTECTED]: * Con Kolivas [EMAIL PROTECTED] wrote: A suggestion for improvement: right now swap-prefetch does a small bit of swapin every 5 seconds and stays idle inbetween. Could this perhaps be made more agressive (optionally perhaps), if the system is

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Tuesday 22 May 2007 20:15, Antonino Ingargiola wrote: 2007/5/21, Ingo Molnar [EMAIL PROTECTED]: * Con Kolivas [EMAIL PROTECTED] wrote: A suggestion for improvement: right now swap-prefetch does a small bit of swapin every 5 seconds and stays idle inbetween. Could this perhaps be

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Con Kolivas [EMAIL PROTECTED] wrote: there was nothing else running on the system - so i suspect the swapin activity flagged 'itself' as some 'other' activity and stopped? The swapins happened in 4 bursts, separated by 5 seconds total idleness. I've noted burst swapins

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Tuesday 22 May 2007 20:25, Ingo Molnar wrote: * Con Kolivas [EMAIL PROTECTED] wrote: there was nothing else running on the system - so i suspect the swapin activity flagged 'itself' as some 'other' activity and stopped? The swapins happened in 4 bursts, separated by 5 seconds

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Con Kolivas [EMAIL PROTECTED] wrote: On Tuesday 22 May 2007 20:25, Ingo Molnar wrote: * Con Kolivas [EMAIL PROTECTED] wrote: there was nothing else running on the system - so i suspect the swapin activity flagged 'itself' as some 'other' activity and stopped? The swapins

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Tuesday 22 May 2007 20:46, Ingo Molnar wrote: It clearly should not consider 'itself' as IO activity. This suggests some bug in the 'detect activity' mechanism, agreed? I'm wondering whether you are seeing the same problem, or is all swap-prefetch IO on your system continuous until it's

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Con Kolivas [EMAIL PROTECTED] wrote: On Tuesday 22 May 2007 20:46, Ingo Molnar wrote: It clearly should not consider 'itself' as IO activity. This suggests some bug in the 'detect activity' mechanism, agreed? I'm wondering whether you are seeing the same problem, or is all

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Tuesday 22 May 2007 20:57, Ingo Molnar wrote: * Con Kolivas [EMAIL PROTECTED] wrote: On Tuesday 22 May 2007 20:46, Ingo Molnar wrote: It clearly should not consider 'itself' as IO activity. This suggests some bug in the 'detect activity' mechanism, agreed? I'm wondering whether you

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Con Kolivas [EMAIL PROTECTED] wrote: hm, what do you call 'journal writeout' here that would be happening on my system? Not really sure what you have in terms of fs, but here even with nothing going on, ext3 writes to disk every 5 seconds with kjournald. i have ext3, but it doesnt

Re: [PATCH] mm: swap prefetch improvements

2007-05-21 Thread Ingo Molnar
* Con Kolivas <[EMAIL PROTECTED]> wrote: > > A suggestion for improvement: right now swap-prefetch does a small > > bit of swapin every 5 seconds and stays idle inbetween. Could this > > perhaps be made more agressive (optionally perhaps), if the system > > is not swapping otherwise? If

Re: [PATCH] mm: swap prefetch improvements

2007-05-21 Thread Con Kolivas
On Monday 21 May 2007 20:03, Ingo Molnar wrote: > * Con Kolivas <[EMAIL PROTECTED]> wrote: > > It turns out that fixing swap prefetch was not that hard to fix and > > improve upon, and since Andrew hasn't dropped swap prefetch, instead > > here are a swag of fixes and improvements, [...] > > it's

Re: [PATCH] mm: swap prefetch improvements

2007-05-21 Thread Ingo Molnar
* Con Kolivas <[EMAIL PROTECTED]> wrote: > It turns out that fixing swap prefetch was not that hard to fix and > improve upon, and since Andrew hasn't dropped swap prefetch, instead > here are a swag of fixes and improvements, [...] it's a reliable win on my testbox too: # echo 1 >

Re: [PATCH] mm: swap prefetch improvements

2007-05-21 Thread Ingo Molnar
* Con Kolivas [EMAIL PROTECTED] wrote: It turns out that fixing swap prefetch was not that hard to fix and improve upon, and since Andrew hasn't dropped swap prefetch, instead here are a swag of fixes and improvements, [...] it's a reliable win on my testbox too: # echo 1

Re: [PATCH] mm: swap prefetch improvements

2007-05-21 Thread Con Kolivas
On Monday 21 May 2007 20:03, Ingo Molnar wrote: * Con Kolivas [EMAIL PROTECTED] wrote: It turns out that fixing swap prefetch was not that hard to fix and improve upon, and since Andrew hasn't dropped swap prefetch, instead here are a swag of fixes and improvements, [...] it's a reliable

Re: [PATCH] mm: swap prefetch improvements

2007-05-21 Thread Ingo Molnar
* Con Kolivas [EMAIL PROTECTED] wrote: A suggestion for improvement: right now swap-prefetch does a small bit of swapin every 5 seconds and stays idle inbetween. Could this perhaps be made more agressive (optionally perhaps), if the system is not swapping otherwise? If block-IO level

Re: [PATCH] mm: swap prefetch improvements

2007-05-12 Thread Paul Jackson
Con wrote: > Ok so change the default value for swap_prefetch to 0 when CPUSETS is > enabled? I don't see why that special case for cpusets is needed. I'm suggesting making no special cases for CPUSETS at all, until and unless we find reason to. In other words, I'm suggesting simply removing

Re: [PATCH] mm: swap prefetch improvements

2007-05-12 Thread Con Kolivas
On Saturday 12 May 2007 18:14, Paul Jackson wrote: > > Ummm this is what I've been saying for over a year now but noone has been > > listening. > > Well ... if there is a problem using prefetch and cpusets together, > it doesn't look like the two of us are going to find it. > > I should probably

Re: [PATCH] mm: swap prefetch improvements

2007-05-12 Thread Paul Jackson
> Ummm this is what I've been saying for over a year now but noone has been > listening. Well ... if there is a problem using prefetch and cpusets together, it doesn't look like the two of us are going to find it. I should probably look at your patch to answer this next question, but being a

Re: [PATCH] mm: swap prefetch improvements

2007-05-12 Thread Con Kolivas
On Saturday 12 May 2007 15:51, Paul Jackson wrote: > Con wrote: > > Hmm I'm not really sure what it takes to make it cpuset aware; > > ... > > It is numa aware to some degree. It stores the node id and when it starts > > prefetching it only prefetches to nodes that are suitable for prefetching > >

Re: [PATCH] mm: swap prefetch improvements

2007-05-12 Thread Con Kolivas
On Saturday 12 May 2007 15:51, Paul Jackson wrote: Con wrote: Hmm I'm not really sure what it takes to make it cpuset aware; ... It is numa aware to some degree. It stores the node id and when it starts prefetching it only prefetches to nodes that are suitable for prefetching to ...

Re: [PATCH] mm: swap prefetch improvements

2007-05-12 Thread Paul Jackson
Ummm this is what I've been saying for over a year now but noone has been listening. Well ... if there is a problem using prefetch and cpusets together, it doesn't look like the two of us are going to find it. I should probably look at your patch to answer this next question, but being a lazy

Re: [PATCH] mm: swap prefetch improvements

2007-05-12 Thread Con Kolivas
On Saturday 12 May 2007 18:14, Paul Jackson wrote: Ummm this is what I've been saying for over a year now but noone has been listening. Well ... if there is a problem using prefetch and cpusets together, it doesn't look like the two of us are going to find it. I should probably look at

Re: [PATCH] mm: swap prefetch improvements

2007-05-12 Thread Paul Jackson
Con wrote: Ok so change the default value for swap_prefetch to 0 when CPUSETS is enabled? I don't see why that special case for cpusets is needed. I'm suggesting making no special cases for CPUSETS at all, until and unless we find reason to. In other words, I'm suggesting simply removing

Re: [PATCH] mm: swap prefetch improvements

2007-05-11 Thread Paul Jackson
Con wrote: > Hmm I'm not really sure what it takes to make it cpuset aware; > ... > It is numa aware to some degree. It stores the node id and when it starts > prefetching it only prefetches to nodes that are suitable for prefetching to > ... > It would be absolutely trivial to add a check for

Re: [PATCH] mm: swap prefetch improvements

2007-05-11 Thread Con Kolivas
On Saturday 12 May 2007 15:03, Paul Jackson wrote: > > Swap prefetch is not cpuset aware so make the config option depend on > > !CPUSETS. > > Ok. > > Could you explain what it means to say "swap prefetch is not cpuset aware", > or could you give a rough idea of what it would take to make it

Re: [PATCH] mm: swap prefetch improvements

2007-05-11 Thread Paul Jackson
> Swap prefetch is not cpuset aware so make the config option depend on > !CPUSETS. Ok. Could you explain what it means to say "swap prefetch is not cpuset aware", or could you give a rough idea of what it would take to make it cpuset aware? I wouldn't go so far as to say that no one would

Re: [PATCH] mm: swap prefetch improvements

2007-05-11 Thread Paul Jackson
Swap prefetch is not cpuset aware so make the config option depend on !CPUSETS. Ok. Could you explain what it means to say swap prefetch is not cpuset aware, or could you give a rough idea of what it would take to make it cpuset aware? I wouldn't go so far as to say that no one would ever

Re: [PATCH] mm: swap prefetch improvements

2007-05-11 Thread Con Kolivas
On Saturday 12 May 2007 15:03, Paul Jackson wrote: Swap prefetch is not cpuset aware so make the config option depend on !CPUSETS. Ok. Could you explain what it means to say swap prefetch is not cpuset aware, or could you give a rough idea of what it would take to make it cpuset aware?

Re: [PATCH] mm: swap prefetch improvements

2007-05-11 Thread Paul Jackson
Con wrote: Hmm I'm not really sure what it takes to make it cpuset aware; ... It is numa aware to some degree. It stores the node id and when it starts prefetching it only prefetches to nodes that are suitable for prefetching to ... It would be absolutely trivial to add a check for