subject:"Re\: swap\-prefetch\: 2.6.22 \-mm merge plans"

On 5/9/07, Nick Piggin <[EMAIL PROTECTED]> wrote:

Ray Lee wrote:
> On 5/9/07, Nick Piggin <[EMAIL PROTECTED]> wrote:
>
>> You said it helped with the updatedb problem. That says we should look at
>> why it is going bad first, and for example improve use-once algorithms.
>> After we do that, then swap prefetching might still help, which is fine.
>
> Nick, if you're volunteering to do that analysis, then great. If not,
> then you're just providing a airy hope with nothing to back up when or
> if that work would ever occur.

I'd like to try helping. Tell me your problem.

Huh? You already stated one version of it above, namely updatedb. But
let's put this another way, shall we? A gedankenexperiment, if you
will.

Say we have a perfect swap-out algorithm that can choose exactly what
needs to be evicted to disk. ('Perfect', of course, is dependent upon
one's metric, but let's go with "maximizes overall system utilization
and minimizes IO wait time." Arbitrary, but hey.)

So, great, the right things got swapped out. Anything else that could
have been chosen would have caused more overall IO Wait. Yay us.

So what happens when those processes that triggered the swap-outs go
away? (Firefox is closed, I stop hitting my local copy of a database,
whatever.) Well, currently, nothing. What happens when I switch
workspaces and try to use my email program? Swap-ins.

Okay, so why didn't the system swap that stuff in preemptively? Why am
I sitting there waiting for something that it could have already done
in the background?

A new swap-out algorithm, be it use-once, Clock-Pro, or perfect
foreknowledge isn't going to change that issue. Swap prefetch does.

> Further, if you or someone else *does* do that work, then guess what,
> we still have the option to rip out the swap prefetching code after
> the hypothetical use-once improvements have been proven and merged.
> Which, by the way, I've watched people talk about since 2.4. That was,
> y'know, a *while* ago.

What's wrong with the use-once we have? What improvements are you talking
about?

You said, effectively: "Use-once could be improved to deal with
updatedb". I said I've been reading emails from Rik and others talking
about that for four years now, and we're still talking about it. Were
it merely updatedb, I'd say us userspace folk should step up and
rewrite the damn thing to amortize its work. However, I and others
feel it's only an example -- glaring, obviously -- of a more pervasive
issue. A small issue, to be sure!, but an issue nevertheless.

In general, I/others are talking about improving the desktop
experience of running too much on a RAM limited machine. (Which, in my
case, is with a gig and a 2.2GHz processor.)

Or restated: the desktop experience occasionally sucks for me, and I
don't think I'm alone. There may be a heuristic, completely isolated
from userspace (and so isn't an API the kernel has to support! -- if
it doesn't work, we can rip it out again), that may mitigate the
suckiness. Let's try it.

> So enough with the stop energy, okay? You're better than that.

I don't think it is about energy or being mean, I'm just stating the
issues I have with it.

Nick, I in no way think you're being mean, and I'm sorry if I've given
you that impression. However, if you're just stating the issues you
have with it, then can I assume that you won't lobby against having
this experiment merged?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans

On Thursday 10 May 2007 13:48, Ray Lee wrote:
> On 5/9/07, Nick Piggin <[EMAIL PROTECTED]> wrote:
> > You said it helped with the updatedb problem. That says we should look at
> > why it is going bad first, and for example improve use-once algorithms.
> > After we do that, then swap prefetching might still help, which is fine.
>
> Nick, if you're volunteering to do that analysis, then great. If not,
> then you're just providing a airy hope with nothing to back up when or
> if that work would ever occur.
>
> Further, if you or someone else *does* do that work, then guess what,
> we still have the option to rip out the swap prefetching code after
> the hypothetical use-once improvements have been proven and merged.
> Which, by the way, I've watched people talk about since 2.4. That was,
> y'know, a *while* ago.
>
> So enough with the stop energy, okay? You're better than that.
>
> Con? He is right about the last feature to go in needs to work
> gracefully with what's there now. However, it's not unheard of for
> authors of other sections of code to help out with incompatibilities
> by answering politely phrased questions for guidance. Though the
> intersection of users between cpusets and desktop systems seems small
> indeed.

Let's just set the record straight. I actually discussed cpusets over a year 
ago in this nonsense and was told by sgi folk there was no need to get my 
head around cpusets and honouring node placement should be enough which, by 
the way, swap prefetch does. So I by no means ignored this; we just hit an 
impasse on just how much more featured it should be for the sake of a goddamn 
home desktop pc feature.

Anyway why the hell am I resurrecting this thread? The code is declared dead 
already. Leave it be.

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


Ray Lee wrote:

On 5/9/07, Nick Piggin <[EMAIL PROTECTED]> wrote:


You said it helped with the updatedb problem. That says we should look at
why it is going bad first, and for example improve use-once algorithms.
After we do that, then swap prefetching might still help, which is fine.



Nick, if you're volunteering to do that analysis, then great. If not,
then you're just providing a airy hope with nothing to back up when or
if that work would ever occur.


I'd like to try helping. Tell me your problem.



Further, if you or someone else *does* do that work, then guess what,
we still have the option to rip out the swap prefetching code after
the hypothetical use-once improvements have been proven and merged.
Which, by the way, I've watched people talk about since 2.4. That was,
y'know, a *while* ago.


What's wrong with the use-once we have? What improvements are you talking
about?



So enough with the stop energy, okay? You're better than that.


I don't think it is about energy or being mean, I'm just stating the
issues I have with it.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


On 5/9/07, Nick Piggin <[EMAIL PROTECTED]> wrote:

You said it helped with the updatedb problem. That says we should look at
why it is going bad first, and for example improve use-once algorithms.
After we do that, then swap prefetching might still help, which is fine.


Nick, if you're volunteering to do that analysis, then great. If not,
then you're just providing a airy hope with nothing to back up when or
if that work would ever occur.

Further, if you or someone else *does* do that work, then guess what,
we still have the option to rip out the swap prefetching code after
the hypothetical use-once improvements have been proven and merged.
Which, by the way, I've watched people talk about since 2.4. That was,
y'know, a *while* ago.

So enough with the stop energy, okay? You're better than that.

Con? He is right about the last feature to go in needs to work
gracefully with what's there now. However, it's not unheard of for
authors of other sections of code to help out with incompatibilities
by answering politely phrased questions for guidance. Though the
intersection of users between cpusets and desktop systems seems small
indeed.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


Con Kolivas wrote:

On Thursday 10 May 2007 10:05, Nick Piggin wrote:



I'm not the gatekeeper and it is completely up to you whether you want
to work on something or not... but I'm sure you understand where I was
coming from when I suggested it doesn't get merged yet.



No matter how you spin it, you're the gatekeeper.


If raising unaddressed issues means closing a gate, then OK. You can
equally open it by answering them.



You may not believe this, but I agree that swap prefetching (and
prefetching in general) has some potential to help desktop workloads :).
But it still should go through the normal process of being tested and
questioned and having a look at options for first improving existing
code in those problematic cases.



Not this again? Proof was there ages ago that it helped and no proof that it 
harmed could be found yet you cunningly pretend it never existed. It's been 
done to death and I'm sick of this.


I said I know it can help. Do you know how many patches I have that help
some workloads but are not merged? That's just the way it works.

What I have seen is it helps the case where you force out a huge amount
of swap. OK, that's nice -- the base case obviously works.

You said it helped with the updatedb problem. That says we should look at
why it is going bad first, and for example improve use-once algorithms.
After we do that, then swap prefetching might still help, which is fine.



Once that process happens and it is shown to work nicely, etc., then I
would not be able to (or want to) keep it from getting merged.

As far as cpusets goes... if your code goes in last, then you have to
make it work with what is there, as a rule. People are using cpusets
for memory resource control, which would have uses on a desktop system.
It is just a really bad precedent to set, having different parts of the
VM not work correctly together. Even if you made them mutually
exclusive CONFIG_ options, that is still not a very nice solution.



That's as close to a 3 as I'm likely to get out of you.


If you're not willing to try making it work with existing code, among other
things, then yes it will be difficult to get it merged. That's not going to
change.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans

On Thursday 10 May 2007 10:05, Nick Piggin wrote:
> Con Kolivas wrote:
> > Well how about that? That was the difference with a swap _file_ as I
> > said, but I went ahead and checked with a swap partition as I used to
> > have. I didn't notice, but somewhere in the last few months, swap
> > prefetch code itself being unchanged for a year, seems to have been
> > broken by other changes in the vm and it doesn't even start up
> > prefetching often and has stale swap entries in its list. Once it breaks
> > like that it does nothing from then on. So that leaves me with a quandry
> > now.
> >
> >
> > Do I:
> >
> > 1. Go ahead and find whatever breakage was introduced and fix it with
> > hopefully a trivial change
> >
> > 2. Do option 1. and then implement support for yet another kernel feature
> > (cpusets) that will be used perhaps never with swap prefetch [No Nick I
> > don't believe you that cpusets have anything to do with normal users on a
> > desktop ever; if it's used on a desktop it will only be by a kernel
> > developer testing the cpusets code].
> >
> > or
> >
> > 3. Dump swap prefetch forever and ignore that it ever worked and was
> > helpful and was a lot of work to implement and so on.
> >
> >
> > Given that even if I do 1 and/or 2 it'll still be blocked from ever going
> > to mainline I think the choice is clear.
> >
> > Nick since you're personally the gatekeeper for this code, would you like
> > to make a call? Just say 3 and put me out of my misery please.
>
> I'm not the gatekeeper and it is completely up to you whether you want
> to work on something or not... but I'm sure you understand where I was
> coming from when I suggested it doesn't get merged yet.

No matter how you spin it, you're the gatekeeper.

> You may not believe this, but I agree that swap prefetching (and
> prefetching in general) has some potential to help desktop workloads :).
> But it still should go through the normal process of being tested and
> questioned and having a look at options for first improving existing
> code in those problematic cases.

Not this again? Proof was there ages ago that it helped and no proof that it 
harmed could be found yet you cunningly pretend it never existed. It's been 
done to death and I'm sick of this.

> Once that process happens and it is shown to work nicely, etc., then I
> would not be able to (or want to) keep it from getting merged.
>
> As far as cpusets goes... if your code goes in last, then you have to
> make it work with what is there, as a rule. People are using cpusets
> for memory resource control, which would have uses on a desktop system.
> It is just a really bad precedent to set, having different parts of the
> VM not work correctly together. Even if you made them mutually
> exclusive CONFIG_ options, that is still not a very nice solution.

That's as close to a 3 as I'm likely to get out of you.

Andrew you'll be relieved to know I would like you to throw swap prefetch and 
related patches into the bin. Thanks.

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


Con Kolivas wrote:

Well how about that? That was the difference with a swap _file_ as I said, but 
I went ahead and checked with a swap partition as I used to have. I didn't 
notice, but somewhere in the last few months, swap prefetch code itself being 
unchanged for a year, seems to have been broken by other changes in the vm 
and it doesn't even start up prefetching often and has stale swap entries in 
its list. Once it breaks like that it does nothing from then on. So that 
leaves me with a quandry now.



Do I:

1. Go ahead and find whatever breakage was introduced and fix it with 
hopefully a trivial change


2. Do option 1. and then implement support for yet another kernel feature 
(cpusets) that will be used perhaps never with swap prefetch [No Nick I don't 
believe you that cpusets have anything to do with normal users on a desktop 
ever; if it's used on a desktop it will only be by a kernel developer testing 
the cpusets code].


or

3. Dump swap prefetch forever and ignore that it ever worked and was helpful 
and was a lot of work to implement and so on.



Given that even if I do 1 and/or 2 it'll still be blocked from ever going to 
mainline I think the choice is clear.


Nick since you're personally the gatekeeper for this code, would you like to 
make a call? Just say 3 and put me out of my misery please.



I'm not the gatekeeper and it is completely up to you whether you want
to work on something or not... but I'm sure you understand where I was
coming from when I suggested it doesn't get merged yet.

You may not believe this, but I agree that swap prefetching (and
prefetching in general) has some potential to help desktop workloads :).
But it still should go through the normal process of being tested and
questioned and having a look at options for first improving existing
code in those problematic cases.

Once that process happens and it is shown to work nicely, etc., then I
would not be able to (or want to) keep it from getting merged.

As far as cpusets goes... if your code goes in last, then you have to
make it work with what is there, as a rule. People are using cpusets
for memory resource control, which would have uses on a desktop system.
It is just a really bad precedent to set, having different parts of the
VM not work correctly together. Even if you made them mutually
exclusive CONFIG_ options, that is still not a very nice solution.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans

On Saturday 05 May 2007 18:42, Con Kolivas wrote:
> On Friday 04 May 2007 22:10, Con Kolivas wrote:
> > On Friday 04 May 2007 18:52, Ingo Molnar wrote:
> > > agreed. Con, IIRC you wrote a testcase for this, right? Could you
> > > please send us the results of that testing?
> >
> > Yes, sorry it's a crappy test app but works on 32bit. Timed with prefetch
> > disabled and then enabled swap prefetch saves ~5 seconds on average
> > hardware on this one test case. I had many users try this and the results
> > were between 2 and 10 seconds, but always showed a saving on this
> > testcase. This effect easily occurs on printing a big picture, editing a
> > large file, compressing an iso image or whatever in real world workloads.
> > Smaller, but much more frequent effects of this over the course of a day
> > obviously also occur and do add up.
>
> Here's a better swap prefetch tester. Instructions in file.
>
> Machine with 2GB ram and 2GB swapfile
>
> Prefetch disabled:
> ./sp_tester

> Timed portion 53397 milliseconds
>
> Enabled:
> ./sp_tester

> Timed portion 26351 milliseconds
>
> Note huge time difference.

Well how about that? That was the difference with a swap _file_ as I said, but 
I went ahead and checked with a swap partition as I used to have. I didn't 
notice, but somewhere in the last few months, swap prefetch code itself being 
unchanged for a year, seems to have been broken by other changes in the vm 
and it doesn't even start up prefetching often and has stale swap entries in 
its list. Once it breaks like that it does nothing from then on. So that 
leaves me with a quandry now.

Do I:

1. Go ahead and find whatever breakage was introduced and fix it with 
hopefully a trivial change

2. Do option 1. and then implement support for yet another kernel feature 
(cpusets) that will be used perhaps never with swap prefetch [No Nick I don't 
believe you that cpusets have anything to do with normal users on a desktop 
ever; if it's used on a desktop it will only be by a kernel developer testing 
the cpusets code].

or

3. Dump swap prefetch forever and ignore that it ever worked and was helpful 
and was a lot of work to implement and so on.

Given that even if I do 1 and/or 2 it'll still be blocked from ever going to 
mainline I think the choice is clear.

Nick since you're personally the gatekeeper for this code, would you like to 
make a call? Just say 3 and put me out of my misery please.

-- 
-ck

P.S. Ingo, thanks (and sorry) for your involvement here.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans

On Saturday 05 May 2007 18:42, Con Kolivas wrote:
 On Friday 04 May 2007 22:10, Con Kolivas wrote:
  On Friday 04 May 2007 18:52, Ingo Molnar wrote:
   agreed. Con, IIRC you wrote a testcase for this, right? Could you
   please send us the results of that testing?
 
  Yes, sorry it's a crappy test app but works on 32bit. Timed with prefetch
  disabled and then enabled swap prefetch saves ~5 seconds on average
  hardware on this one test case. I had many users try this and the results
  were between 2 and 10 seconds, but always showed a saving on this
  testcase. This effect easily occurs on printing a big picture, editing a
  large file, compressing an iso image or whatever in real world workloads.
  Smaller, but much more frequent effects of this over the course of a day
  obviously also occur and do add up.

 Here's a better swap prefetch tester. Instructions in file.

 Machine with 2GB ram and 2GB swapfile

 Prefetch disabled:
 ./sp_tester

 Timed portion 53397 milliseconds

 Enabled:
 ./sp_tester

 Timed portion 26351 milliseconds

 Note huge time difference.

Well how about that? That was the difference with a swap _file_ as I said, but 
I went ahead and checked with a swap partition as I used to have. I didn't 
notice, but somewhere in the last few months, swap prefetch code itself being 
unchanged for a year, seems to have been broken by other changes in the vm 
and it doesn't even start up prefetching often and has stale swap entries in 
its list. Once it breaks like that it does nothing from then on. So that 
leaves me with a quandry now.


Do I:

1. Go ahead and find whatever breakage was introduced and fix it with 
hopefully a trivial change

2. Do option 1. and then implement support for yet another kernel feature 
(cpusets) that will be used perhaps never with swap prefetch [No Nick I don't 
believe you that cpusets have anything to do with normal users on a desktop 
ever; if it's used on a desktop it will only be by a kernel developer testing 
the cpusets code].

or

3. Dump swap prefetch forever and ignore that it ever worked and was helpful 
and was a lot of work to implement and so on.


Given that even if I do 1 and/or 2 it'll still be blocked from ever going to 
mainline I think the choice is clear.

Nick since you're personally the gatekeeper for this code, would you like to 
make a call? Just say 3 and put me out of my misery please.

-- 
-ck

P.S. Ingo, thanks (and sorry) for your involvement here.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


Con Kolivas wrote:

Well how about that? That was the difference with a swap _file_ as I said, but 
I went ahead and checked with a swap partition as I used to have. I didn't 
notice, but somewhere in the last few months, swap prefetch code itself being 
unchanged for a year, seems to have been broken by other changes in the vm 
and it doesn't even start up prefetching often and has stale swap entries in 
its list. Once it breaks like that it does nothing from then on. So that 
leaves me with a quandry now.



Do I:

1. Go ahead and find whatever breakage was introduced and fix it with 
hopefully a trivial change


2. Do option 1. and then implement support for yet another kernel feature 
(cpusets) that will be used perhaps never with swap prefetch [No Nick I don't 
believe you that cpusets have anything to do with normal users on a desktop 
ever; if it's used on a desktop it will only be by a kernel developer testing 
the cpusets code].


or

3. Dump swap prefetch forever and ignore that it ever worked and was helpful 
and was a lot of work to implement and so on.



Given that even if I do 1 and/or 2 it'll still be blocked from ever going to 
mainline I think the choice is clear.


Nick since you're personally the gatekeeper for this code, would you like to 
make a call? Just say 3 and put me out of my misery please.



I'm not the gatekeeper and it is completely up to you whether you want
to work on something or not... but I'm sure you understand where I was
coming from when I suggested it doesn't get merged yet.

You may not believe this, but I agree that swap prefetching (and
prefetching in general) has some potential to help desktop workloads :).
But it still should go through the normal process of being tested and
questioned and having a look at options for first improving existing
code in those problematic cases.

Once that process happens and it is shown to work nicely, etc., then I
would not be able to (or want to) keep it from getting merged.

As far as cpusets goes... if your code goes in last, then you have to
make it work with what is there, as a rule. People are using cpusets
for memory resource control, which would have uses on a desktop system.
It is just a really bad precedent to set, having different parts of the
VM not work correctly together. Even if you made them mutually
exclusive CONFIG_ options, that is still not a very nice solution.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans

On Thursday 10 May 2007 10:05, Nick Piggin wrote:
 Con Kolivas wrote:
  Well how about that? That was the difference with a swap _file_ as I
  said, but I went ahead and checked with a swap partition as I used to
  have. I didn't notice, but somewhere in the last few months, swap
  prefetch code itself being unchanged for a year, seems to have been
  broken by other changes in the vm and it doesn't even start up
  prefetching often and has stale swap entries in its list. Once it breaks
  like that it does nothing from then on. So that leaves me with a quandry
  now.
 
 
  Do I:
 
  1. Go ahead and find whatever breakage was introduced and fix it with
  hopefully a trivial change
 
  2. Do option 1. and then implement support for yet another kernel feature
  (cpusets) that will be used perhaps never with swap prefetch [No Nick I
  don't believe you that cpusets have anything to do with normal users on a
  desktop ever; if it's used on a desktop it will only be by a kernel
  developer testing the cpusets code].
 
  or
 
  3. Dump swap prefetch forever and ignore that it ever worked and was
  helpful and was a lot of work to implement and so on.
 
 
  Given that even if I do 1 and/or 2 it'll still be blocked from ever going
  to mainline I think the choice is clear.
 
  Nick since you're personally the gatekeeper for this code, would you like
  to make a call? Just say 3 and put me out of my misery please.

 I'm not the gatekeeper and it is completely up to you whether you want
 to work on something or not... but I'm sure you understand where I was
 coming from when I suggested it doesn't get merged yet.

No matter how you spin it, you're the gatekeeper.

 You may not believe this, but I agree that swap prefetching (and
 prefetching in general) has some potential to help desktop workloads :).
 But it still should go through the normal process of being tested and
 questioned and having a look at options for first improving existing
 code in those problematic cases.

Not this again? Proof was there ages ago that it helped and no proof that it 
harmed could be found yet you cunningly pretend it never existed. It's been 
done to death and I'm sick of this.

 Once that process happens and it is shown to work nicely, etc., then I
 would not be able to (or want to) keep it from getting merged.

 As far as cpusets goes... if your code goes in last, then you have to
 make it work with what is there, as a rule. People are using cpusets
 for memory resource control, which would have uses on a desktop system.
 It is just a really bad precedent to set, having different parts of the
 VM not work correctly together. Even if you made them mutually
 exclusive CONFIG_ options, that is still not a very nice solution.

That's as close to a 3 as I'm likely to get out of you.

Andrew you'll be relieved to know I would like you to throw swap prefetch and 
related patches into the bin. Thanks.

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


Con Kolivas wrote:

On Thursday 10 May 2007 10:05, Nick Piggin wrote:



I'm not the gatekeeper and it is completely up to you whether you want
to work on something or not... but I'm sure you understand where I was
coming from when I suggested it doesn't get merged yet.



No matter how you spin it, you're the gatekeeper.


If raising unaddressed issues means closing a gate, then OK. You can
equally open it by answering them.



You may not believe this, but I agree that swap prefetching (and
prefetching in general) has some potential to help desktop workloads :).
But it still should go through the normal process of being tested and
questioned and having a look at options for first improving existing
code in those problematic cases.



Not this again? Proof was there ages ago that it helped and no proof that it 
harmed could be found yet you cunningly pretend it never existed. It's been 
done to death and I'm sick of this.


I said I know it can help. Do you know how many patches I have that help
some workloads but are not merged? That's just the way it works.

What I have seen is it helps the case where you force out a huge amount
of swap. OK, that's nice -- the base case obviously works.

You said it helped with the updatedb problem. That says we should look at
why it is going bad first, and for example improve use-once algorithms.
After we do that, then swap prefetching might still help, which is fine.



Once that process happens and it is shown to work nicely, etc., then I
would not be able to (or want to) keep it from getting merged.

As far as cpusets goes... if your code goes in last, then you have to
make it work with what is there, as a rule. People are using cpusets
for memory resource control, which would have uses on a desktop system.
It is just a really bad precedent to set, having different parts of the
VM not work correctly together. Even if you made them mutually
exclusive CONFIG_ options, that is still not a very nice solution.



That's as close to a 3 as I'm likely to get out of you.


If you're not willing to try making it work with existing code, among other
things, then yes it will be difficult to get it merged. That's not going to
change.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


On 5/9/07, Nick Piggin [EMAIL PROTECTED] wrote:

You said it helped with the updatedb problem. That says we should look at
why it is going bad first, and for example improve use-once algorithms.
After we do that, then swap prefetching might still help, which is fine.


Nick, if you're volunteering to do that analysis, then great. If not,
then you're just providing a airy hope with nothing to back up when or
if that work would ever occur.

Further, if you or someone else *does* do that work, then guess what,
we still have the option to rip out the swap prefetching code after
the hypothetical use-once improvements have been proven and merged.
Which, by the way, I've watched people talk about since 2.4. That was,
y'know, a *while* ago.

So enough with the stop energy, okay? You're better than that.

Con? He is right about the last feature to go in needs to work
gracefully with what's there now. However, it's not unheard of for
authors of other sections of code to help out with incompatibilities
by answering politely phrased questions for guidance. Though the
intersection of users between cpusets and desktop systems seems small
indeed.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


Ray Lee wrote:

On 5/9/07, Nick Piggin [EMAIL PROTECTED] wrote:


You said it helped with the updatedb problem. That says we should look at
why it is going bad first, and for example improve use-once algorithms.
After we do that, then swap prefetching might still help, which is fine.



Nick, if you're volunteering to do that analysis, then great. If not,
then you're just providing a airy hope with nothing to back up when or
if that work would ever occur.


I'd like to try helping. Tell me your problem.



Further, if you or someone else *does* do that work, then guess what,
we still have the option to rip out the swap prefetching code after
the hypothetical use-once improvements have been proven and merged.
Which, by the way, I've watched people talk about since 2.4. That was,
y'know, a *while* ago.


What's wrong with the use-once we have? What improvements are you talking
about?



So enough with the stop energy, okay? You're better than that.


I don't think it is about energy or being mean, I'm just stating the
issues I have with it.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans

On Thursday 10 May 2007 13:48, Ray Lee wrote:
 On 5/9/07, Nick Piggin [EMAIL PROTECTED] wrote:
  You said it helped with the updatedb problem. That says we should look at
  why it is going bad first, and for example improve use-once algorithms.
  After we do that, then swap prefetching might still help, which is fine.

 Nick, if you're volunteering to do that analysis, then great. If not,
 then you're just providing a airy hope with nothing to back up when or
 if that work would ever occur.

 Further, if you or someone else *does* do that work, then guess what,
 we still have the option to rip out the swap prefetching code after
 the hypothetical use-once improvements have been proven and merged.
 Which, by the way, I've watched people talk about since 2.4. That was,
 y'know, a *while* ago.

 So enough with the stop energy, okay? You're better than that.

 Con? He is right about the last feature to go in needs to work
 gracefully with what's there now. However, it's not unheard of for
 authors of other sections of code to help out with incompatibilities
 by answering politely phrased questions for guidance. Though the
 intersection of users between cpusets and desktop systems seems small
 indeed.

Let's just set the record straight. I actually discussed cpusets over a year 
ago in this nonsense and was told by sgi folk there was no need to get my 
head around cpusets and honouring node placement should be enough which, by 
the way, swap prefetch does. So I by no means ignored this; we just hit an 
impasse on just how much more featured it should be for the sake of a goddamn 
home desktop pc feature.

Anyway why the hell am I resurrecting this thread? The code is declared dead 
already. Leave it be.

-- 
-ck
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


On 5/9/07, Nick Piggin [EMAIL PROTECTED] wrote:

Ray Lee wrote:
 On 5/9/07, Nick Piggin [EMAIL PROTECTED] wrote:

 You said it helped with the updatedb problem. That says we should look at
 why it is going bad first, and for example improve use-once algorithms.
 After we do that, then swap prefetching might still help, which is fine.

 Nick, if you're volunteering to do that analysis, then great. If not,
 then you're just providing a airy hope with nothing to back up when or
 if that work would ever occur.

I'd like to try helping. Tell me your problem.


Huh? You already stated one version of it above, namely updatedb. But
let's put this another way, shall we? A gedankenexperiment, if you
will.

Say we have a perfect swap-out algorithm that can choose exactly what
needs to be evicted to disk. ('Perfect', of course, is dependent upon
one's metric, but let's go with maximizes overall system utilization
and minimizes IO wait time. Arbitrary, but hey.)

So, great, the right things got swapped out. Anything else that could
have been chosen would have caused more overall IO Wait. Yay us.

So what happens when those processes that triggered the swap-outs go
away? (Firefox is closed, I stop hitting my local copy of a database,
whatever.) Well, currently, nothing. What happens when I switch
workspaces and try to use my email program? Swap-ins.

Okay, so why didn't the system swap that stuff in preemptively? Why am
I sitting there waiting for something that it could have already done
in the background?

A new swap-out algorithm, be it use-once, Clock-Pro, or perfect
foreknowledge isn't going to change that issue. Swap prefetch does.


 Further, if you or someone else *does* do that work, then guess what,
 we still have the option to rip out the swap prefetching code after
 the hypothetical use-once improvements have been proven and merged.
 Which, by the way, I've watched people talk about since 2.4. That was,
 y'know, a *while* ago.

What's wrong with the use-once we have? What improvements are you talking
about?


You said, effectively: Use-once could be improved to deal with
updatedb. I said I've been reading emails from Rik and others talking
about that for four years now, and we're still talking about it. Were
it merely updatedb, I'd say us userspace folk should step up and
rewrite the damn thing to amortize its work. However, I and others
feel it's only an example -- glaring, obviously -- of a more pervasive
issue. A small issue, to be sure!, but an issue nevertheless.

In general, I/others are talking about improving the desktop
experience of running too much on a RAM limited machine. (Which, in my
case, is with a gig and a 2.2GHz processor.)

Or restated: the desktop experience occasionally sucks for me, and I
don't think I'm alone. There may be a heuristic, completely isolated
from userspace (and so isn't an API the kernel has to support! -- if
it doesn't work, we can rip it out again), that may mitigate the
suckiness. Let's try it.


 So enough with the stop energy, okay? You're better than that.

I don't think it is about energy or being mean, I'm just stating the
issues I have with it.


Nick, I in no way think you're being mean, and I'm sorry if I've given
you that impression. However, if you're just stating the issues you
have with it, then can I assume that you won't lobby against having
this experiment merged?
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


Con Kolivas wrote:

On Friday 04 May 2007 18:52, Ingo Molnar wrote:

agreed. Con, IIRC you wrote a testcase for this, right? Could you please
send us the results of that testing?


Yes, sorry it's a crappy test app but works on 32bit. Timed with prefetch 
disabled and then enabled swap prefetch saves ~5 seconds on average hardware 
on this one test case. I had many users try this and the results were between 
2 and 10 seconds, but always showed a saving on this testcase. This effect 
easily occurs on printing a big picture, editing a large file, compressing an 
iso image or whatever in real world workloads. Smaller, but much more 
frequent effects of this over the course of a day obviously also occur and do 
add up.


I'll try this when I get the scheduler stuff done, and also dig out the 
"resp1" stuff for "back when." I see the most recent datasets were 
comparing 2.5.43-mm2 responsiveness with 2.4.19-ck7, you know I always 
test your stuff ;-)


Guess it might need a bit of polish for current hardware, I was testing 
on *small* machines, deliberately.


--
Bill Davidsen <[EMAIL PROTECTED]>
  "We have more to fear from the bungling of the incompetent than from
the machinations of the wicked."  - from Slashdot

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


Ingo Molnar wrote:

* Nick Piggin <[EMAIL PROTECTED]> wrote:


i'm wondering about swap-prefetch:


Being able to config all these core heuristics changes is really not 
that much of a positive. The fact that we might _need_ to config 
something out, and double the configuration range isn't too pleasing.


Well, to the desktop user this is a speculative performance feature that 
he is willing to potentially waste CPU and IO capacity, in expectation 
of better performance.


On the conceptual level it is _precisely the same thing as regular file 
readahead_. (with the difference that to me swapahead seems to be quite 
a bit more intelligent than our current file readahead logic.)


This feature has no API or ABI impact at all, it's a pure performance 
feature. (besides the trivial sysctl to turn it runtime on/off).



Here were some of my concerns, and where our discussion got up to.


[...snip...]

i see no real problem here. We've had heuristics for a _long_ time in 
various areas of the code. Sometimes they work, sometimes they suck.


the flow of this is really easy: distro looking for a feature edge turns 
it on and announces it, if the feature does not work out for users then 
user turns it off and complains to distro, if enough users complain then 
distro turns it off for next release, upstream forgets about this 
performance feature and eventually removes it once someone notices that 
it wouldnt even compile in the past 2 main releases. I see no problem 
here, we did that in the past too with performance features. The 
networking stack has literally dozens of such small tunable things which 
get experimented with, and whose defaults do get tuned carefully. Some 
of the knobs help bandwidth, some help latency.


I haven't looked at this code since it first came out and didn't impress 
me, but I think it would be good to get the current version in. However, 
when you say "user turns it off" I hope you mean "in /proc/sys with a 
switch or knob" and not by expecting people to recompile and install a 
kernel. Then it might take a little memory but wouldn't do something 
undesirable.


Note: I had no bad effect from the code, it just didn't feel faster. On 
a low memory machine it might help. Of course I have wanted to have a 
hard limit on memory used for i/o buffers, just to avoid swapping 
programs to make room for i/o, so to some extent I feel as if this is a 
fix for a problem we shouldn't have.


--
Bill Davidsen <[EMAIL PROTECTED]>
  "We have more to fear from the bungling of the incompetent than from
the machinations of the wicked."  - from Slashdot

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


Ingo Molnar wrote:

* Nick Piggin [EMAIL PROTECTED] wrote:


i'm wondering about swap-prefetch:


Being able to config all these core heuristics changes is really not 
that much of a positive. The fact that we might _need_ to config 
something out, and double the configuration range isn't too pleasing.


Well, to the desktop user this is a speculative performance feature that 
he is willing to potentially waste CPU and IO capacity, in expectation 
of better performance.


On the conceptual level it is _precisely the same thing as regular file 
readahead_. (with the difference that to me swapahead seems to be quite 
a bit more intelligent than our current file readahead logic.)


This feature has no API or ABI impact at all, it's a pure performance 
feature. (besides the trivial sysctl to turn it runtime on/off).



Here were some of my concerns, and where our discussion got up to.


[...snip...]

i see no real problem here. We've had heuristics for a _long_ time in 
various areas of the code. Sometimes they work, sometimes they suck.


the flow of this is really easy: distro looking for a feature edge turns 
it on and announces it, if the feature does not work out for users then 
user turns it off and complains to distro, if enough users complain then 
distro turns it off for next release, upstream forgets about this 
performance feature and eventually removes it once someone notices that 
it wouldnt even compile in the past 2 main releases. I see no problem 
here, we did that in the past too with performance features. The 
networking stack has literally dozens of such small tunable things which 
get experimented with, and whose defaults do get tuned carefully. Some 
of the knobs help bandwidth, some help latency.


I haven't looked at this code since it first came out and didn't impress 
me, but I think it would be good to get the current version in. However, 
when you say user turns it off I hope you mean in /proc/sys with a 
switch or knob and not by expecting people to recompile and install a 
kernel. Then it might take a little memory but wouldn't do something 
undesirable.


Note: I had no bad effect from the code, it just didn't feel faster. On 
a low memory machine it might help. Of course I have wanted to have a 
hard limit on memory used for i/o buffers, just to avoid swapping 
programs to make room for i/o, so to some extent I feel as if this is a 
fix for a problem we shouldn't have.


--
Bill Davidsen [EMAIL PROTECTED]
  We have more to fear from the bungling of the incompetent than from
the machinations of the wicked.  - from Slashdot

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


Con Kolivas wrote:

On Friday 04 May 2007 18:52, Ingo Molnar wrote:

agreed. Con, IIRC you wrote a testcase for this, right? Could you please
send us the results of that testing?


Yes, sorry it's a crappy test app but works on 32bit. Timed with prefetch 
disabled and then enabled swap prefetch saves ~5 seconds on average hardware 
on this one test case. I had many users try this and the results were between 
2 and 10 seconds, but always showed a saving on this testcase. This effect 
easily occurs on printing a big picture, editing a large file, compressing an 
iso image or whatever in real world workloads. Smaller, but much more 
frequent effects of this over the course of a day obviously also occur and do 
add up.


I'll try this when I get the scheduler stuff done, and also dig out the 
resp1 stuff for back when. I see the most recent datasets were 
comparing 2.5.43-mm2 responsiveness with 2.4.19-ck7, you know I always 
test your stuff ;-)


Guess it might need a bit of polish for current hardware, I was testing 
on *small* machines, deliberately.


--
Bill Davidsen [EMAIL PROTECTED]
  We have more to fear from the bungling of the incompetent than from
the machinations of the wicked.  - from Slashdot

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ck] Re: swap-prefetch: 2.6.22 -mm merge plans

2007-05-06 Thread Jory A. Pratt

Con Kolivas wrote:
> Here's a better swap prefetch tester. Instructions in file.
>
> Machine with 2GB ram and 2GB swapfile
>
> Prefetch disabled:
> ./sp_tester
> Ram 2060352000  Swap 197342
> Total ram to be malloced: 3047062000 bytes
> Starting first malloc of 1523531000 bytes
> Starting 1st read of first malloc
> Touching this much ram takes 809 milliseconds
> Starting second malloc of 1523531000 bytes
> Completed second malloc and free
> Sleeping for 600 seconds
> Important part - starting reread of first malloc
> Completed read of first malloc
> Timed portion 53397 milliseconds
>
> Enabled:
> ./sp_tester
> Ram 2060352000  Swap 197342
> Total ram to be malloced: 3047062000 bytes
> Starting first malloc of 1523531000 bytes
> Starting 1st read of first malloc
> Touching this much ram takes 676 milliseconds
> Starting second malloc of 1523531000 bytes
> Completed second malloc and free
> Sleeping for 600 seconds
> Important part - starting reread of first malloc
> Completed read of first malloc
> Timed portion 26351 milliseconds
>   
echo 1 > /proc/sys/vm/overcommit_memory
swapoff -a
swapon -a
./sp_tester

Ram 1153644000  Swap 1004052000
Total ram to be malloced: 165567 bytes
Starting first malloc of 827835000 bytes
Starting 1st read of first malloc
Touching this much ram takes 937 milliseconds
Starting second malloc of 827835000 bytes
Completed second malloc and free
Sleeping for 600 seconds
Important part - starting reread of first malloc
Completed read of first malloc
Timed portion 15011 milliseconds

echo 0 > /proc/sys/vm/overcommit_memory
swapoff -a
swapon -a
./sp_tester

Ram 1153644000  Swap 1004052000
Total ram to be malloced: 165567 bytes
Starting first malloc of 827835000 bytes
Starting 1st read of first malloc
Touching this much ram takes 1125 milliseconds
Starting second malloc of 827835000 bytes
Completed second malloc and free
Sleeping for 600 seconds
Important part - starting reread of first malloc
Completed read of first malloc
Timed portion 14611 milliseconds

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ck] Re: swap-prefetch: 2.6.22 -mm merge plans

2007-05-06 Thread Antonino Ingargiola


2007/5/5, Con Kolivas <[EMAIL PROTECTED]>:
[cut]

Here's a better swap prefetch tester. Instructions in file.


The system should be leaved totally inactive for the test duration (10m) right?


Regards,

 ~ Antonio
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ck] Re: swap-prefetch: 2.6.22 -mm merge plans

2007-05-06 Thread Antonino Ingargiola


2007/5/5, Con Kolivas [EMAIL PROTECTED]:
[cut]

Here's a better swap prefetch tester. Instructions in file.


The system should be leaved totally inactive for the test duration (10m) right?


Regards,

 ~ Antonio
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ck] Re: swap-prefetch: 2.6.22 -mm merge plans

2007-05-06 Thread Jory A. Pratt

Con Kolivas wrote:
 Here's a better swap prefetch tester. Instructions in file.

 Machine with 2GB ram and 2GB swapfile

 Prefetch disabled:
 ./sp_tester
 Ram 2060352000  Swap 197342
 Total ram to be malloced: 3047062000 bytes
 Starting first malloc of 1523531000 bytes
 Starting 1st read of first malloc
 Touching this much ram takes 809 milliseconds
 Starting second malloc of 1523531000 bytes
 Completed second malloc and free
 Sleeping for 600 seconds
 Important part - starting reread of first malloc
 Completed read of first malloc
 Timed portion 53397 milliseconds

 Enabled:
 ./sp_tester
 Ram 2060352000  Swap 197342
 Total ram to be malloced: 3047062000 bytes
 Starting first malloc of 1523531000 bytes
 Starting 1st read of first malloc
 Touching this much ram takes 676 milliseconds
 Starting second malloc of 1523531000 bytes
 Completed second malloc and free
 Sleeping for 600 seconds
 Important part - starting reread of first malloc
 Completed read of first malloc
 Timed portion 26351 milliseconds
   
echo 1  /proc/sys/vm/overcommit_memory
swapoff -a
swapon -a
./sp_tester

Ram 1153644000  Swap 1004052000
Total ram to be malloced: 165567 bytes
Starting first malloc of 827835000 bytes
Starting 1st read of first malloc
Touching this much ram takes 937 milliseconds
Starting second malloc of 827835000 bytes
Completed second malloc and free
Sleeping for 600 seconds
Important part - starting reread of first malloc
Completed read of first malloc
Timed portion 15011 milliseconds

echo 0  /proc/sys/vm/overcommit_memory
swapoff -a
swapon -a
./sp_tester

Ram 1153644000  Swap 1004052000
Total ram to be malloced: 165567 bytes
Starting first malloc of 827835000 bytes
Starting 1st read of first malloc
Touching this much ram takes 1125 milliseconds
Starting second malloc of 827835000 bytes
Completed second malloc and free
Sleeping for 600 seconds
Important part - starting reread of first malloc
Completed read of first malloc
Timed portion 14611 milliseconds

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans

2007-05-05 Thread Con Kolivas

On Friday 04 May 2007 22:10, Con Kolivas wrote:
> On Friday 04 May 2007 18:52, Ingo Molnar wrote:
> > agreed. Con, IIRC you wrote a testcase for this, right? Could you please
> > send us the results of that testing?
>
> Yes, sorry it's a crappy test app but works on 32bit. Timed with prefetch
> disabled and then enabled swap prefetch saves ~5 seconds on average
> hardware on this one test case. I had many users try this and the results
> were between 2 and 10 seconds, but always showed a saving on this testcase.
> This effect easily occurs on printing a big picture, editing a large file,
> compressing an iso image or whatever in real world workloads. Smaller, but
> much more frequent effects of this over the course of a day obviously also
> occur and do add up.

Here's a better swap prefetch tester. Instructions in file.

Machine with 2GB ram and 2GB swapfile

Prefetch disabled:
./sp_tester
Ram 2060352000  Swap 197342
Total ram to be malloced: 3047062000 bytes
Starting first malloc of 1523531000 bytes
Starting 1st read of first malloc
Touching this much ram takes 809 milliseconds
Starting second malloc of 1523531000 bytes
Completed second malloc and free
Sleeping for 600 seconds
Important part - starting reread of first malloc
Completed read of first malloc
Timed portion 53397 milliseconds

Enabled:
./sp_tester
Ram 2060352000  Swap 197342
Total ram to be malloced: 3047062000 bytes
Starting first malloc of 1523531000 bytes
Starting 1st read of first malloc
Touching this much ram takes 676 milliseconds
Starting second malloc of 1523531000 bytes
Completed second malloc and free
Sleeping for 600 seconds
Important part - starting reread of first malloc
Completed read of first malloc
Timed portion 26351 milliseconds

Note huge time difference.

-- 
-ck
/*
sp_tester.c

Build with:
gcc -o sp_tester sp_tester.c -lrt -W -Wall -O2

How to use:
echo 1 > /proc/sys/vm/overcommit_memory
swapoff -a
swapon -a
./sp_tester

then repeat with changed conditions eg
echo 0 > /proc/sys/vm/swap_prefetch

Each Test takes 10 minutes
*/

#include 
#include 
#include 
#include 
#include 
#include 
#include 

void fatal(const char *format, ...)
{
	va_list ap;

	if (format) {
		va_start(ap, format);
		vfprintf(stderr, format, ap);
		va_end(ap);
	}

	fprintf(stderr, "Fatal error - exiting\n");
	exit(1);
}

unsigned long ramsize, swapsize;

size_t get_ram(void)
{
	FILE *meminfo;
char aux[256];

	if(!(meminfo = fopen("/proc/meminfo", "r")))
		fatal("fopen\n");

	while( !feof(meminfo) && !fscanf(meminfo, "MemTotal: %lu kB", ) )
		fgets(aux,sizeof(aux),meminfo);
	while( !feof(meminfo) && !fscanf(meminfo, "SwapTotal: %lu kB", ) )
		fgets(aux,sizeof(aux),meminfo);
	if (fclose(meminfo) == -1)
		fatal("fclose");
	ramsize *= 1000;
	swapsize *= 1000;
	printf("Ram %lu  Swap %lu\n", ramsize, swapsize);
	return ramsize + (swapsize / 2);
}

unsigned long get_usecs(struct timespec *myts)
{
	if (clock_gettime(CLOCK_REALTIME, myts))
		fatal("clock_gettime");
	return (myts->tv_sec * 100 + myts->tv_nsec / 1000 );
}

int main(void)
{
	unsigned long current_time, time_diff;
	struct timespec myts;
	char *buf1, *buf2, *buf3, *buf4;
	size_t size = get_ram();
	int sleep_seconds = 600;

	if (size > ramsize / 2 * 3)
		size = ramsize / 2 * 3;
	printf("Total ram to be malloced: %u bytes\n", size);
	size /= 2;
	printf("Starting first malloc of %u bytes\n", size);
	buf1 = malloc(size);
	buf4 = malloc(1);
	if (buf1 == (char *)-1)
		fatal("Failed to malloc 1st buffer\n");
	memset(buf1, 0, size);
	time_diff = current_time = get_usecs();
	for (buf3 = buf1; buf3 < buf1 + size; buf3++)
		*buf4 = *buf3;
	printf("Starting 1st read of first malloc\n");
	current_time = get_usecs();
	time_diff = current_time - time_diff;
	printf("Touching this much ram takes %lu milliseconds\n",time_diff / 1000);
	printf("Starting second malloc of %u bytes\n", size);

	buf2 = malloc(size);
	if (buf2 == (char *)-1)
		fatal("Failed to malloc 2nd buffer\n");
	memset(buf2, 0, size);
	for (buf3 = buf2 + size; buf3 > buf2; buf3--)
		*buf4 = *buf3;
	free(buf2);
	printf("Completed second malloc and free\n");

	printf("Sleeping for %u seconds\n", sleep_seconds);
	sleep(sleep_seconds);

	printf("Important part - starting reread of first malloc\n");
	time_diff = current_time = get_usecs();
	for (buf3 = buf1; buf3 < buf1 + size; buf3++)
		*buf4 = *buf3;
	current_time = get_usecs();
	time_diff = current_time - time_diff;
	printf("Completed read of first malloc\n");
	printf("Timed portion %lu milliseconds\n",time_diff / 1000);

	free(buf1);
	free(buf4);

	return 0;
}

Re: swap-prefetch: 2.6.22 -mm merge plans

2007-05-05 Thread Con Kolivas

On Friday 04 May 2007 22:10, Con Kolivas wrote:
 On Friday 04 May 2007 18:52, Ingo Molnar wrote:
  agreed. Con, IIRC you wrote a testcase for this, right? Could you please
  send us the results of that testing?

 Yes, sorry it's a crappy test app but works on 32bit. Timed with prefetch
 disabled and then enabled swap prefetch saves ~5 seconds on average
 hardware on this one test case. I had many users try this and the results
 were between 2 and 10 seconds, but always showed a saving on this testcase.
 This effect easily occurs on printing a big picture, editing a large file,
 compressing an iso image or whatever in real world workloads. Smaller, but
 much more frequent effects of this over the course of a day obviously also
 occur and do add up.

Here's a better swap prefetch tester. Instructions in file.

Machine with 2GB ram and 2GB swapfile

Prefetch disabled:
./sp_tester
Ram 2060352000  Swap 197342
Total ram to be malloced: 3047062000 bytes
Starting first malloc of 1523531000 bytes
Starting 1st read of first malloc
Touching this much ram takes 809 milliseconds
Starting second malloc of 1523531000 bytes
Completed second malloc and free
Sleeping for 600 seconds
Important part - starting reread of first malloc
Completed read of first malloc
Timed portion 53397 milliseconds

Enabled:
./sp_tester
Ram 2060352000  Swap 197342
Total ram to be malloced: 3047062000 bytes
Starting first malloc of 1523531000 bytes
Starting 1st read of first malloc
Touching this much ram takes 676 milliseconds
Starting second malloc of 1523531000 bytes
Completed second malloc and free
Sleeping for 600 seconds
Important part - starting reread of first malloc
Completed read of first malloc
Timed portion 26351 milliseconds

Note huge time difference.

-- 
-ck
/*
sp_tester.c

Build with:
gcc -o sp_tester sp_tester.c -lrt -W -Wall -O2

How to use:
echo 1  /proc/sys/vm/overcommit_memory
swapoff -a
swapon -a
./sp_tester

then repeat with changed conditions eg
echo 0  /proc/sys/vm/swap_prefetch

Each Test takes 10 minutes
*/

#include stdio.h
#include stdarg.h
#include stdlib.h
#include string.h
#include unistd.h
#include sys/mman.h
#include time.h

void fatal(const char *format, ...)
{
	va_list ap;

	if (format) {
		va_start(ap, format);
		vfprintf(stderr, format, ap);
		va_end(ap);
	}

	fprintf(stderr, Fatal error - exiting\n);
	exit(1);
}

unsigned long ramsize, swapsize;

size_t get_ram(void)
{
	FILE *meminfo;
char aux[256];

	if(!(meminfo = fopen(/proc/meminfo, r)))
		fatal(fopen\n);

	while( !feof(meminfo)  !fscanf(meminfo, MemTotal: %lu kB, ramsize) )
		fgets(aux,sizeof(aux),meminfo);
	while( !feof(meminfo)  !fscanf(meminfo, SwapTotal: %lu kB, swapsize) )
		fgets(aux,sizeof(aux),meminfo);
	if (fclose(meminfo) == -1)
		fatal(fclose);
	ramsize *= 1000;
	swapsize *= 1000;
	printf(Ram %lu  Swap %lu\n, ramsize, swapsize);
	return ramsize + (swapsize / 2);
}

unsigned long get_usecs(struct timespec *myts)
{
	if (clock_gettime(CLOCK_REALTIME, myts))
		fatal(clock_gettime);
	return (myts-tv_sec * 100 + myts-tv_nsec / 1000 );
}

int main(void)
{
	unsigned long current_time, time_diff;
	struct timespec myts;
	char *buf1, *buf2, *buf3, *buf4;
	size_t size = get_ram();
	int sleep_seconds = 600;

	if (size  ramsize / 2 * 3)
		size = ramsize / 2 * 3;
	printf(Total ram to be malloced: %u bytes\n, size);
	size /= 2;
	printf(Starting first malloc of %u bytes\n, size);
	buf1 = malloc(size);
	buf4 = malloc(1);
	if (buf1 == (char *)-1)
		fatal(Failed to malloc 1st buffer\n);
	memset(buf1, 0, size);
	time_diff = current_time = get_usecs(myts);
	for (buf3 = buf1; buf3  buf1 + size; buf3++)
		*buf4 = *buf3;
	printf(Starting 1st read of first malloc\n);
	current_time = get_usecs(myts);
	time_diff = current_time - time_diff;
	printf(Touching this much ram takes %lu milliseconds\n,time_diff / 1000);
	printf(Starting second malloc of %u bytes\n, size);

	buf2 = malloc(size);
	if (buf2 == (char *)-1)
		fatal(Failed to malloc 2nd buffer\n);
	memset(buf2, 0, size);
	for (buf3 = buf2 + size; buf3  buf2; buf3--)
		*buf4 = *buf3;
	free(buf2);
	printf(Completed second malloc and free\n);

	printf(Sleeping for %u seconds\n, sleep_seconds);
	sleep(sleep_seconds);

	printf(Important part - starting reread of first malloc\n);
	time_diff = current_time = get_usecs(myts);
	for (buf3 = buf1; buf3  buf1 + size; buf3++)
		*buf4 = *buf3;
	current_time = get_usecs(myts);
	time_diff = current_time - time_diff;
	printf(Completed read of first malloc\n);
	printf(Timed portion %lu milliseconds\n,time_diff / 1000);

	free(buf1);
	free(buf4);

	return 0;
}

Re: swap-prefetch: 2.6.22 -mm merge plans

2007-05-04 Thread Con Kolivas

On Friday 04 May 2007 18:52, Ingo Molnar wrote:
> agreed. Con, IIRC you wrote a testcase for this, right? Could you please
> send us the results of that testing?

Yes, sorry it's a crappy test app but works on 32bit. Timed with prefetch 
disabled and then enabled swap prefetch saves ~5 seconds on average hardware 
on this one test case. I had many users try this and the results were between 
2 and 10 seconds, but always showed a saving on this testcase. This effect 
easily occurs on printing a big picture, editing a large file, compressing an 
iso image or whatever in real world workloads. Smaller, but much more 
frequent effects of this over the course of a day obviously also occur and do 
add up.

-- 
-ck
#include 
#include 
#include 
#include 
#include 
#include 
#include 

void fatal(const char *format, ...)
{
	va_list ap;

	if (format) {
		va_start(ap, format);
		vfprintf(stderr, format, ap);
		va_end(ap);
	}

	fprintf(stderr, "Fatal error - exiting\n");
	exit(1);
}

size_t get_ram(void)
{
unsigned long ramsize;
	FILE *meminfo;
char aux[256];

	if(!(meminfo = fopen("/proc/meminfo", "r")))
		fatal("fopen\n");

	while( !feof(meminfo) && !fscanf(meminfo, "MemTotal: %lu kB", ) )
		fgets(aux,sizeof(aux),meminfo);
	if (fclose(meminfo) == -1)
		fatal("fclose");
	return ramsize * 1000;
}

unsigned long get_usecs(struct timespec *myts)
{
	if (clock_gettime(CLOCK_REALTIME, myts))
		fatal("clock_gettime");
	return (myts->tv_sec * 100 + myts->tv_nsec / 1000 );
}

int main(void)
{
	unsigned long current_time, time_diff;
	struct timespec myts;
	char *buf1, *buf2, *buf3, *buf4;
	size_t size, full_size = get_ram();
	int sleep_seconds = 600;

	size = full_size * 7 / 10;
	printf("Starting first malloc of %d bytes\n", size);
	buf1 = malloc(size);
	if (buf1 == (char *)-1)
		fatal("Failed to malloc 1st buffer\n");
	memset(buf1, 0, size);
	printf("Completed first malloc and starting second malloc of %d bytes\n", full_size);

	buf2 = malloc(full_size);
	if (buf2 == (char *)-1)
		fatal("Failed to malloc 2nd buffer\n");
	memset(buf2, 0, full_size);
	buf4 = malloc(1);
	for (buf3 = buf2 + full_size; buf3 > buf2; buf3--)
		*buf4 = *buf3;
	free(buf2);
	printf("Completed second malloc and free\n");

	printf("Sleeping for %d seconds\n", sleep_seconds);
	sleep(sleep_seconds);

	printf("Important part - starting read of first malloc\n");
	time_diff = current_time = get_usecs();
	for (buf3 = buf1; buf3 < buf1 + size; buf3++)
		*buf4 = *buf3;
	current_time = get_usecs();
	free(buf4);
	free(buf1);
	printf("Completed read and freeing of first malloc\n");
	time_diff = current_time - time_diff;
	printf("Timed portion %lu microseconds\n",time_diff);

	return 0;
}

Re: swap-prefetch: 2.6.22 -mm merge plans


Ingo Molnar wrote:

* Nick Piggin <[EMAIL PROTECTED]> wrote:



Here were some of my concerns, and where our discussion got up to.



Yes.  Perhaps it just doesn't help with the updatedb thing.  Or 
maybe with normal system activity we get enough free pages to kick 
the thing off and running.  Perhaps updatedb itself has a lot of 
rss, for example.


Could be, but I don't know. I'd think it unlikely to allow _much_ 
swapin, if huge amounts of the desktop have been swapped out. But 
maybe... as I said, nobody seems to have a recipe for these things.



can i take this one as a "no fundamental objection"? There are really 
only 2 maintainance options left:


  1) either you can do it better or at least have a _very_ clearly
 described idea outlined about how to do it differently

  2) or you should let others try it

#1 you've not done for 2-3 years since swap-prefetch was waiting for
integration so it's not an option at this stage anymore. Then you are 
pretty much obliged to do #2. ;-)


The burden is not on me to get someone else's feature merged. If it
can be shown to work well and people's concerns addressed, then anything
will get merged. The reason Linux is so good is because of what we don't
merge, figuratively speaking.

I wanted to see some basic regression tests to show that it hasn't caused
obvious problems, and some basic scenarios where it helps, so that we can
analyse them. It is really simple, but I haven't got any since first
asking.

And note that I don't think I ever explicitly "nacked" anything, just
voiced my concerns. If my concerns had been addressed, then I couldn't
have stopped anybody from merging anything.



2) It is a _highly_ speculative operation, and in workloads where periods
  of low and high page usage with genuinely unused anonymous / tmpfs
  pages, it could waste power, memory bandwidth, bus bandwidth, disk
  bandwidth...


Yes.  I suspect that's a matter of waiting for the corner-case 
reporters to complain, then add more heuristics.


Ugh. Well it is a pretty fundamental problem. Basically swap-prefetch 
is happy to do a _lot_ of work for these things which we have already 
decided are least likely to be used again.



i see no real problem here. We've had heuristics for a _long_ time in 
various areas of the code. Sometimes they work, sometimes they suck.


So that's one of my issues with the code. If all you have to support a
merge is anecodal evidence, then I find it interesting that you would
easily discount something like this.



4) If this is helpful, wouldn't it be equally important for things like
  mapped file pages? Seems like half a solution.


[...]


(otoh the akpm usersapce implementation is swapoff -a;swapon -a)


Perhaps. You may need a few indicators to see whether the system is 
idle... but OTOH, we've already got a lot of indicators for memory, 
disk usage, etc. So, maybe :)



The time has passed for this. Let others play too. Please :-)


Play with what? Prefetching mmaped file pages as well? Sure.


I could be wrong, but IIRC there is no good way to know which cpuset 
to bring the page back into, (and I guess similarly it would be hard 
to know what container to account it to, if doing 
account-on-allocate).



(i think cpusets are totally uninteresting in this context: nobody in 
their right mind is going to use swap-prefetch on a big NUMA box. Nor 
can i see any fundamental impediment to making this more cpuset-aware, 
just like other subsystems were made cpuset-aware, once the requests 
from actual users came in and people started getting interested in it.)


OK, so make it more cpuset aware. This isn't a new issue, I raised it
a long time ago. And trust me, it is a nightmare to just assume that
nobody will use cpusets on a small box for example (AFAIK the resource
control guys are looking at doing just that).

All core VM features should play nicely with each other without *really*
good reason.


I think the "lack of testcase and numbers" is the only valid technical 
objection i've seen so far.


Well you're entitled to your opinion too.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans

2007-05-04 Thread Ingo Molnar

* Nick Piggin <[EMAIL PROTECTED]> wrote:

> > i'm wondering about swap-prefetch:

> Being able to config all these core heuristics changes is really not 
> that much of a positive. The fact that we might _need_ to config 
> something out, and double the configuration range isn't too pleasing.

Well, to the desktop user this is a speculative performance feature that 
he is willing to potentially waste CPU and IO capacity, in expectation 
of better performance.

On the conceptual level it is _precisely the same thing as regular file 
readahead_. (with the difference that to me swapahead seems to be quite 
a bit more intelligent than our current file readahead logic.)

This feature has no API or ABI impact at all, it's a pure performance 
feature. (besides the trivial sysctl to turn it runtime on/off).

> Here were some of my concerns, and where our discussion got up to.

> > Yes.  Perhaps it just doesn't help with the updatedb thing.  Or 
> > maybe with normal system activity we get enough free pages to kick 
> > the thing off and running.  Perhaps updatedb itself has a lot of 
> > rss, for example.
> 
> Could be, but I don't know. I'd think it unlikely to allow _much_ 
> swapin, if huge amounts of the desktop have been swapped out. But 
> maybe... as I said, nobody seems to have a recipe for these things.

can i take this one as a "no fundamental objection"? There are really 
only 2 maintainance options left:

  1) either you can do it better or at least have a _very_ clearly
 described idea outlined about how to do it differently

  2) or you should let others try it

#1 you've not done for 2-3 years since swap-prefetch was waiting for
integration so it's not an option at this stage anymore. Then you are 
pretty much obliged to do #2. ;-)

And let me be really blunt about this, there is no option #3 to say: "I 
have no real better idea, I have no code, I have no time, but hey, lets 
not merge this because it 'could in theory' be possible to do it better" 
=B-)

really, we are likely be better off by risking the merge of _bad_ code 
(which in the swap-prefetch case is the exact opposite of the truth), 
than to let code stagnate. People are clearly unhappy about certain 
desktop aspects of swapping, and the only way out of that is to let more 
people hack that code. Merging code involves more people. It will cause 
'noise' and could cause regressions, but at least in this case the only 
impact is 'performance' and the feature is trivial to disable.

The maintainance drag outside of swap_prefetch.c is essentially _zero_. 
If the feature doesnt work it ends up on Con's desk. If it turns out to 
not work at all (despite years of testing and happy users) it still only 
ends up on Con's desk. A clear win/win scenario for you i think :-)

> > Would be useful to see this claim substantiated with a real 
> > testcase, description of results and an explanation of how and why 
> > it worked.
> 
> Yes... and then try to first improve regular page reclaim and use-once 
> handling.

agreed. Con, IIRC you wrote a testcase for this, right? Could you please 
send us the results of that testing?

> >>2) It is a _highly_ speculative operation, and in workloads where periods
> >>of low and high page usage with genuinely unused anonymous / tmpfs
> >>pages, it could waste power, memory bandwidth, bus bandwidth, disk
> >>bandwidth...
> > 
> > Yes.  I suspect that's a matter of waiting for the corner-case 
> > reporters to complain, then add more heuristics.
> 
> Ugh. Well it is a pretty fundamental problem. Basically swap-prefetch 
> is happy to do a _lot_ of work for these things which we have already 
> decided are least likely to be used again.

i see no real problem here. We've had heuristics for a _long_ time in 
various areas of the code. Sometimes they work, sometimes they suck.

the flow of this is really easy: distro looking for a feature edge turns 
it on and announces it, if the feature does not work out for users then 
user turns it off and complains to distro, if enough users complain then 
distro turns it off for next release, upstream forgets about this 
performance feature and eventually removes it once someone notices that 
it wouldnt even compile in the past 2 main releases. I see no problem 
here, we did that in the past too with performance features. The 
networking stack has literally dozens of such small tunable things which 
get experimented with, and whose defaults do get tuned carefully. Some 
of the knobs help bandwidth, some help latency.

I do not even see any risk of "splitup of mindshare" - swap-prefetch is 
so clearly speculative that it's not really a different view about how 
to do swapping (which would split the tester base, etc.), it's simply a 
"do you want your system to speculate about the future or not" add-on 
decision. Every system has a pretty clear idea about that: desktops 
generally want to do it, clusters generally dont want to do it.

> >>3) I haven't seen a single

Re: swap-prefetch: 2.6.22 -mm merge plans

Ingo Molnar wrote:

* Andrew Morton <[EMAIL PROTECTED]> wrote:

- If replying, please be sure to cc the appropriate individuals.  
 Please also consider rewriting the Subject: to something 
 appropriate.

i'm wondering about swap-prefetch:

Well I had some issues with it that I don't think were fully discussed,
and Andrew prompted me to say something, but it went off list for a
couple of posts (my fault, sorry). Posting it below with Andrew's
permission...

  mm-implement-swap-prefetching.patch
  swap-prefetch-avoid-repeating-entry.patch

add-__gfp_movable-for-callers-to-flag-allocations-from-high-memory-that-may-be-migrated-swap-prefetch.patch

The swap-prefetch feature is relatively compact:

   10 files changed, 745 insertions(+), 1 deletion(-)

it is contained mostly to itself:

   mm/swap_prefetch.c|  581 

i've reviewed it once again and in the !CONFIG_SWAP_PREFETCH case it's a 
clear NOP, while in the CONFIG_SWAP_PREFETCH=y case all the feedback 
i've seen so far was positive. Time to have this upstream and time for a 
desktop-oriented distro to pick it up.

I think this has been held back way too long. It's .config selectable 
and it is as ready for integration as it ever is going to be. So it's a 
win/win scenario.

Being able to config all these core heuristics changes is really not that
much of a positive. The fact that we might _need_ to config something out,
and double the configuration range isn't too pleasing.

Here were some of my concerns, and where our discussion got up to.

Andrew Morton wrote:
> On Fri, 04 May 2007 14:34:45 +1000 Nick Piggin <[EMAIL PROTECTED]> wrote:
>
>
>>Andrew Morton wrote:
>>
>>>istr you had issues with swap-prefetch?
>>>
>>>If so, now's a good time to reiterate them ;)
>>
>>1) It is said to help with the updatedb overnight problem, however it
>>actually _doesn't_ prefetch swap when there are low free pages, which
>>is how updatedb will leave the system. So this really puzzles me how
>>it would work. However if updatedb is causing excessive swapout, I
>>think we should try improving use-once algorithms first, for example.
>
>
> Yes.  Perhaps it just doesn't help with the updatedb thing.  Or maybe with
> normal system activity we get enough free pages to kick the thing off and
> running.  Perhaps updatedb itself has a lot of rss, for example.

Could be, but I don't know. I'd think it unlikely to allow _much_ swapin,
if huge amounts of the desktop have been swapped out. But maybe... as I
said, nobody seems to have a recipe for these things.

> Would be useful to see this claim substantiated with a real testcase,
> description of results and an explanation of how and why it worked.

Yes... and then try to first improve regular page reclaim and use-once
handling.

>>2) It is a _highly_ speculative operation, and in workloads where periods
>>of low and high page usage with genuinely unused anonymous / tmpfs
>>pages, it could waste power, memory bandwidth, bus bandwidth, disk
>>bandwidth...
>
>
> Yes.  I suspect that's a matter of waiting for the corner-case reporters to
> complain, then add more heuristics.

Ugh. Well it is a pretty fundamental problem. Basically swap-prefetch is
happy to do a _lot_ of work for these things which we have already decided
are least likely to be used again.

>>3) I haven't seen a single set of numbers out of it. Feedback seems to
>>have mostly come from people who
>
>
> Yup.  But can we come up with a testcase?  It's hard.

I guess it is hard firstly because swapping is quite random to start with.
But I haven't even seen basic things like "make -jhuge swapstorm has no
regressions".

>>4) If this is helpful, wouldn't it be equally important for things like
>>mapped file pages? Seems like half a solution.
>
>
> True.
>
> Without thinking about it, I almost wonder if one could do a userspace
> implementation with something which pokes around in /proc/pid/pagemap and
> /proc/pid/kpagemap, perhaps with some additional interfaces added to
> do a swapcache read.  (Give userspace the ability to get at swapcache
> via a regular fd?)
>
> (otoh the akpm usersapce implementation is swapoff -a;swapon -a)

Perhaps. You may need a few indicators to see whether the system is idle...
but OTOH, we've already got a lot of indicators for memory, disk usage,
etc. So, maybe :)

>>5) New one: it is possibly going to interact badly with MADV_FREE lazy
>>freeing. The more complex we make page reclaim, the worse IMO.
>
>
> That's a bit vague.  What sort of problems do you envisage?

Well MADV_FREE pages aren't technically free, are they? So it might be
possible for a significant number of them to build up and prevent
swap prefetch from working. Maybe.

>>...) I had a few issues with implementation, like interaction with
>>cpusets. Don't know if these are all fixed or not. I sort of gave
>>up looking at it.
>
>
> Ah yes, I remember some mention of cpusets.

Re: swap-prefetch: 2.6.22 -mm merge plans

Ingo Molnar wrote:

* Andrew Morton [EMAIL PROTECTED] wrote:

- If replying, please be sure to cc the appropriate individuals.
Please also consider rewriting the Subject: to something
appropriate.

i'm wondering about swap-prefetch:

Well I had some issues with it that I don't think were fully discussed,
and Andrew prompted me to say something, but it went off list for a
couple of posts (my fault, sorry). Posting it below with Andrew's
permission...

mm-implement-swap-prefetching.patch
swap-prefetch-avoid-repeating-entry.patch

add-__gfp_movable-for-callers-to-flag-allocations-from-high-memory-that-may-be-migrated-swap-prefetch.patch

The swap-prefetch feature is relatively compact:

10 files changed, 745 insertions(+), 1 deletion(-)

it is contained mostly to itself:

mm/swap_prefetch.c| 581

i've reviewed it once again and in the !CONFIG_SWAP_PREFETCH case it's a
clear NOP, while in the CONFIG_SWAP_PREFETCH=y case all the feedback
i've seen so far was positive. Time to have this upstream and time for a
desktop-oriented distro to pick it up.

I think this has been held back way too long. It's .config selectable
and it is as ready for integration as it ever is going to be. So it's a
win/win scenario.

Being able to config all these core heuristics changes is really not that
much of a positive. The fact that we might _need_ to config something out,
and double the configuration range isn't too pleasing.

Here were some of my concerns, and where our discussion got up to.

Andrew Morton wrote:
On Fri, 04 May 2007 14:34:45 +1000 Nick Piggin [EMAIL PROTECTED] wrote:

Andrew Morton wrote:

istr you had issues with swap-prefetch?

If so, now's a good time to reiterate them ;)

1) It is said to help with the updatedb overnight problem, however it
actually _doesn't_ prefetch swap when there are low free pages, which
is how updatedb will leave the system. So this really puzzles me how
it would work. However if updatedb is causing excessive swapout, I
think we should try improving use-once algorithms first, for example.

Yes. Perhaps it just doesn't help with the updatedb thing. Or maybe with
normal system activity we get enough free pages to kick the thing off and
running. Perhaps updatedb itself has a lot of rss, for example.

Could be, but I don't know. I'd think it unlikely to allow _much_ swapin,
if huge amounts of the desktop have been swapped out. But maybe... as I
said, nobody seems to have a recipe for these things.

Would be useful to see this claim substantiated with a real testcase,
description of results and an explanation of how and why it worked.

Yes... and then try to first improve regular page reclaim and use-once
handling.

2) It is a _highly_ speculative operation, and in workloads where periods
of low and high page usage with genuinely unused anonymous / tmpfs
pages, it could waste power, memory bandwidth, bus bandwidth, disk
bandwidth...

Yes. I suspect that's a matter of waiting for the corner-case reporters to
complain, then add more heuristics.

Ugh. Well it is a pretty fundamental problem. Basically swap-prefetch is
happy to do a _lot_ of work for these things which we have already decided
are least likely to be used again.

3) I haven't seen a single set of numbers out of it. Feedback seems to
have mostly come from people who

Yup. But can we come up with a testcase? It's hard.

I guess it is hard firstly because swapping is quite random to start with.
But I haven't even seen basic things like make -jhuge swapstorm has no
regressions.

4) If this is helpful, wouldn't it be equally important for things like
mapped file pages? Seems like half a solution.

True.

Without thinking about it, I almost wonder if one could do a userspace
implementation with something which pokes around in /proc/pid/pagemap and
/proc/pid/kpagemap, perhaps with some additional interfaces added to
do a swapcache read. (Give userspace the ability to get at swapcache
via a regular fd?)

(otoh the akpm usersapce implementation is swapoff -a;swapon -a)

Perhaps. You may need a few indicators to see whether the system is idle...
but OTOH, we've already got a lot of indicators for memory, disk usage,
etc. So, maybe :)

5) New one: it is possibly going to interact badly with MADV_FREE lazy
freeing. The more complex we make page reclaim, the worse IMO.

That's a bit vague. What sort of problems do you envisage?

Well MADV_FREE pages aren't technically free, are they? So it might be
possible for a significant number of them to build up and prevent
swap prefetch from working. Maybe.

...) I had a few issues with implementation, like interaction with
cpusets. Don't know if these are all fixed or not. I sort of gave
up looking at it.

Ah yes, I remember some mention of cpusets. I forget what it was though.

I could be wrong, but IIRC there is no good way to know which

Re: swap-prefetch: 2.6.22 -mm merge plans

2007-05-04 Thread Ingo Molnar


* Nick Piggin [EMAIL PROTECTED] wrote:

  i'm wondering about swap-prefetch:

 Being able to config all these core heuristics changes is really not 
 that much of a positive. The fact that we might _need_ to config 
 something out, and double the configuration range isn't too pleasing.

Well, to the desktop user this is a speculative performance feature that 
he is willing to potentially waste CPU and IO capacity, in expectation 
of better performance.

On the conceptual level it is _precisely the same thing as regular file 
readahead_. (with the difference that to me swapahead seems to be quite 
a bit more intelligent than our current file readahead logic.)

This feature has no API or ABI impact at all, it's a pure performance 
feature. (besides the trivial sysctl to turn it runtime on/off).

 Here were some of my concerns, and where our discussion got up to.

  Yes.  Perhaps it just doesn't help with the updatedb thing.  Or 
  maybe with normal system activity we get enough free pages to kick 
  the thing off and running.  Perhaps updatedb itself has a lot of 
  rss, for example.
 
 Could be, but I don't know. I'd think it unlikely to allow _much_ 
 swapin, if huge amounts of the desktop have been swapped out. But 
 maybe... as I said, nobody seems to have a recipe for these things.

can i take this one as a no fundamental objection? There are really 
only 2 maintainance options left:

  1) either you can do it better or at least have a _very_ clearly
 described idea outlined about how to do it differently

  2) or you should let others try it

#1 you've not done for 2-3 years since swap-prefetch was waiting for
integration so it's not an option at this stage anymore. Then you are 
pretty much obliged to do #2. ;-)

And let me be really blunt about this, there is no option #3 to say: I 
have no real better idea, I have no code, I have no time, but hey, lets 
not merge this because it 'could in theory' be possible to do it better 
=B-)

really, we are likely be better off by risking the merge of _bad_ code 
(which in the swap-prefetch case is the exact opposite of the truth), 
than to let code stagnate. People are clearly unhappy about certain 
desktop aspects of swapping, and the only way out of that is to let more 
people hack that code. Merging code involves more people. It will cause 
'noise' and could cause regressions, but at least in this case the only 
impact is 'performance' and the feature is trivial to disable.

The maintainance drag outside of swap_prefetch.c is essentially _zero_. 
If the feature doesnt work it ends up on Con's desk. If it turns out to 
not work at all (despite years of testing and happy users) it still only 
ends up on Con's desk. A clear win/win scenario for you i think :-)

  Would be useful to see this claim substantiated with a real 
  testcase, description of results and an explanation of how and why 
  it worked.
 
 Yes... and then try to first improve regular page reclaim and use-once 
 handling.

agreed. Con, IIRC you wrote a testcase for this, right? Could you please 
send us the results of that testing?

 2) It is a _highly_ speculative operation, and in workloads where periods
 of low and high page usage with genuinely unused anonymous / tmpfs
 pages, it could waste power, memory bandwidth, bus bandwidth, disk
 bandwidth...
  
  Yes.  I suspect that's a matter of waiting for the corner-case 
  reporters to complain, then add more heuristics.
 
 Ugh. Well it is a pretty fundamental problem. Basically swap-prefetch 
 is happy to do a _lot_ of work for these things which we have already 
 decided are least likely to be used again.

i see no real problem here. We've had heuristics for a _long_ time in 
various areas of the code. Sometimes they work, sometimes they suck.

the flow of this is really easy: distro looking for a feature edge turns 
it on and announces it, if the feature does not work out for users then 
user turns it off and complains to distro, if enough users complain then 
distro turns it off for next release, upstream forgets about this 
performance feature and eventually removes it once someone notices that 
it wouldnt even compile in the past 2 main releases. I see no problem 
here, we did that in the past too with performance features. The 
networking stack has literally dozens of such small tunable things which 
get experimented with, and whose defaults do get tuned carefully. Some 
of the knobs help bandwidth, some help latency.

I do not even see any risk of splitup of mindshare - swap-prefetch is 
so clearly speculative that it's not really a different view about how 
to do swapping (which would split the tester base, etc.), it's simply a 
do you want your system to speculate about the future or not add-on 
decision. Every system has a pretty clear idea about that: desktops 
generally want to do it, clusters generally dont want to do it.

 3) I haven't seen a single set of numbers out of it. Feedback seems to
 have mostly

Re: swap-prefetch: 2.6.22 -mm merge plans


Ingo Molnar wrote:

* Nick Piggin [EMAIL PROTECTED] wrote:



Here were some of my concerns, and where our discussion got up to.



Yes.  Perhaps it just doesn't help with the updatedb thing.  Or 
maybe with normal system activity we get enough free pages to kick 
the thing off and running.  Perhaps updatedb itself has a lot of 
rss, for example.


Could be, but I don't know. I'd think it unlikely to allow _much_ 
swapin, if huge amounts of the desktop have been swapped out. But 
maybe... as I said, nobody seems to have a recipe for these things.



can i take this one as a no fundamental objection? There are really 
only 2 maintainance options left:


  1) either you can do it better or at least have a _very_ clearly
 described idea outlined about how to do it differently

  2) or you should let others try it

#1 you've not done for 2-3 years since swap-prefetch was waiting for
integration so it's not an option at this stage anymore. Then you are 
pretty much obliged to do #2. ;-)


The burden is not on me to get someone else's feature merged. If it
can be shown to work well and people's concerns addressed, then anything
will get merged. The reason Linux is so good is because of what we don't
merge, figuratively speaking.

I wanted to see some basic regression tests to show that it hasn't caused
obvious problems, and some basic scenarios where it helps, so that we can
analyse them. It is really simple, but I haven't got any since first
asking.

And note that I don't think I ever explicitly nacked anything, just
voiced my concerns. If my concerns had been addressed, then I couldn't
have stopped anybody from merging anything.



2) It is a _highly_ speculative operation, and in workloads where periods
  of low and high page usage with genuinely unused anonymous / tmpfs
  pages, it could waste power, memory bandwidth, bus bandwidth, disk
  bandwidth...


Yes.  I suspect that's a matter of waiting for the corner-case 
reporters to complain, then add more heuristics.


Ugh. Well it is a pretty fundamental problem. Basically swap-prefetch 
is happy to do a _lot_ of work for these things which we have already 
decided are least likely to be used again.



i see no real problem here. We've had heuristics for a _long_ time in 
various areas of the code. Sometimes they work, sometimes they suck.


So that's one of my issues with the code. If all you have to support a
merge is anecodal evidence, then I find it interesting that you would
easily discount something like this.



4) If this is helpful, wouldn't it be equally important for things like
  mapped file pages? Seems like half a solution.


[...]


(otoh the akpm usersapce implementation is swapoff -a;swapon -a)


Perhaps. You may need a few indicators to see whether the system is 
idle... but OTOH, we've already got a lot of indicators for memory, 
disk usage, etc. So, maybe :)



The time has passed for this. Let others play too. Please :-)


Play with what? Prefetching mmaped file pages as well? Sure.


I could be wrong, but IIRC there is no good way to know which cpuset 
to bring the page back into, (and I guess similarly it would be hard 
to know what container to account it to, if doing 
account-on-allocate).



(i think cpusets are totally uninteresting in this context: nobody in 
their right mind is going to use swap-prefetch on a big NUMA box. Nor 
can i see any fundamental impediment to making this more cpuset-aware, 
just like other subsystems were made cpuset-aware, once the requests 
from actual users came in and people started getting interested in it.)


OK, so make it more cpuset aware. This isn't a new issue, I raised it
a long time ago. And trust me, it is a nightmare to just assume that
nobody will use cpusets on a small box for example (AFAIK the resource
control guys are looking at doing just that).

All core VM features should play nicely with each other without *really*
good reason.


I think the lack of testcase and numbers is the only valid technical 
objection i've seen so far.


Well you're entitled to your opinion too.

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans

2007-05-04 Thread Con Kolivas

On Friday 04 May 2007 18:52, Ingo Molnar wrote:
 agreed. Con, IIRC you wrote a testcase for this, right? Could you please
 send us the results of that testing?

Yes, sorry it's a crappy test app but works on 32bit. Timed with prefetch 
disabled and then enabled swap prefetch saves ~5 seconds on average hardware 
on this one test case. I had many users try this and the results were between 
2 and 10 seconds, but always showed a saving on this testcase. This effect 
easily occurs on printing a big picture, editing a large file, compressing an 
iso image or whatever in real world workloads. Smaller, but much more 
frequent effects of this over the course of a day obviously also occur and do 
add up.

-- 
-ck
#include stdio.h
#include stdarg.h
#include stdlib.h
#include string.h
#include unistd.h
#include sys/mman.h
#include time.h

void fatal(const char *format, ...)
{
	va_list ap;

	if (format) {
		va_start(ap, format);
		vfprintf(stderr, format, ap);
		va_end(ap);
	}

	fprintf(stderr, Fatal error - exiting\n);
	exit(1);
}

size_t get_ram(void)
{
unsigned long ramsize;
	FILE *meminfo;
char aux[256];

	if(!(meminfo = fopen(/proc/meminfo, r)))
		fatal(fopen\n);

	while( !feof(meminfo)  !fscanf(meminfo, MemTotal: %lu kB, ramsize) )
		fgets(aux,sizeof(aux),meminfo);
	if (fclose(meminfo) == -1)
		fatal(fclose);
	return ramsize * 1000;
}

unsigned long get_usecs(struct timespec *myts)
{
	if (clock_gettime(CLOCK_REALTIME, myts))
		fatal(clock_gettime);
	return (myts-tv_sec * 100 + myts-tv_nsec / 1000 );
}

int main(void)
{
	unsigned long current_time, time_diff;
	struct timespec myts;
	char *buf1, *buf2, *buf3, *buf4;
	size_t size, full_size = get_ram();
	int sleep_seconds = 600;

	size = full_size * 7 / 10;
	printf(Starting first malloc of %d bytes\n, size);
	buf1 = malloc(size);
	if (buf1 == (char *)-1)
		fatal(Failed to malloc 1st buffer\n);
	memset(buf1, 0, size);
	printf(Completed first malloc and starting second malloc of %d bytes\n, full_size);

	buf2 = malloc(full_size);
	if (buf2 == (char *)-1)
		fatal(Failed to malloc 2nd buffer\n);
	memset(buf2, 0, full_size);
	buf4 = malloc(1);
	for (buf3 = buf2 + full_size; buf3  buf2; buf3--)
		*buf4 = *buf3;
	free(buf2);
	printf(Completed second malloc and free\n);

	printf(Sleeping for %d seconds\n, sleep_seconds);
	sleep(sleep_seconds);

	printf(Important part - starting read of first malloc\n);
	time_diff = current_time = get_usecs(myts);
	for (buf3 = buf1; buf3  buf1 + size; buf3++)
		*buf4 = *buf3;
	current_time = get_usecs(myts);
	free(buf4);
	free(buf1);
	printf(Completed read and freeing of first malloc\n);
	time_diff = current_time - time_diff;
	printf(Timed portion %lu microseconds\n,time_diff);

	return 0;
}

Re: swap-prefetch: 2.6.22 -mm merge plans

2007-05-03 Thread Con Kolivas

On Friday 04 May 2007 01:54, Ingo Molnar wrote:
> * Andrew Morton <[EMAIL PROTECTED]> wrote:
> > - If replying, please be sure to cc the appropriate individuals.
> >   Please also consider rewriting the Subject: to something
> >   appropriate.

> i've reviewed it once again and in the !CONFIG_SWAP_PREFETCH case it's a
> clear NOP, while in the CONFIG_SWAP_PREFETCH=y case all the feedback
> i've seen so far was positive. Time to have this upstream and time for a
> desktop-oriented distro to pick it up.
>
> I think this has been held back way too long. It's .config selectable
> and it is as ready for integration as it ever is going to be. So it's a
> win/win scenario.
>
> Acked-by: Ingo Molnar <[EMAIL PROTECTED]>

Thank you very much for code review, ack and support!

-- 
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


On 03/05/07, Michal Piotrowski <[EMAIL PROTECTED]> wrote:

Hi,

On 03/05/07, Ingo Molnar <[EMAIL PROTECTED]> wrote:
>
> * Andrew Morton <[EMAIL PROTECTED]> wrote:
>
> > - If replying, please be sure to cc the appropriate individuals.
> >   Please also consider rewriting the Subject: to something
> >   appropriate.
>
> i'm wondering about swap-prefetch:
>
>   mm-implement-swap-prefetching.patch
>   swap-prefetch-avoid-repeating-entry.patch
>   
add-__gfp_movable-for-callers-to-flag-allocations-from-high-memory-that-may-be-migrated-swap-prefetch.patch
>
> The swap-prefetch feature is relatively compact:
>
>10 files changed, 745 insertions(+), 1 deletion(-)
>
> it is contained mostly to itself:
>
>mm/swap_prefetch.c|  581 
>
> i've reviewed it once again and in the !CONFIG_SWAP_PREFETCH case it's a
> clear NOP, while in the CONFIG_SWAP_PREFETCH=y case all the feedback
> i've seen so far was positive. Time to have this upstream and time for a
> desktop-oriented distro to pick it up.
>
> I think this has been held back way too long. It's .config selectable
> and it is as ready for integration as it ever is going to be. So it's a
> win/win scenario.

I'm using SWAP_PREFETCH since 2.6.17-mm1 (I don't have earlier configs)


since 2.6.14-mm2 :)
http://lkml.org/lkml/2005/11/11/260

Regards,
Michal

--
Michal K. K. Piotrowski
Kernel Monkeys
(http://kernel.wikidot.com/start)
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans

Hi,

On 03/05/07, Ingo Molnar <[EMAIL PROTECTED]> wrote:

* Andrew Morton <[EMAIL PROTECTED]> wrote:

> - If replying, please be sure to cc the appropriate individuals.
>   Please also consider rewriting the Subject: to something
>   appropriate.

i'm wondering about swap-prefetch:

  mm-implement-swap-prefetching.patch
  swap-prefetch-avoid-repeating-entry.patch

add-__gfp_movable-for-callers-to-flag-allocations-from-high-memory-that-may-be-migrated-swap-prefetch.patch

The swap-prefetch feature is relatively compact:

   10 files changed, 745 insertions(+), 1 deletion(-)

it is contained mostly to itself:

   mm/swap_prefetch.c|  581 

i've reviewed it once again and in the !CONFIG_SWAP_PREFETCH case it's a
clear NOP, while in the CONFIG_SWAP_PREFETCH=y case all the feedback
i've seen so far was positive. Time to have this upstream and time for a
desktop-oriented distro to pick it up.

I think this has been held back way too long. It's .config selectable
and it is as ready for integration as it ever is going to be. So it's a
win/win scenario.

I'm using SWAP_PREFETCH since 2.6.17-mm1 (I don't have earlier configs)
http://www.stardust.webpages.pl/files/tbf/euridica/2.6.17-mm1/mm-config
and I don't recall _any_ problems. It's very stable for me.

Acked-by: Ingo Molnar <[EMAIL PROTECTED]>

Ingo

Regards,
Michal

--
Michal K. K. Piotrowski
Kernel Monkeys
(http://kernel.wikidot.com/start)
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans

2007-05-03 Thread Ingo Molnar

* Andrew Morton <[EMAIL PROTECTED]> wrote:

> - If replying, please be sure to cc the appropriate individuals.  
>   Please also consider rewriting the Subject: to something 
>   appropriate.

i'm wondering about swap-prefetch:

  mm-implement-swap-prefetching.patch
  swap-prefetch-avoid-repeating-entry.patch

add-__gfp_movable-for-callers-to-flag-allocations-from-high-memory-that-may-be-migrated-swap-prefetch.patch

The swap-prefetch feature is relatively compact:

   10 files changed, 745 insertions(+), 1 deletion(-)

it is contained mostly to itself:

   mm/swap_prefetch.c|  581 

i've reviewed it once again and in the !CONFIG_SWAP_PREFETCH case it's a 
clear NOP, while in the CONFIG_SWAP_PREFETCH=y case all the feedback 
i've seen so far was positive. Time to have this upstream and time for a 
desktop-oriented distro to pick it up.

I think this has been held back way too long. It's .config selectable 
and it is as ready for integration as it ever is going to be. So it's a 
win/win scenario.

Acked-by: Ingo Molnar <[EMAIL PROTECTED]>

Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans

2007-05-03 Thread Ingo Molnar


* Andrew Morton [EMAIL PROTECTED] wrote:

 - If replying, please be sure to cc the appropriate individuals.  
   Please also consider rewriting the Subject: to something 
   appropriate.

i'm wondering about swap-prefetch:

  mm-implement-swap-prefetching.patch
  swap-prefetch-avoid-repeating-entry.patch
  
add-__gfp_movable-for-callers-to-flag-allocations-from-high-memory-that-may-be-migrated-swap-prefetch.patch

The swap-prefetch feature is relatively compact:

   10 files changed, 745 insertions(+), 1 deletion(-)

it is contained mostly to itself:

   mm/swap_prefetch.c|  581 

i've reviewed it once again and in the !CONFIG_SWAP_PREFETCH case it's a 
clear NOP, while in the CONFIG_SWAP_PREFETCH=y case all the feedback 
i've seen so far was positive. Time to have this upstream and time for a 
desktop-oriented distro to pick it up.

I think this has been held back way too long. It's .config selectable 
and it is as ready for integration as it ever is going to be. So it's a 
win/win scenario.

Acked-by: Ingo Molnar [EMAIL PROTECTED]

Ingo
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans


Hi,

On 03/05/07, Ingo Molnar [EMAIL PROTECTED] wrote:


* Andrew Morton [EMAIL PROTECTED] wrote:

 - If replying, please be sure to cc the appropriate individuals.
   Please also consider rewriting the Subject: to something
   appropriate.

i'm wondering about swap-prefetch:

  mm-implement-swap-prefetching.patch
  swap-prefetch-avoid-repeating-entry.patch
  
add-__gfp_movable-for-callers-to-flag-allocations-from-high-memory-that-may-be-migrated-swap-prefetch.patch

The swap-prefetch feature is relatively compact:

   10 files changed, 745 insertions(+), 1 deletion(-)

it is contained mostly to itself:

   mm/swap_prefetch.c|  581 

i've reviewed it once again and in the !CONFIG_SWAP_PREFETCH case it's a
clear NOP, while in the CONFIG_SWAP_PREFETCH=y case all the feedback
i've seen so far was positive. Time to have this upstream and time for a
desktop-oriented distro to pick it up.

I think this has been held back way too long. It's .config selectable
and it is as ready for integration as it ever is going to be. So it's a
win/win scenario.


I'm using SWAP_PREFETCH since 2.6.17-mm1 (I don't have earlier configs)
http://www.stardust.webpages.pl/files/tbf/euridica/2.6.17-mm1/mm-config
and I don't recall _any_ problems. It's very stable for me.



Acked-by: Ingo Molnar [EMAIL PROTECTED]

Ingo


Regards,
Michal

--
Michal K. K. Piotrowski
Kernel Monkeys
(http://kernel.wikidot.com/start)
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: swap-prefetch: 2.6.22 -mm merge plans