Re: Linux 2.4.1-ac7
On Tue, 13 Feb 2001, Rik van Riel wrote: > On Tue, 13 Feb 2001, Mike Galbraith wrote: > > On Mon, 12 Feb 2001, Marcelo Tosatti wrote: > > > > > Could you please try the attached patch on top of latest Rik's patch? > > > > Sure thing.. (few minutes later) no change. > > That's because your problem requires a change to the > balancing between swap_out() and refill_inactive_scan() > in refill_inactive()... > > The big problem here is that no matter which magic > proportion between the two functions we use, it'll always > be wrong for a large proportion of the people out there. > > This means we need to have a good way to auto-tune this > thing. I'm thinking of letting swap_out() start out way > less active than refill_inactive_scan() with extra calls > to swapout being made from refill_inactive_scan when we > think it's needed... > > (... I'm writing a patch right now ...) We're using nr_async_pages to calculate the number of pages which should be flushed, but nr_async_pages counts on flight swap _readaheads_ (each swapin increases nr_async_pages by (1 << page_cluster)) and writes, not only writes. That makes the "pageout free shortage and sleep" kswapd behaviour you wanted a bit messy. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Tue, 13 Feb 2001, Rik van Riel wrote: On Tue, 13 Feb 2001, Mike Galbraith wrote: On Mon, 12 Feb 2001, Marcelo Tosatti wrote: Could you please try the attached patch on top of latest Rik's patch? Sure thing.. (few minutes later) no change. That's because your problem requires a change to the balancing between swap_out() and refill_inactive_scan() in refill_inactive()... The big problem here is that no matter which magic proportion between the two functions we use, it'll always be wrong for a large proportion of the people out there. This means we need to have a good way to auto-tune this thing. I'm thinking of letting swap_out() start out way less active than refill_inactive_scan() with extra calls to swapout being made from refill_inactive_scan when we think it's needed... (... I'm writing a patch right now ...) We're using nr_async_pages to calculate the number of pages which should be flushed, but nr_async_pages counts on flight swap _readaheads_ (each swapin increases nr_async_pages by (1 page_cluster)) and writes, not only writes. That makes the "pageout free shortage and sleep" kswapd behaviour you wanted a bit messy. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Tue, 13 Feb 2001, Mike Galbraith wrote: > On Mon, 12 Feb 2001, Marcelo Tosatti wrote: > > > Could you please try the attached patch on top of latest Rik's patch? > > Sure thing.. (few minutes later) no change. That's because your problem requires a change to the balancing between swap_out() and refill_inactive_scan() in refill_inactive()... The big problem here is that no matter which magic proportion between the two functions we use, it'll always be wrong for a large proportion of the people out there. This means we need to have a good way to auto-tune this thing. I'm thinking of letting swap_out() start out way less active than refill_inactive_scan() with extra calls to swapout being made from refill_inactive_scan when we think it's needed... (... I'm writing a patch right now ...) regards, Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com.br/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Tue, 13 Feb 2001, Mike Galbraith wrote: On Mon, 12 Feb 2001, Marcelo Tosatti wrote: Could you please try the attached patch on top of latest Rik's patch? Sure thing.. (few minutes later) no change. That's because your problem requires a change to the balancing between swap_out() and refill_inactive_scan() in refill_inactive()... The big problem here is that no matter which magic proportion between the two functions we use, it'll always be wrong for a large proportion of the people out there. This means we need to have a good way to auto-tune this thing. I'm thinking of letting swap_out() start out way less active than refill_inactive_scan() with extra calls to swapout being made from refill_inactive_scan when we think it's needed... (... I'm writing a patch right now ...) regards, Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com.br/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Mon, 12 Feb 2001, Marcelo Tosatti wrote: > On Sun, 11 Feb 2001, Mike Galbraith wrote: > > > On Sun, 11 Feb 2001, Rik van Riel wrote: > > > > > On Sun, 11 Feb 2001, Mike Galbraith wrote: > > > > On Sun, 11 Feb 2001, Mike Galbraith wrote: > > > > > > > > > Something else I see while watching it run: MUCH more swapout than > > > > > swapin. Does that mean we're sending pages to swap only to find out > > > > > that we never need them again? > > > > > > > > (numbers might be more descriptive) > > > > > > > > user : 0:07:21.70 54.3% page in : 142613 > > > > nice : 0:00:00.00 0.0% page out: 155454 > > > > system: 0:03:40.63 27.1% swap in :56334 > > > > idle : 0:02:30.50 18.5% swap out: 149872 > > > > uptime: 0:13:32.83 context : 519726 > > > > > > Indeed, in this case we send a lot more pages to swap > > > than we read back in from swap, this means that the > > > data is still sitting in swap space and was never needed > > > again. > > > > But it looks and feels (box is I/O hyper-saturated) like > > it wrote it all to disk. > > > > (btw, ac5 does more disk read.. ie the reduced cache size > > of earlier kernels under heavy pressure does have it's price > > with this workload.. quite visible in agregates. looks to > > be much cheaper than swap though.. for this workload) > > Mike, > > Could you please try the attached patch on top of latest Rik's patch? Sure thing.. (few minutes later) no change. -Mike P.S. (said I'd try other loads...) I tried a different workload yesterday (only one because it took the entire day to finish). Glimpseindexing my webserver test area played well. As it was running, all of it's growth was pushed to swap (very slow trickle for some hours). I figured it would get beaten up when it started actually using the dataset it was building, but the way it used it, sending it to swap was perfect. When it started using, it paged in a small burst and then used this data combined with a truckload of I/O (actual database building bit). Since there were hours between dataset generation and subsequent use, I had free use of about ~30 mb of pages for those hours. Doing other things (like letting lynx webcrawl over my other box's server area while it was running) were not even visible.. ie caused no additional paging, so huge cache/buffers space (~100mb of 128mb) was not plugging up the mana supply in any way. What was in cache _looked_ to be old cruft, and the system released these resources just fine. (so why won't it give up cache to gcc? either the vm doesn't like gcc, or I flat don't understand page aging yet. nod;) Very boring test session. I hope the results mean more to you than they do to me ;-) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sun, 11 Feb 2001, Mike Galbraith wrote:
> On Sun, 11 Feb 2001, Rik van Riel wrote:
>
> > On Sun, 11 Feb 2001, Mike Galbraith wrote:
> > > On Sun, 11 Feb 2001, Mike Galbraith wrote:
> > >
> > > > Something else I see while watching it run: MUCH more swapout than
> > > > swapin. Does that mean we're sending pages to swap only to find out
> > > > that we never need them again?
> > >
> > > (numbers might be more descriptive)
> > >
> > > user : 0:07:21.70 54.3% page in : 142613
> > > nice : 0:00:00.00 0.0% page out: 155454
> > > system: 0:03:40.63 27.1% swap in :56334
> > > idle : 0:02:30.50 18.5% swap out: 149872
> > > uptime: 0:13:32.83 context : 519726
> >
> > Indeed, in this case we send a lot more pages to swap
> > than we read back in from swap, this means that the
> > data is still sitting in swap space and was never needed
> > again.
>
> But it looks and feels (box is I/O hyper-saturated) like
> it wrote it all to disk.
>
> (btw, ac5 does more disk read.. ie the reduced cache size
> of earlier kernels under heavy pressure does have it's price
> with this workload.. quite visible in agregates. looks to
> be much cheaper than swap though.. for this workload)
Mike,
Could you please try the attached patch on top of latest Rik's patch?
Thanks!
--- linux.orig/mm/vmscan.c Sun Feb 11 07:56:29 2001
+++ linux/mm/vmscan.c Sun Feb 11 11:05:30 2001
@@ -563,7 +566,8 @@
/* The buffers were not freed. */
if (!clearedbuf) {
add_page_to_inactive_dirty_list(page);
- flushed_pages++;
+ if (wait)
+ flushed_pages++;
/* The page was only in the buffer cache. */
} else if (!page->mapping) {
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://vger.kernel.org/lkml/
Re: Linux 2.4.1-ac7
On Sun, 11 Feb 2001, Mike Galbraith wrote:
On Sun, 11 Feb 2001, Rik van Riel wrote:
On Sun, 11 Feb 2001, Mike Galbraith wrote:
On Sun, 11 Feb 2001, Mike Galbraith wrote:
Something else I see while watching it run: MUCH more swapout than
swapin. Does that mean we're sending pages to swap only to find out
that we never need them again?
(numbers might be more descriptive)
user : 0:07:21.70 54.3% page in : 142613
nice : 0:00:00.00 0.0% page out: 155454
system: 0:03:40.63 27.1% swap in :56334
idle : 0:02:30.50 18.5% swap out: 149872
uptime: 0:13:32.83 context : 519726
Indeed, in this case we send a lot more pages to swap
than we read back in from swap, this means that the
data is still sitting in swap space and was never needed
again.
But it looks and feels (box is I/O hyper-saturated) like
it wrote it all to disk.
(btw, ac5 does more disk read.. ie the reduced cache size
of earlier kernels under heavy pressure does have it's price
with this workload.. quite visible in agregates. looks to
be much cheaper than swap though.. for this workload)
Mike,
Could you please try the attached patch on top of latest Rik's patch?
Thanks!
--- linux.orig/mm/vmscan.c Sun Feb 11 07:56:29 2001
+++ linux/mm/vmscan.c Sun Feb 11 11:05:30 2001
@@ -563,7 +566,8 @@
/* The buffers were not freed. */
if (!clearedbuf) {
add_page_to_inactive_dirty_list(page);
- flushed_pages++;
+ if (wait)
+ flushed_pages++;
/* The page was only in the buffer cache. */
} else if (!page-mapping) {
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://vger.kernel.org/lkml/
Re: Linux 2.4.1-ac7
On Mon, 12 Feb 2001, Marcelo Tosatti wrote: On Sun, 11 Feb 2001, Mike Galbraith wrote: On Sun, 11 Feb 2001, Rik van Riel wrote: On Sun, 11 Feb 2001, Mike Galbraith wrote: On Sun, 11 Feb 2001, Mike Galbraith wrote: Something else I see while watching it run: MUCH more swapout than swapin. Does that mean we're sending pages to swap only to find out that we never need them again? (numbers might be more descriptive) user : 0:07:21.70 54.3% page in : 142613 nice : 0:00:00.00 0.0% page out: 155454 system: 0:03:40.63 27.1% swap in :56334 idle : 0:02:30.50 18.5% swap out: 149872 uptime: 0:13:32.83 context : 519726 Indeed, in this case we send a lot more pages to swap than we read back in from swap, this means that the data is still sitting in swap space and was never needed again. But it looks and feels (box is I/O hyper-saturated) like it wrote it all to disk. (btw, ac5 does more disk read.. ie the reduced cache size of earlier kernels under heavy pressure does have it's price with this workload.. quite visible in agregates. looks to be much cheaper than swap though.. for this workload) Mike, Could you please try the attached patch on top of latest Rik's patch? Sure thing.. (few minutes later) no change. -Mike P.S. (said I'd try other loads...) I tried a different workload yesterday (only one because it took the entire day to finish). Glimpseindexing my webserver test area played well. As it was running, all of it's growth was pushed to swap (very slow trickle for some hours). I figured it would get beaten up when it started actually using the dataset it was building, but the way it used it, sending it to swap was perfect. When it started using, it paged in a small burst and then used this data combined with a truckload of I/O (actual database building bit). Since there were hours between dataset generation and subsequent use, I had free use of about ~30 mb of pages for those hours. Doing other things (like letting lynx webcrawl over my other box's server area while it was running) were not even visible.. ie caused no additional paging, so huge cache/buffers space (~100mb of 128mb) was not plugging up the mana supply in any way. What was in cache _looked_ to be old cruft, and the system released these resources just fine. (so why won't it give up cache to gcc? either the vm doesn't like gcc, or I flat don't understand page aging yet. nod;) Very boring test session. I hope the results mean more to you than they do to me ;-) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sun, 11 Feb 2001, Rik van Riel wrote: > On Sun, 11 Feb 2001, Mike Galbraith wrote: > > On Sun, 11 Feb 2001, Mike Galbraith wrote: > > > > > Something else I see while watching it run: MUCH more swapout than > > > swapin. Does that mean we're sending pages to swap only to find out > > > that we never need them again? > > > > (numbers might be more descriptive) > > > > user : 0:07:21.70 54.3% page in : 142613 > > nice : 0:00:00.00 0.0% page out: 155454 > > system: 0:03:40.63 27.1% swap in :56334 > > idle : 0:02:30.50 18.5% swap out: 149872 > > uptime: 0:13:32.83 context : 519726 > > Indeed, in this case we send a lot more pages to swap > than we read back in from swap, this means that the > data is still sitting in swap space and was never needed > again. But it looks and feels (box is I/O hyper-saturated) like it wrote it all to disk. (btw, ac5 does more disk read.. ie the reduced cache size of earlier kernels under heavy pressure does have it's price with this workload.. quite visible in agregates. looks to be much cheaper than swap though.. for this workload) -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sun, 11 Feb 2001, Mike Galbraith wrote: > On Sun, 11 Feb 2001, Mike Galbraith wrote: > > > Something else I see while watching it run: MUCH more swapout than > > swapin. Does that mean we're sending pages to swap only to find out > > that we never need them again? > > (numbers might be more descriptive) > > user : 0:07:21.70 54.3% page in : 142613 > nice : 0:00:00.00 0.0% page out: 155454 > system: 0:03:40.63 27.1% swap in :56334 > idle : 0:02:30.50 18.5% swap out: 149872 > uptime: 0:13:32.83 context : 519726 Indeed, in this case we send a lot more pages to swap than we read back in from swap, this means that the data is still sitting in swap space and was never needed again. regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sun, 11 Feb 2001 [EMAIL PROTECTED] wrote: > On Sat, 10 Feb 2001, Mike Galbraith wrote: > > > > > o Rebalance the 2.4.1 VM (Rik van Riel) > > This change makes my box swap madly under load. It appears to be > > keeping more cache around than is really needed, and therefore > > having to resort to swap instead. The result is MUCH more I/O than > > previous kernels while doing the same exact job. > > I concur this, > It appeared, that rather than free the cached buffers and reuse > the memory, it preferred to hit swap space. Streaming I/O > performance seems to have taken a hit lately. OK, so we're back to the old VM magic number game again ;( In short, we have to be more agressive towards unmapped cache pages than towards mapped pages in processes, except that this horribly breaks down when somebody does streaming IO using mmap while somebody else is at the same time re-using data from cached files (say, .h files)... Now the question is ... WHY do we need to change this behaviour and HOW exactly should it be changed ? I don't really feel comfortable just tweaking stuff until we get a half-dozen benchmarks right, I think we need to understand what is happening and change things accordingly. It's fine with me to put some temporary thing in place to get at least -ac5 behaviour back, but I don't think we should have this as a long-term thing. regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sun, 11 Feb 2001, Mike Galbraith wrote: > Something else I see while watching it run: MUCH more swapout than > swapin. Does that mean we're sending pages to swap only to find out > that we never need them again? (numbers might be more descriptive) user : 0:07:21.70 54.3% page in : 142613 nice : 0:00:00.00 0.0% page out: 155454 system: 0:03:40.63 27.1% swap in :56334 idle : 0:02:30.50 18.5% swap out: 149872 uptime: 0:13:32.83 context : 519726 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sun, 11 Feb 2001, Mike Galbraith wrote: Something else I see while watching it run: MUCH more swapout than swapin. Does that mean we're sending pages to swap only to find out that we never need them again? (numbers might be more descriptive) user : 0:07:21.70 54.3% page in : 142613 nice : 0:00:00.00 0.0% page out: 155454 system: 0:03:40.63 27.1% swap in :56334 idle : 0:02:30.50 18.5% swap out: 149872 uptime: 0:13:32.83 context : 519726 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sun, 11 Feb 2001 [EMAIL PROTECTED] wrote: On Sat, 10 Feb 2001, Mike Galbraith wrote: o Rebalance the 2.4.1 VM (Rik van Riel) This change makes my box swap madly under load. It appears to be keeping more cache around than is really needed, and therefore having to resort to swap instead. The result is MUCH more I/O than previous kernels while doing the same exact job. I concur this, It appeared, that rather than free the cached buffers and reuse the memory, it preferred to hit swap space. Streaming I/O performance seems to have taken a hit lately. OK, so we're back to the old VM magic number game again ;( In short, we have to be more agressive towards unmapped cache pages than towards mapped pages in processes, except that this horribly breaks down when somebody does streaming IO using mmap while somebody else is at the same time re-using data from cached files (say, .h files)... Now the question is ... WHY do we need to change this behaviour and HOW exactly should it be changed ? I don't really feel comfortable just tweaking stuff until we get a half-dozen benchmarks right, I think we need to understand what is happening and change things accordingly. It's fine with me to put some temporary thing in place to get at least -ac5 behaviour back, but I don't think we should have this as a long-term thing. regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sun, 11 Feb 2001, Mike Galbraith wrote: On Sun, 11 Feb 2001, Mike Galbraith wrote: Something else I see while watching it run: MUCH more swapout than swapin. Does that mean we're sending pages to swap only to find out that we never need them again? (numbers might be more descriptive) user : 0:07:21.70 54.3% page in : 142613 nice : 0:00:00.00 0.0% page out: 155454 system: 0:03:40.63 27.1% swap in :56334 idle : 0:02:30.50 18.5% swap out: 149872 uptime: 0:13:32.83 context : 519726 Indeed, in this case we send a lot more pages to swap than we read back in from swap, this means that the data is still sitting in swap space and was never needed again. regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sun, 11 Feb 2001, Rik van Riel wrote: On Sun, 11 Feb 2001, Mike Galbraith wrote: On Sun, 11 Feb 2001, Mike Galbraith wrote: Something else I see while watching it run: MUCH more swapout than swapin. Does that mean we're sending pages to swap only to find out that we never need them again? (numbers might be more descriptive) user : 0:07:21.70 54.3% page in : 142613 nice : 0:00:00.00 0.0% page out: 155454 system: 0:03:40.63 27.1% swap in :56334 idle : 0:02:30.50 18.5% swap out: 149872 uptime: 0:13:32.83 context : 519726 Indeed, in this case we send a lot more pages to swap than we read back in from swap, this means that the data is still sitting in swap space and was never needed again. But it looks and feels (box is I/O hyper-saturated) like it wrote it all to disk. (btw, ac5 does more disk read.. ie the reduced cache size of earlier kernels under heavy pressure does have it's price with this workload.. quite visible in agregates. looks to be much cheaper than swap though.. for this workload) -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sat, 10 Feb 2001, Rik van Riel wrote: > On Sat, 10 Feb 2001, Mike Galbraith wrote: > > On Sat, 10 Feb 2001, Rik van Riel wrote: > > > On Sat, 10 Feb 2001, Marcelo Tosatti wrote: > > > > On Sat, 10 Feb 2001, Mike Galbraith wrote: > > > > > > > > > This change makes my box swap madly under load. > > > > > > > > Swapped out pages were not being counted in the flushing limitation. > > > > > > > > Could you try the following patch? > > > > > > Marcelo's patch should do the trick wrt. to making page_launder() > > > well-behaved again. It should fix the problems some people have > > > seen with bursty swap behaviour. > > > > It's still reluctant to shrink cache. I'm hitting I/O saturation > > at 20 jobs vs 30 with ac5. (difference seems to be the delta in > > space taken by cache.. ~same space shows as additional swap volume). > > Indeed, to "fix" that we'll need to work at refill_inactive(). If this reluctance to munch cache can be relaxed a little, I think we'll see the end of a long standing problem. I often see a scenario wherein we flush everything flushable, then steal the entire cache before doing any paging. The result (we hit a wall) is a mondo swapout followed immediately by swapping it all right back in. We seem to have done a complete turnaround wrt paging vs flush/cache reap preference, and that does effectively cure this scenario.. but methinks optimal (-ENOENT?) lies somewhere in between. > However, I am very much against tuning the VM for one particular > workload. If you can show me that this problem also happens under > other workloads we can work at changing it, but I don't think it's > right to optimise the VM for a specific workload... I'll watch behavior under other loads. (I don't have enough network capacity to do anything stressful there, and whatever load I pick has to be compute bound as to not end up benchmarking my modest I/O capacity.. suggestions welcome. I use make -j primarily because it doesn't need much I/O bandwidth for itself, but does allocate quite a bit.. that leaves most I/O capacity free for vm usage) Something else I see while watching it run: MUCH more swapout than swapin. Does that mean we're sending pages to swap only to find out that we never need them again? -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sat, 10 Feb 2001, Mike Galbraith wrote: > > > o Rebalance the 2.4.1 VM (Rik van Riel) > This change makes my box swap madly under load. It appears to be > keeping more cache around than is really needed, and therefore > having to resort to swap instead. The result is MUCH more I/O than > previous kernels while doing the same exact job. I concur this, I watched a DVD tonight, and actually it got so bad I had to reboot at one point as the it became too jerky to watch. free output looked like this at this point... total used free sharedbuffers cached Mem:254960 253252 1708 0 24116 174500 -/+ buffers/cache: 54636 200324 Swap: 248996 20848 228148 It appeared, that rather than free the cached buffers and reuse the memory, it preferred to hit swap space. Streaming I/O performance seems to have taken a hit lately. (This was 2.4.1-ac9 btw) regards, Dave. -- | Dave Jones.http://www.suse.de/~davej | SuSE Labs - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
Hello Rik , As an aside to the below conversation . Is there a URL/doc/... that gives basic tuning examples for various types workloads ? Tia , JimL On Sat, 10 Feb 2001, Rik van Riel wrote: ...snip... > > It's still reluctant to shrink cache. I'm hitting I/O saturation > > at 20 jobs vs 30 with ac5. (difference seems to be the delta in > > space taken by cache.. ~same space shows as additional swap volume). > > Indeed, to "fix" that we'll need to work at refill_inactive(). > > However, I am very much against tuning the VM for one particular > workload. If you can show me that this problem also happens under > other workloads we can work at changing it, but I don't think it's ...snip... ++ | James W. Laferriere | System Techniques | Give me VMS | | NetworkEngineer | 25416 22nd So | Give me Linux | | [EMAIL PROTECTED] | DesMoines WA 98198 | only on AXP | ++ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sat, 10 Feb 2001, Mike Galbraith wrote: > On Sat, 10 Feb 2001, Rik van Riel wrote: > > On Sat, 10 Feb 2001, Marcelo Tosatti wrote: > > > On Sat, 10 Feb 2001, Mike Galbraith wrote: > > > > > > > This change makes my box swap madly under load. > > > > > > Swapped out pages were not being counted in the flushing limitation. > > > > > > Could you try the following patch? > > > > Marcelo's patch should do the trick wrt. to making page_launder() > > well-behaved again. It should fix the problems some people have > > seen with bursty swap behaviour. > > It's still reluctant to shrink cache. I'm hitting I/O saturation > at 20 jobs vs 30 with ac5. (difference seems to be the delta in > space taken by cache.. ~same space shows as additional swap volume). Indeed, to "fix" that we'll need to work at refill_inactive(). However, I am very much against tuning the VM for one particular workload. If you can show me that this problem also happens under other workloads we can work at changing it, but I don't think it's right to optimise the VM for a specific workload... regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sat, 10 Feb 2001, Rik van Riel wrote: > On Sat, 10 Feb 2001, Marcelo Tosatti wrote: > > On Sat, 10 Feb 2001, Mike Galbraith wrote: > > > > > This change makes my box swap madly under load. > > > > Swapped out pages were not being counted in the flushing limitation. > > > > Could you try the following patch? > > Marcelo's patch should do the trick wrt. to making page_launder() > well-behaved again. It should fix the problems some people have > seen with bursty swap behaviour. It's still reluctant to shrink cache. I'm hitting I/O saturation at 20 jobs vs 30 with ac5. (difference seems to be the delta in space taken by cache.. ~same space shows as additional swap volume). -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sat, 10 Feb 2001, Marcelo Tosatti wrote: > On Sat, 10 Feb 2001, Mike Galbraith wrote: > > > This change makes my box swap madly under load. > > Swapped out pages were not being counted in the flushing limitation. > > Could you try the following patch? Marcelo's patch should do the trick wrt. to making page_launder() well-behaved again. It should fix the problems some people have seen with bursty swap behaviour. > --- linux.orig/mm/vmscan.c Sat Feb 10 08:26:17 2001 > +++ linux/mm/vmscan.c Sat Feb 10 09:34:20 2001 > @@ -515,6 +515,7 @@ > > writepage(page); > flushed_pages++; > + max_launder--; > page_cache_release(page); > > /* And re-start the thing.. */ Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sat, 10 Feb 2001, Mike Galbraith wrote: > Hi Rik, > > This change makes my box swap madly under load. It appears to be > keeping more cache around than is really needed, and therefore > having to resort to swap instead. The result is MUCH more I/O than > previous kernels while doing the same exact job. > > My test load is make -jN bzImage. Previous kernels kept cache at > an average of ~20ish mb at a job level N at which level I had nearly > zero measurable throughput loss compared to single task compile. > > >>From that, I surmise that the cachable component of this job must > fit in that roughly 20ish mb of space. (for otherwise, I would be > suffering throughput loss). With this vm change, cache is nearly > three times as large as usual. Where 30 tasks will run with only > modest throughput loss in ac5, ac8 throughput tapers off rapidly > at half of that. Swapped out pages were not being counted in the flushing limitation. Could you try the following patch? Thanks --- linux.orig/mm/vmscan.c Sat Feb 10 08:26:17 2001 +++ linux/mm/vmscan.c Sat Feb 10 09:34:20 2001 @@ -515,6 +515,7 @@ writepage(page); flushed_pages++; + max_launder--; page_cache_release(page); /* And re-start the thing.. */ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Thu, 8 Feb 2001, Rik van Riel wrote: > On Thu, 8 Feb 2001, Alan Cox wrote: > > > ftp://ftp.kernel.org/pub/linux/kernel/people/alan/2.4/ > > > > 2.4.1-ac7 > > o Rebalance the 2.4.1 VM (Rik van Riel) > > | This should make things feel a lot faster especially > > | on small boxes .. feedback to Rik > > I'd really like feedback from people when it comes to this > change. The change /should/ fix most paging performance bugs > because it makes kswapd do the right amount of work in order > to solve the free memory shortage every time it is run. Hi Rik, This change makes my box swap madly under load. It appears to be keeping more cache around than is really needed, and therefore having to resort to swap instead. The result is MUCH more I/O than previous kernels while doing the same exact job. My test load is make -jN bzImage. Previous kernels kept cache at an average of ~20ish mb at a job level N at which level I had nearly zero measurable throughput loss compared to single task compile. >From that, I surmise that the cachable component of this job must fit in that roughly 20ish mb of space. (for otherwise, I would be suffering throughput loss). With this vm change, cache is nearly three times as large as usual. Where 30 tasks will run with only modest throughput loss in ac5, ac8 throughput tapers off rapidly at half of that. -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Thu, 8 Feb 2001, Rik van Riel wrote: On Thu, 8 Feb 2001, Alan Cox wrote: ftp://ftp.kernel.org/pub/linux/kernel/people/alan/2.4/ 2.4.1-ac7 o Rebalance the 2.4.1 VM (Rik van Riel) | This should make things feel a lot faster especially | on small boxes .. feedback to Rik I'd really like feedback from people when it comes to this change. The change /should/ fix most paging performance bugs because it makes kswapd do the right amount of work in order to solve the free memory shortage every time it is run. Hi Rik, This change makes my box swap madly under load. It appears to be keeping more cache around than is really needed, and therefore having to resort to swap instead. The result is MUCH more I/O than previous kernels while doing the same exact job. My test load is make -jN bzImage. Previous kernels kept cache at an average of ~20ish mb at a job level N at which level I had nearly zero measurable throughput loss compared to single task compile. From that, I surmise that the cachable component of this job must fit in that roughly 20ish mb of space. (for otherwise, I would be suffering throughput loss). With this vm change, cache is nearly three times as large as usual. Where 30 tasks will run with only modest throughput loss in ac5, ac8 throughput tapers off rapidly at half of that. -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sat, 10 Feb 2001, Mike Galbraith wrote: Hi Rik, This change makes my box swap madly under load. It appears to be keeping more cache around than is really needed, and therefore having to resort to swap instead. The result is MUCH more I/O than previous kernels while doing the same exact job. My test load is make -jN bzImage. Previous kernels kept cache at an average of ~20ish mb at a job level N at which level I had nearly zero measurable throughput loss compared to single task compile. From that, I surmise that the cachable component of this job must fit in that roughly 20ish mb of space. (for otherwise, I would be suffering throughput loss). With this vm change, cache is nearly three times as large as usual. Where 30 tasks will run with only modest throughput loss in ac5, ac8 throughput tapers off rapidly at half of that. Swapped out pages were not being counted in the flushing limitation. Could you try the following patch? Thanks --- linux.orig/mm/vmscan.c Sat Feb 10 08:26:17 2001 +++ linux/mm/vmscan.c Sat Feb 10 09:34:20 2001 @@ -515,6 +515,7 @@ writepage(page); flushed_pages++; + max_launder--; page_cache_release(page); /* And re-start the thing.. */ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sat, 10 Feb 2001, Marcelo Tosatti wrote: On Sat, 10 Feb 2001, Mike Galbraith wrote: This change makes my box swap madly under load. Swapped out pages were not being counted in the flushing limitation. Could you try the following patch? Marcelo's patch should do the trick wrt. to making page_launder() well-behaved again. It should fix the problems some people have seen with bursty swap behaviour. --- linux.orig/mm/vmscan.c Sat Feb 10 08:26:17 2001 +++ linux/mm/vmscan.c Sat Feb 10 09:34:20 2001 @@ -515,6 +515,7 @@ writepage(page); flushed_pages++; + max_launder--; page_cache_release(page); /* And re-start the thing.. */ Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sat, 10 Feb 2001, Rik van Riel wrote: On Sat, 10 Feb 2001, Marcelo Tosatti wrote: On Sat, 10 Feb 2001, Mike Galbraith wrote: This change makes my box swap madly under load. Swapped out pages were not being counted in the flushing limitation. Could you try the following patch? Marcelo's patch should do the trick wrt. to making page_launder() well-behaved again. It should fix the problems some people have seen with bursty swap behaviour. It's still reluctant to shrink cache. I'm hitting I/O saturation at 20 jobs vs 30 with ac5. (difference seems to be the delta in space taken by cache.. ~same space shows as additional swap volume). -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sat, 10 Feb 2001, Mike Galbraith wrote: On Sat, 10 Feb 2001, Rik van Riel wrote: On Sat, 10 Feb 2001, Marcelo Tosatti wrote: On Sat, 10 Feb 2001, Mike Galbraith wrote: This change makes my box swap madly under load. Swapped out pages were not being counted in the flushing limitation. Could you try the following patch? Marcelo's patch should do the trick wrt. to making page_launder() well-behaved again. It should fix the problems some people have seen with bursty swap behaviour. It's still reluctant to shrink cache. I'm hitting I/O saturation at 20 jobs vs 30 with ac5. (difference seems to be the delta in space taken by cache.. ~same space shows as additional swap volume). Indeed, to "fix" that we'll need to work at refill_inactive(). However, I am very much against tuning the VM for one particular workload. If you can show me that this problem also happens under other workloads we can work at changing it, but I don't think it's right to optimise the VM for a specific workload... regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
Hello Rik , As an aside to the below conversation . Is there a URL/doc/... that gives basic tuning examples for various types workloads ? Tia , JimL On Sat, 10 Feb 2001, Rik van Riel wrote: ...snip... It's still reluctant to shrink cache. I'm hitting I/O saturation at 20 jobs vs 30 with ac5. (difference seems to be the delta in space taken by cache.. ~same space shows as additional swap volume). Indeed, to "fix" that we'll need to work at refill_inactive(). However, I am very much against tuning the VM for one particular workload. If you can show me that this problem also happens under other workloads we can work at changing it, but I don't think it's ...snip... ++ | James W. Laferriere | System Techniques | Give me VMS | | NetworkEngineer | 25416 22nd So | Give me Linux | | [EMAIL PROTECTED] | DesMoines WA 98198 | only on AXP | ++ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sat, 10 Feb 2001, Mike Galbraith wrote: o Rebalance the 2.4.1 VM (Rik van Riel) This change makes my box swap madly under load. It appears to be keeping more cache around than is really needed, and therefore having to resort to swap instead. The result is MUCH more I/O than previous kernels while doing the same exact job. I concur this, I watched a DVD tonight, and actually it got so bad I had to reboot at one point as the it became too jerky to watch. free output looked like this at this point... total used free sharedbuffers cached Mem:254960 253252 1708 0 24116 174500 -/+ buffers/cache: 54636 200324 Swap: 248996 20848 228148 It appeared, that rather than free the cached buffers and reuse the memory, it preferred to hit swap space. Streaming I/O performance seems to have taken a hit lately. (This was 2.4.1-ac9 btw) regards, Dave. -- | Dave Jones.http://www.suse.de/~davej | SuSE Labs - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Sat, 10 Feb 2001, Rik van Riel wrote: On Sat, 10 Feb 2001, Mike Galbraith wrote: On Sat, 10 Feb 2001, Rik van Riel wrote: On Sat, 10 Feb 2001, Marcelo Tosatti wrote: On Sat, 10 Feb 2001, Mike Galbraith wrote: This change makes my box swap madly under load. Swapped out pages were not being counted in the flushing limitation. Could you try the following patch? Marcelo's patch should do the trick wrt. to making page_launder() well-behaved again. It should fix the problems some people have seen with bursty swap behaviour. It's still reluctant to shrink cache. I'm hitting I/O saturation at 20 jobs vs 30 with ac5. (difference seems to be the delta in space taken by cache.. ~same space shows as additional swap volume). Indeed, to "fix" that we'll need to work at refill_inactive(). If this reluctance to munch cache can be relaxed a little, I think we'll see the end of a long standing problem. I often see a scenario wherein we flush everything flushable, then steal the entire cache before doing any paging. The result (we hit a wall) is a mondo swapout followed immediately by swapping it all right back in. We seem to have done a complete turnaround wrt paging vs flush/cache reap preference, and that does effectively cure this scenario.. but methinks optimal (-ENOENT?) lies somewhere in between. However, I am very much against tuning the VM for one particular workload. If you can show me that this problem also happens under other workloads we can work at changing it, but I don't think it's right to optimise the VM for a specific workload... I'll watch behavior under other loads. (I don't have enough network capacity to do anything stressful there, and whatever load I pick has to be compute bound as to not end up benchmarking my modest I/O capacity.. suggestions welcome. I use make -j primarily because it doesn't need much I/O bandwidth for itself, but does allocate quite a bit.. that leaves most I/O capacity free for vm usage) Something else I see while watching it run: MUCH more swapout than swapin. Does that mean we're sending pages to swap only to find out that we never need them again? -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Fri, 9 Feb 2001, Kurt Roeckx wrote: > I just tested ac8. > > If I run this test, the system gets really slow. It takes about > a second between the time I press a key, and the time it appears > on the screen. The load goes way up. Everything seems to block. I'm sorry, but ... what test ? And how do older kernels run the same thing ? > It ran for serval minutes. The process itself took about 1 > minutes of CPU time, and so did kswapd. It took atleast 5 > minutes real time. > > I once did just the same with 2.4.0, it took more like 30 > minutes then, and I ended up killing the process myself. So the kernel's behaviour has improved ? regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Fri, 9 Feb 2001, Kurt Roeckx wrote: I just tested ac8. If I run this test, the system gets really slow. It takes about a second between the time I press a key, and the time it appears on the screen. The load goes way up. Everything seems to block. I'm sorry, but ... what test ? And how do older kernels run the same thing ? It ran for serval minutes. The process itself took about 1 minutes of CPU time, and so did kswapd. It took atleast 5 minutes real time. I once did just the same with 2.4.0, it took more like 30 minutes then, and I ended up killing the process myself. So the kernel's behaviour has improved ? regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Thu, Feb 08, 2001 at 08:12:39PM -0200, Rik van Riel wrote: > On Thu, 8 Feb 2001, Alan Cox wrote: > > > ftp://ftp.kernel.org/pub/linux/kernel/people/alan/2.4/ > > > > 2.4.1-ac7 > > o Rebalance the 2.4.1 VM (Rik van Riel) > > | This should make things feel a lot faster especially > > | on small boxes .. feedback to Rik > > I'd really like feedback from people when it comes to this > change. The change /should/ fix most paging performance bugs > because it makes kswapd do the right amount of work in order > to solve the free memory shortage every time it is run. I just tested ac8. If I run this test, the system gets really slow. It takes about a second between the time I press a key, and the time it appears on the screen. The load goes way up. Everything seems to block. This is a box with 64 MB or RAM, and 32 MB of swap. There isn't much running on the box while doing this, only dnetc. It starts to get slow from the time the process starts is about 70 MB. Then you really hear the disk work. It ended up at about 75 MB, where it got killed by the OOM killer. (For once it killed the right thing!) I ran a vmstat 1, while doing this, and have attached the output. It ran for serval minutes. The process itself took about 1 minutes of CPU time, and so did kswapd. It took atleast 5 minutes real time. I once did just the same with 2.4.0, it took more like 30 minutes then, and I ended up killing the process myself. Kurt procs memoryswapiosystem cpu r b w swpd free buff cache si so bi bo in cs us sy id 1 0 0 16580 1808 1200 43336 0 0 834 130 25 93 2 5 1 0 0 16704 1804 1200 43460 0 000 1164 100 0 0 1 0 0 16704 1804 1200 43460 0 000 1204 100 0 0 1 0 0 16784 1808 1200 43540 0 000 1026 99 1 0 1 0 0 16904 1808 1200 43660 0 000 1017 99 1 0 1 0 0 16988 1808 1200 43744 0 000 130 32 99 1 0 1 0 0 16988 1808 1200 43744 0 000 136 18 100 0 0 1 0 0 17152 1808 1200 43908 0 000 110 14 100 0 0 1 0 0 17152 1808 1200 43908 0 003 110 10 100 0 0 2 0 0 17112 1428 1204 43960 0 0 3844 142 63 98 2 0 2 0 0 17268 1428 1204 43016 0 0 1300 133 46 100 0 0 2 0 0 17344 1428 1204 41980 0 0 1280 110 47 97 3 0 2 0 0 17548 1428 1204 41104 0 0 1290 116 48 98 2 0 2 0 0 17604 1428 1204 40056 0 0 1280 102 45 97 3 0 2 0 0 17820 1428 1208 39188 0 0 1290 106 49 97 3 0 2 0 0 17972 1428 1208 38288 0 0 1280 108 48 99 1 0 2 0 0 18136 1428 1208 37352 0 0 1290 104 46 95 5 0 2 0 0 18140 1428 1208 36296 0 0 1280 107 49 98 2 0 2 0 0 18140 1428 1208 35212 0 0 1290 108 54 96 4 0 2 0 0 18140 1428 1208 34108 0 0 1280 105 44 99 1 0 2 0 0 18140 1428 1208 33208 0 000 126 47 96 4 0 procs memoryswapiosystem cpu r b w swpd free buff cache si so bi bo in cs us sy id 2 0 0 18116 1456 1168 32144 0 0 1290 127 49 96 4 0 2 0 0 18116 1368 1100 31236 0 0 1280 110 46 97 3 0 2 0 0 18752 1304 1016 30944 0 0 1291 107 51 97 3 0 2 0 0 18752 2020 908 29944 0 0 1280 109 49 100 0 0 2 0 0 18752 1296 904 29596 0 0 1290 104 45 95 5 0 2 0 0 18752 1300 880 28788 0 248 128 62 106 42 97 3 0 2 1 0 18752 1236 880 27864 0 1896 73 531 162 52 95 5 0 2 0 0 18752 948 880 27056 0 0 1840 128 43 96 4 0 2 0 0 18752 948 856 26008 0 000 105 41 98 2 0 2 0 0 19688 980 840 26024 0 1360 129 340 133 50 97 3 0 2 0 0 19688 948 828 25028 0 624 128 156 114 46 98 2 0 2 0 0 20628 948 812 25008 0 192 129 48 108 45 95 5 0 2 0 0 23420 948 784 26956 0 1140 128 285 125 49 97 3 0 2 0 0 23464 948 676 26604 0 0 1300 105 49 99 1 0 2 0 0 23560 948 564 25780 0 820 128 214 132 43 95 5 0 2 1 0 23840 948 560 25036 0 244 73 61 109 47 96 4 0 2 0 0 23940 948 556 24184 0 1768 56 445 152 61 94 6 0 2 0 0 23928 948 540 23144 0 0 1430 111 55 98 2 0 2 0 0 24168 948 532 22352 0 164 129 41 106 48 96 4 0 2 0 0 29044 948 504 26240 0 120 128 30 105 42 97 3 0 2 0 0 31476 948 412 28256 0 316 129 79 126 48 96 4 0 procs memoryswapiosystem cpu r b w swpd free buff cache si so bi bo in cs us sy id 3 0 0 31476 948 416 27300 0 1580 193 413 269 68 97 3 0 2 0 0 32580 948
Re: Linux 2.4.1-ac7
On Thu, 8 Feb 2001, Torben Mathiasen wrote: > On Thu, Feb 08 2001, Rik van Riel wrote: > > On Thu, 8 Feb 2001, Alan Cox wrote: > > > > > ftp://ftp.kernel.org/pub/linux/kernel/people/alan/2.4/ > > > > > > 2.4.1-ac7 > > > o Rebalance the 2.4.1 VM (Rik van Riel) > > > | This should make things feel a lot faster especially > > > | on small boxes .. feedback to Rik > Just installed ac7 and after some 30 minutes of unpacking > kernel-sources and diffing patches, I left my computer unattended > for about 1 hour. When I came back the system was unusable (like it > was frozen), and /var/log/messages just displayed messages of the > type: > > Feb 8 22:54:40 fry kernel: Out of Memory: Killed process 455 (xmms). > ... > > The OOM killer killed most of my apps, and finally X. I had to reboot > in order to get the system back. I've been running ac1-ac6 since they > came out with no problems, so I guess its the VM hack that is buggy. Highly unlikely since the VM rebalancing patch doesn't change any of the actual swapout mechanisms. All it does is change how often the particular algorithms get called by kswapd and by user programs. As for trigerring the OOM killer, this strongly suggest a memory leak since there's a bug in the code which makes it very hard to trigger the OOM killer under normal situations (I'm working on a fix for that now). regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Thu, Feb 08 2001, Rik van Riel wrote: > On Thu, 8 Feb 2001, Alan Cox wrote: > > > ftp://ftp.kernel.org/pub/linux/kernel/people/alan/2.4/ > > > > 2.4.1-ac7 > > o Rebalance the 2.4.1 VM (Rik van Riel) > > | This should make things feel a lot faster especially > > | on small boxes .. feedback to Rik > > I'd really like feedback from people when it comes to this > change. The change /should/ fix most paging performance bugs > because it makes kswapd do the right amount of work in order > to solve the free memory shortage every time it is run. Rik, Just installed ac7 and after some 30 minutes of unpacking kernel-sources and diffing patches, I left my computer unattended for about 1 hour. When I came back the system was unusable (like it was frozen), and /var/log/messages just displayed messages of the type: Feb 8 22:54:40 fry kernel: Out of Memory: Killed process 455 (xmms). ... The OOM killer killed most of my apps, and finally X. I had to reboot in order to get the system back. I've been running ac1-ac6 since they came out with no problems, so I guess its the VM hack that is buggy. This is on an AMD K7 1200Mhz, 512MB Ram, ATA100. Nothing big was running at the time (xchat, xmms, mozilla, gnome, x, a few xterms). I'll do some more testing tomorrow and provide any further information you might need. -- Torben Mathiasen <[EMAIL PROTECTED]> Linux ThunderLAN maintainer http://opensource.compaq.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Thu, 8 Feb 2001, Alan Cox wrote: > ftp://ftp.kernel.org/pub/linux/kernel/people/alan/2.4/ > > 2.4.1-ac7 > o Rebalance the 2.4.1 VM (Rik van Riel) > | This should make things feel a lot faster especially > | on small boxes .. feedback to Rik I'd really like feedback from people when it comes to this change. The change /should/ fix most paging performance bugs because it makes kswapd do the right amount of work in order to solve the free memory shortage every time it is run. This, in turn, should make it far less likely that user processes will *ever* need to call try_to_free_pages() themselves, unless the system really goes into overload mode. It would be good to know if this change really fixes the bug or if it only helps for certain workloads and not for others. I'd really like to close the following bug but need confirmation that it works first ;) http://distro.conectiva.com/bugzilla/show_bug.cgi?id=1178 regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Thu, 8 Feb 2001, Doug Ledford wrote: > Thanks, I hoped it would ;-) It's amazing what happens when you have a bcopy > in assembly that's missing the source address initialization :-( Yes! The output from the description of my SCSI hds when the driver initialised was highly amusing (containing extremely random garbage)... (-; Best regards, Anton -- Anton Altaparmakov (replace at with @) Linux NTFS maintainer / WWW: http://sourceforge.net/projects/linux-ntfs/ ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
Tigran Aivazian wrote: > > two mistakes: > > a) [EMAIL PROTECTED], not veritas.com! (it was pine, not me -- default > domain etc :) > > b) it was ac6 which fixed it, not ac7 (but I am running ac7) > > On Thu, 8 Feb 2001, Tigran Aivazian wrote: > > > Doug, > > > > I confirm that ac7 fixed all the aic7xxx problems on my machine. Thanks, I hoped it would ;-) It's amazing what happens when you have a bcopy in assembly that's missing the source address initialization :-( -- Doug Ledford <[EMAIL PROTECTED]> http://people.redhat.com/dledford Please check my web site for aic7xxx updates/answers before e-mailing me about problems - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
At 19:33 08/02/01, Tigran Aivazian wrote: >On Thu, 8 Feb 2001, Tigran Aivazian wrote: > > I confirm that ac7 fixed all the aic7xxx problems on my machine. me,too (-: AHA2940UW dual channel adapter (on board a SMP Tyan Thunder Pro 100 GX440 mobo). -ac5 crashed on boot at SCSI init. -ac6 untested -ac7 all working fine again Best regards, Anton -- Anton Altaparmakov (replace at with @) Linux NTFS Maintainer / WWW: http://sourceforge.net/projects/linux-ntfs/ ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
Doug, I confirm that ac7 fixed all the aic7xxx problems on my machine. Thanks, Tigran - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
two mistakes: a) [EMAIL PROTECTED], not veritas.com! (it was pine, not me -- default domain etc :) b) it was ac6 which fixed it, not ac7 (but I am running ac7) On Thu, 8 Feb 2001, Tigran Aivazian wrote: > Doug, > > I confirm that ac7 fixed all the aic7xxx problems on my machine. > > Thanks, > Tigran > > - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
two mistakes: a) [EMAIL PROTECTED], not veritas.com! (it was pine, not me -- default domain etc :) b) it was ac6 which fixed it, not ac7 (but I am running ac7) On Thu, 8 Feb 2001, Tigran Aivazian wrote: Doug, I confirm that ac7 fixed all the aic7xxx problems on my machine. Thanks, Tigran - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
Doug, I confirm that ac7 fixed all the aic7xxx problems on my machine. Thanks, Tigran - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
At 19:33 08/02/01, Tigran Aivazian wrote: On Thu, 8 Feb 2001, Tigran Aivazian wrote: I confirm that ac7 fixed all the aic7xxx problems on my machine. AOLme,too/AOL (-: AHA2940UW dual channel adapter (on board a SMP Tyan Thunder Pro 100 GX440 mobo). -ac5 crashed on boot at SCSI init. -ac6 untested -ac7 all working fine again Best regards, Anton -- Anton Altaparmakov aia21 at cam.ac.uk (replace at with @) Linux NTFS Maintainer / WWW: http://sourceforge.net/projects/linux-ntfs/ ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
Tigran Aivazian wrote: two mistakes: a) [EMAIL PROTECTED], not veritas.com! (it was pine, not me -- default domain etc :) b) it was ac6 which fixed it, not ac7 (but I am running ac7) On Thu, 8 Feb 2001, Tigran Aivazian wrote: Doug, I confirm that ac7 fixed all the aic7xxx problems on my machine. Thanks, I hoped it would ;-) It's amazing what happens when you have a bcopy in assembly that's missing the source address initialization :-( -- Doug Ledford [EMAIL PROTECTED] http://people.redhat.com/dledford Please check my web site for aic7xxx updates/answers before e-mailing me about problems - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Thu, 8 Feb 2001, Doug Ledford wrote: Thanks, I hoped it would ;-) It's amazing what happens when you have a bcopy in assembly that's missing the source address initialization :-( Yes! The output from the description of my SCSI hds when the driver initialised was highly amusing (containing extremely random garbage)... (-; Best regards, Anton -- Anton Altaparmakov aia21 at cam.ac.uk (replace at with @) Linux NTFS maintainer / WWW: http://sourceforge.net/projects/linux-ntfs/ ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Thu, 8 Feb 2001, Alan Cox wrote: ftp://ftp.kernel.org/pub/linux/kernel/people/alan/2.4/ 2.4.1-ac7 o Rebalance the 2.4.1 VM (Rik van Riel) | This should make things feel a lot faster especially | on small boxes .. feedback to Rik I'd really like feedback from people when it comes to this change. The change /should/ fix most paging performance bugs because it makes kswapd do the right amount of work in order to solve the free memory shortage every time it is run. This, in turn, should make it far less likely that user processes will *ever* need to call try_to_free_pages() themselves, unless the system really goes into overload mode. It would be good to know if this change really fixes the bug or if it only helps for certain workloads and not for others. I'd really like to close the following bug but need confirmation that it works first ;) http://distro.conectiva.com/bugzilla/show_bug.cgi?id=1178 regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Thu, Feb 08 2001, Rik van Riel wrote: On Thu, 8 Feb 2001, Alan Cox wrote: ftp://ftp.kernel.org/pub/linux/kernel/people/alan/2.4/ 2.4.1-ac7 o Rebalance the 2.4.1 VM (Rik van Riel) | This should make things feel a lot faster especially | on small boxes .. feedback to Rik I'd really like feedback from people when it comes to this change. The change /should/ fix most paging performance bugs because it makes kswapd do the right amount of work in order to solve the free memory shortage every time it is run. Rik, Just installed ac7 and after some 30 minutes of unpacking kernel-sources and diffing patches, I left my computer unattended for about 1 hour. When I came back the system was unusable (like it was frozen), and /var/log/messages just displayed messages of the type: Feb 8 22:54:40 fry kernel: Out of Memory: Killed process 455 (xmms). ... The OOM killer killed most of my apps, and finally X. I had to reboot in order to get the system back. I've been running ac1-ac6 since they came out with no problems, so I guess its the VM hack that is buggy. This is on an AMD K7 1200Mhz, 512MB Ram, ATA100. Nothing big was running at the time (xchat, xmms, mozilla, gnome, x, a few xterms). I'll do some more testing tomorrow and provide any further information you might need. -- Torben Mathiasen [EMAIL PROTECTED] Linux ThunderLAN maintainer http://opensource.compaq.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Thu, 8 Feb 2001, Torben Mathiasen wrote: On Thu, Feb 08 2001, Rik van Riel wrote: On Thu, 8 Feb 2001, Alan Cox wrote: ftp://ftp.kernel.org/pub/linux/kernel/people/alan/2.4/ 2.4.1-ac7 o Rebalance the 2.4.1 VM (Rik van Riel) | This should make things feel a lot faster especially | on small boxes .. feedback to Rik Just installed ac7 and after some 30 minutes of unpacking kernel-sources and diffing patches, I left my computer unattended for about 1 hour. When I came back the system was unusable (like it was frozen), and /var/log/messages just displayed messages of the type: Feb 8 22:54:40 fry kernel: Out of Memory: Killed process 455 (xmms). ... The OOM killer killed most of my apps, and finally X. I had to reboot in order to get the system back. I've been running ac1-ac6 since they came out with no problems, so I guess its the VM hack that is buggy. Highly unlikely since the VM rebalancing patch doesn't change any of the actual swapout mechanisms. All it does is change how often the particular algorithms get called by kswapd and by user programs. As for trigerring the OOM killer, this strongly suggest a memory leak since there's a bug in the code which makes it very hard to trigger the OOM killer under normal situations (I'm working on a fix for that now). regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux 2.4.1-ac7
On Thu, Feb 08, 2001 at 08:12:39PM -0200, Rik van Riel wrote: On Thu, 8 Feb 2001, Alan Cox wrote: ftp://ftp.kernel.org/pub/linux/kernel/people/alan/2.4/ 2.4.1-ac7 o Rebalance the 2.4.1 VM (Rik van Riel) | This should make things feel a lot faster especially | on small boxes .. feedback to Rik I'd really like feedback from people when it comes to this change. The change /should/ fix most paging performance bugs because it makes kswapd do the right amount of work in order to solve the free memory shortage every time it is run. I just tested ac8. If I run this test, the system gets really slow. It takes about a second between the time I press a key, and the time it appears on the screen. The load goes way up. Everything seems to block. This is a box with 64 MB or RAM, and 32 MB of swap. There isn't much running on the box while doing this, only dnetc. It starts to get slow from the time the process starts is about 70 MB. Then you really hear the disk work. It ended up at about 75 MB, where it got killed by the OOM killer. (For once it killed the right thing!) I ran a vmstat 1, while doing this, and have attached the output. It ran for serval minutes. The process itself took about 1 minutes of CPU time, and so did kswapd. It took atleast 5 minutes real time. I once did just the same with 2.4.0, it took more like 30 minutes then, and I ended up killing the process myself. Kurt procs memoryswapiosystem cpu r b w swpd free buff cache si so bi bo in cs us sy id 1 0 0 16580 1808 1200 43336 0 0 834 130 25 93 2 5 1 0 0 16704 1804 1200 43460 0 000 1164 100 0 0 1 0 0 16704 1804 1200 43460 0 000 1204 100 0 0 1 0 0 16784 1808 1200 43540 0 000 1026 99 1 0 1 0 0 16904 1808 1200 43660 0 000 1017 99 1 0 1 0 0 16988 1808 1200 43744 0 000 130 32 99 1 0 1 0 0 16988 1808 1200 43744 0 000 136 18 100 0 0 1 0 0 17152 1808 1200 43908 0 000 110 14 100 0 0 1 0 0 17152 1808 1200 43908 0 003 110 10 100 0 0 2 0 0 17112 1428 1204 43960 0 0 3844 142 63 98 2 0 2 0 0 17268 1428 1204 43016 0 0 1300 133 46 100 0 0 2 0 0 17344 1428 1204 41980 0 0 1280 110 47 97 3 0 2 0 0 17548 1428 1204 41104 0 0 1290 116 48 98 2 0 2 0 0 17604 1428 1204 40056 0 0 1280 102 45 97 3 0 2 0 0 17820 1428 1208 39188 0 0 1290 106 49 97 3 0 2 0 0 17972 1428 1208 38288 0 0 1280 108 48 99 1 0 2 0 0 18136 1428 1208 37352 0 0 1290 104 46 95 5 0 2 0 0 18140 1428 1208 36296 0 0 1280 107 49 98 2 0 2 0 0 18140 1428 1208 35212 0 0 1290 108 54 96 4 0 2 0 0 18140 1428 1208 34108 0 0 1280 105 44 99 1 0 2 0 0 18140 1428 1208 33208 0 000 126 47 96 4 0 procs memoryswapiosystem cpu r b w swpd free buff cache si so bi bo in cs us sy id 2 0 0 18116 1456 1168 32144 0 0 1290 127 49 96 4 0 2 0 0 18116 1368 1100 31236 0 0 1280 110 46 97 3 0 2 0 0 18752 1304 1016 30944 0 0 1291 107 51 97 3 0 2 0 0 18752 2020 908 29944 0 0 1280 109 49 100 0 0 2 0 0 18752 1296 904 29596 0 0 1290 104 45 95 5 0 2 0 0 18752 1300 880 28788 0 248 128 62 106 42 97 3 0 2 1 0 18752 1236 880 27864 0 1896 73 531 162 52 95 5 0 2 0 0 18752 948 880 27056 0 0 1840 128 43 96 4 0 2 0 0 18752 948 856 26008 0 000 105 41 98 2 0 2 0 0 19688 980 840 26024 0 1360 129 340 133 50 97 3 0 2 0 0 19688 948 828 25028 0 624 128 156 114 46 98 2 0 2 0 0 20628 948 812 25008 0 192 129 48 108 45 95 5 0 2 0 0 23420 948 784 26956 0 1140 128 285 125 49 97 3 0 2 0 0 23464 948 676 26604 0 0 1300 105 49 99 1 0 2 0 0 23560 948 564 25780 0 820 128 214 132 43 95 5 0 2 1 0 23840 948 560 25036 0 244 73 61 109 47 96 4 0 2 0 0 23940 948 556 24184 0 1768 56 445 152 61 94 6 0 2 0 0 23928 948 540 23144 0 0 1430 111 55 98 2 0 2 0 0 24168 948 532 22352 0 164 129 41 106 48 96 4 0 2 0 0 29044 948 504 26240 0 120 128 30 105 42 97 3 0 2 0 0 31476 948 412 28256 0 316 129 79 126 48 96 4 0 procs memoryswapiosystem cpu r b w swpd free buff cache si so bi bo in cs us sy id 3 0 0 31476 948 416 27300 0 1580 193 413 269 68 97 3 0 2 0 0 32580 948 416 27404 0
