Re: kswapd high CPU usage with no swap
>> When this is sorted out, should I keep the previous patch [1] applied >> as well? > > That doesn't hurt. OK, I've used just the latter patch (because I somehow believe the first one lowers the probability of bad behavior), so let's see if kswapd consumes CPU again. I don't have any test patter to trigger that CPU usage, though... Cheers and thanks for your time, -jkt -- cd /local/pub && more beer > /dev/mouth signature.asc Description: OpenPGP digital signature
Re: kswapd high CPU usage with no swap
On Wed, 26 Sep 2007 09:27:36 +0200 Jan Kundrát <[EMAIL PROTECTED]> wrote: > Rik van Riel wrote: > > Could you try out the attached patch, too? > > Sorry, I wasn't able to apply it against 2.6.22-gentoo-r3 and vanilla > 2.6.22.7; I don't have the "order" member in the "struct scan_control" > and also the bit about "if (sync_writeback == PAGEOUT_IO_SYNC && > may_enter_fs)" was missing so there was no "else" branch to add (and > it seemd pretty significant to me). What version should I patch > against? Sorry, the patch is against 2.6.23-rc7. > When this is sorted out, should I keep the previous patch [1] applied > as well? That doesn't hurt. -- All Rights Reversed - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: kswapd high CPU usage with no swap
Rik van Riel wrote: > Could you try out the attached patch, too? Sorry, I wasn't able to apply it against 2.6.22-gentoo-r3 and vanilla 2.6.22.7; I don't have the "order" member in the "struct scan_control" and also the bit about "if (sync_writeback == PAGEOUT_IO_SYNC && may_enter_fs)" was missing so there was no "else" branch to add (and it seemd pretty significant to me). What version should I patch against? When this is sorted out, should I keep the previous patch [1] applied as well? [1] http://www.redhat.com/archives/nahant-list/2006-March/msg00033.html Cheers, -jkt -- cd /local/pub && more beer > /dev/mouth signature.asc Description: OpenPGP digital signature
Re: kswapd high CPU usage with no swap
Rik van Riel wrote: Could you try out the attached patch, too? Sorry, I wasn't able to apply it against 2.6.22-gentoo-r3 and vanilla 2.6.22.7; I don't have the order member in the struct scan_control and also the bit about if (sync_writeback == PAGEOUT_IO_SYNC may_enter_fs) was missing so there was no else branch to add (and it seemd pretty significant to me). What version should I patch against? When this is sorted out, should I keep the previous patch [1] applied as well? [1] http://www.redhat.com/archives/nahant-list/2006-March/msg00033.html Cheers, -jkt -- cd /local/pub more beer /dev/mouth signature.asc Description: OpenPGP digital signature
Re: kswapd high CPU usage with no swap
On Wed, 26 Sep 2007 09:27:36 +0200 Jan Kundrát [EMAIL PROTECTED] wrote: Rik van Riel wrote: Could you try out the attached patch, too? Sorry, I wasn't able to apply it against 2.6.22-gentoo-r3 and vanilla 2.6.22.7; I don't have the order member in the struct scan_control and also the bit about if (sync_writeback == PAGEOUT_IO_SYNC may_enter_fs) was missing so there was no else branch to add (and it seemd pretty significant to me). What version should I patch against? Sorry, the patch is against 2.6.23-rc7. When this is sorted out, should I keep the previous patch [1] applied as well? That doesn't hurt. -- All Rights Reversed - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: kswapd high CPU usage with no swap
When this is sorted out, should I keep the previous patch [1] applied as well? That doesn't hurt. OK, I've used just the latter patch (because I somehow believe the first one lowers the probability of bad behavior), so let's see if kswapd consumes CPU again. I don't have any test patter to trigger that CPU usage, though... Cheers and thanks for your time, -jkt -- cd /local/pub more beer /dev/mouth signature.asc Description: OpenPGP digital signature
Re: kswapd high CPU usage with no swap
On Tue, 25 Sep 2007 12:13:41 +0200
Jan Kundrát <[EMAIL PROTECTED]> wrote:
> Rik van Riel wrote:
> > How much memory did you have in "cached" when you looked
> > with top (and no swap enabled) ?
>
> Hi Rik,
> it was pretty low number (several thousands, or maybe tens of
> thousands).
>
> In the meanwhile, I've come across your patch [1] ("prevent kswapd
> from freeing excessive amounts of lowmem") and applied it locally.
Could you try out the attached patch, too?
Kswapd and try_to_free_pages() have a built-in pause, where
it waits for IO to complete. However, the current code also
calls blk_congestion_wait() when there is no IO in flight!
This patch should only make the pageout code wait for IO when
there actually is a significant amount of pageout IO in flight.
Signed-off-by: Rik van Riel <[EMAIL PROTECTED]>
diff -up linux-2.6.22.x86_64/mm/vmscan.c.wait linux-2.6.22.x86_64/mm/vmscan.c
--- linux-2.6.22.x86_64/mm/vmscan.c.wait 2007-09-25 11:33:30.0 -0400
+++ linux-2.6.22.x86_64/mm/vmscan.c 2007-09-25 21:27:08.0 -0400
@@ -68,6 +68,8 @@ struct scan_control {
int all_unreclaimable;
int order;
+
+ int nr_io_pages;
};
#define lru_to_page(_head) (list_entry((_head)->prev, struct page, lru))
@@ -489,8 +491,10 @@ static unsigned long shrink_page_list(st
*/
if (sync_writeback == PAGEOUT_IO_SYNC && may_enter_fs)
wait_on_page_writeback(page);
- else
+ else {
+sc->nr_io_pages++;
goto keep_locked;
+ }
}
referenced = page_referenced(page, 1);
@@ -541,8 +545,10 @@ static unsigned long shrink_page_list(st
case PAGE_ACTIVATE:
goto activate_locked;
case PAGE_SUCCESS:
-if (PageWriteback(page) || PageDirty(page))
+if (PageWriteback(page) || PageDirty(page)) {
+ sc->nr_io_pages++;
goto keep;
+}
/*
* A synchronous write - probably a ramdisk. Go
* ahead and try to reclaim the page.
@@ -1201,6 +1207,7 @@ unsigned long try_to_free_pages(struct z
for (priority = DEF_PRIORITY; priority >= 0; priority--) {
sc.nr_scanned = 0;
+ sc.nr_io_pages = 0;
if (!priority)
disable_swap_token();
nr_reclaimed += shrink_zones(priority, zones, );
@@ -1229,7 +1236,8 @@ unsigned long try_to_free_pages(struct z
}
/* Take a nap, wait for some writeback to complete */
- if (sc.nr_scanned && priority < DEF_PRIORITY - 2)
+ if (sc.nr_scanned && priority < DEF_PRIORITY - 2 &&
+sc.nr_io_pages > sc.swap_cluster_max)
congestion_wait(WRITE, HZ/10);
}
/* top priority shrink_caches still had more to do? don't OOM, then */
@@ -1315,6 +1323,7 @@ loop_again:
if (!priority)
disable_swap_token();
+ sc.nr_io_pages = 0;
all_zones_ok = 1;
/*
@@ -1398,7 +1407,8 @@ loop_again:
* OK, kswapd is getting into trouble. Take a nap, then take
* another pass across the zones.
*/
- if (total_scanned && priority < DEF_PRIORITY - 2)
+ if (total_scanned && priority < DEF_PRIORITY - 2 &&
+ sc.nr_io_pages > sc.swap_cluster_max)
congestion_wait(WRITE, HZ/10);
/*
Re: kswapd high CPU usage with no swap
Rik van Riel wrote:
> How much memory did you have in "cached" when you looked
> with top (and no swap enabled) ?
Hi Rik,
it was pretty low number (several thousands, or maybe tens of thousands).
In the meanwhile, I've come across your patch [1] ("prevent kswapd from
freeing excessive amounts of lowmem") and applied it locally. I'll see
if it fixes my problem, but at a first glance, it seems that it might
actually slow other things down -- when I switch windows, konqueror's
screen redrawing seems to be pretty slow and I can see it progressing
from top to bottom...
[1] http://lkml.org/lkml/2007/9/5/289
Cheers,
-jkt
--
cd /local/pub && more beer > /dev/mouth
signature.asc
Description: OpenPGP digital signature
Re: kswapd high CPU usage with no swap
Rik van Riel wrote: How much memory did you have in cached when you looked with top (and no swap enabled) ? Hi Rik, it was pretty low number (several thousands, or maybe tens of thousands). In the meanwhile, I've come across your patch [1] (prevent kswapd from freeing excessive amounts of lowmem) and applied it locally. I'll see if it fixes my problem, but at a first glance, it seems that it might actually slow other things down -- when I switch windows, konqueror's screen redrawing seems to be pretty slow and I can see it progressing from top to bottom... [1] http://lkml.org/lkml/2007/9/5/289 Cheers, -jkt -- cd /local/pub more beer /dev/mouth signature.asc Description: OpenPGP digital signature
Re: kswapd high CPU usage with no swap
On Tue, 25 Sep 2007 12:13:41 +0200
Jan Kundrát [EMAIL PROTECTED] wrote:
Rik van Riel wrote:
How much memory did you have in cached when you looked
with top (and no swap enabled) ?
Hi Rik,
it was pretty low number (several thousands, or maybe tens of
thousands).
In the meanwhile, I've come across your patch [1] (prevent kswapd
from freeing excessive amounts of lowmem) and applied it locally.
Could you try out the attached patch, too?
Kswapd and try_to_free_pages() have a built-in pause, where
it waits for IO to complete. However, the current code also
calls blk_congestion_wait() when there is no IO in flight!
This patch should only make the pageout code wait for IO when
there actually is a significant amount of pageout IO in flight.
Signed-off-by: Rik van Riel [EMAIL PROTECTED]
diff -up linux-2.6.22.x86_64/mm/vmscan.c.wait linux-2.6.22.x86_64/mm/vmscan.c
--- linux-2.6.22.x86_64/mm/vmscan.c.wait 2007-09-25 11:33:30.0 -0400
+++ linux-2.6.22.x86_64/mm/vmscan.c 2007-09-25 21:27:08.0 -0400
@@ -68,6 +68,8 @@ struct scan_control {
int all_unreclaimable;
int order;
+
+ int nr_io_pages;
};
#define lru_to_page(_head) (list_entry((_head)-prev, struct page, lru))
@@ -489,8 +491,10 @@ static unsigned long shrink_page_list(st
*/
if (sync_writeback == PAGEOUT_IO_SYNC may_enter_fs)
wait_on_page_writeback(page);
- else
+ else {
+sc-nr_io_pages++;
goto keep_locked;
+ }
}
referenced = page_referenced(page, 1);
@@ -541,8 +545,10 @@ static unsigned long shrink_page_list(st
case PAGE_ACTIVATE:
goto activate_locked;
case PAGE_SUCCESS:
-if (PageWriteback(page) || PageDirty(page))
+if (PageWriteback(page) || PageDirty(page)) {
+ sc-nr_io_pages++;
goto keep;
+}
/*
* A synchronous write - probably a ramdisk. Go
* ahead and try to reclaim the page.
@@ -1201,6 +1207,7 @@ unsigned long try_to_free_pages(struct z
for (priority = DEF_PRIORITY; priority = 0; priority--) {
sc.nr_scanned = 0;
+ sc.nr_io_pages = 0;
if (!priority)
disable_swap_token();
nr_reclaimed += shrink_zones(priority, zones, sc);
@@ -1229,7 +1236,8 @@ unsigned long try_to_free_pages(struct z
}
/* Take a nap, wait for some writeback to complete */
- if (sc.nr_scanned priority DEF_PRIORITY - 2)
+ if (sc.nr_scanned priority DEF_PRIORITY - 2
+sc.nr_io_pages sc.swap_cluster_max)
congestion_wait(WRITE, HZ/10);
}
/* top priority shrink_caches still had more to do? don't OOM, then */
@@ -1315,6 +1323,7 @@ loop_again:
if (!priority)
disable_swap_token();
+ sc.nr_io_pages = 0;
all_zones_ok = 1;
/*
@@ -1398,7 +1407,8 @@ loop_again:
* OK, kswapd is getting into trouble. Take a nap, then take
* another pass across the zones.
*/
- if (total_scanned priority DEF_PRIORITY - 2)
+ if (total_scanned priority DEF_PRIORITY - 2
+ sc.nr_io_pages sc.swap_cluster_max)
congestion_wait(WRITE, HZ/10);
/*
Re: kswapd high CPU usage with no swap
On Tue, 25 Sep 2007 02:13:42 +0200 Jan Kundrát <[EMAIL PROTECTED]> wrote: > Hi folks, > I use a 2.6.22-gentoo-r2 SMP kernel with fglrx 8.40.4 [1], > tp_smapi-0.32 and ipw3945-1.2.0 on a Thinkpad T60 with dual core > Intel Core CPU. My root filesystem is XFS stored on an internal SATA > disk, and I have 1GB of RAM and no swap. > > After several suspend to RAM/resume cycles, the X interface got pretty > slow today. Looking at the top output, I see that one core was > completely busy in the "wa" state and according to `ps auxww`, the > kswapd0 process was in uninterruptable deep sleep. > > I don't use any swap because my applications rarely need more memory > than I have in my system and using 1GB of swap yielded poor > performance when it was actually accessed. I know I can tweak VM's > preferences, but I simply don't feel the need to use swap when I have > "plenty" of memory. How much memory did you have in "cached" when you looked with top (and no swap enabled) ? If the amount of "cached" memory is very low, it could mean that your shared libraries are being pushed out of memory, instead of the kernel swapping out some page that belongs to only one process. As for kswapd getting into uninterruptible sleep state, it will do that all by itself sometimes, without even having any disk IO going on... that code looks a little suspect, I will look into it. -- "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Brian W. Kernighan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
kswapd high CPU usage with no swap
Hi folks, I use a 2.6.22-gentoo-r2 SMP kernel with fglrx 8.40.4 [1], tp_smapi-0.32 and ipw3945-1.2.0 on a Thinkpad T60 with dual core Intel Core CPU. My root filesystem is XFS stored on an internal SATA disk, and I have 1GB of RAM and no swap. After several suspend to RAM/resume cycles, the X interface got pretty slow today. Looking at the top output, I see that one core was completely busy in the "wa" state and according to `ps auxww`, the kswapd0 process was in uninterruptable deep sleep. I don't use any swap because my applications rarely need more memory than I have in my system and using 1GB of swap yielded poor performance when it was actually accessed. I know I can tweak VM's preferences, but I simply don't feel the need to use swap when I have "plenty" of memory. That said, I've googled a thread [2] which recommends using at least a small swap because VM somehow performs better with it. So I created a small (12MB) swap on a physical partition on the same SATA disk and swapon-ed it. The swap got full immediately but the load didn't get lower. After that, I've turned it off again, adjusted its size to be about 1GB and enabled it again. Several tens of seconds later, swap usage was at approximately 350MB and RAM usage at about 60MB. There was no unusual activity at the background (no intensive cronned jobs, no locate, nothing, just an idle KDE session with bunch of idling applications). Touching the swappiness value (0, 10, 40, 60 and 100) didn't change anything on the fact that switching from one application to another was slow, and even moving a mouse cursor (USB-attached mouse) in X was delayed and not smooth. Could you please provide a hint about what might cause such problems, what am I doing wrong and how can I avoid it? If there are any details like kernel config that are relevant here, I'll happily provide them. I'd also appreciate if you keep me on the CC as I'm not subscribed to this list. If this is not an appropriate place to ask, please accept my apologies. [1] I'm aware of the fact that fglrx is a binary closed source and that the kernel is tainted by its usage. Unfortunately no other driver besides VESA supports my VGA card. [2] http://www.redhat.com/archives/nahant-list/2006-March/msg00033.html Cheers, -jkt -- cd /local/pub && more beer > /dev/mouth signature.asc Description: OpenPGP digital signature
kswapd high CPU usage with no swap
Hi folks, I use a 2.6.22-gentoo-r2 SMP kernel with fglrx 8.40.4 [1], tp_smapi-0.32 and ipw3945-1.2.0 on a Thinkpad T60 with dual core Intel Core CPU. My root filesystem is XFS stored on an internal SATA disk, and I have 1GB of RAM and no swap. After several suspend to RAM/resume cycles, the X interface got pretty slow today. Looking at the top output, I see that one core was completely busy in the wa state and according to `ps auxww`, the kswapd0 process was in uninterruptable deep sleep. I don't use any swap because my applications rarely need more memory than I have in my system and using 1GB of swap yielded poor performance when it was actually accessed. I know I can tweak VM's preferences, but I simply don't feel the need to use swap when I have plenty of memory. That said, I've googled a thread [2] which recommends using at least a small swap because VM somehow performs better with it. So I created a small (12MB) swap on a physical partition on the same SATA disk and swapon-ed it. The swap got full immediately but the load didn't get lower. After that, I've turned it off again, adjusted its size to be about 1GB and enabled it again. Several tens of seconds later, swap usage was at approximately 350MB and RAM usage at about 60MB. There was no unusual activity at the background (no intensive cronned jobs, no locate, nothing, just an idle KDE session with bunch of idling applications). Touching the swappiness value (0, 10, 40, 60 and 100) didn't change anything on the fact that switching from one application to another was slow, and even moving a mouse cursor (USB-attached mouse) in X was delayed and not smooth. Could you please provide a hint about what might cause such problems, what am I doing wrong and how can I avoid it? If there are any details like kernel config that are relevant here, I'll happily provide them. I'd also appreciate if you keep me on the CC as I'm not subscribed to this list. If this is not an appropriate place to ask, please accept my apologies. [1] I'm aware of the fact that fglrx is a binary closed source and that the kernel is tainted by its usage. Unfortunately no other driver besides VESA supports my VGA card. [2] http://www.redhat.com/archives/nahant-list/2006-March/msg00033.html Cheers, -jkt -- cd /local/pub more beer /dev/mouth signature.asc Description: OpenPGP digital signature
Re: kswapd high CPU usage with no swap
On Tue, 25 Sep 2007 02:13:42 +0200 Jan Kundrát [EMAIL PROTECTED] wrote: Hi folks, I use a 2.6.22-gentoo-r2 SMP kernel with fglrx 8.40.4 [1], tp_smapi-0.32 and ipw3945-1.2.0 on a Thinkpad T60 with dual core Intel Core CPU. My root filesystem is XFS stored on an internal SATA disk, and I have 1GB of RAM and no swap. After several suspend to RAM/resume cycles, the X interface got pretty slow today. Looking at the top output, I see that one core was completely busy in the wa state and according to `ps auxww`, the kswapd0 process was in uninterruptable deep sleep. I don't use any swap because my applications rarely need more memory than I have in my system and using 1GB of swap yielded poor performance when it was actually accessed. I know I can tweak VM's preferences, but I simply don't feel the need to use swap when I have plenty of memory. How much memory did you have in cached when you looked with top (and no swap enabled) ? If the amount of cached memory is very low, it could mean that your shared libraries are being pushed out of memory, instead of the kernel swapping out some page that belongs to only one process. As for kswapd getting into uninterruptible sleep state, it will do that all by itself sometimes, without even having any disk IO going on... that code looks a little suspect, I will look into it. -- Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. - Brian W. Kernighan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
