Re: 100% iowait on one of cpus in current -git

2007-11-01 Thread Torsten Kaiser
On 11/1/07, Torsten Kaiser <[EMAIL PROTECTED]> wrote: > On 11/1/07, Fengguang Wu <[EMAIL PROTECTED]> wrote: > > Thank you. Maybe we can start by the applied debug patch :-) > > Will applied it and try to recreate this. Patch applied, used emerge to install a 2.6.24-rc1 kernel. I had no complete

Re: 100% iowait on one of cpus in current -git

2007-11-01 Thread Torsten Kaiser
On 11/1/07, Fengguang Wu <[EMAIL PROTECTED]> wrote: > On Wed, Oct 31, 2007 at 04:22:10PM +0100, Torsten Kaiser wrote: > > Since 2.6.23-mm1 I also experience strange hangs during heavy writeouts. > > Each time I noticed this I was using emerge (package util from the > > gentoo distribution) to

Re: 100% iowait on one of cpus in current -git

2007-11-01 Thread Fengguang Wu
On Wed, Oct 31, 2007 at 04:22:10PM +0100, Torsten Kaiser wrote: > On 10/22/07, Fengguang Wu <[EMAIL PROTECTED]> wrote: > > On Mon, Oct 22, 2007 at 09:10:45PM +0800, Fengguang Wu wrote: > > > Hmm, Maybe it's an reiserfs related issue. Do you have the full log file? > > > > Bingo! It can be

Re: 100% iowait on one of cpus in current -git

2007-11-01 Thread Fengguang Wu
On Wed, Oct 31, 2007 at 04:22:10PM +0100, Torsten Kaiser wrote: On 10/22/07, Fengguang Wu [EMAIL PROTECTED] wrote: On Mon, Oct 22, 2007 at 09:10:45PM +0800, Fengguang Wu wrote: Hmm, Maybe it's an reiserfs related issue. Do you have the full log file? Bingo! It can be reproduced in -mm

Re: 100% iowait on one of cpus in current -git

2007-11-01 Thread Torsten Kaiser
On 11/1/07, Fengguang Wu [EMAIL PROTECTED] wrote: On Wed, Oct 31, 2007 at 04:22:10PM +0100, Torsten Kaiser wrote: Since 2.6.23-mm1 I also experience strange hangs during heavy writeouts. Each time I noticed this I was using emerge (package util from the gentoo distribution) to

Re: 100% iowait on one of cpus in current -git

2007-11-01 Thread Torsten Kaiser
On 11/1/07, Torsten Kaiser [EMAIL PROTECTED] wrote: On 11/1/07, Fengguang Wu [EMAIL PROTECTED] wrote: Thank you. Maybe we can start by the applied debug patch :-) Will applied it and try to recreate this. Patch applied, used emerge to install a 2.6.24-rc1 kernel. I had no complete stalls,

Re: 100% iowait on one of cpus in current -git

2007-10-31 Thread Torsten Kaiser
On 10/22/07, Fengguang Wu <[EMAIL PROTECTED]> wrote: > On Mon, Oct 22, 2007 at 09:10:45PM +0800, Fengguang Wu wrote: > > Hmm, Maybe it's an reiserfs related issue. Do you have the full log file? > > Bingo! It can be reproduced in -mm on reiserfs: > > # mkfs.reiserfs /dev/sdb1 > # mount /dev/sdb1

Re: 100% iowait on one of cpus in current -git

2007-10-31 Thread Torsten Kaiser
On 10/22/07, Fengguang Wu [EMAIL PROTECTED] wrote: On Mon, Oct 22, 2007 at 09:10:45PM +0800, Fengguang Wu wrote: Hmm, Maybe it's an reiserfs related issue. Do you have the full log file? Bingo! It can be reproduced in -mm on reiserfs: # mkfs.reiserfs /dev/sdb1 # mount /dev/sdb1 /test #

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Fengguang Wu
On Mon, Oct 22, 2007 at 09:10:45PM +0800, Fengguang Wu wrote: > Hmm, Maybe it's an reiserfs related issue. Do you have the full log file? Bingo! It can be reproduced in -mm on reiserfs: # mkfs.reiserfs /dev/sdb1 # mount /dev/sdb1 /test # cp bin /test # dmesg [...] [ 418.346113] requeue_io

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Maxim Levitsky
On Monday 22 October 2007 15:10:45 Fengguang Wu wrote: > On Mon, Oct 22, 2007 at 03:05:35PM +0200, Maxim Levitsky wrote: > > Hi, > > Thank you very much too, for helping me. > > You are welcome :-) > > > Applied. > > Had to kill klogd, since kernel generates tons of redirtied inode messages. > >

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Fengguang Wu
On Mon, Oct 22, 2007 at 03:05:35PM +0200, Maxim Levitsky wrote: > Hi, > Thank you very much too, for helping me. You are welcome :-) > Applied. > Had to kill klogd, since kernel generates tons of redirtied inode messages. > Size of the kern.log is 863 KB, thus I I don't think it is polite to

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Maxim Levitsky
On Monday 22 October 2007 14:37:07 Fengguang Wu wrote: > On Mon, Oct 22, 2007 at 02:21:21PM +0200, Maxim Levitsky wrote: > > I Bisected this bug to exactly this commit: > > > > 2e6883bdf49abd0e7f0d9b6297fc3be7ebb2250b > > writeback: introduce writeback_control.more_io to indicate more io > > > >

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Fengguang Wu
On Mon, Oct 22, 2007 at 02:21:21PM +0200, Maxim Levitsky wrote: > I Bisected this bug to exactly this commit: > > 2e6883bdf49abd0e7f0d9b6297fc3be7ebb2250b > writeback: introduce writeback_control.more_io to indicate more io > > Reverting it and compiling latest git shows no more signs of that

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Maxim Levitsky
On Monday 22 October 2007 13:19:08 Fengguang Wu wrote: > On Mon, Oct 22, 2007 at 12:58:11PM +0200, Maxim Levitsky wrote: > > On Monday 22 October 2007 12:55:25 Fengguang Wu wrote: > > > On Mon, Oct 22, 2007 at 12:40:24PM +0200, Maxim Levitsky wrote: > > > > On Monday 22 October 2007 12:22:10 Peter

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Fengguang Wu
On Mon, Oct 22, 2007 at 12:58:11PM +0200, Maxim Levitsky wrote: > On Monday 22 October 2007 12:55:25 Fengguang Wu wrote: > > On Mon, Oct 22, 2007 at 12:40:24PM +0200, Maxim Levitsky wrote: > > > On Monday 22 October 2007 12:22:10 Peter Zijlstra wrote: > > > > > [ 673.365631] pdflush D

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Maxim Levitsky
On Monday 22 October 2007 12:55:25 Fengguang Wu wrote: > On Mon, Oct 22, 2007 at 12:40:24PM +0200, Maxim Levitsky wrote: > > On Monday 22 October 2007 12:22:10 Peter Zijlstra wrote: > > > > [ 673.365631] pdflush D c21bdecc 0 221 2 > > > > [ 673.365635]c21bdee0 0046

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Fengguang Wu
On Mon, Oct 22, 2007 at 12:40:24PM +0200, Maxim Levitsky wrote: > On Monday 22 October 2007 12:22:10 Peter Zijlstra wrote: > > > [ 673.365631] pdflush D c21bdecc 0 221 2 > > > [ 673.365635]c21bdee0 0046 0002 c21bdecc c21bdec4 > > > c21b3000 0002 > >

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Maxim Levitsky
On Monday 22 October 2007 12:22:10 Peter Zijlstra wrote: > On Mon, 2007-10-22 at 11:59 +0200, Maxim Levitsky wrote: > > On Monday 22 October 2007 11:41:57 Peter Zijlstra wrote: > > > On Mon, 2007-10-22 at 08:22 +0200, Maxim Levitsky wrote: > > > > Hi, > > > > > > > > I found a bug in current

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Peter Zijlstra
On Mon, 2007-10-22 at 11:59 +0200, Maxim Levitsky wrote: > On Monday 22 October 2007 11:41:57 Peter Zijlstra wrote: > > On Mon, 2007-10-22 at 08:22 +0200, Maxim Levitsky wrote: > > > Hi, > > > > > > I found a bug in current -git: > > > > > > On my system on of cpus stays 100% in iowait mode (I

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Maxim Levitsky
On Monday 22 October 2007 11:41:57 Peter Zijlstra wrote: > On Mon, 2007-10-22 at 08:22 +0200, Maxim Levitsky wrote: > > Hi, > > > > I found a bug in current -git: > > > > On my system on of cpus stays 100% in iowait mode (I have core 2 duo) > > Otherwise the system works OK, no disk activity

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Maxim Levitsky
On Monday 22 October 2007 11:11:52 Paolo Ornati wrote: > On Mon, 22 Oct 2007 08:22:52 +0200 > Maxim Levitsky <[EMAIL PROTECTED]> wrote: > > > I tried to bisect this, but eventually I run into other bugs that cause > > system to oops early. > > You can pick a different revision to test with: >

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Peter Zijlstra
On Mon, 2007-10-22 at 08:22 +0200, Maxim Levitsky wrote: > Hi, > > I found a bug in current -git: > > On my system on of cpus stays 100% in iowait mode (I have core 2 duo) > Otherwise the system works OK, no disk activity and/or slowdown. > Suspecting that this is a swap-related problem I tried

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Paolo Ornati
On Mon, 22 Oct 2007 08:22:52 +0200 Maxim Levitsky <[EMAIL PROTECTED]> wrote: > I tried to bisect this, but eventually I run into other bugs that cause > system to oops early. You can pick a different revision to test with: git-reset --hard "SHA1" Choose one with "git-bisect

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Paolo Ornati
On Mon, 22 Oct 2007 08:22:52 +0200 Maxim Levitsky [EMAIL PROTECTED] wrote: I tried to bisect this, but eventually I run into other bugs that cause system to oops early. You can pick a different revision to test with: git-reset --hard SHA1 Choose one with git-bisect visualize. --

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Peter Zijlstra
On Mon, 2007-10-22 at 08:22 +0200, Maxim Levitsky wrote: Hi, I found a bug in current -git: On my system on of cpus stays 100% in iowait mode (I have core 2 duo) Otherwise the system works OK, no disk activity and/or slowdown. Suspecting that this is a swap-related problem I tried to turn

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Maxim Levitsky
On Monday 22 October 2007 11:11:52 Paolo Ornati wrote: On Mon, 22 Oct 2007 08:22:52 +0200 Maxim Levitsky [EMAIL PROTECTED] wrote: I tried to bisect this, but eventually I run into other bugs that cause system to oops early. You can pick a different revision to test with:

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Maxim Levitsky
On Monday 22 October 2007 11:41:57 Peter Zijlstra wrote: On Mon, 2007-10-22 at 08:22 +0200, Maxim Levitsky wrote: Hi, I found a bug in current -git: On my system on of cpus stays 100% in iowait mode (I have core 2 duo) Otherwise the system works OK, no disk activity and/or slowdown.

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Peter Zijlstra
On Mon, 2007-10-22 at 11:59 +0200, Maxim Levitsky wrote: On Monday 22 October 2007 11:41:57 Peter Zijlstra wrote: On Mon, 2007-10-22 at 08:22 +0200, Maxim Levitsky wrote: Hi, I found a bug in current -git: On my system on of cpus stays 100% in iowait mode (I have core 2 duo)

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Maxim Levitsky
On Monday 22 October 2007 12:22:10 Peter Zijlstra wrote: On Mon, 2007-10-22 at 11:59 +0200, Maxim Levitsky wrote: On Monday 22 October 2007 11:41:57 Peter Zijlstra wrote: On Mon, 2007-10-22 at 08:22 +0200, Maxim Levitsky wrote: Hi, I found a bug in current -git: On my

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Fengguang Wu
On Mon, Oct 22, 2007 at 12:40:24PM +0200, Maxim Levitsky wrote: On Monday 22 October 2007 12:22:10 Peter Zijlstra wrote: [ 673.365631] pdflush D c21bdecc 0 221 2 [ 673.365635]c21bdee0 0046 0002 c21bdecc c21bdec4 c21b3000 0002 [

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Maxim Levitsky
On Monday 22 October 2007 12:55:25 Fengguang Wu wrote: On Mon, Oct 22, 2007 at 12:40:24PM +0200, Maxim Levitsky wrote: On Monday 22 October 2007 12:22:10 Peter Zijlstra wrote: [ 673.365631] pdflush D c21bdecc 0 221 2 [ 673.365635]c21bdee0 0046 0002

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Fengguang Wu
On Mon, Oct 22, 2007 at 12:58:11PM +0200, Maxim Levitsky wrote: On Monday 22 October 2007 12:55:25 Fengguang Wu wrote: On Mon, Oct 22, 2007 at 12:40:24PM +0200, Maxim Levitsky wrote: On Monday 22 October 2007 12:22:10 Peter Zijlstra wrote: [ 673.365631] pdflush D c21bdecc 0

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Maxim Levitsky
On Monday 22 October 2007 13:19:08 Fengguang Wu wrote: On Mon, Oct 22, 2007 at 12:58:11PM +0200, Maxim Levitsky wrote: On Monday 22 October 2007 12:55:25 Fengguang Wu wrote: On Mon, Oct 22, 2007 at 12:40:24PM +0200, Maxim Levitsky wrote: On Monday 22 October 2007 12:22:10 Peter Zijlstra

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Fengguang Wu
On Mon, Oct 22, 2007 at 02:21:21PM +0200, Maxim Levitsky wrote: I Bisected this bug to exactly this commit: 2e6883bdf49abd0e7f0d9b6297fc3be7ebb2250b writeback: introduce writeback_control.more_io to indicate more io Reverting it and compiling latest git shows no more signs of that bug.

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Maxim Levitsky
On Monday 22 October 2007 14:37:07 Fengguang Wu wrote: On Mon, Oct 22, 2007 at 02:21:21PM +0200, Maxim Levitsky wrote: I Bisected this bug to exactly this commit: 2e6883bdf49abd0e7f0d9b6297fc3be7ebb2250b writeback: introduce writeback_control.more_io to indicate more io Reverting it

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Fengguang Wu
On Mon, Oct 22, 2007 at 03:05:35PM +0200, Maxim Levitsky wrote: Hi, Thank you very much too, for helping me. You are welcome :-) Applied. Had to kill klogd, since kernel generates tons of redirtied inode messages. Size of the kern.log is 863 KB, thus I I don't think it is polite to

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Maxim Levitsky
On Monday 22 October 2007 15:10:45 Fengguang Wu wrote: On Mon, Oct 22, 2007 at 03:05:35PM +0200, Maxim Levitsky wrote: Hi, Thank you very much too, for helping me. You are welcome :-) Applied. Had to kill klogd, since kernel generates tons of redirtied inode messages. Size of the

Re: 100% iowait on one of cpus in current -git

2007-10-22 Thread Fengguang Wu
On Mon, Oct 22, 2007 at 09:10:45PM +0800, Fengguang Wu wrote: Hmm, Maybe it's an reiserfs related issue. Do you have the full log file? Bingo! It can be reproduced in -mm on reiserfs: # mkfs.reiserfs /dev/sdb1 # mount /dev/sdb1 /test # cp bin /test wait for a while # dmesg [...] [ 418.346113]