Re: [PATCH 00/17] per device dirty throttling -v7

2007-07-18 Thread Martin Knoblauch
Miklos Szeredi wrote:

>> Latest version of the per bdi dirty throttling patches.
>>
>> Most of the changes since last time are little cleanups and more
>> detail in the split out of the floating proportion into their
>> own little lib.
>>
>> Patches are against 2.6.22-rc4-mm2
>>
>> A rollup of all this against 2.6.21 is available here:
>>
http://programming.kicks-ass.net/kernel-patches/balance_dirty_pages/2.6.21-per_bdi_dirty_pages.patch
>>
>> This patch-set passes the starve an USB stick test..
>
>I've done some testing of several problem cases.

 just curious - what are the plans towards inclusion in mainline?

Cheers
Martin

--
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www:   http://www.knobisoft.de
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 00/17] per device dirty throttling -v7

2007-07-18 Thread Martin Knoblauch
Miklos Szeredi wrote:

 Latest version of the per bdi dirty throttling patches.

 Most of the changes since last time are little cleanups and more
 detail in the split out of the floating proportion into their
 own little lib.

 Patches are against 2.6.22-rc4-mm2

 A rollup of all this against 2.6.21 is available here:

http://programming.kicks-ass.net/kernel-patches/balance_dirty_pages/2.6.21-per_bdi_dirty_pages.patch

 This patch-set passes the starve an USB stick test..

I've done some testing of several problem cases.

 just curious - what are the plans towards inclusion in mainline?

Cheers
Martin

--
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www:   http://www.knobisoft.de
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 00/17] per device dirty throttling -v7

2007-07-17 Thread Miklos Szeredi
> Latest version of the per bdi dirty throttling patches.
> 
> Most of the changes since last time are little cleanups and more
> detail in the split out of the floating proportion into their
> own little lib.
> 
> Patches are against 2.6.22-rc4-mm2
> 
> A rollup of all this against 2.6.21 is available here:
>   
> http://programming.kicks-ass.net/kernel-patches/balance_dirty_pages/2.6.21-per_bdi_dirty_pages.patch
> 
> This patch-set passes the starve an USB stick test..

I've done some testing of several problem cases.

1) fuse writable mmap patches + bash_shared_mapping
2) writes in a setup involving a loop dev
  a) ext3 over loop over ext3
  b) ext3 over loop over fuse-passthrough over ext3
  c) ext3 over loop over ntfs-3g

Without the patch, in all the cases I've seen deadlocks or long
stalls.  With the patch, I could not reproduce this in any of the
cases.  As predicted, the patch is performing well in this respect :)

2a is the simplest to reproduce (2.6.22, dual core, 1GB ram)

  dd if=/dev/zero of=/tmp/p5 bs=1M seek=4999 count=1
  mkfs.ext3 -F /tmp/p5
  mkdir /tmp/m5
  mount -oloop /tmp/p5 /tmp/m5
  dd if=/dev/zero of=/tmp/m5/foo bs=1M count=4000

The second dd can stall for indefinite amounts of time.  Kicking it
with sync can get it moving, but it relapses after some time.

Even with the per-device-throttling patch, case 2 shows an nr_dirty
elevated far above the 10% limit, reaching 40% or higher.  I believe,
this is due to a missing balance_dirty_pages() call in the loop
device.  And indeed the anomaly can be solved by adding this patch:

  http://lkml.org/lkml/2007/3/24/101

Miklos
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 00/17] per device dirty throttling -v7

2007-07-17 Thread Miklos Szeredi
 Latest version of the per bdi dirty throttling patches.
 
 Most of the changes since last time are little cleanups and more
 detail in the split out of the floating proportion into their
 own little lib.
 
 Patches are against 2.6.22-rc4-mm2
 
 A rollup of all this against 2.6.21 is available here:
   
 http://programming.kicks-ass.net/kernel-patches/balance_dirty_pages/2.6.21-per_bdi_dirty_pages.patch
 
 This patch-set passes the starve an USB stick test..

I've done some testing of several problem cases.

1) fuse writable mmap patches + bash_shared_mapping
2) writes in a setup involving a loop dev
  a) ext3 over loop over ext3
  b) ext3 over loop over fuse-passthrough over ext3
  c) ext3 over loop over ntfs-3g

Without the patch, in all the cases I've seen deadlocks or long
stalls.  With the patch, I could not reproduce this in any of the
cases.  As predicted, the patch is performing well in this respect :)

2a is the simplest to reproduce (2.6.22, dual core, 1GB ram)

  dd if=/dev/zero of=/tmp/p5 bs=1M seek=4999 count=1
  mkfs.ext3 -F /tmp/p5
  mkdir /tmp/m5
  mount -oloop /tmp/p5 /tmp/m5
  dd if=/dev/zero of=/tmp/m5/foo bs=1M count=4000

The second dd can stall for indefinite amounts of time.  Kicking it
with sync can get it moving, but it relapses after some time.

Even with the per-device-throttling patch, case 2 shows an nr_dirty
elevated far above the 10% limit, reaching 40% or higher.  I believe,
this is due to a missing balance_dirty_pages() call in the loop
device.  And indeed the anomaly can be solved by adding this patch:

  http://lkml.org/lkml/2007/3/24/101

Miklos
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 00/17] per device dirty throttling -v7

2007-06-14 Thread Andrew Morton
> On Thu, 14 Jun 2007 23:58:17 +0200 Peter Zijlstra <[EMAIL PROTECTED]> wrote:
> Latest version of the per bdi dirty throttling patches.

Thanks.  I've got some travel coming up and will be rather intermittent and
laggy for a week or two.  I'll save this patchset for when the in-flight
movies get dull ;)
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


[PATCH 00/17] per device dirty throttling -v7

2007-06-14 Thread Peter Zijlstra
Latest version of the per bdi dirty throttling patches.

Most of the changes since last time are little cleanups and more
detail in the split out of the floating proportion into their
own little lib.

Patches are against 2.6.22-rc4-mm2

A rollup of all this against 2.6.21 is available here:
  
http://programming.kicks-ass.net/kernel-patches/balance_dirty_pages/2.6.21-per_bdi_dirty_pages.patch

This patch-set passes the starve an USB stick test..
-- 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


[PATCH 00/17] per device dirty throttling -v7

2007-06-14 Thread Peter Zijlstra
Latest version of the per bdi dirty throttling patches.

Most of the changes since last time are little cleanups and more
detail in the split out of the floating proportion into their
own little lib.

Patches are against 2.6.22-rc4-mm2

A rollup of all this against 2.6.21 is available here:
  
http://programming.kicks-ass.net/kernel-patches/balance_dirty_pages/2.6.21-per_bdi_dirty_pages.patch

This patch-set passes the starve an USB stick test..
-- 

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH 00/17] per device dirty throttling -v7

2007-06-14 Thread Andrew Morton
 On Thu, 14 Jun 2007 23:58:17 +0200 Peter Zijlstra [EMAIL PROTECTED] wrote:
 Latest version of the per bdi dirty throttling patches.

Thanks.  I've got some travel coming up and will be rather intermittent and
laggy for a week or two.  I'll save this patchset for when the in-flight
movies get dull ;)
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/