Re: [PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-04-03 Thread Tomoki Sekiyama
Hi, Thanks for your comments. I'm sorry for my late reply. Bill Davidsen wrote: >> Then, what do you think of the following idea? >> >> (4) add `dirty_start_writeback_ratio' as percentage of memory, >> at which a generator of dirty pages itself starts writeback >> (that is, non-blocking ra

Re: [PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-03-26 Thread Bill Davidsen
Tomoki Sekiyama wrote: Hi, Thanks for your reply. 3) Use "dirty_ratio" as the blocking ratio. And add "start_writeback_ratio", and start writeback at start_writeback_ratio(default:90) * dirty_ratio / 100 [%]. In this way, specifying blocking ratio can be done in the same way as curre

Re: [PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-03-26 Thread Tomoki Sekiyama
Hi, Thanks for your reply. >>3) Use "dirty_ratio" as the blocking ratio. And add >> "start_writeback_ratio", and start writeback at >> start_writeback_ratio(default:90) * dirty_ratio / 100 [%]. >> In this way, specifying blocking ratio can be done in the same way >> as current kernel, but

Re: [PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-03-22 Thread Bill Davidsen
Tomoki Sekiyama wrote: Hi, Thanks for your comments. I'm sorry for my late reply. Bill Davidsen wrote: > Andrew Morton wrote: >> - I wonder if dirty_limit_ratio is the best name we could choose. >> vm_dirty_blocking_ratio, perhaps? Dunno. >> > I don't like it, but I dislike it less than "dirt

Re: [PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-03-21 Thread Tomoki Sekiyama
Hi, Thanks for your comments. I'm sorry for my late reply. Bill Davidsen wrote: > Andrew Morton wrote: >>> On Wed, 14 Mar 2007 21:42:46 +0900 Tomoki Sekiyama >>> <[EMAIL PROTECTED]> wrote: >>> >>> ... >>> >>> >>> -Solution: >>> >>> I consider that all of the dirty pages for the disk have been wr

Re: [PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-03-18 Thread Bill Davidsen
Andrew Morton wrote: On Wed, 14 Mar 2007 21:42:46 +0900 Tomoki Sekiyama <[EMAIL PROTECTED]> wrote: ... -Solution: I consider that all of the dirty pages for the disk have been written back and that the disk is clean if a process cannot write 'write_chunk' pages in balance_dirty_pages(). To a

Re: [PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-03-15 Thread Andrew Morton
> On Wed, 14 Mar 2007 21:42:46 +0900 Tomoki Sekiyama <[EMAIL PROTECTED]> wrote: > > ... > > > -Solution: > > I consider that all of the dirty pages for the disk have been written > back and that the disk is clean if a process cannot write 'write_chunk' > pages in balance_dirty_pages(). > > To av

Re: [PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-03-14 Thread Peter Zijlstra
Hi, I've been working on an alternative solution (see patch below). However I haven't posted yet because I'm not quite satisfied and haven't done a lot of testing. The patch relies on the per backing dev dirty/writeback counts currently in -mm to which David Chinner objected. I plan to rework tho

[PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-03-14 Thread Tomoki Sekiyama
Hi, I ported the patch sent before to 2.6.21-rc3-mm2, so I'm resending it. ( Previous patch is available at http://marc.info/?l=linux-kernel&m=117223267512340&w=2 ) -Summary: I have observed a problem that write(2) can be blocked for a long time if a system has several disks and is under heavy

Re: [RFC][PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-03-07 Thread Yuji Kakutani
Hi, Thank you for your comments. Leroy van Logchem wrote: >The default dirty_ratio on most 2.6 kernels tend to be too large imo. >If you are going to do sustained writes multiple times the size of >the memory you have at least two problems. > >1) The precious dentry and inodecache will be dropped

Re: [RFC][PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-03-02 Thread Brice Figureau
Hi, On Fri, 2007-03-02 at 13:06 +, Leroy van Logchem wrote: > > I'm sorry to piggy-back this thread. > > > > Could it be what I'm experiencing in the following bugzilla report: > > http://bugzilla.kernel.org/show_bug.cgi?id=7372 > > > > As I explained in the report, I see this issue only sin

Re: [RFC][PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-03-02 Thread Leroy van Logchem
> I'm sorry to piggy-back this thread. > > Could it be what I'm experiencing in the following bugzilla report: > http://bugzilla.kernel.org/show_bug.cgi?id=7372 > > As I explained in the report, I see this issue only since 2.6.18. > So if your concern is related to mine, what could have changed b

Re: [RFC][PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-03-02 Thread Brice Figureau
Hi, On Thu, 2007-03-01 at 12:47 +, Leroy van Logchem wrote: > Tomoki Sekiyama hitachi.com> writes: > > thanks for your comments. > > The default dirty_ratio on most 2.6 kernels tend to be too large imo. > If you are going to do sustained writes multiple times the size of > the memory you hav

Re: [RFC][PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-03-01 Thread Tomoki Sekiyama
Hi Kamezawa-san, KAMEZAWA Hiroyuki wrote: >>> Interesting, but how about adjust this parameter like below instead of >>> adding new control knob ?(this kind of knob is not easy to use.) >>> count_dirty_pages_on_device_limited(bdi, writechunk) above returns >>> dirty pages on bdi. if # of dirt

Re: [RFC][PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-03-01 Thread Leroy van Logchem
Tomoki Sekiyama hitachi.com> writes: > thanks for your comments. The default dirty_ratio on most 2.6 kernels tend to be too large imo. If you are going to do sustained writes multiple times the size of the memory you have at least two problems. 1) The precious dentry and inodecache will be dro

Re: [RFC][PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-02-26 Thread KAMEZAWA Hiroyuki
On Tue, 27 Feb 2007 09:50:16 +0900 Tomoki Sekiyama <[EMAIL PROTECTED]> wrote: > Hi Kamezawa-san, > > thanks for your reply. > > KAMEZAWA Hiroyuki wrote: > > Interesting, but how about adjust this parameter like below instead of > > adding new control knob ?(this kind of knob is not easy to use.)

Re: [RFC][PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-02-26 Thread Tomoki Sekiyama
Hi Nikita, thanks for your comments. Nikita Danilov wrote: >> While Dirty+Writeback pages get more than 40% of memory, process-B is >> blocked in balance_dirty_pages() until writeback of some (`write_chunk', >> typically = 1536) dirty pages on disk-b is started. > > May be the simpler solution is

Re: [RFC][PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-02-26 Thread Tomoki Sekiyama
Hi Kamezawa-san, thanks for your reply. KAMEZAWA Hiroyuki wrote: > Interesting, but how about adjust this parameter like below instead of > adding new control knob ?(this kind of knob is not easy to use.) > > == > struct writeback_control wbc = { > .bdi

Re: [RFC][PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-02-24 Thread Nikita Danilov
Tomoki Sekiyama writes: > Hi, Hello, > [...] > > While Dirty+Writeback pages get more than 40% of memory, process-B is > blocked in balance_dirty_pages() until writeback of some (`write_chunk', > typically = 1536) dirty pages on disk-b is started. May be the simpler solution is to use

Re: [RFC][PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-02-23 Thread KAMEZAWA Hiroyuki
On Fri, 23 Feb 2007 21:03:37 +0900 Tomoki Sekiyama <[EMAIL PROTECTED]> wrote: > Hi, > > I have observed a problem that write(2) can be blocked for a long time > if a system has several disks and is under heavy I/O pressure. This > patchset is to avoid the problem. > > Example of the probrem: >

[RFC][PATCH 0/3] VM throttling: avoid blocking occasional writers

2007-02-23 Thread Tomoki Sekiyama
Hi, I have observed a problem that write(2) can be blocked for a long time if a system has several disks and is under heavy I/O pressure. This patchset is to avoid the problem. Example of the probrem: There are two processes on a system which has two disks. Process-A writes heavily to disk-a, an