[Devel] Re: [PATCH 0/5] blk-throttle: writeback and swap IO control

2011-02-23 Thread Vivek Goyal
On Thu, Feb 24, 2011 at 12:14:11AM +0100, Andrea Righi wrote: > On Wed, Feb 23, 2011 at 10:23:54AM -0500, Vivek Goyal wrote: > > > > Agreed. Granularity of per inode level might be accetable in many > > > > cases. Again, I am worried faster group getting stuck

[Devel] Re: [PATCH 0/5] blk-throttle: writeback and swap IO control

2011-02-23 Thread Vivek Goyal
> > Agreed. Granularity of per inode level might be accetable in many > > cases. Again, I am worried faster group getting stuck behind slower > > group. > > > > I am wondering if we are trying to solve the problem of ASYNC write > > throttling > > at wrong layer. Should ASYNC IO be throttled bef

[Devel] Re: [PATCH 3/5] page_cgroup: make page tracking available for blkio

2011-02-22 Thread Vivek Goyal
On Wed, Feb 23, 2011 at 12:01:47AM +0100, Andrea Righi wrote: > On Tue, Feb 22, 2011 at 01:01:45PM -0700, Jonathan Corbet wrote: > > On Tue, 22 Feb 2011 18:12:54 +0100 > > Andrea Righi wrote: > > > > > The page_cgroup infrastructure, currently available only for the memory > > > cgroup controller

[Devel] Re: [PATCH 0/5] blk-throttle: writeback and swap IO control

2011-02-22 Thread Vivek Goyal
On Tue, Feb 22, 2011 at 11:41:41PM +0100, Andrea Righi wrote: > On Tue, Feb 22, 2011 at 02:34:03PM -0500, Vivek Goyal wrote: > > On Tue, Feb 22, 2011 at 06:12:51PM +0100, Andrea Righi wrote: > > > Currently the blkio.throttle controller only support synchronous IO > > >

[Devel] Re: [PATCH 4/5] blk-throttle: track buffered and anonymous pages

2011-02-22 Thread Vivek Goyal
On Wed, Feb 23, 2011 at 12:05:34AM +0100, Andrea Righi wrote: > On Tue, Feb 22, 2011 at 04:00:30PM -0500, Vivek Goyal wrote: > > On Tue, Feb 22, 2011 at 06:12:55PM +0100, Andrea Righi wrote: > > > Add the tracking of buffered (writeback) and anonymous pages. > > > &g

[Devel] Re: [PATCH 3/5] page_cgroup: make page tracking available for blkio

2011-02-22 Thread Vivek Goyal
On Tue, Feb 22, 2011 at 01:01:45PM -0700, Jonathan Corbet wrote: > On Tue, 22 Feb 2011 18:12:54 +0100 > Andrea Righi wrote: > > > The page_cgroup infrastructure, currently available only for the memory > > cgroup controller, can be used to store the owner of each page and > > opportunely track th

[Devel] Re: [PATCH 4/5] blk-throttle: track buffered and anonymous pages

2011-02-22 Thread Vivek Goyal
On Tue, Feb 22, 2011 at 06:12:55PM +0100, Andrea Righi wrote: > Add the tracking of buffered (writeback) and anonymous pages. > > Dirty pages in the page cache can be processed asynchronously by the > per-bdi flusher kernel threads or by any other thread in the system, > according to the writeback

[Devel] Re: [PATCH 3/5] page_cgroup: make page tracking available for blkio

2011-02-22 Thread Vivek Goyal
On Tue, Feb 22, 2011 at 06:12:54PM +0100, Andrea Righi wrote: > The page_cgroup infrastructure, currently available only for the memory > cgroup controller, can be used to store the owner of each page and > opportunely track the writeback IO. This information is encoded in > the upper 16-bits of th

[Devel] Re: [PATCH 4/5] blk-throttle: track buffered and anonymous pages

2011-02-22 Thread Vivek Goyal
On Tue, Feb 22, 2011 at 10:42:41AM -0800, Chad Talbott wrote: > On Tue, Feb 22, 2011 at 9:12 AM, Andrea Righi wrote: > > Add the tracking of buffered (writeback) and anonymous pages. > ... > > --- > >  block/blk-throttle.c   |   87 > > +++- > >  include

[Devel] Re: [PATCH 0/5] blk-throttle: writeback and swap IO control

2011-02-22 Thread Vivek Goyal
On Tue, Feb 22, 2011 at 06:12:51PM +0100, Andrea Righi wrote: > Currently the blkio.throttle controller only support synchronous IO requests. > This means that we always look at the current task to identify the "owner" of > each IO request. > > However dirty pages in the page cache can be wrote to

[Devel] Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

2010-07-22 Thread Vivek Goyal
On Thu, Jul 22, 2010 at 04:14:01PM -0700, Greg KH wrote: > On Fri, Jul 23, 2010 at 01:08:12AM +0200, Kay Sievers wrote: > > On Thu, Jul 22, 2010 at 23:18, Greg KH wrote: > > >> For my testing I now always use /cgroup/ and create directories under it > > >> /cgroup/blkio /cgroup/cpu etc and mount c

[Devel] Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

2010-07-22 Thread Vivek Goyal
On Thu, Jul 22, 2010 at 02:18:56PM -0700, Greg KH wrote: > On Thu, Jul 22, 2010 at 03:37:41PM -0400, Vivek Goyal wrote: > > On Thu, Jul 22, 2010 at 11:36:15AM -0700, Greg KH wrote: > > > On Thu, Jul 22, 2010 at 11:31:07AM -0700, Paul Menage wrote: > > > > On Thu, Jul

[Devel] Re: [PATCH] cgroupfs: create /sys/fs/cgroup to mount cgroupfs on

2010-07-22 Thread Vivek Goyal
On Thu, Jul 22, 2010 at 11:36:15AM -0700, Greg KH wrote: > On Thu, Jul 22, 2010 at 11:31:07AM -0700, Paul Menage wrote: > > On Thu, Jul 22, 2010 at 11:26 AM, Greg KH wrote: > > > We really shouldn't be asking userspace to create new root filesystems. > > > So follow along with all of the other in-

[Devel] Re: [PATCH 5/5] blk-cgroup: Fix an RCU warning in blkiocg_create()

2010-04-22 Thread Vivek Goyal
n! > ... > IIUC, so blkiocg_create() is being called with cgroup_mutex held and not with rcu read lock held. Hence rcu_dereference() in css_depth() gives warning. So one easy solution is to don't use css_depth() at all. In this case simple check like cgroup->top_cgroup should suffice. Makese

[Devel] Re: [PATCH -mmotm 1/5] memcg: disable irq at page cgroup lock

2010-04-14 Thread Vivek Goyal
On Tue, Apr 13, 2010 at 11:55:12PM -0700, Greg Thelen wrote: > On Thu, Mar 18, 2010 at 8:00 PM, KAMEZAWA Hiroyuki > wrote: > > On Fri, 19 Mar 2010 08:10:39 +0530 > > Balbir Singh wrote: > > > >> * KAMEZAWA Hiroyuki [2010-03-19 10:23:32]: > >> > >> > On Thu, 18 Mar 2010 21:58:55 +0530 > >> > Bal

[Devel] Re: [PATCH -mmotm 1/5] memcg: disable irq at page cgroup lock

2010-04-14 Thread Vivek Goyal
On Wed, Apr 14, 2010 at 06:29:04PM +0900, KAMEZAWA Hiroyuki wrote: > On Tue, 13 Apr 2010 23:55:12 -0700 > Greg Thelen wrote: > > > On Thu, Mar 18, 2010 at 8:00 PM, KAMEZAWA Hiroyuki > > wrote: > > > On Fri, 19 Mar 2010 08:10:39 +0530 > > > Balbir Singh wrote: > > > > > >> * KAMEZAWA Hiroyuki

[Devel] Re: [PATCH -mmotm 0/5] memcg: per cgroup dirty limit (v7)

2010-03-17 Thread Vivek Goyal
On Thu, Mar 18, 2010 at 12:47:43AM +0530, Balbir Singh wrote: > * Vivek Goyal [2010-03-17 09:34:07]: > > > On Wed, Mar 17, 2010 at 05:24:28PM +0530, Balbir Singh wrote: > > > * Vivek Goyal [2010-03-15 13:19:21]: > > > > > > > On Mon, Mar 15, 2

[Devel] Re: [PATCH -mmotm 0/5] memcg: per cgroup dirty limit (v7)

2010-03-17 Thread Vivek Goyal
On Thu, Mar 18, 2010 at 12:23:27AM +0530, Balbir Singh wrote: > * Vivek Goyal [2010-03-17 09:34:07]: > > > > > > > > > root cgroup > > > > == > > > > #time dd if=/dev/zero of=/root/zerofile bs=4K count=1M > &g

[Devel] Re: [PATCH -mmotm 0/5] memcg: per cgroup dirty limit (v7)

2010-03-17 Thread Vivek Goyal
On Wed, Mar 17, 2010 at 05:24:28PM +0530, Balbir Singh wrote: > * Vivek Goyal [2010-03-15 13:19:21]: > > > On Mon, Mar 15, 2010 at 01:12:09PM -0400, Vivek Goyal wrote: > > > On Mon, Mar 15, 2010 at 12:26:37AM +0100, Andrea Righi wrote: > > > > Control the maxim

[Devel] Re: [PATCH -mmotm 4/5] memcg: dirty pages accounting and limiting infrastructure

2010-03-16 Thread Vivek Goyal
On Tue, Mar 16, 2010 at 11:32:38AM +0900, Daisuke Nishimura wrote: [..] > > + * mem_cgroup_page_stat() - get memory cgroup file cache statistics > > + * @item: memory statistic item exported to the kernel > > + * > > + * Return the accounted statistic value, or a negative value in case of > > er

[Devel] Re: [PATCH] block: make CONFIG_BLK_CGROUP visible

2010-03-16 Thread Vivek Goyal
On Tue, Mar 16, 2010 at 09:36:41AM +0800, Li Zefan wrote: > >>---help--- > >>Generic block IO controller cgroup interface. This is the common > >> @@ -91,7 +92,7 @@ config BLK_CGROUP > >>to such task groups. > >> > >> config DEBUG_BLK_CGROUP > >> - bool > >> + bool "Block cgroup de

[Devel] Re: [PATCH -mmotm 0/5] memcg: per cgroup dirty limit (v7)

2010-03-15 Thread Vivek Goyal
On Mon, Mar 15, 2010 at 01:12:09PM -0400, Vivek Goyal wrote: > On Mon, Mar 15, 2010 at 12:26:37AM +0100, Andrea Righi wrote: > > Control the maximum amount of dirty pages a cgroup can have at any given > > time. > > > > Per cgroup dirty limit is like fixing the m

[Devel] Re: [PATCH -mmotm 0/5] memcg: per cgroup dirty limit (v7)

2010-03-15 Thread Vivek Goyal
On Mon, Mar 15, 2010 at 12:26:37AM +0100, Andrea Righi wrote: > Control the maximum amount of dirty pages a cgroup can have at any given time. > > Per cgroup dirty limit is like fixing the max amount of dirty (hard to > reclaim) > page cache used by any cgroup. So, in case of multiple cgroup writ

[Devel] Re: [PATCH -mmotm 0/5] memcg: per cgroup dirty limit (v6)

2010-03-15 Thread Vivek Goyal
On Fri, Mar 12, 2010 at 11:24:33AM +0900, KAMEZAWA Hiroyuki wrote: > On Fri, 12 Mar 2010 10:14:11 +0900 > Daisuke Nishimura wrote: > > > On Thu, 11 Mar 2010 18:42:44 +0900, KAMEZAWA Hiroyuki > > wrote: > > > On Thu, 11 Mar 2010 18:25:00 +0900 > > > KAMEZAWA Hiroyuki wrote: > > > > Then, it's n

[Devel] Re: [PATCH -mmotm 0/5] memcg: per cgroup dirty limit (v6)

2010-03-15 Thread Vivek Goyal
On Fri, Mar 12, 2010 at 12:59:22AM +0100, Andrea Righi wrote: > On Thu, Mar 11, 2010 at 01:07:53PM -0500, Vivek Goyal wrote: > > On Wed, Mar 10, 2010 at 12:00:31AM +0100, Andrea Righi wrote: > > > Control the maximum amount of dirty pages a cgroup can have at any

[Devel] Re: [PATCH -mmotm 0/5] memcg: per cgroup dirty limit (v6)

2010-03-15 Thread Vivek Goyal
On Fri, Mar 12, 2010 at 08:42:30AM +0900, KAMEZAWA Hiroyuki wrote: > On Thu, 11 Mar 2010 10:03:07 -0500 > Vivek Goyal wrote: > > > On Thu, Mar 11, 2010 at 06:25:00PM +0900, KAMEZAWA Hiroyuki wrote: > > > On Thu, 11 Mar 2010 10:14:25 +0100 > > > Peter Zijlstra w

[Devel] Re: [PATCH -mmotm 0/5] memcg: per cgroup dirty limit (v6)

2010-03-15 Thread Vivek Goyal
On Fri, Mar 12, 2010 at 12:27:09AM +0100, Andrea Righi wrote: > On Thu, Mar 11, 2010 at 10:03:07AM -0500, Vivek Goyal wrote: > > On Thu, Mar 11, 2010 at 06:25:00PM +0900, KAMEZAWA Hiroyuki wrote: > > > On Thu, 11 Mar 2010 10:14:25 +0100 > > > Peter Zijlstra wrote: >

[Devel] Re: [PATCH] block: make CONFIG_BLK_CGROUP visible

2010-03-15 Thread Vivek Goyal
On Mon, Mar 15, 2010 at 11:18:13AM +0800, Li Zefan wrote: > Make the config visible, so we can choose from CONFIG_BLK_CGROUP=y > and CONFIG_BLK_CGROUP=m when CONFIG_IOSCHED_CFQ=m. > > Signed-off-by: Li Zefan > --- > block/Kconfig |5 +++-- > 1 files changed, 3 insertions(+), 2 deletions(-) >

[Devel] Re: [PATCH -mmotm 0/5] memcg: per cgroup dirty limit (v6)

2010-03-11 Thread Vivek Goyal
On Wed, Mar 10, 2010 at 12:00:31AM +0100, Andrea Righi wrote: > Control the maximum amount of dirty pages a cgroup can have at any given time. > > Per cgroup dirty limit is like fixing the max amount of dirty (hard to > reclaim) > page cache used by any cgroup. So, in case of multiple cgroup writ

[Devel] Re: [PATCH mmotm 2.5/4] memcg: disable irq at page cgroup lock (Re: [PATCH -mmotm 3/4] memcg: dirty pages accounting and limiting infrastructure)

2010-03-11 Thread Vivek Goyal
On Thu, Mar 11, 2010 at 01:49:08PM +0900, KAMEZAWA Hiroyuki wrote: > On Thu, 11 Mar 2010 13:31:23 +0900 > Daisuke Nishimura wrote: > > > On Wed, 10 Mar 2010 09:26:24 +0530, Balbir Singh > > wrote: > > > * nishim...@mxp.nes.nec.co.jp [2010-03-10 > > > 10:43:09]: > > > I made a patch(attached)

[Devel] Re: [PATCH -mmotm 0/5] memcg: per cgroup dirty limit (v6)

2010-03-11 Thread Vivek Goyal
On Thu, Mar 11, 2010 at 06:25:00PM +0900, KAMEZAWA Hiroyuki wrote: > On Thu, 11 Mar 2010 10:14:25 +0100 > Peter Zijlstra wrote: > > > On Thu, 2010-03-11 at 10:17 +0900, KAMEZAWA Hiroyuki wrote: > > > On Thu, 11 Mar 2010 09:39:13 +0900 > > > KAMEZAWA Hiroyuki wrote: > > > > > The performance over

[Devel] Re: [PATCH -mmotm 4/5] memcg: dirty pages accounting and limiting infrastructure

2010-03-10 Thread Vivek Goyal
On Wed, Mar 10, 2010 at 12:00:35AM +0100, Andrea Righi wrote: [..] > - * Currently used to update mapped file statistics, but the routine can be > - * generalized to update other statistics as well. > + * mem_cgroup_update_page_stat() - update memcg file cache's accounting > + * @page:the pag

[Devel] Re: [PATCH -mmotm 4/4] memcg: dirty pages instrumentation

2010-03-04 Thread Vivek Goyal
On Thu, Mar 04, 2010 at 11:40:15AM +0100, Andrea Righi wrote: [..] > diff --git a/mm/page-writeback.c b/mm/page-writeback.c > index 5a0f8f3..c5d14ea 100644 > --- a/mm/page-writeback.c > +++ b/mm/page-writeback.c > @@ -137,13 +137,16 @@ static struct prop_descriptor vm_dirties; > */ > static int

[Devel] Re: [PATCH -mmotm 4/4] memcg: dirty pages instrumentation

2010-03-04 Thread Vivek Goyal
On Thu, Mar 04, 2010 at 11:40:15AM +0100, Andrea Righi wrote: [..] > diff --git a/mm/page-writeback.c b/mm/page-writeback.c > index 5a0f8f3..c5d14ea 100644 > --- a/mm/page-writeback.c > +++ b/mm/page-writeback.c > @@ -137,13 +137,16 @@ static struct prop_descriptor vm_dirties; > */ > static int

[Devel] Re: [PATCH -mmotm 3/3] memcg: dirty pages instrumentation

2010-03-02 Thread Vivek Goyal
On Tue, Mar 02, 2010 at 11:22:48PM +0100, Andrea Righi wrote: > On Tue, Mar 02, 2010 at 10:05:29AM -0500, Vivek Goyal wrote: > > On Mon, Mar 01, 2010 at 11:18:31PM +0100, Andrea Righi wrote: > > > On Mon, Mar 01, 2010 at 05:02:08PM -0500, Vivek Goyal wrote: > > > >

[Devel] Re: [PATCH -mmotm 3/3] memcg: dirty pages instrumentation

2010-03-02 Thread Vivek Goyal
On Mon, Mar 01, 2010 at 11:18:31PM +0100, Andrea Righi wrote: > On Mon, Mar 01, 2010 at 05:02:08PM -0500, Vivek Goyal wrote: > > > @@ -686,10 +699,14 @@ void throttle_vm_writeout(gfp_t gfp_mask) > > > */ > > > dirty_thresh += dir

[Devel] Re: [PATCH -mmotm 3/3] memcg: dirty pages instrumentation

2010-03-01 Thread Vivek Goyal
On Mon, Mar 01, 2010 at 10:23:40PM +0100, Andrea Righi wrote: > Apply the cgroup dirty pages accounting and limiting infrastructure to > the opportune kernel functions. > > Signed-off-by: Andrea Righi > --- > fs/fuse/file.c |5 +++ > fs/nfs/write.c |4 ++ > fs/nilfs2/segment.c

[Devel] Re: [PATCH 2/2] memcg: dirty pages instrumentation

2010-02-26 Thread Vivek Goyal
On Fri, Feb 26, 2010 at 11:21:21PM +0100, Andrea Righi wrote: > On Fri, Feb 26, 2010 at 04:48:11PM -0500, Vivek Goyal wrote: > > On Thu, Feb 25, 2010 at 04:12:11PM +0100, Andrea Righi wrote: > > > On Tue, Feb 23, 2010 at 04:29:43PM -0500, Vivek Goyal wrote: > > > >

[Devel] Re: [PATCH 2/2] memcg: dirty pages instrumentation

2010-02-26 Thread Vivek Goyal
On Thu, Feb 25, 2010 at 04:12:11PM +0100, Andrea Righi wrote: > On Tue, Feb 23, 2010 at 04:29:43PM -0500, Vivek Goyal wrote: > > On Sun, Feb 21, 2010 at 04:18:45PM +0100, Andrea Righi wrote: > > > > [..] > > > diff --git a/mm/page-writeback.c b/mm/page-writeback.

[Devel] Re: [PATCH 2/2] memcg: dirty pages instrumentation

2010-02-23 Thread Vivek Goyal
On Sun, Feb 21, 2010 at 04:18:45PM +0100, Andrea Righi wrote: [..] > diff --git a/mm/page-writeback.c b/mm/page-writeback.c > index 0b19943..c9ff1cd 100644 > --- a/mm/page-writeback.c > +++ b/mm/page-writeback.c > @@ -137,10 +137,11 @@ static struct prop_descriptor vm_dirties; > */ > static int

[Devel] Re: [RFC] [PATCH 0/2] memcg: per cgroup dirty limit

2010-02-23 Thread Vivek Goyal
On Tue, Feb 23, 2010 at 10:55:55AM +0100, Andrea Righi wrote: > On Mon, Feb 22, 2010 at 01:29:34PM -0500, Vivek Goyal wrote: > > > I would't like to add many different interfaces to do the same thing. > > > I'd prefer to choose just one interface and always use i

[Devel] Re: [PATCH 2/2] memcg: dirty pages instrumentation

2010-02-23 Thread Vivek Goyal
On Tue, Feb 23, 2010 at 10:40:40AM +0100, Andrea Righi wrote: > On Mon, Feb 22, 2010 at 11:52:15AM -0500, Vivek Goyal wrote: > > > unsigned long determine_dirtyable_memory(void) > > > { > > > - unsigned long x; > > > - > > > - x = global_page

[Devel] Re: [RFC] [PATCH 0/2] memcg: per cgroup dirty limit

2010-02-23 Thread Vivek Goyal
On Tue, Feb 23, 2010 at 09:07:04AM +0900, KAMEZAWA Hiroyuki wrote: > On Mon, 22 Feb 2010 12:58:33 -0500 > Vivek Goyal wrote: > > > On Mon, Feb 22, 2010 at 11:06:40PM +0530, Balbir Singh wrote: > > > * Vivek Goyal [2010-02-22 09:27:45]: > > > > > >

[Devel] Re: [PATCH 1/2] memcg: dirty pages accounting and limiting infrastructure

2010-02-22 Thread Vivek Goyal
On Mon, Feb 22, 2010 at 09:22:42AM +0900, KAMEZAWA Hiroyuki wrote: [..] > > +static int mem_cgroup_dirty_bytes_write(struct cgroup *cgrp, struct cftype > > *cft, > > + u64 val) > > +{ > > + struct mem_cgroup *memcg = mem_cgroup_from_cont(cgrp); > > + struct me

[Devel] Re: [RFC] [PATCH 0/2] memcg: per cgroup dirty limit

2010-02-22 Thread Vivek Goyal
On Mon, Feb 22, 2010 at 07:12:27PM +0100, Andrea Righi wrote: > On Mon, Feb 22, 2010 at 09:27:45AM -0500, Vivek Goyal wrote: > > On Sun, Feb 21, 2010 at 04:18:43PM +0100, Andrea Righi wrote: > > > Control the maximum amount of dirty pages a cgroup can have at any

[Devel] Re: [PATCH 1/2] memcg: dirty pages accounting and limiting infrastructure

2010-02-22 Thread Vivek Goyal
On Sun, Feb 21, 2010 at 11:17:01PM +0100, Andrea Righi wrote: > On Sun, Feb 21, 2010 at 01:28:35PM -0800, David Rientjes wrote: > > [snip] > > > > +static struct mem_cgroup *get_mem_cgroup_from_page(struct page *page) > > > +{ > > > + struct page_cgroup *pc; > > > + struct mem_cgroup *mem = NULL;

[Devel] Re: [RFC] [PATCH 0/2] memcg: per cgroup dirty limit

2010-02-22 Thread Vivek Goyal
On Mon, Feb 22, 2010 at 11:06:40PM +0530, Balbir Singh wrote: > * Vivek Goyal [2010-02-22 09:27:45]: > > > On Sun, Feb 21, 2010 at 04:18:43PM +0100, Andrea Righi wrote: > > > Control the maximum amount of dirty pages a cgroup can have at any given > > > time. >

[Devel] Re: [PATCH 2/2] memcg: dirty pages instrumentation

2010-02-22 Thread Vivek Goyal
On Sun, Feb 21, 2010 at 04:18:45PM +0100, Andrea Righi wrote: > Apply the cgroup dirty pages accounting and limiting infrastructure to > the opportune kernel functions. > > Signed-off-by: Andrea Righi > --- > fs/fuse/file.c |3 ++ > fs/nfs/write.c |3 ++ > fs/nilfs2/segment.c |

[Devel] Re: [PATCH 1/2] memcg: dirty pages accounting and limiting infrastructure

2010-02-22 Thread Vivek Goyal
On Sun, Feb 21, 2010 at 04:18:44PM +0100, Andrea Righi wrote: > Infrastructure to account dirty pages per cgroup + add memory.dirty_bytes > limit > in cgroupfs. > > Signed-off-by: Andrea Righi > --- > include/linux/memcontrol.h | 31 ++ > mm/memcontrol.c| 218 > +

[Devel] Re: [RFC] [PATCH 0/2] memcg: per cgroup dirty limit

2010-02-22 Thread Vivek Goyal
On Sun, Feb 21, 2010 at 04:18:43PM +0100, Andrea Righi wrote: > Control the maximum amount of dirty pages a cgroup can have at any given time. > > Per cgroup dirty limit is like fixing the max amount of dirty (hard to > reclaim) > page cache used by any cgroup. So, in case of multiple cgroup writ

[Devel] Re: [RFC] [PATCH 2/2] cgroups: blkio subsystem as module

2010-01-14 Thread Vivek Goyal
On Thu, Jan 14, 2010 at 03:02:09PM +0530, Balbir Singh wrote: > On Tue, Jan 12, 2010 at 5:51 AM, KAMEZAWA Hiroyuki > wrote: > > On Fri, 8 Jan 2010 10:10:38 -0500 > > Vivek Goyal wrote: > > > >> On Fri, Jan 08, 2010 at 12:30:21AM -0500, Ben Blum wrote: > >&g

[Devel] Re: [RFC] [PATCH 2/2] cgroups: blkio subsystem as module

2010-01-08 Thread Vivek Goyal
On Fri, Jan 08, 2010 at 12:30:21AM -0500, Ben Blum wrote: > Convert blk-cgroup to be buildable as a module > > From: Ben Blum > > This patch modifies the Block I/O cgroup subsystem to be able to be built as a > module. As the CFQ disk scheduler optionally depends on blk-cgroup, config > options

[Devel] Re: [RFC] [PATCH 2/2] cgroups: blkio subsystem as module

2010-01-08 Thread Vivek Goyal
On Fri, Jan 08, 2010 at 12:30:21AM -0500, Ben Blum wrote: > Convert blk-cgroup to be buildable as a module > > From: Ben Blum > > This patch modifies the Block I/O cgroup subsystem to be able to be built as a > module. As the CFQ disk scheduler optionally depends on blk-cgroup, config > options

[Devel] Re: IO controller Mini-Summit 2009

2009-10-14 Thread Vivek Goyal
On Wed, Oct 14, 2009 at 09:24:44PM +0900, Ryo Tsuruta wrote: > Hello, > Hi Ryo, CCing people who are planning to attend the mini summit either in person or phone. (As per your list on io mini summit wiki page). Not sure if everybody is scanning mailing list for update on mini summit. I am check

[Devel] Re: Performance numbers with IO throttling patches (Was: Re: IO scheduler based IO controller V10)

2009-10-12 Thread Vivek Goyal
On Sun, Oct 11, 2009 at 12:27:30AM +0200, Andrea Righi wrote: [..] > > Multiple Random Reader vs Sequential Reader > > === > > Generally random readers bring the throughput down of others in the > > system. Ran a test to see the impact of increasing numb

[Devel] Re: Performance numbers with IO throttling patches (Was: Re: IO scheduler based IO controller V10)

2009-10-11 Thread Vivek Goyal
On Sun, Oct 11, 2009 at 12:27:30AM +0200, Andrea Righi wrote: [..] > > > > - Andrea, can you please also run similar tests to see if you see same > > results or not. This is to rule out any testing methodology errors or > > scripting bugs. :-). I also have collected the snapshot of some cgrou

[Devel] Performance numbers with IO throttling patches (Was: Re: IO scheduler based IO controller V10)

2009-10-10 Thread Vivek Goyal
On Thu, Sep 24, 2009 at 02:33:15PM -0700, Andrew Morton wrote: [..] > > Environment > > == > > A 7200 RPM SATA drive with queue depth of 31. Ext3 filesystem. > > That's a bit of a toy. > > Do we have testing results for more enterprisey hardware? Big storage > arrays? SSD? Infiniband?

[Devel] More performance numbers (Was: Re: IO scheduler based IO controller V10)

2009-10-07 Thread Vivek Goyal
On Thu, Sep 24, 2009 at 02:33:15PM -0700, Andrew Morton wrote: [..] > > > > Testing > > === > > > > Environment > > == > > A 7200 RPM SATA drive with queue depth of 31. Ext3 filesystem. > > That's a bit of a toy. > > Do we have testing results for more enterprisey hardware? Big sto

[Devel] Re: IO scheduler based IO controller V10

2009-10-07 Thread Vivek Goyal
On Wed, Oct 07, 2009 at 11:38:05PM +0900, Ryo Tsuruta wrote: > Hi Vivek, > > Vivek Goyal wrote: > > > > >> If one would like to > > > > >> combine some physical disks into one logical device like a dm-linear, > > > > >> I thi

[Devel] Re: Do we support ioprio on SSDs with NCQ (Was: Re: IO scheduler based IO controller V10)

2009-10-06 Thread Vivek Goyal
On Sun, Oct 04, 2009 at 02:46:44PM +0200, Corrado Zoccolo wrote: > Hi Vivek, > On Sun, Oct 4, 2009 at 2:11 PM, Vivek Goyal wrote: > > On Sun, Oct 04, 2009 at 11:15:24AM +0200, Corrado Zoccolo wrote: > >> Hi Vivek, > >> My guess is that the formula that is used to han

[Devel] Re: IO scheduler based IO controller V10

2009-10-06 Thread Vivek Goyal
On Tue, Oct 06, 2009 at 04:17:44PM +0900, Ryo Tsuruta wrote: > Hi Vivek and Nauman, > > Nauman Rafique wrote: > > >> > > How about adding a callback function to the higher level controller? > > >> > > CFQ calls it when the active queue runs out of time, then the higer > > >> > > level controller

[Devel] Re: IO scheduler based IO controller V10

2009-10-05 Thread Vivek Goyal
On Mon, Oct 05, 2009 at 11:55:35PM +0900, Ryo Tsuruta wrote: > Hi Vivek, > > Vivek Goyal wrote: > > On Mon, Oct 05, 2009 at 07:38:08PM +0900, Ryo Tsuruta wrote: > > > Hi, > > > > > > Munehiro Ikeda wrote: > > > > Vivek Goyal wrote, on 10/

[Devel] Re: IO scheduler based IO controller V10

2009-10-05 Thread Vivek Goyal
On Mon, Oct 05, 2009 at 07:38:08PM +0900, Ryo Tsuruta wrote: > Hi, > > Munehiro Ikeda wrote: > > Vivek Goyal wrote, on 10/01/2009 10:57 PM: > > > Before finishing this mail, will throw a whacky idea in the ring. I was > > > going through the request based dm-mu

[Devel] Re: Do we support ioprio on SSDs with NCQ (Was: Re: IO scheduler based IO controller V10)

2009-10-04 Thread Vivek Goyal
On Sun, Oct 04, 2009 at 11:15:24AM +0200, Corrado Zoccolo wrote: > Hi Vivek, > On Sat, Oct 3, 2009 at 3:38 PM, Vivek Goyal wrote: > > On Sat, Oct 03, 2009 at 02:43:14PM +0200, Corrado Zoccolo wrote: > >> On Sat, Oct 3, 2009 at 12:27 AM, Vivek Goyal wrote: > >> &

[Devel] Re: Do not overload dispatch queue (Was: Re: IO scheduler based IO controller V10)

2009-10-03 Thread Vivek Goyal
On Sat, Oct 03, 2009 at 03:21:15PM +0200, Jens Axboe wrote: > On Sat, Oct 03 2009, Vivek Goyal wrote: > > On Sat, Oct 03, 2009 at 07:29:15AM -0400, Vivek Goyal wrote: > > > On Sat, Oct 03, 2009 at 07:56:18AM +0200, Mike Galbraith wrote: > > > > On Sat, 2009-10-03

[Devel] Do we support ioprio on SSDs with NCQ (Was: Re: IO scheduler based IO controller V10)

2009-10-03 Thread Vivek Goyal
On Sat, Oct 03, 2009 at 02:43:14PM +0200, Corrado Zoccolo wrote: > On Sat, Oct 3, 2009 at 12:27 AM, Vivek Goyal wrote: > > On Sat, Oct 03, 2009 at 12:14:28AM +0200, Corrado Zoccolo wrote: > >> In fact I think that the 'rotating' flag name is misleading. > >

[Devel] Do not overload dispatch queue (Was: Re: IO scheduler based IO controller V10)

2009-10-03 Thread Vivek Goyal
On Sat, Oct 03, 2009 at 07:29:15AM -0400, Vivek Goyal wrote: > On Sat, Oct 03, 2009 at 07:56:18AM +0200, Mike Galbraith wrote: > > On Sat, 2009-10-03 at 07:49 +0200, Mike Galbraith wrote: > > > On Fri, 2009-10-02 at 20:19 +0200, Jens Axboe wrote: > > > > > > &g

[Devel] Re: IO scheduler based IO controller V10

2009-10-03 Thread Vivek Goyal
On Sat, Oct 03, 2009 at 07:56:18AM +0200, Mike Galbraith wrote: > On Sat, 2009-10-03 at 07:49 +0200, Mike Galbraith wrote: > > On Fri, 2009-10-02 at 20:19 +0200, Jens Axboe wrote: > > > > > If you could do a cleaned up version of your overload patch based on > > > this: > > > > > > http://git.ker

[Devel] Re: IO scheduler based IO controller V10

2009-10-02 Thread Vivek Goyal
On Sat, Oct 03, 2009 at 12:14:28AM +0200, Corrado Zoccolo wrote: > On Fri, Oct 2, 2009 at 9:58 PM, Vivek Goyal wrote: > > On Fri, Oct 02, 2009 at 12:50:17PM -0400, valdis.kletni...@vt.edu wrote: > >> On Fri, 02 Oct 2009 11:40:20 EDT, Vivek Goyal said: > >> > >>

[Devel] Re: IO scheduler based IO controller V10

2009-10-02 Thread Vivek Goyal
On Fri, Oct 02, 2009 at 12:50:17PM -0400, valdis.kletni...@vt.edu wrote: > On Fri, 02 Oct 2009 11:40:20 EDT, Vivek Goyal said: > > > In that case, Corrado's suggestion of refining it further and disabling > > idling > > for seeky process only on non-rotational

[Devel] Re: IO scheduler based IO controller V10

2009-10-02 Thread Vivek Goyal
On Fri, Oct 02, 2009 at 05:32:00PM +0200, Mike Galbraith wrote: > On Fri, 2009-10-02 at 17:27 +0200, Corrado Zoccolo wrote: > > On Fri, Oct 2, 2009 at 2:49 PM, Vivek Goyal wrote: > > > On Fri, Oct 02, 2009 at 12:55:25PM +0200, Corrado Zoccolo wrote: > > > > >

[Devel] Re: IO scheduler based IO controller V10

2009-10-02 Thread Vivek Goyal
On Fri, Oct 02, 2009 at 05:27:55PM +0200, Corrado Zoccolo wrote: > On Fri, Oct 2, 2009 at 2:49 PM, Vivek Goyal wrote: > > On Fri, Oct 02, 2009 at 12:55:25PM +0200, Corrado Zoccolo wrote: > > > > Actually I am not touching this code. Looking at the V10, I have not > &g

[Devel] Re: IO scheduler based IO controller V10

2009-10-02 Thread Vivek Goyal
On Fri, Oct 02, 2009 at 12:55:25PM +0200, Corrado Zoccolo wrote: > Hi Jens, > On Fri, Oct 2, 2009 at 11:28 AM, Jens Axboe wrote: > > On Fri, Oct 02 2009, Ingo Molnar wrote: > >> > >> * Jens Axboe wrote: > >> > > > > It's really not that simple, if we go and do easy latency bits, then > > throughp

[Devel] Re: IO scheduler based IO controller V10

2009-10-01 Thread Vivek Goyal
On Thu, Oct 01, 2009 at 09:31:09AM -0400, Vivek Goyal wrote: > On Thu, Oct 01, 2009 at 03:41:25PM +0900, Ryo Tsuruta wrote: > > Hi Vivek, > > > > Vivek Goyal wrote: > > > On Wed, Sep 30, 2009 at 05:43:19PM +0900, Ryo Tsuruta wrote: > > > > Hi

[Devel] Re: IO scheduler based IO controller V10

2009-10-01 Thread Vivek Goyal
On Thu, Oct 01, 2009 at 03:41:25PM +0900, Ryo Tsuruta wrote: > Hi Vivek, > > Vivek Goyal wrote: > > On Wed, Sep 30, 2009 at 05:43:19PM +0900, Ryo Tsuruta wrote: > > > Hi Vivek, > > > > > > Vivek Goyal wrote: > > > > I was thinkin

[Devel] Re: IO scheduler based IO controller V10

2009-09-30 Thread Vivek Goyal
On Wed, Sep 30, 2009 at 10:05:39PM +0200, Mike Galbraith wrote: > > > > /* > > +* We may have seeky queues, don't throttle up just yet. > > +*/ > > + if (time_before(jiffies, cfqd->last_seeker + CIC_SEEK_THR)) > > + return 0; > > +

[Devel] Re: IO scheduler based IO controller V10

2009-09-30 Thread Vivek Goyal
On Wed, Sep 30, 2009 at 05:43:19PM +0900, Ryo Tsuruta wrote: > Hi Vivek, > > Vivek Goyal wrote: > > I was thinking that elevator layer will do the merge of bios. So IO > > scheduler/elevator can time stamp the first bio in the request as it goes > > into the disk and

[Devel] Re: IO scheduler based IO controller V10

2009-09-29 Thread Vivek Goyal
On Tue, Sep 29, 2009 at 06:56:53PM +0900, Ryo Tsuruta wrote: > Hi Vivek and all, > > Vivek Goyal wrote: > > On Mon, Sep 28, 2009 at 05:37:28PM -0700, Nauman Rafique wrote: > > > > We are starting from a point where there is no cgroup based IO > > > scheduli

[Devel] Re: IO scheduler based IO controller V10

2009-09-29 Thread Vivek Goyal
On Tue, Sep 29, 2009 at 06:56:53PM +0900, Ryo Tsuruta wrote: > Hi Vivek and all, > > Vivek Goyal wrote: > > On Mon, Sep 28, 2009 at 05:37:28PM -0700, Nauman Rafique wrote: > > > > We are starting from a point where there is no cgroup based IO > > > scheduli

[Devel] Re: IO scheduler based IO controller V10

2009-09-28 Thread Vivek Goyal
On Mon, Sep 28, 2009 at 05:37:28PM -0700, Nauman Rafique wrote: > Hi Vivek, > Me, Divyesh, Fernando and Yoshikawa had a chance to have a chat with > Jens about IO controller during Linux Plumbers Conference '09. Jens > expressed his concerns about the size and complexity of the patches. I > believe

[Devel] Re: IO scheduler based IO controller V10

2009-09-28 Thread Vivek Goyal
On Mon, Sep 28, 2009 at 07:51:14PM +0200, Mike Galbraith wrote: > On Mon, 2009-09-28 at 17:35 +0200, Corrado Zoccolo wrote: > > > Great. > > Can you try the attached patch (on top of 2.6.31)? > > It implements the alternative approach we discussed privately in july, > > and it addresses the possib

[Devel] Re: IO scheduler based IO controller V10

2009-09-28 Thread Vivek Goyal
On Mon, Sep 28, 2009 at 06:04:08AM +0200, Mike Galbraith wrote: > On Sun, 2009-09-27 at 20:16 +0200, Mike Galbraith wrote: > > On Sun, 2009-09-27 at 18:42 +0200, Jens Axboe wrote: > > > I'll give it a shot first thing in the A.M. > > > > diff --git a/block/elevator.c b/block/elevator.c > > > inde

[Devel] Re: IO scheduler based IO controller V10

2009-09-28 Thread Vivek Goyal
On Mon, Sep 28, 2009 at 05:35:02PM +0200, Corrado Zoccolo wrote: > On Mon, Sep 28, 2009 at 4:56 PM, Vivek Goyal wrote: > > On Sun, Sep 27, 2009 at 07:00:08PM +0200, Corrado Zoccolo wrote: > >> Hi Vivek, > >> On Fri, Sep 25, 2009 at 10:26 PM, Vivek Goyal wrote: > &

[Devel] Re: IO scheduler based IO controller V10

2009-09-28 Thread Vivek Goyal
On Sun, Sep 27, 2009 at 07:00:08PM +0200, Corrado Zoccolo wrote: > Hi Vivek, > On Fri, Sep 25, 2009 at 10:26 PM, Vivek Goyal wrote: > > On Fri, Sep 25, 2009 at 04:20:14AM +0200, Ulrich Lukas wrote: > >> Vivek Goyal wrote: > >> > Notes: > >> > - Wit

[Devel] Re: IO scheduler based IO controller V10

2009-09-25 Thread Vivek Goyal
On Fri, Sep 25, 2009 at 04:20:14AM +0200, Ulrich Lukas wrote: > Vivek Goyal wrote: > > Notes: > > - With vanilla CFQ, random writers can overwhelm a random reader. > > Bring down its throughput and bump up latencies significantly. > > > IIRC, with vanilla CFQ, se

[Devel] Re: IO scheduler based IO controller V10

2009-09-25 Thread Vivek Goyal
On Fri, Sep 25, 2009 at 06:07:24PM +0900, Ryo Tsuruta wrote: > Hi Vivek, > > Vivek Goyal wrote: > > Higher level solutions are not keeping track of time slices. Time slices > > will > > be allocated by CFQ which does not have any idea about grouping. Higher > >

[Devel] Re: IO scheduler based IO controller V10

2009-09-24 Thread Vivek Goyal
On Thu, Sep 24, 2009 at 02:33:15PM -0700, Andrew Morton wrote: > On Thu, 24 Sep 2009 15:25:04 -0400 > Vivek Goyal wrote: > > > > > Hi All, > > > > Here is the V10 of the IO controller patches generated on top of 2.6.31. > > > > Thanks for the wri

[Devel] Re: IO scheduler based IO controller V10

2009-09-24 Thread Vivek Goyal
On Fri, Sep 25, 2009 at 10:09:52AM +0900, KAMEZAWA Hiroyuki wrote: > On Thu, 24 Sep 2009 14:33:15 -0700 > Andrew Morton wrote: > > > Test5 (Fairness for async writes, Buffered Write Vs Buffered Write) > > > === > > > Fairness for asyn

[Devel] [PATCH 28/28] io-controller: debug elevator fair queuing support

2009-09-24 Thread Vivek Goyal
o More debugging help to debug elevator fair queuing support. Enabled under CONFIG_DEBUG_ELV_FAIR_QUEUING. Currently it prints vdisktime related trace messages in blktrace. Signed-off-by: Vivek Goyal Acked-by: Rik van Riel --- block/Kconfig.iosched |9 + block/elevator-fq.c

[Devel] [PATCH 08/28] io-controller: Common hierarchical fair queuing code in elevaotor layer

2009-09-24 Thread Vivek Goyal
by: Paolo Valente Signed-off-by: Aristeu Rozanski Signed-off-by: Gui Jianfeng Signed-off-by: Vivek Goyal Acked-by: Rik van Riel --- block/cfq-iosched.c |2 + block/elevator-fq.c | 500 +-- block/elevator-fq.h | 35 block/elevator.c

[Devel] [PATCH 06/28] io-controller: Core scheduler changes to support hierarhical scheduling

2009-09-24 Thread Vivek Goyal
o This patch introduces core changes in fair queuing scheduler to support hierarhical/group scheduling. It is enabled by CONFIG_GROUP_IOSCHED. Signed-off-by: Fabio Checconi Signed-off-by: Paolo Valente Signed-off-by: Nauman Rafique Signed-off-by: Vivek Goyal Acked-by: Rik van Riel

[Devel] [PATCH 16/28] io-controller: Wait for requests to complete from last queue before new queue is scheduled

2009-09-24 Thread Vivek Goyal
write workloads. Signed-off-by: Vivek Goyal Acked-by: Rik van Riel --- block/cfq-iosched.c |1 + block/elevator-fq.c | 30 +- block/elevator-fq.h | 10 +- 3 files changed, 35 insertions(+), 6 deletions(-) diff --git a/block/cfq-iosched.c b/block/cfq-iosched.

[Devel] [PATCH 13/28] io-controller: Implement wait busy for io queues

2009-09-24 Thread Vivek Goyal
is disabled. Signed-off-by: Vivek Goyal --- block/elevator-fq.c | 56 ++- 1 files changed, 55 insertions(+), 1 deletions(-) diff --git a/block/elevator-fq.c b/block/elevator-fq.c index 5511256..b8862d3 100644 --- a/block/elevator-fq.c +++ b

[Devel] [PATCH 17/28] io-controller: Separate out queue and data

2009-09-24 Thread Vivek Goyal
can check for q->nr_sorted in elevator layer to see if ioscheduler queues are empty or not. Signed-off-by: Nauman Rafique Signed-off-by: Gui Jianfeng Signed-off-by: Vivek Goyal Acked-by: Rik van Riel --- block/as-iosched.c | 208 ++ bl

[Devel] [PATCH 26/28] io-controller: Per io group bdi congestion interface

2009-09-24 Thread Vivek Goyal
up the page will go into is congested or not. o Currently I have introduced the core functions and migrated most of the users. But there might be still some left. This is an ongoing TODO item. Signed-off-by: Vivek Goyal Acked-by: Rik van Riel --- block/blk-core.c| 26 block

[Devel] [PATCH 21/28] io-controller: deadline changes for hierarchical fair queuing

2009-09-24 Thread Vivek Goyal
This patch changes deadline to use queue scheduling code from elevator layer. One can go back to old deadline by selecting CONFIG_IOSCHED_DEADLINE_HIER. Signed-off-by: Nauman Rafique Signed-off-by: Vivek Goyal Acked-by: Rik van Riel --- block/Kconfig.iosched| 11 +++ block

[Devel] [PATCH 03/28] io-controller: Keep a cache of recently expired queues

2009-09-24 Thread Vivek Goyal
id of idle trees in V9 and now I am bringing it back. (Now I understand it better. :-)). Signed-off-by: Vivek Goyal Acked-by: Rik van Riel --- block/elevator-fq.c | 188 ++- block/elevator-fq.h |7 ++ 2 files changed, 179 insertions(+), 16 de

[Devel] [PATCH 20/28] io-controller: noop changes for hierarchical fair queuing

2009-09-24 Thread Vivek Goyal
This patch changes noop to use queue scheduling code from elevator layer. One can go back to old noop by deselecting CONFIG_IOSCHED_NOOP_HIER. Signed-off-by: Nauman Rafique Signed-off-by: Vivek Goyal Acked-by: Rik van Riel --- block/Kconfig.iosched | 11 +++ block/noop-iosched.c

[Devel] [PATCH 22/28] io-controller: anticipatory changes for hierarchical fair queuing

2009-09-24 Thread Vivek Goyal
ion if last request in the queue completed and associated io context is eligible to anticipate. Also AS lets elevatory layer know that it is anticipating (elv_ioq_wait_request()) This solves above mentioned issues. Signed-off-by: Nauman Rafique Signed-off-by: Vivek Goyal Acked-by: Rik va

[Devel] [PATCH 24/28] io-controller: map async requests to appropriate cgroup

2009-09-24 Thread Vivek Goyal
tracking patches to determine group bio belongs to and then map it to async queue of that group. Signed-off-by: Nauman Rafique Signed-off-by: Gui Jianfeng Signed-off-by: Vivek Goyal Acked-by: Rik van Riel --- block/Kconfig.iosched| 16 + block/as-iosched.c |2 +- block/blk

  1   2   3   4   5   6   >