[Devel] [PATCH -mm 4/6] memcg: interface to charge the right cgroup of asynchronous i/o activity

2008-10-07 Thread Andrea Righi
this functionality could/should be provided by bio-cgroup [1], instead of directly use the memcg page-tracking functionality. [1] http://people.valinux.co.jp/~ryov/bio-cgroup/ Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- include/linux/memcontrol.h |3 +++ mm/memcontrol.c

[Devel] Re: [RFC] [PATCH -mm 0/2] memcg: per cgroup dirty_ratio

2008-10-07 Thread Andrea Righi
Balbir Singh wrote: Michael Rubin wrote: On Fri, Sep 12, 2008 at 1:18 PM, Andrew Morton [EMAIL PROTECTED] wrote: One thing to think about please: Michael Rubin is hitting problems with the existing /proc/sys/vm/dirty-ratio. Its present granularity of 1% is just too coarse for really large

[Devel] Re: [PATCH -mm 2/6] introduce struct res_counter_ratelimit

2008-10-06 Thread Andrea Righi
Paul Menage wrote: Hi Andrea, The principle seems useful, but you seem to be duplicating a lot of the existing res_counter code. Agree. Could you not either: - include these two extra fields in res_counter? - include res_counter as the first field in a res_counter_ratelimit? The

[Devel] Re: [PATCH -mm 2/6] introduce struct res_counter_ratelimit

2008-10-06 Thread Andrea Righi
Balbir Singh wrote: Paul Menage wrote: On Mon, Oct 6, 2008 at 1:03 AM, Andrea Righi [EMAIL PROTECTED] wrote: Could you not either: - include these two extra fields in res_counter? - include res_counter as the first field in a res_counter_ratelimit? The second solution would save some space

[Devel] Re: dm-ioband + bio-cgroup benchmarks

2008-09-26 Thread Andrea Righi
Vivek Goyal wrote: [snip] Ok, I will give more details of the thought process. I was thinking of maintaing an rb-tree per request queue and not an rb-tree per cgroup. This tree can contain all the bios submitted to that request queue through __make_request(). Every node in the tree will

[Devel] Re: dm-ioband + bio-cgroup benchmarks

2008-09-26 Thread Andrea Righi
Andrea Righi wrote: Vivek Goyal wrote: [snip] Ok, I will give more details of the thought process. I was thinking of maintaing an rb-tree per request queue and not an rb-tree per cgroup. This tree can contain all the bios submitted to that request queue through __make_request(). Every node

[Devel] Re: dm-ioband + bio-cgroup benchmarks

2008-09-26 Thread Andrea Righi
Andrea Righi wrote: Andrea Righi wrote: Vivek Goyal wrote: [snip] Ok, I will give more details of the thought process. I was thinking of maintaing an rb-tree per request queue and not an rb-tree per cgroup. This tree can contain all the bios submitted to that request queue through

[Devel] Re: [RFC] [PATCH -mm 0/2] memcg: per cgroup dirty_ratio

2008-09-23 Thread Andrea Righi
Michael Rubin wrote: On Fri, Sep 12, 2008 at 1:18 PM, Andrew Morton [EMAIL PROTECTED] wrote: One thing to think about please: Michael Rubin is hitting problems with the existing /proc/sys/vm/dirty-ratio. Its present granularity of 1% is just too coarse for really large machines, and as

[Devel] Re: dm-ioband + bio-cgroup benchmarks

2008-09-19 Thread Andrea Righi
Vivek Goyal wrote: On Fri, Sep 19, 2008 at 08:20:31PM +0900, Hirokazu Takahashi wrote: To avoid creation of stacking another device (dm-ioband) on top of every device we want to subject to rules, I was thinking of maintaining an rb-tree per request queue. Requests will first go into this

[Devel] Re: [PATCH -mm 2/6] introduce struct res_counter_ratelimit

2008-09-19 Thread Andrea Righi
Randy Dunlap wrote: On Wed, 17 Sep 2008 13:05:24 +0200 Andrea Righi wrote: Introduce res_counter_ratelimit as a generic structure to implement throttling-based cgroup subsystems. [ Only the interfaces needed by the IO controller are implemented right now ] Signed-off-by: Andrea Righi

[Devel] Re: [RFC][PATCH -mm 0/5] cgroup: block device i/o controller (v9)

2008-09-18 Thread Andrea Righi
Hirokazu Takahashi wrote: Hi, Hi, TODO: * Try to push down the throttling and implement it directly in the I/O schedulers, using bio-cgroup (http://people.valinux.co.jp/~ryov/bio-cgroup/) to keep track of the right cgroup context. This approach could lead to more memory

[Devel] Re: [RFC][PATCH -mm 0/5] cgroup: block device i/o controller (v9)

2008-09-18 Thread Andrea Righi
Vivek Goyal wrote: On Wed, Sep 17, 2008 at 10:47:54AM +0200, Andrea Righi wrote: Hirokazu Takahashi wrote: Hi, TODO: * Try to push down the throttling and implement it directly in the I/O schedulers, using bio-cgroup (http://people.valinux.co.jp/~ryov/bio-cgroup/) to keep track

[Devel] Re: [RFC][PATCH -mm 1/5] i/o controller documentation

2008-09-18 Thread Andrea Righi
Vivek Goyal wrote: On Wed, Aug 27, 2008 at 06:07:33PM +0200, Andrea Righi wrote: Documentation of the block device I/O controller: description, usage, advantages and design. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- Documentation/controllers/io-throttle.txt | 377

[Devel] Re: dm-ioband + bio-cgroup benchmarks

2008-09-18 Thread Andrea Righi
Vivek Goyal wrote: On Thu, Sep 18, 2008 at 04:37:41PM +0200, Andrea Righi wrote: Vivek Goyal wrote: On Thu, Sep 18, 2008 at 09:04:18PM +0900, Ryo Tsuruta wrote: Hi All, I have got excellent results of dm-ioband, that controls the disk I/O bandwidth even when it accepts delayed write

[Devel] Re: [RFC][PATCH -mm 1/5] i/o controller documentation

2008-09-18 Thread Andrea Righi
Vivek Goyal wrote: On Thu, Sep 18, 2008 at 05:03:59PM +0200, Andrea Righi wrote: Vivek Goyal wrote: On Wed, Aug 27, 2008 at 06:07:33PM +0200, Andrea Righi wrote: Documentation of the block device I/O controller: description, usage, advantages and design. Signed-off-by: Andrea Righi [EMAIL

[Devel] [PATCH -mm] memrlimit: fix task_lock() recursive locking

2008-09-18 Thread Andrea Righi
] do_exit+0x167/0x930 [ 5346.421610] [8047604a] _spin_unlock_irq+0x2a/0x50 [ 5346.421618] [8023fd46] do_group_exit+0x36/0xa0 [ 5346.421626] [8020b7cb] system_call_fastpath+0x16/0x1b Here is a possible fix, introducing get_task_mm_task_locked(). Signed-off-by: Andrea Righi

[Devel] Re: dm-ioband + bio-cgroup benchmarks

2008-09-18 Thread Andrea Righi
Vivek Goyal wrote: On Thu, Sep 18, 2008 at 05:18:50PM +0200, Andrea Righi wrote: Vivek Goyal wrote: On Thu, Sep 18, 2008 at 04:37:41PM +0200, Andrea Righi wrote: Vivek Goyal wrote: On Thu, Sep 18, 2008 at 09:04:18PM +0900, Ryo Tsuruta wrote: Hi All, I have got excellent results of dm

[Devel] Re: [PATCH -mm] memrlimit: fix task_lock() recursive locking

2008-09-18 Thread Andrea Righi
-by: Andrea Righi [EMAIL PROTECTED] --- kernel/cgroup.c |3 ++- mm/memrlimitcgroup.c |6 +++--- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/kernel/cgroup.c b/kernel/cgroup.c index 678a680..03cc925 100644 --- a/kernel/cgroup.c +++ b/kernel/cgroup.c @@ -2757,7 +2757,8

[Devel] [PATCH -mm] memrlimit: fix task_lock() recursive locking (v2)

2008-09-18 Thread Andrea Righi
incrementing mm_users. So, just use p-mm directly and check that we've not picked a kernel thread. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- kernel/cgroup.c |3 ++- mm/memrlimitcgroup.c | 10 -- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/kernel

[Devel] Re: [RFC][PATCH -mm 0/5] cgroup: block device i/o controller (v9)

2008-09-17 Thread Andrea Righi
Hirokazu Takahashi wrote: Hi, TODO: * Try to push down the throttling and implement it directly in the I/O schedulers, using bio-cgroup (http://people.valinux.co.jp/~ryov/bio-cgroup/) to keep track of the right cgroup context. This approach could lead to more memory consumption

[Devel] Re: [RFC][PATCH -mm 0/5] cgroup: block device i/o controller (v9)

2008-09-17 Thread Andrea Righi
Takuya Yoshikawa wrote: Hi, Andrea Righi wrote: TODO: * Try to push down the throttling and implement it directly in the I/O schedulers, using bio-cgroup (http://people.valinux.co.jp/~ryov/bio-cgroup/) to keep track of the right cgroup context. This approach could lead to more

[Devel] Re: [RFC][PATCH -mm 0/5] cgroup: block device i/o controller (v9)

2008-09-17 Thread Andrea Righi
Takuya Yoshikawa wrote: Hi, Andrea Righi wrote: TODO: * Try to push down the throttling and implement it directly in the I/O schedulers, using bio-cgroup (http://people.valinux.co.jp/~ryov/bio-cgroup/) to keep track of the right cgroup context. This approach could lead to more

[Devel] [PATCH -mm 4/6] memcg: interface to charge the right cgroup of asynchronous i/o activity

2008-09-17 Thread Andrea Righi
Implement get_cgroup_from_page() / put_cgroup_from_page() in the cgroup memory controller used to retrieve the owner of a page during writes in submit_bio() processed asynchronously by kernel threads (i.e. pdflush). Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- include/linux/memcontrol.h

[Devel] [PATCH -mm 2/6] introduce struct res_counter_ratelimit

2008-09-17 Thread Andrea Righi
Introduce res_counter_ratelimit as a generic structure to implement throttling-based cgroup subsystems. [ Only the interfaces needed by the IO controller are implemented right now ] Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- include/linux/res_counter.h | 70

[Devel] [PATCH -mm 3/6] i/o controller infrastructure

2008-09-17 Thread Andrea Righi
This is the core of the cgroup-io-throttle kernel infrastructure. It creates the basic interfaces to cgroups and implements the I/O measurement and throttling functionality. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/Makefile |2 + block/blk-io-throttle.c

[Devel] [PATCH -mm 1/6] i/o controller documentation

2008-09-17 Thread Andrea Righi
Documentation of the block device I/O controller: description, usage, advantages and design. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- Documentation/controllers/io-throttle.txt | 395 + 1 files changed, 395 insertions(+), 0 deletions(-) create mode 100644

[Devel] [PATCH -mm 5/6] i/o controller instrumentation: accounting and throttling

2008-09-17 Thread Andrea Righi
Apply the cgroup-io-throttle controller to the opportune kernel functions. Both accounting and throttling functionalities are performed by cgroup_io_throttle(). Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/blk-core.c |4 fs/aio.c | 12 fs

[Devel] [PATCH -mm 0/6] cgroup: block device i/o controller (v10)

2008-09-17 Thread Andrea Righi
The objective of the i/o controller is to improve i/o performance predictability of different cgroups sharing the same block devices. Respect to other priority/weight-based solutions the approach used by this controller is to explicitly choke applications' requests that directly (or indirectly)

[Devel] [PATCH -mm 6/6] export per-task i/o throttling statistics to userspace

2008-09-17 Thread Andrea Righi
Export the throttling statistics collected for each task through /proc/PID/io-throttle-stat: - total number of delayed I/O request - total amount of time in clock ticks spent to wait for the delayed requests Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- fs/proc/base.c | 15

[Devel] [RFC] [PATCH -mm 0/2] memcg: per cgroup dirty_ratio

2008-09-12 Thread Andrea Righi
The goal of the patch is to control how much dirty file pages a cgroup can have at any given time (see also [1]). Dirty file and writeback pages are accounted for each cgroup using the memory controller statistics. Moreover, the dirty_ratio parameter is added to the memory controller. It

[Devel] [RFC] [PATCH -mm 2/2] use collected memory cgroup statistics for page writeback

2008-09-12 Thread Andrea Righi
according to the cgroup statistics. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- fs/super.c|4 +- fs/sync.c |7 ++- include/linux/writeback.h | 11 +++-- kernel/trace/trace.c |2 +- mm/backing-dev.c |3 +- mm/page-writeback.c

[Devel] [RFC] [PATCH -mm 1/2] memcg dirty_ratio and additional page statistics

2008-09-12 Thread Andrea Righi
changing too much stuff. ] Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- fs/buffer.c|2 + fs/nfs/write.c |4 + fs/nilfs2/page.h |9 ++- fs/reiser4/as_ops.c|5 +- fs/reiser4/page_cache.c|5 +- include/linux/memcontrol.h

[Devel] Re: [RFC] [PATCH -mm 0/2] memcg: per cgroup dirty_ratio

2008-09-12 Thread Andrea Righi
Andrew Morton wrote: On Fri, 12 Sep 2008 17:09:50 +0200 Andrea Righi [EMAIL PROTECTED] wrote: The goal of the patch is to control how much dirty file pages a cgroup can have at any given time (see also [1]). Dirty file and writeback pages are accounted for each cgroup using the memory

[Devel] Re: [RFC] [PATCH -mm] cgroup: limit the amount of dirty file pages

2008-09-10 Thread Andrea Righi
Dave Hansen wrote: On Tue, 2008-09-09 at 17:38 +0200, Andrea Righi wrote: It allows to control how much dirty file pages a cgroup can have at any given time. This feature is supposed to be strictly connected to a generic cgroup IO controller (see below). So, this functions similarly to our

[Devel] [RFC] [PATCH -mm] cgroup: limit the amount of dirty file pages

2008-09-09 Thread Andrea Righi
to the allowed dirty limit. Suggestions are welcome. ] Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- fs/buffer.c|1 + fs/reiser4/as_ops.c|4 ++- fs/reiser4/page_cache.c|4 ++- include/linux/memcontrol.h | 12 ++ mm/filemap.c |1 + mm

[Devel] Re: [RFC] [PATCH -mm] cgroup: limit the amount of dirty file pages

2008-09-09 Thread Andrea Righi
[EMAIL PROTECTED] wrote: - Original Message - This is a totally experimental patch against 2.6.27-rc5-mm1. It allows to control how much dirty file pages a cgroup can have at any given time. This feature is supposed to be strictly connected to a generic cgroup IO controller (see

[Devel] Re: [RFC][PATCH -mm 0/5] cgroup: block device i/o controller (v9)

2008-09-02 Thread Andrea Righi
Vivek Goyal wrote: On Wed, Aug 27, 2008 at 06:07:32PM +0200, Andrea Righi wrote: The objective of the i/o controller is to improve i/o performance predictability of different cgroups sharing the same block devices. Respect to other priority/weight-based solutions the approach used

[Devel] [RFC][PATCH -mm 1/5] i/o controller documentation

2008-08-27 Thread Andrea Righi
Documentation of the block device I/O controller: description, usage, advantages and design. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- Documentation/controllers/io-throttle.txt | 377 + 1 files changed, 377 insertions(+), 0 deletions(-) create mode 100644

[Devel] [RFC][PATCH -mm 3/5] i/o controller infrastructure

2008-08-27 Thread Andrea Righi
This is the core io-throttle kernel infrastructure. It creates the basic interfaces to cgroups and implements the I/O measurement and throttling functions. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/Makefile |2 + block/blk-io-throttle.c | 672

[Devel] [RFC][PATCH -mm 0/5] cgroup: block device i/o controller (v9)

2008-08-27 Thread Andrea Righi
The objective of the i/o controller is to improve i/o performance predictability of different cgroups sharing the same block devices. Respect to other priority/weight-based solutions the approach used by this controller is to explicitly choke applications' requests that directly (or indirectly)

[Devel] [RFC][PATCH -mm 5/5] export per-task i/o throttling statistics to userspace

2008-08-27 Thread Andrea Righi
Export the throttling statistics collected for each task through /proc/PID/io-throttle-stat: - total number of delayed I/O request - total amount of time in jiffies spent to wait for the delayed requests Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- fs/proc/base.c | 15 +++ 1

[Devel] [RFC][PATCH -mm 4/5] i/o controller instrumentation: accounting and throttling

2008-08-27 Thread Andrea Righi
Apply the io-throttle controller to the opportune kernel functions. Both accounting and throttling functionalities are performed by cgroup_io_throttle(). Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/blk-core.c |2 ++ fs/aio.c | 14 ++ include/linux

[Devel] [RFC][PATCH -mm 2/5] introduce struct res_counter_ratelimit

2008-08-27 Thread Andrea Righi
Introduce res_counter_ratelimit as a generic structure to implement throttling-based cgroup subsystems. [ Only the interfaces needed by the IO controller are implemented right now ] Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- include/linux/res_counter.h | 70

[Devel] Re: RFC: I/O bandwidth controller

2008-08-13 Thread Andrea Righi
Dong-Jae Kang wrote: Hi, 2008/8/13 Andrea Righi [EMAIL PROTECTED]: Fernando Luis Vázquez Cao wrote: On Tue, 2008-08-12 at 22:29 +0900, Andrea Righi wrote: Andrea Righi wrote: Hirokazu Takahashi wrote: 3. 4. 5. - I/O bandwidth shaping General design aspects The implementation of an I

[Devel] Re: RFC: I/O bandwidth controller

2008-08-12 Thread Andrea Righi
Hirokazu Takahashi wrote: 3. 4. 5. - I/O bandwidth shaping General design aspects The implementation of an I/O scheduling algorithm is to a certain extent influenced by what we are trying to achieve in terms of I/O bandwidth shaping, but, as discussed below, the required accuracy can

[Devel] Re: RFC: I/O bandwidth controller

2008-08-12 Thread Andrea Righi
Fernando Luis Vázquez Cao wrote: On Tue, 2008-08-12 at 22:29 +0900, Andrea Righi wrote: Andrea Righi wrote: Hirokazu Takahashi wrote: 3. 4. 5. - I/O bandwidth shaping General design aspects The implementation of an I/O scheduling algorithm is to a certain extent influenced by what we

[Devel] Re: RFC: I/O bandwidth controller

2008-08-12 Thread Andrea Righi
[EMAIL PROTECTED] wrote: Fernando Luis Vázquez Cao wrote: BTW as I said in a previous email, an interesting path to be explored IMHO could be to think in terms of IO time. So, look at the time an IO request is issued to the drive, look at the time the request is served, evaluate the

[Devel] Re: RFC: I/O bandwidth controller (was Re: Too many I/O controller patches)

2008-08-11 Thread Andrea Righi
Fernando Luis Vázquez Cao wrote: This seems to be the easiest part, but the current cgroups infrastructure has some limitations when it comes to dealing with block devices: impossibility of creating/removing certain control structures dynamically and hardcoding of subsystems (i.e. resource

[Devel] Re: RFC: I/O bandwidth controller (was Re: Too many I/O controller patches)

2008-08-07 Thread Andrea Righi
Fernando Luis Vázquez Cao wrote: This RFC ended up being a bit longer than I had originally intended, but hopefully it will serve as the start of a fruitful discussion. Thanks for posting this detailed RFC! A few comments below. As you pointed out, it seems that there is not much consensus

[Devel] Re: [PATCH 4/7] bio-cgroup: Split the cgroup memory subsystem into two parts

2008-08-05 Thread Andrea Righi
Ryo Tsuruta wrote: +static int mem_cgroup_charge_common(struct page *page, struct mm_struct *mm, + gfp_t gfp_mask, enum charge_type ctype, + struct mem_cgroup *memcg) +{ + struct page_cgroup *pc; +#ifdef CONFIG_CGROUP_MEM_RES_CTLR

[Devel] Re: [PATCH -mm 2/3] i/o controller infrastructure

2008-08-04 Thread Andrea Righi
Li Zefan wrote: Andrea Righi wrote: This is the core io-throttle kernel infrastructure. It creates the basic interfaces to cgroups and implements the I/O measurement and throttling functions. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/Makefile|2

[Devel] Re: Too many I/O controller patches

2008-08-04 Thread Andrea Righi
Dave Hansen wrote: On Mon, 2008-08-04 at 17:51 +0900, Ryo Tsuruta wrote: This series of patches of dm-ioband now includes The bio tracking mechanism, which has been posted individually to this mailing list. This makes it easy for anybody to control the I/O bandwidth even when the I/O is one

[Devel] Re: Too many I/O controller patches

2008-08-04 Thread Andrea Righi
Balbir Singh wrote: Dave Hansen wrote: On Mon, 2008-08-04 at 17:51 +0900, Ryo Tsuruta wrote: This series of patches of dm-ioband now includes The bio tracking mechanism, which has been posted individually to this mailing list. This makes it easy for anybody to control the I/O bandwidth even

[Devel] Re: Too many I/O controller patches

2008-08-04 Thread Andrea Righi
Dave Hansen wrote: On Mon, 2008-08-04 at 20:22 +0200, Andrea Righi wrote: But I'm not yet convinced that limiting the IO writes at the device mapper layer is the best solution. IMHO it would be better to throttle applications' writes when they're dirtying pages in the page cache (the io

[Devel] [PATCH -mm 0/3] cgroup: block device i/o controller (v8)

2008-08-03 Thread Andrea Righi
The objective of the i/o controller is to improve i/o performance predictability of different cgroups sharing the same block devices. Respect to other priority/weight-based solutions the approach used by this controller is to explicitly choke applications' requests that directly (or indirectly)

[Devel] [PATCH -mm 2/3] i/o controller infrastructure

2008-08-03 Thread Andrea Righi
This is the core io-throttle kernel infrastructure. It creates the basic interfaces to cgroups and implements the I/O measurement and throttling functions. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/Makefile|2 ++ include/linux/cgroup_subsys.h |6 ++ init

[Devel] [PATCH -mm 3/3] i/o controller instrumentation: accounting and throttling

2008-08-03 Thread Andrea Righi
Apply the io-throttle controller to the opportune kernel functions. Both accounting and throttling functionalities are performed by cgroup_io_throttle(). Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/blk-core.c |9 + fs/aio.c | 31

[Devel] [PATCH -mm 1/3] i/o controller documentation

2008-08-03 Thread Andrea Righi
Documentation of the block device I/O controller: description, usage, advantages and design. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- Documentation/controllers/io-throttle.txt | 312 + block/blk-io-throttle.c | 719 + include

[Devel] [PATCH -mm 1/3] i/o bandwidth controller documentation

2008-07-22 Thread Andrea Righi
Documentation of the block device I/O bandwidth controller: description, usage, advantages and design. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- Documentation/controllers/io-throttle.txt | 328 + 1 files changed, 328 insertions(+), 0 deletions(-) create mode

[Devel] [PATCH -mm 2/3] i/o bandwidth controller infrastructure

2008-07-22 Thread Andrea Righi
This is the core io-throttle kernel infrastructure. It creates the basic interfaces to cgroups and implements the I/O measurement and throttling functions. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/Makefile |2 + block/blk-io-throttle.c | 668

[Devel] [PATCH -mm 3/3] i/o accounting and control

2008-07-22 Thread Andrea Righi
Apply the io-throttle controller to the opportune kernel functions. Both accounting and throttling functionalities are performed by cgroup_io_throttle(). Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/blk-core.c |9 + fs/aio.c | 31

[Devel] [PATCH -mm 0/3] cgroup: block device i/o bandwidth controller (v7)

2008-07-22 Thread Andrea Righi
The objective of the i/o bandwidth controller is to improve i/o performance predictability of different cgroups sharing the same block devices. Respect to other priority/weight-based solutions the approach used by this controller is to explicitly choke applications' requests that directly (or

[Devel] [PATCH -mm 0/3] cgroup: block device i/o bandwidth controller (v6)

2008-07-15 Thread Andrea Righi
The objective of the i/o bandwidth controller is to improve i/o performance predictability of different cgroups sharing the same block devices. Respect to other priority/weight-based solutions the approach used by this controller is to explicitly choke applications' requests that directly (or

[Devel] Re: [PATCH -mm 2/3] i/o bandwidth controller infrastructure

2008-07-14 Thread Andrea Righi
Li Zefan wrote: +/* The various types of throttling algorithms */ +enum iothrottle_strategy { +IOTHROTTLE_LEAKY_BUCKET, It's better to explicitly assigned 0 to IOTHROTTLE_LEAKY_BUCKET. OK. +IOTHROTTLE_TOKEN_BUCKET, +}; +static int iothrottle_parse_args(char *buf, size_t

[Devel] [PATCH -mm 0/3] cgroup: block device i/o bandwidth controller (v5)

2008-07-12 Thread Andrea Righi
The objective of the i/o bandwidth controller is to improve i/o performance predictability of different cgroups sharing the same block devices. Respect to other priority/weight-based solutions the approach used by this controller is to explicitly choke applications' requests that directly (or

[Devel] [PATCH -mm 1/3] i/o bandwidth controller documentation

2008-07-12 Thread Andrea Righi
Documentation of the block device I/O bandwidth controller: description, usage, advantages and design. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- Documentation/controllers/io-throttle.txt | 282 + 1 files changed, 282 insertions(+), 0 deletions(-) create mode

[Devel] [PATCH -mm 2/3] i/o bandwidth controller infrastructure

2008-07-12 Thread Andrea Righi
This is the core io-throttle kernel infrastructure. It creates the basic interfaces to cgroups and implements the I/O measurement and throttling functions. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/Makefile |2 + block/blk-io-throttle.c | 549

[Devel] [PATCH -mm 3/3] i/o accounting and control

2008-07-12 Thread Andrea Righi
Apply the io-throttle controller to the opportune kernel functions. Both accounting and throttling functionalities are performed by cgroup_io_throttle(). Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/blk-core.c |2 ++ fs/aio.c | 31

[Devel] Re: [PATCH] ltp controllers: block device i/o bandwidth controller testcase

2008-07-07 Thread Andrea Righi
at 12:05 +0200, Andrea Righi wrote: Add the block device I/O bandwidth controller (io-throttle) testcase. See testcase documentation for design and implementation details. See also: http://lkml.org/lkml/2008/7/4/143 Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- ... diff --exclude CVS

[Devel] Re: [PATCH 2/3] i/o bandwidth controller infrastructure

2008-07-05 Thread Andrea Righi
Li Zefan wrote: +/** + * struct iothrottle_node - throttling rule of a single block device + * @node: list of per block device throttling rules + * @dev: block device number, used as key in the list + * @iorate: max i/o bandwidth (in bytes/s) + * @strategy: throttling strategy (0 = leaky

[Devel] [PATCH 0/3] cgroup: block device i/o bandwidth controller (v4)

2008-07-04 Thread Andrea Righi
The objective of the i/o bandwidth controller is to improve i/o performance predictability of different cgroups sharing the same block devices. Respect to other priority/weight-based solutions the approach used by this controller is to explicitly choke applications' requests that directly (or

[Devel] [PATCH 2/3] i/o bandwidth controller infrastructure

2008-07-04 Thread Andrea Righi
This is the core io-throttle kernel infrastructure. It creates the basic interfaces to cgroups and implements the I/O measurement and throttling functions. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/Makefile |2 + block/blk-io-throttle.c | 529

[Devel] [PATCH 1/3] i/o bandwidth controller documentation

2008-07-04 Thread Andrea Righi
Documentation of the block device I/O bandwidth controller: description, usage, advantages and design. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- Documentation/controllers/io-throttle.txt | 265 + 1 files changed, 265 insertions(+), 0 deletions(-) create mode

[Devel] [PATCH 3/3] i/o accounting and control

2008-07-04 Thread Andrea Righi
Apply the io-throttle controller to the opportune kernel functions. Both accounting and throttling functionalities are performed by cgroup_io_throttle(). Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/blk-core.c|2 ++ fs/buffer.c | 10 ++ fs/direct-io.c

[Devel] Re: [PATCH -mmotm] res_counter: fix building failure

2008-07-03 Thread Andrea Righi
Li Zefan wrote: Paul Menage wrote: On Tue, Jul 1, 2008 at 11:16 PM, Li Zefan [EMAIL PROTECTED] wrote: kernel/res_counter.c: In function 'res_counter_memparse_write_strategy': kernel/res_counter.c:115: error: implicit declaration of function 'PAGE_ALIGN' Signed-off-by: Li Zefan [EMAIL

[Devel] Re: Attaching PID 0 to a cgroup

2008-07-01 Thread Andrea Righi
Li Zefan wrote: CC: Paul Jackson [EMAIL PROTECTED] Dhaval Giani wrote: [put in the wrong alias for containers list correcting it.] On Tue, Jul 01, 2008 at 03:15:45PM +0530, Dhaval Giani wrote: Hi Paul, Attaching PID 0 to a cgroup caused the current task to be attached to the cgroup.

[Devel] Re: [PATCH 2/3] i/o bandwidth controller infrastructure

2008-06-30 Thread Andrea Righi
Andrea Righi wrote: Andrew Morton wrote: On Fri, 27 Jun 2008 00:36:46 +0200 Andrea Righi [EMAIL PROTECTED] wrote: Does all this code treat /dev/sda1 as a separate device from /dev/sda2? If so, that would be broken. Yes, all the partitions are treated as separate devices with (potentially

[Devel] Re: [PATCH 2/3] i/o bandwidth controller infrastructure

2008-06-27 Thread Andrea Righi
Andrew Morton wrote: On Fri, 27 Jun 2008 00:36:46 +0200 Andrea Righi [EMAIL PROTECTED] wrote: Does all this code treat /dev/sda1 as a separate device from /dev/sda2? If so, that would be broken. Yes, all the partitions are treated as separate devices with (potentially) different limiting

[Devel] Re: [PATCH 3/3] i/o accounting and control

2008-06-26 Thread Andrea Righi
Andrew Morton wrote: On Fri, 20 Jun 2008 12:05:35 +0200 Andrea Righi [EMAIL PROTECTED] wrote: Apply the io-throttle controller to the opportune kernel functions. Both accounting and throttling functionalities are performed by cgroup_io_throttle(). Signed-off-by: Andrea Righi [EMAIL

[Devel] Re: [PATCH 2/3] i/o bandwidth controller infrastructure

2008-06-26 Thread Andrea Righi
Andrew, thanks for your time and the detailed revision. I'll try to fix everything you reported and better document the code according to your suggestions. I'll re-submit a new patchset version ASAP. A few comments below. Andrew Morton wrote: [snip] +static ssize_t iothrottle_read(struct

[Devel] Re: [PATCH 2/3] i/o bandwidth controller infrastructure

2008-06-22 Thread Andrea Righi
do more accurate tests ASAP. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/Makefile |2 + block/blk-io-throttle.c | 490 +++ include/linux/blk-io-throttle.h | 12 + include/linux/cgroup_subsys.h |6 + init/Kconfig

[Devel] Re: [RFC][v2][patch 0/12][CFQ-cgroup]Yet another I/O bandwidth controlling subsystem for CGroups based on CFQ

2008-06-22 Thread Andrea Righi
Satoshi UCHIDA wrote: Hi, Andrea. Thanks for bug reports. I fix this problem. This problem causes by miss of trace for children groups. Please adopt and test this patch. OK, I can confirm the fix resolves the problem. Tested-by: Andrea Righi [EMAIL PROTECTED] If OK, this amendment

[Devel] Re: [PATCH 1/3] i/o bandwidth controller documentation

2008-06-21 Thread Andrea Righi
Thanks Randy, I've applied all your fixes to my local documentation, next patchset version will include them. A few small comments below. Randy Dunlap wrote: +* Run a benchmark doing I/O on /dev/sda1 and /dev/sda5; I/O limits and usage + defined for cgroup foo can be shown as following: + #

[Devel] Re: [PATCH 0/3] cgroup: block device i/o bandwidth controller (v3)

2008-06-21 Thread Andrea Righi
Subrata Modak wrote: Dear Andrea, We have been tracking Controllers developement for quite some time, and in order to make them work better, we have followed test-driven development for both CPU and Memory Controllers. We already have much of the CPU controllers and few Memory

[Devel] [PATCH 3/3] i/o accounting and control

2008-06-20 Thread Andrea Righi
Apply the io-throttle controller to the opportune kernel functions. Both accounting and throttling functionalities are performed by cgroup_io_throttle(). Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/blk-core.c|2 ++ fs/buffer.c | 10 ++ fs/direct-io.c

[Devel] [PATCH 0/3] cgroup: block device i/o bandwidth controller (v3)

2008-06-20 Thread Andrea Righi
The goal of the i/o bandwidth controller is to improve i/o performance predictability and provide better QoS for different cgroups sharing the same block devices. Respect to other priority/weight-based solutions the approach used by this controller is to explicitly choke applications' requests

[Devel] [PATCH 2/3] i/o bandwidth controller infrastructure

2008-06-20 Thread Andrea Righi
This is the core io-throttle kernel infrastructure. It creates the basic interfaces to cgroups and implements the I/O measurement and throttling functions. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- block/Makefile |2 + block/blk-io-throttle.c | 393

[Devel] [PATCH 1/3] i/o bandwidth controller documentation

2008-06-20 Thread Andrea Righi
Documentation of the block device I/O bandwidth controller: description, usage, advantages and design. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- Documentation/controllers/io-throttle.txt | 163 + 1 files changed, 163 insertions(+), 0 deletions(-) create mode

[Devel] Re: [RFC][v2][patch 0/12][CFQ-cgroup]Yet another I/O bandwidth controlling subsystem for CGroups based on CFQ

2008-06-18 Thread Andrea Righi
Satoshi UCHIDA wrote: Hi, Andrea. Thanks for bug reports. I fix this problem. This problem causes by miss of trace for children groups. Please adopt and test this patch. Thanks Satoshi, I'll test the fix in this weekend and I'll let you know. -Andrea If OK, this amendment is adopted

[Devel] Re: [RFC PATCH 5/5] refresh VM committed space after a task migration

2008-06-11 Thread Andrea Righi
Dave Hansen wrote: On Tue, 2008-06-10 at 01:33 +0200, Andrea Righi wrote: + preempt_disable(); + committed = atomic_long_read(p-vm_committed_space); + atomic_long_sub(committed, old_mem-vmacct.vm_committed_space); + atomic_long_add(committed, mem

[Devel] Re: [RFC PATCH 3/5] mgmcg VM overcommit interface

2008-06-11 Thread Andrea Righi
Dave Hansen wrote: On Tue, 2008-06-10 at 01:33 +0200, Andrea Righi wrote: + cgroup_unlock(); + + count = sprintf(page, CommitLimit: %8lu kB\n + Committed_AS: %8lu kB\n, + K(allowed), K(committed)); + ret

[Devel] Re: [RFC PATCH 0/5] memcg: VM overcommit accounting and handling

2008-06-10 Thread Andrea Righi
Pavel Emelyanov wrote: Balbir Singh wrote: KAMEZAWA Hiroyuki wrote: On Tue, 10 Jun 2008 01:32:58 +0200 Andrea Righi [EMAIL PROTECTED] wrote: Provide distinct cgroup VM overcommit accounting and handling using the memory resource controller. Could you explain the benefits of this even

[Devel] [RFC PATCH 5/5] refresh VM committed space after a task migration

2008-06-09 Thread Andrea Righi
Update VM committed space statistics when a task is migrated from a cgroup to another. To implement this feature we must keep track of the space committed by each task (that is directly accounted in the task_struct). Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- include/linux/sched.h |3

[Devel] [RFC PATCH 3/5] mgmcg VM overcommit interface

2008-06-09 Thread Andrea Righi
-off-by: Andrea Righi [EMAIL PROTECTED] --- mm/memcontrol.c | 116 ++- 1 files changed, 115 insertions(+), 1 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4100e24..e3e34e9 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c

[Devel] [RFC PATCH 0/5] memcg: VM overcommit accounting and handling

2008-06-09 Thread Andrea Righi
Provide distinct cgroup VM overcommit accounting and handling using the memory resource controller. Patchset against latest Linus git tree. This patchset allows to set different per-cgroup overcommit rules and, according to them, it's possible to return a memory allocation failure (ENOMEM) to

[Devel] [RFC PATCH 2/5] overcommit accounting and handling functions

2008-06-09 Thread Andrea Righi
-by: Andrea Righi [EMAIL PROTECTED] --- include/linux/mman.h | 148 -- mm/memcontrol.c | 139 ++- mm/mmap.c| 85 - mm/nommu.c | 84

[Devel] [RFC PATCH 1/5] memcg VM overcommit documentation

2008-06-09 Thread Andrea Righi
Documentation of the VM overcommit memory controller included in the generic memory controller documentation: basic description and usage. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- Documentation/controllers/memory.txt | 29 + 1 files changed, 29 insertions

[Devel] [RFC PATCH 4/5] use the memcg VM overcommit prototypes

2008-06-09 Thread Andrea Righi
Apply the new memory controller VM prototypes to the opportune kernel routines. Signed-off-by: Andrea Righi [EMAIL PROTECTED] --- ipc/shm.c |6 -- kernel/fork.c |2 +- mm/mmap.c | 10 ++ mm/mprotect.c |2 +- mm/mremap.c |4 ++-- mm/shmem.c| 49

[Devel] Re: [PATCH 0/3] cgroup: block device i/o bandwidth controller

2008-06-06 Thread Andrea Righi
Andrea Righi wrote: So, my approach has the disadvantage or the advantage (depending on the context and the requirements) to explicitly choke applications' requests. Other solutions that operates in the subsystem used to dispatch i/o requests are probably better to maximize overall

[Devel] Re: [PATCH 0/3] cgroup: block device i/o bandwidth controller

2008-05-26 Thread Andrea Righi
Balbir Singh wrote: Andrea Righi wrote: I'm resending an updated version of the cgroup I/O bandwidth controller that I wrote some months ago (http://lwn.net/Articles/265944), since someone asked me recently. The goal of this patchset is to implement a block device I/O bandwidth controller

<    1   2   3   4   >