[Devel] Re: [PATCH 3/9] bio-cgroup controller

2009-04-17 Thread Ryo Tsuruta
Hi, From: KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com Date: Fri, 17 Apr 2009 11:24:33 +0900 On Fri, 17 Apr 2009 10:49:43 +0900 Takuya Yoshikawa yoshikawa.tak...@oss.ntt.co.jp wrote: Hi, I have a few question. - I have not yet fully understood how your controller are using

[Devel] Re: [PATCH 1/9] io-throttle documentation

2009-04-17 Thread KAMEZAWA Hiroyuki
On Fri, 17 Apr 2009 15:34:53 +0800 Gui Jianfeng guijianf...@cn.fujitsu.com wrote: KAMEZAWA Hiroyuki wrote: On Tue, 14 Apr 2009 22:21:12 +0200 Andrea Righi righi.and...@gmail.com wrote: +Example: +* Create an association between an io-throttle group and a bio-cgroup group + with

[Devel] Re: [PATCH 3/9] bio-cgroup controller

2009-04-17 Thread Ryo Tsuruta
Hi, From: Andrew Morton a...@linux-foundation.org Subject: Re: [PATCH 3/9] bio-cgroup controller Date: Thu, 16 Apr 2009 21:15:14 -0700 On Fri, 17 Apr 2009 10:44:32 +0900 (JST) Ryo Tsuruta r...@valinux.co.jp wrote: Hmm, how about iotrack-cgroup ? Well. blockio_cgroup has the

[Devel] Re: [PATCH 3/9] bio-cgroup controller

2009-04-17 Thread Ryo Tsuruta
Hi, From: Takuya Yoshikawa yoshikawa.tak...@oss.ntt.co.jp Subject: Re: [PATCH 3/9] bio-cgroup controller Date: Fri, 17 Apr 2009 10:49:43 +0900 +config CGROUP_BIO + bool Block I/O cgroup subsystem + depends on CGROUPS BLOCK + select MM_OWNER + help + Provides a Resource

[Devel] Re: [PATCH 3/9] bio-cgroup controller

2009-04-17 Thread KAMEZAWA Hiroyuki
On Fri, 17 Apr 2009 16:22:01 +0900 (JST) Ryo Tsuruta r...@valinux.co.jp wrote: In the case where the bio-cgroup data is allocated dynamically, - Sometimes quite a large amount of memory get marked dirty. In this case it requires more kernel memory than that of the current

[Devel] Re: C/R without leaks (was: Re: Creating tasks on restart: userspace vs kernel)

2009-04-17 Thread Greg Kurz
On Thu, 2009-04-16 at 20:12 +0400, Alexey Dobriyan wrote: On Thu, Apr 16, 2009 at 12:42:17AM +0200, Greg Kurz wrote: On Wed, 2009-04-15 at 23:56 +0400, Alexey Dobriyan wrote: There are sockets and live netns as the most complex example. I'm not prepared to describe it exactly, but

[Devel] Re: [PATCH 1/9] io-throttle documentation

2009-04-17 Thread Gui Jianfeng
KAMEZAWA Hiroyuki wrote: On Tue, 14 Apr 2009 22:21:12 +0200 Andrea Righi righi.and...@gmail.com wrote: +Example: +* Create an association between an io-throttle group and a bio-cgroup group + with bio and blockio subsystems mounted in different mount points: + # mount -t cgroup -o bio

[Devel] Re: [PATCH 1/9] io-throttle documentation

2009-04-17 Thread Gui Jianfeng
KAMEZAWA Hiroyuki wrote: On Fri, 17 Apr 2009 15:34:53 +0800 Gui Jianfeng guijianf...@cn.fujitsu.com wrote: KAMEZAWA Hiroyuki wrote: On Tue, 14 Apr 2009 22:21:12 +0200 Andrea Righi righi.and...@gmail.com wrote: +Example: +* Create an association between an io-throttle group and a

[Devel] Re: [PATCH 01/10] Documentation

2009-04-17 Thread Andrea Righi
On Thu, Apr 16, 2009 at 02:37:53PM -0400, Vivek Goyal wrote: I think it would be possible to implement both proportional and limiting rules at the same level (e.g., the IO scheduler), but we need also to address the memory consumption problem (I still need to review your patchset in

[Devel] Re: [PATCH 0/9] cgroup: io-throttle controller (v13)

2009-04-17 Thread Andrea Righi
On Thu, Apr 16, 2009 at 03:24:33PM -0700, Andrew Morton wrote: On Tue, 14 Apr 2009 22:21:11 +0200 Andrea Righi righi.and...@gmail.com wrote: Objective ~ The objective of the io-throttle controller is to improve IO performance predictability of different cgroups that share the

[Devel] Re: [PATCH 3/9] bio-cgroup controller

2009-04-17 Thread Andrea Righi
On Thu, Apr 16, 2009 at 03:29:37PM -0700, Andrew Morton wrote: On Tue, 14 Apr 2009 22:21:14 +0200 Andrea Righi righi.and...@gmail.com wrote: Subject: [PATCH 3/9] bio-cgroup controller Sorry, but I have to register extreme distress at the name of this. The term bio is well-established in

[Devel] Re: [PATCH 3/9] bio-cgroup controller

2009-04-17 Thread Andrea Righi
On Fri, Apr 17, 2009 at 09:04:51AM +0900, KAMEZAWA Hiroyuki wrote: On Thu, 16 Apr 2009 12:42:36 +0200 Andrea Righi righi.and...@gmail.com wrote: On Thu, Apr 16, 2009 at 08:58:14AM +0900, KAMEZAWA Hiroyuki wrote: On Wed, 15 Apr 2009 15:23:57 +0200 Andrea Righi righi.and...@gmail.com

[Devel] Re: [PATCH 3/9] bio-cgroup controller

2009-04-17 Thread KAMEZAWA Hiroyuki
On Fri, 17 Apr 2009 17:48:54 +0900 KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com wrote: On Fri, 17 Apr 2009 17:00:16 +0900 KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com wrote: On Fri, 17 Apr 2009 16:22:01 +0900 (JST) Ryo Tsuruta r...@valinux.co.jp wrote: In the case where the

[Devel] Re: [PATCH 1/9] io-throttle documentation

2009-04-17 Thread Andrea Righi
On Fri, Apr 17, 2009 at 09:56:31AM +0800, Li Zefan wrote: KAMEZAWA Hiroyuki wrote: On Tue, 14 Apr 2009 22:21:12 +0200 Andrea Righi righi.and...@gmail.com wrote: +Example: +* Create an association between an io-throttle group and a bio-cgroup group + with bio and blockio

[Devel] Re: C/R without leaks

2009-04-17 Thread Greg Kurz
On Thu, 2009-04-16 at 14:39 -0400, Oren Laadan wrote: Any connection in that case is, of course, lost, and it's up to the application to do something about it. If the application relies on the state of the connection, it will have to give up (e.g. sshd, and ssh, die). And that's a good

[Devel] Re: [PATCH 1/9] io-throttle documentation

2009-04-17 Thread Andrea Righi
On Fri, Apr 17, 2009 at 12:25:40PM +0200, Andrea Righi wrote: So, what about implementing the bio-cgroup functionality as cgroup page tracking infrastructure and provide the following interfaces: /* * Encode the cgrp-css.id in page_group-flags sorry, I meant css_id(struct

[Devel] Re: [PATCH 1/9] io-throttle documentation

2009-04-17 Thread Fernando Luis Vázquez Cao
Andrea Righi wrote: On Fri, Apr 17, 2009 at 09:56:31AM +0800, Li Zefan wrote: KAMEZAWA Hiroyuki wrote: On Tue, 14 Apr 2009 22:21:12 +0200 Andrea Righi righi.and...@gmail.com wrote: +Example: +* Create an association between an io-throttle group and a bio-cgroup group + with bio and

[Devel] Re: C/R without leaks

2009-04-17 Thread Oren Laadan
Greg Kurz wrote: On Thu, 2009-04-16 at 14:39 -0400, Oren Laadan wrote: Any connection in that case is, of course, lost, and it's up to the application to do something about it. If the application relies on the state of the connection, it will have to give up (e.g. sshd, and ssh, die).

[Devel] Re: [PATCH 1/9] io-throttle documentation

2009-04-17 Thread Andrea Righi
On Fri, Apr 17, 2009 at 10:24:17AM +0900, KAMEZAWA Hiroyuki wrote: On Tue, 14 Apr 2009 22:21:12 +0200 Andrea Righi righi.and...@gmail.com wrote: +Example: +* Create an association between an io-throttle group and a bio-cgroup group + with bio and blockio subsystems mounted in different

[Devel] Re: C/R without leaks

2009-04-17 Thread Greg Kurz
On Fri, 2009-04-17 at 05:48 -0400, Oren Laadan wrote: You mean an sshd with an open connection probably; the server itself is clearly useful to be able to c/r. Yes I mean C/R of sshd with active connections. A canonical example would a virtual-private-server: instead of doing server

[Devel] Block I/O tracking (was Re: [PATCH 3/9] bio-cgroup controller)

2009-04-17 Thread Fernando Luis Vázquez Cao
Ryo Tsuruta wrote: Hi, From: KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com Date: Fri, 17 Apr 2009 11:24:33 +0900 On Fri, 17 Apr 2009 10:49:43 +0900 Takuya Yoshikawa yoshikawa.tak...@oss.ntt.co.jp wrote: Hi, I have a few question. - I have not yet fully understood how your

[Devel] Re: [PATCH 9/9] ext3: do not throttle metadata and journal IO

2009-04-17 Thread Jens Axboe
On Fri, Apr 17 2009, Theodore Tso wrote: On Tue, Apr 14, 2009 at 10:21:20PM +0200, Andrea Righi wrote: Delaying journal IO can unnecessarily delay other independent IO operations from different cgroups. Add BIO_RW_META flag to the ext3 journal IO that informs the io-throttle subsystem

[Devel] IO Controller discussion (Was: Re: [PATCH 01/10] Documentation)

2009-04-17 Thread Vivek Goyal
On Fri, Apr 17, 2009 at 11:05:17AM +0530, Dhaval Giani wrote: On Thu, Apr 16, 2009 at 02:37:53PM -0400, Vivek Goyal wrote: On Wed, Apr 08, 2009 at 10:37:59PM +0200, Andrea Righi wrote: [..] - I can think of atleast one usage of uppper limit controller where we might have

[Devel] IO controller discussion (Was: Re: [PATCH 01/10] Documentation)

2009-04-17 Thread Vivek Goyal
On Fri, Apr 17, 2009 at 11:37:28AM +0200, Andrea Righi wrote: On Thu, Apr 16, 2009 at 02:37:53PM -0400, Vivek Goyal wrote: I think it would be possible to implement both proportional and limiting rules at the same level (e.g., the IO scheduler), but we need also to address the memory

[Devel] [PATCH] c/r: Take uts_sem during checkpoint (v2)

2009-04-17 Thread Dan Smith
Fix the potential for breakage if our UTS changes during checkpoint by grabbing uts_sem and copying those strings to temporary buffers. Cc: or...@cs.columbia.edu Signed-off-by: Dan Smith da...@us.ibm.com Changes in v2: - Be less stupid about holding the system-wide uts_sem during

[Devel] Re: [PATCH 9/9] ext3: do not throttle metadata and journal IO

2009-04-17 Thread Andrea Righi
On Fri, Apr 17, 2009 at 02:50:04PM +0200, Jens Axboe wrote: On Fri, Apr 17 2009, Theodore Tso wrote: On Tue, Apr 14, 2009 at 10:21:20PM +0200, Andrea Righi wrote: Delaying journal IO can unnecessarily delay other independent IO operations from different cgroups. Add BIO_RW_META

[Devel] Re: [PATCH] c/r: Take uts_sem during checkpoint (v2)

2009-04-17 Thread Dan Smith
SH So it would be robust to future code changes if you SH #define MAX_UTS_LEN (__NEW_UTS_LEN+1) SH in utsname.h, so that anyone expanding the size of hostname SH doesn't need to look for this usage. Yeah, I think that came up in a prior round of comments on the UTS stuff, but got dropped when

[Devel] Re: [PATCH] c/r: Take uts_sem during checkpoint (v2)

2009-04-17 Thread Serge E. Hallyn
Quoting Dan Smith (da...@us.ibm.com): Fix the potential for breakage if our UTS changes during checkpoint by grabbing uts_sem and copying those strings to temporary buffers. Cc: or...@cs.columbia.edu Signed-off-by: Dan Smith da...@us.ibm.com Looks good. The only thing I'd add is that you

[Devel] Re: [PATCH] c/r: Take uts_sem during checkpoint (v2)

2009-04-17 Thread Serge E. Hallyn
Quoting Dan Smith (da...@us.ibm.com): SH So it would be robust to future code changes if you SH #define MAX_UTS_LEN (__NEW_UTS_LEN+1) SH in utsname.h, so that anyone expanding the size of hostname SH doesn't need to look for this usage. Yeah, I think that came up in a prior round of

[Devel] [PATCH] Make new_utsname fields use the proper length constant

2009-04-17 Thread Dan Smith
The members of the new_utsname structure are defined with magic numbers that *should* correspond to the constant __NEW_UTS_LEN+1. Everywhere else, code assumes this and uses the constant, so this patch makes the structure match. Originally suggested by Serge here:

[Devel] Re: [PATCH 1/9] io-throttle documentation

2009-04-17 Thread Vivek Goyal
On Tue, Apr 14, 2009 at 10:21:12PM +0200, Andrea Righi wrote: [..] +4.2. Buffered I/O (write-back) tracking + +For buffered writes the scenario is a bit more complex, because the writes in +the page cache are processed asynchronously by kernel threads (pdflush), using +a write-back policy.

[Devel] Re: [PATCH] Make new_utsname fields use the proper length constant

2009-04-17 Thread Serge E. Hallyn
Quoting Dan Smith (da...@us.ibm.com): The members of the new_utsname structure are defined with magic numbers that *should* correspond to the constant __NEW_UTS_LEN+1. Everywhere else, code assumes this and uses the constant, so this patch makes the structure match. Originally suggested by

[Devel] Re: IO controller discussion (Was: Re: [PATCH 01/10] Documentation)

2009-04-17 Thread Nauman Rafique
On Fri, Apr 17, 2009 at 7:13 AM, Vivek Goyal vgo...@redhat.com wrote: On Fri, Apr 17, 2009 at 11:37:28AM +0200, Andrea Righi wrote: On Thu, Apr 16, 2009 at 02:37:53PM -0400, Vivek Goyal wrote: I think it would be possible to implement both proportional and limiting rules at the same level

[Devel] Re: [PATCH 3/4] deferqueue: generic queue to defer work

2009-04-17 Thread Oren Laadan
Thanks, taken. Serge E. Hallyn wrote: Add a interface to postpone an action until the end of the entire checkpoint or restart operation. This is useful when during the scan of tasks an operation cannot be performed in place, to avoid the need for a second scan. One use case is when

[Devel] Re: Block I/O tracking (was Re: [PATCH 3/9] bio-cgroup controller)

2009-04-17 Thread Andrea Righi
On Fri, Apr 17, 2009 at 08:27:25PM +0900, Fernando Luis Vázquez Cao wrote: Ryo Tsuruta wrote: Hi, From: KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com Date: Fri, 17 Apr 2009 11:24:33 +0900 On Fri, 17 Apr 2009 10:49:43 +0900 Takuya Yoshikawa yoshikawa.tak...@oss.ntt.co.jp wrote: Hi, I

[Devel] Re: IO controller discussion (Was: Re: [PATCH 01/10] Documentation)

2009-04-17 Thread Andrea Righi
On Fri, Apr 17, 2009 at 10:13:58AM -0400, Vivek Goyal wrote: I think setting a maximum limit on dirty pages is an interesting thought. It sounds like as if memory controller can handle it? Exactly, the same above. Thinking more about it. Memory controller can probably enforce the

[Devel] Re: [PATCH 1/9] io-throttle documentation

2009-04-17 Thread Andrea Righi
On Fri, Apr 17, 2009 at 01:39:55PM -0400, Vivek Goyal wrote: On Tue, Apr 14, 2009 at 10:21:12PM +0200, Andrea Righi wrote: [..] +4.2. Buffered I/O (write-back) tracking + +For buffered writes the scenario is a bit more complex, because the writes in +the page cache are processed