[Devel] Re: [PATCH v3 06/11] memcg: add kernel calls for memcg dirty page stats

2010-10-19 Thread Daisuke Nishimura
On Mon, 18 Oct 2010 17:39:39 -0700 Greg Thelen gthe...@google.com wrote: Add calls into memcg dirty page accounting. Notify memcg when pages transition between clean, file dirty, writeback, and unstable nfs. This allows the memory controller to maintain an accurate view of the amount of its

[Devel] Re: [PATCH v3 02/11] memcg: document cgroup dirty memory interfaces

2010-10-19 Thread Daisuke Nishimura
On Mon, 18 Oct 2010 17:39:35 -0700 Greg Thelen gthe...@google.com wrote: Document cgroup dirty memory interfaces and statistics. Signed-off-by: Andrea Righi ari...@develer.com Signed-off-by: Greg Thelen gthe...@google.com --- Changelog since v1: - Renamed nfs/total_nfs to

[Devel] [PATCH] Clear the objhash before completing restart, but delay free (v2)

2010-10-19 Thread Dan Smith
This patch causes the restart coordinator to clear the object hash before releasing the restarted tasks. It does this to make sure that any objects being held exclusively by the hash are released before the tasks start running again. If we continue to postpone clearing the object hash until

[Devel] cgroup debug seems dead?

2010-10-19 Thread Stephen Hemminger
While running namespace checks to look for dead code, I found that debug_subsys is declared global but never used in current code. This leads to the fact that since debug_subsys is never used, the whole set of debugging functions is also never used. Is the whole CGROUP_DEBUG config option dead?

[Devel] Re: [PATCH] Clear the objhash before completing restart, but delay free (v2)

2010-10-19 Thread Matt Helsley
On Tue, Oct 19, 2010 at 07:03:11AM -0700, Dan Smith wrote: This patch causes the restart coordinator to clear the object hash before releasing the restarted tasks. It does this to make sure that any objects being held exclusively by the hash are released before the tasks start running again.

[Devel] Re: [PATCH v3 02/11] memcg: document cgroup dirty memory interfaces

2010-10-19 Thread Greg Thelen
Daisuke Nishimura nishim...@mxp.nes.nec.co.jp writes: On Mon, 18 Oct 2010 17:39:35 -0700 Greg Thelen gthe...@google.com wrote: Document cgroup dirty memory interfaces and statistics. Signed-off-by: Andrea Righi ari...@develer.com Signed-off-by: Greg Thelen gthe...@google.com ---

[Devel] Re: [PATCH v3 02/11] memcg: document cgroup dirty memory interfaces

2010-10-19 Thread KAMEZAWA Hiroyuki
On Tue, 19 Oct 2010 14:00:58 -0700 Greg Thelen gthe...@google.com wrote: Daisuke Nishimura nishim...@mxp.nes.nec.co.jp writes: On Mon, 18 Oct 2010 17:39:35 -0700 Greg Thelen gthe...@google.com wrote: Document cgroup dirty memory interfaces and statistics. Signed-off-by: Andrea

[Devel] Re: [PATCH v3 02/11] memcg: document cgroup dirty memory interfaces

2010-10-19 Thread Daisuke Nishimura
On Wed, 20 Oct 2010 09:11:09 +0900 KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com wrote: On Tue, 19 Oct 2010 14:00:58 -0700 Greg Thelen gthe...@google.com wrote: (snip) +When use_hierarchy=0, each cgroup has independent dirty memory usage and limits. + +When use_hierarchy=1, a

[Devel] Re: [PATCH v3 07/11] memcg: add dirty limits to mem_cgroup

2010-10-19 Thread Daisuke Nishimura
+static unsigned long long +memcg_hierarchical_free_pages(struct mem_cgroup *mem) +{ + struct cgroup *cgroup; + unsigned long long min_free, free; + + min_free = res_counter_read_u64(mem-res, RES_LIMIT) - + res_counter_read_u64(mem-res, RES_USAGE); + cgroup =

[Devel] Re: [PATCH v3 05/11] memcg: add dirty page accounting infrastructure

2010-10-19 Thread Daisuke Nishimura
On Mon, 18 Oct 2010 17:39:38 -0700 Greg Thelen gthe...@google.com wrote: Add memcg routines to track dirty, writeback, and unstable_NFS pages. These routines are not yet used by the kernel to count such pages. A later change adds kernel calls to these new routines. Signed-off-by: Greg

[Devel] Re: cgroup debug seems dead?

2010-10-19 Thread Li Zefan
Stephen Hemminger wrote: While running namespace checks to look for dead code, I found that debug_subsys is declared global but never used in current code. This leads to the fact that since debug_subsys is never used, the whole set of debugging functions is also never used. Is the whole

[Devel] Re: cgroup debug seems dead?

2010-10-19 Thread Stephen Hemminger
On Wed, 20 Oct 2010 09:11:10 +0800 Li Zefan l...@cn.fujitsu.com wrote: Stephen Hemminger wrote: While running namespace checks to look for dead code, I found that debug_subsys is declared global but never used in current code. This leads to the fact that since debug_subsys is never

[Devel] Re: cgroup debug seems dead?

2010-10-19 Thread Paul Menage
On Tue, Oct 19, 2010 at 6:12 PM, Stephen Hemminger shemmin...@vyatta.com wrote: Where is it stored in an array? ~/kernel/linux-2.6$ git grep debug_subsys kernel/cgroup.c:        kfree(cont-subsys[debug_subsys_id]); kernel/cgroup.c:struct cgroup_subsys debug_subsys = { kernel/cgroup.c:      

[Devel] Re: [PATCH v3 02/11] memcg: document cgroup dirty memory interfaces

2010-10-19 Thread KAMEZAWA Hiroyuki
On Wed, 20 Oct 2010 09:48:21 +0900 Daisuke Nishimura nishim...@mxp.nes.nec.co.jp wrote: On Wed, 20 Oct 2010 09:11:09 +0900 KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com wrote: On Tue, 19 Oct 2010 14:00:58 -0700 Greg Thelen gthe...@google.com wrote: (snip) +When use_hierarchy=0,

[Devel] Re: cgroup debug seems dead?

2010-10-19 Thread Stephen Hemminger
On Tue, 19 Oct 2010 18:14:10 -0700 Paul Menage men...@google.com wrote: On Tue, Oct 19, 2010 at 6:12 PM, Stephen Hemminger shemmin...@vyatta.com wrote: Where is it stored in an array? ~/kernel/linux-2.6$ git grep debug_subsys kernel/cgroup.c:        

[Devel] Re: cgroup debug seems dead?

2010-10-19 Thread Paul Menage
On Tue, Oct 19, 2010 at 6:37 PM, Stephen Hemminger shemmin...@vyatta.com wrote: That would work but doesn't because the following is missing in cgroup_subsys.h! #ifdef CONFIG_CGROUP_DEBUG SUBSYS(debug) #endif /* */ I see that fragment at line 16 in cgroup_subsys.h (see e.g.

[Devel] Re: cgroup debug seems dead?

2010-10-19 Thread Li Zefan
Stephen Hemminger wrote: On Tue, 19 Oct 2010 18:14:10 -0700 Paul Menage men...@google.com wrote: On Tue, Oct 19, 2010 at 6:12 PM, Stephen Hemminger shemmin...@vyatta.com wrote: Where is it stored in an array? ~/kernel/linux-2.6$ git grep debug_subsys kernel/cgroup.c:

[Devel] Re: cgroup debug seems dead?

2010-10-19 Thread Stephen Hemminger
On Wed, 20 Oct 2010 09:43:06 +0800 Li Zefan l...@cn.fujitsu.com wrote: Stephen Hemminger wrote: On Tue, 19 Oct 2010 18:14:10 -0700 Paul Menage men...@google.com wrote: On Tue, Oct 19, 2010 at 6:12 PM, Stephen Hemminger shemmin...@vyatta.com wrote: Where is it stored in an array?

[Devel] Re: cgroup debug seems dead?

2010-10-19 Thread Li Zefan
Stephen Hemminger wrote: On Wed, 20 Oct 2010 09:43:06 +0800 Li Zefan l...@cn.fujitsu.com wrote: Stephen Hemminger wrote: On Tue, 19 Oct 2010 18:14:10 -0700 Paul Menage men...@google.com wrote: On Tue, Oct 19, 2010 at 6:12 PM, Stephen Hemminger shemmin...@vyatta.com wrote: Where is it

[Devel] Re: [PATCH v3 02/11] memcg: document cgroup dirty memory interfaces

2010-10-19 Thread KAMEZAWA Hiroyuki
On Wed, 20 Oct 2010 10:14:21 +0900 KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com wrote: On Wed, 20 Oct 2010 09:48:21 +0900 Daisuke Nishimura nishim...@mxp.nes.nec.co.jp wrote: On Wed, 20 Oct 2010 09:11:09 +0900 KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com wrote: On Tue, 19

[Devel] [PATCH][memcg+dirtylimit] Fix overwriting global vm dirty limit setting by memcg (Re: [PATCH v3 00/11] memcg: per cgroup dirty page accounting

2010-10-19 Thread KAMEZAWA Hiroyuki
One bug fix here. == From: KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com Now, at calculating dirty limit, vm_dirty_param() is called. This function returns dirty-limit related parameters considering memory cgroup settings. Now, assume that vm_dirty_bytes=100M (global dirty limit) and memory

[Devel] Re: [PATCH v3 09/11] memcg: add cgroupfs interface to memcg dirty limits

2010-10-19 Thread Daisuke Nishimura
On Mon, 18 Oct 2010 17:39:42 -0700 Greg Thelen gthe...@google.com wrote: Add cgroupfs interface to memcg dirty page limits: Direct write-out is controlled with: - memory.dirty_ratio - memory.dirty_limit_in_bytes Background write-out is controlled with: -

[Devel] Re: [PATCH v3 09/11] memcg: add cgroupfs interface to memcg dirty limits

2010-10-19 Thread Daisuke Nishimura
On Wed, 20 Oct 2010 12:31:10 +0900 Daisuke Nishimura nishim...@mxp.nes.nec.co.jp wrote: On Mon, 18 Oct 2010 17:39:42 -0700 Greg Thelen gthe...@google.com wrote: Add cgroupfs interface to memcg dirty page limits: Direct write-out is controlled with: - memory.dirty_ratio -

[Devel] Re: [PATCH v3 02/11] memcg: document cgroup dirty memory interfaces

2010-10-19 Thread Daisuke Nishimura
On Wed, 20 Oct 2010 11:24:31 +0900 KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com wrote: On Wed, 20 Oct 2010 10:14:21 +0900 KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com wrote: On Wed, 20 Oct 2010 09:48:21 +0900 Daisuke Nishimura nishim...@mxp.nes.nec.co.jp wrote: On Wed, 20

[Devel] Re: [PATCH v3 10/11] writeback: make determine_dirtyable_memory() static.

2010-10-19 Thread Daisuke Nishimura
On Mon, 18 Oct 2010 17:39:43 -0700 Greg Thelen gthe...@google.com wrote: The determine_dirtyable_memory() function is not used outside of page writeback. Make the routine static. No functional change. Just a cleanup in preparation for a change that adds memcg dirty limits consideration into

[Devel] Re: [PATCH v3 09/11] memcg: add cgroupfs interface to memcg dirty limits

2010-10-19 Thread KAMEZAWA Hiroyuki
On Wed, 20 Oct 2010 12:31:10 +0900 Daisuke Nishimura nishim...@mxp.nes.nec.co.jp wrote: On Mon, 18 Oct 2010 17:39:42 -0700 Greg Thelen gthe...@google.com wrote: Add cgroupfs interface to memcg dirty page limits: Direct write-out is controlled with: - memory.dirty_ratio -

[Devel] Re: [PATCH v3 08/11] memcg: CPU hotplug lockdep warning fix

2010-10-19 Thread Daisuke Nishimura
On Mon, 18 Oct 2010 17:39:41 -0700 Greg Thelen gthe...@google.com wrote: From: Balbir Singh bal...@linux.vnet.ibm.com memcg has lockdep warnings (sleep inside rcu lock) From: Balbir Singh bal...@linux.vnet.ibm.com Recent move to get_online_cpus() ends up calling get_online_cpus() from

[Devel] Re: [PATCH v3 02/11] memcg: document cgroup dirty memory interfaces

2010-10-19 Thread Greg Thelen
KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com writes: On Tue, 19 Oct 2010 14:00:58 -0700 Greg Thelen gthe...@google.com wrote: Daisuke Nishimura nishim...@mxp.nes.nec.co.jp writes: On Mon, 18 Oct 2010 17:39:35 -0700 Greg Thelen gthe...@google.com wrote: Document cgroup dirty

[Devel] Re: [PATCH v3 07/11] memcg: add dirty limits to mem_cgroup

2010-10-19 Thread Greg Thelen
Daisuke Nishimura nishim...@mxp.nes.nec.co.jp writes: +static unsigned long long +memcg_hierarchical_free_pages(struct mem_cgroup *mem) +{ +struct cgroup *cgroup; +unsigned long long min_free, free; + +min_free = res_counter_read_u64(mem-res, RES_LIMIT) - +

[Devel] Re: [PATCH v3 02/11] memcg: document cgroup dirty memory interfaces

2010-10-19 Thread KAMEZAWA Hiroyuki
On Tue, 19 Oct 2010 17:45:08 -0700 Greg Thelen gthe...@google.com wrote: KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com writes: BTW, how about supporing dirty_limit_in_bytes when use_hierarchy=0 or leave it as broken when use_hierarchy=1 ? It seems we can only support dirty_ratio when

[Devel] Re: [PATCH][memcg+dirtylimit] Fix overwriting global vm dirty limit setting by memcg (Re: [PATCH v3 00/11] memcg: per cgroup dirty page accounting

2010-10-19 Thread KAMEZAWA Hiroyuki
On Wed, 20 Oct 2010 12:21:44 +0900 KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com wrote: One bug fix here. == From: KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com Now, at calculating dirty limit, vm_dirty_param() is called. This function returns dirty-limit related parameters

[Devel] Re: [PATCH v3 11/11] memcg: check memcg dirty limits in page writeback

2010-10-19 Thread KAMEZAWA Hiroyuki
On Tue, 19 Oct 2010 10:00:15 +0900 KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com wrote: On Mon, 18 Oct 2010 17:39:44 -0700 Greg Thelen gthe...@google.com wrote: If the current process is in a non-root memcg, then global_dirty_limits() will consider the memcg dirty limit. This allows

[Devel] Re: [PATCH v3 02/11] memcg: document cgroup dirty memory interfaces

2010-10-19 Thread Greg Thelen
KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com writes: On Tue, 19 Oct 2010 17:45:08 -0700 Greg Thelen gthe...@google.com wrote: KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com writes: BTW, how about supporing dirty_limit_in_bytes when use_hierarchy=0 or leave it as broken when

[Devel] Re: [PATCH v3 11/11] memcg: check memcg dirty limits in page writeback

2010-10-19 Thread Greg Thelen
KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com writes: On Tue, 19 Oct 2010 10:00:15 +0900 KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com wrote: On Mon, 18 Oct 2010 17:39:44 -0700 Greg Thelen gthe...@google.com wrote: If the current process is in a non-root memcg, then

[Devel] Re: [PATCH v3 02/11] memcg: document cgroup dirty memory interfaces

2010-10-19 Thread KAMEZAWA Hiroyuki
On Tue, 19 Oct 2010 21:25:53 -0700 Greg Thelen gthe...@google.com wrote: KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com writes: On Tue, 19 Oct 2010 17:45:08 -0700 Greg Thelen gthe...@google.com wrote: KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com writes: BTW, how about

[Devel] Re: [PATCH v3 11/11] memcg: check memcg dirty limits in page writeback

2010-10-19 Thread KAMEZAWA Hiroyuki
On Tue, 19 Oct 2010 21:33:21 -0700 Greg Thelen gthe...@google.com wrote: KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com writes: On Tue, 19 Oct 2010 10:00:15 +0900 KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com wrote: On Mon, 18 Oct 2010 17:39:44 -0700 Greg Thelen

[Devel] Re: [PATCH v3 11/11] memcg: check memcg dirty limits in page writeback

2010-10-19 Thread Daisuke Nishimura
On Wed, 20 Oct 2010 13:18:57 +0900 KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com wrote: On Tue, 19 Oct 2010 10:00:15 +0900 KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com wrote: On Mon, 18 Oct 2010 17:39:44 -0700 Greg Thelen gthe...@google.com wrote: If the current process is in

[Devel] Re: cgroup debug seems dead?

2010-10-19 Thread Américo Wang
On Wed, Oct 20, 2010 at 10:23:42AM +0800, Li Zefan wrote: Stephen Hemminger wrote: On Wed, 20 Oct 2010 09:43:06 +0800 Li Zefan l...@cn.fujitsu.com wrote: Stephen Hemminger wrote: On Tue, 19 Oct 2010 18:14:10 -0700 Paul Menage men...@google.com wrote: On Tue, Oct 19, 2010 at 6:12 PM,

[Devel] [PATCH v2][memcg+dirtylimit] Fix overwriting global vm dirty limit setting by memcg (Re: [PATCH v3 00/11] memcg: per cgroup dirty page accounting

2010-10-19 Thread KAMEZAWA Hiroyuki
Fixed one here. == From: KAMEZAWA Hiroyuki kamezawa.hir...@jp.fujitsu.com Now, at calculating dirty limit, vm_dirty_param() is called. This function returns dirty-limit related parameters considering memory cgroup settings. Now, assume that vm_dirty_bytes=100M (global dirty limit) and memory

[Devel] Re: [PATCH v3 11/11] memcg: check memcg dirty limits in page writeback

2010-10-19 Thread Daisuke Nishimura
On Mon, 18 Oct 2010 17:39:44 -0700 Greg Thelen gthe...@google.com wrote: If the current process is in a non-root memcg, then global_dirty_limits() will consider the memcg dirty limit. This allows different cgroups to have distinct dirty limits which trigger direct and background writeback at