[Devel] Re: [PATCH 00/10] Containers(V10): Generic Process Containers

2007-05-30 Thread Andrew Morton
On Tue, 29 May 2007 06:01:04 -0700 [EMAIL PROTECTED] wrote: This is an update to my multi-hierarchy multi-subsystem generic process containers patch. ... Still TODO: ... - lots more testing So how do we do this? Is there any sneaky way in which we can modify the kernel so that

[Devel] Re: [PATCH 01/10] Containers(V10): Basic container framework

2007-05-30 Thread Andrew Morton
On Tue, 29 May 2007 06:01:05 -0700 [EMAIL PROTECTED] wrote: +For example, the following sequence of commands will setup a container +named Charlie, containing just CPUs 2 and 3, and Memory Node 1, +and then start a subshell 'sh' in that container: + + mount -t container cpuset -ocpuset

[Devel] Re: [PATCH 02/10] Containers(V10): Example CPU accounting subsystem

2007-05-30 Thread Andrew Morton
On Tue, 29 May 2007 06:01:06 -0700 [EMAIL PROTECTED] wrote: This example demonstrates how to use the generic container subsystem for a simple resource tracker that counts, for the processes in a container, the total CPU time used and the %CPU used in the last complete 10 second interval.

[Devel] Re: [PATCH 05/10] Containers(V10): Add container_clone() interface

2007-05-30 Thread Andrew Morton
On Tue, 29 May 2007 06:01:09 -0700 [EMAIL PROTECTED] wrote: This patch adds support for container_clone(), a speculative interface to creating new containers intended to be used for systems such as namespace unsharing. ... + +static atomic_t namecnt; +static void get_unused_name(char

[Devel] Re: [PATCH 01/10] Containers(V10): Basic container framework

2007-05-30 Thread Andrew Morton
On Tue, 29 May 2007 06:01:05 -0700 [EMAIL PROTECTED] wrote: This patch adds the main containers framework - the container filesystem, and the basic structures for tracking membership and associating subsystem state objects to tasks. ... --- /dev/null +++

[Devel] Re: [PATCH 00/10] Containers(V10): Generic Process Containers

2007-05-30 Thread William Lee Irwin III
On Wed, May 30, 2007 at 12:14:55AM -0700, Andrew Morton wrote: So how do we do this? Is there any sneaky way in which we can modify the kernel so that this new code gets exercised more? Obviously, tossing init into some default system-wide container would be a start. But I wonder if we can

[Devel] Re: Pid namespaces approaches testing results

2007-05-30 Thread Pavel Emelianov
Dave Hansen wrote: On Tue, 2007-05-29 at 15:45 +0400, Pavel Emelianov wrote: The detailed results are the following: Test name:spawn execlshellps (sys time) 1(no ns) :579.1 618.31623.2 3.052s 2(suka's):570.7 610.81600.2 3.107s Slowdown :1.5%

[Devel] Re: [PATCH 00/10] Containers(V10): Generic Process Containers

2007-05-30 Thread Balbir Singh
Andrew Morton wrote: On Tue, 29 May 2007 06:01:04 -0700 [EMAIL PROTECTED] wrote: This is an update to my multi-hierarchy multi-subsystem generic process containers patch. ... Still TODO: ... - lots more testing So how do we do this? Is there any sneaky way in which we can

[Devel] Re: [PATCH] /proc/*/environ: wrong placing of ptrace_may_attach() check

2007-05-30 Thread Alexey Dobriyan
On Tue, May 29, 2007 at 05:16:23PM -0700, Andrew Morton wrote: On Mon, 28 May 2007 17:41:57 +0400 Alexey Dobriyan [EMAIL PROTECTED] wrote: --- a/fs/proc/base.c +++ b/fs/proc/base.c @@ -204,12 +204,17 @@ static int proc_pid_environ(struct task_ int res = 0; struct mm_struct *mm

[Devel] Re: [PATCH 00/10] Containers(V10): Generic Process Containers

2007-05-30 Thread Pavel Emelianov
Balbir Singh wrote: Andrew Morton wrote: On Tue, 29 May 2007 06:01:04 -0700 [EMAIL PROTECTED] wrote: This is an update to my multi-hierarchy multi-subsystem generic process containers patch. ... Still TODO: ... - lots more testing So how do we do this? Is there any sneaky way in

[Devel] Re: [PATCH 00/10] Containers(V10): Generic Process Containers

2007-05-30 Thread Balbir Singh
Pavel Emelianov wrote: Is there any sneaky way in which we can modify the kernel so that this new code gets exercised more? Obviously, tossing init into some default system-wide container would be a start. But I wonder if we can be sneakier - for example, create a new container on each

[Devel] Re: [PATCH 00/10] Containers(V10): Generic Process Containers

2007-05-30 Thread Pavel Emelianov
Hi Paul. I have faced a warning during testing your patches. The testcase is simple: # ssh to the node mount -t container none /cnt/rss/ -o rss mkdir /cnt/rss/0 /bin/echo $$ /cnt/rss/0/tasks # exit with ^d and ssh again rmdir /cnt/rss/0 dmesg BUG: at mm/slab.c:777

[Devel] Re: Pid namespaces approaches testing results

2007-05-30 Thread Serge E. Hallyn
Quoting Pavel Emelianov ([EMAIL PROTECTED]): Dave Hansen wrote: On Tue, 2007-05-29 at 15:45 +0400, Pavel Emelianov wrote: The detailed results are the following: Test name:spawn execlshellps (sys time) 1(no ns) :579.1 618.31623.2 3.052s 2(suka's):570.7

[Devel] Re: Pid namespaces approaches testing results

2007-05-30 Thread Pavel Emelianov
Serge E. Hallyn wrote: Quoting Pavel Emelianov ([EMAIL PROTECTED]): Dave Hansen wrote: On Tue, 2007-05-29 at 15:45 +0400, Pavel Emelianov wrote: The detailed results are the following: Test name:spawn execlshellps (sys time) 1(no ns) :579.1 618.31623.2 3.052s

[Devel] Re: [PATCH 01/10] Containers(V10): Basic container framework

2007-05-30 Thread Paul Menage
On 5/30/07, Andrew Morton [EMAIL PROTECTED] wrote: Holy cow, do we need all those? I'll experiment to see which ones we can get rid of. +typedef enum { + CONT_REMOVED, +} container_flagbits_t; typedefs are verboten. Fortunately this one is never referred to - only the values are

[Devel] [PATCH 2/8] Add container pointer on struct page

2007-05-30 Thread Pavel Emelianov
Each page is supposed to have an owner - the container that touched the page first. The owner stays alive during the page lifetime even if the task that touched the page dies or moves to another container. This ownership is the forerunner for the fair page sharing accounting, in which page has as

[Devel] [PATCH 1/8] Resource counters

2007-05-30 Thread Pavel Emelianov
Introduce generic structures and routines for resource accounting. Each resource accounting container is supposed to aggregate it, container_subsystem_state and its resource-specific members within. Signed-off-by: Pavel Emelianov [EMAIL PROTECTED] --- diff -upr

[Devel] [PATCH 0/8] RSS controller based on process containers (v3)

2007-05-30 Thread Pavel Emelianov
Adds RSS accounting and control within a container. Ported above Paul's containers V10 with fixes received from Balbir. RSS container includes the per-container RSS accounting and reclamation, and out-of-memory killer. The container behaves like a standalone machine - when it runs out of

[Devel] [PATCH 4/8] RSS container core

2007-05-30 Thread Pavel Emelianov
The core routines for tracking the page ownership, registration of RSS subsystem in the containers and the definition of rss_container as container subsystem combined with resource counter. Includes fixes from Balbir Singh [EMAIL PROTECTED] Signed-off-by: Pavel Emelianov [EMAIL PROTECTED] ---

[Devel] [PATCH 6/8] Per container OOM killer

2007-05-30 Thread Pavel Emelianov
When container is completely out of memory some tasks should die. This is unfair to kill the current task, so a task with the largest RSS is chosen and killed. The code re-uses current OOM killer select_bad_process() for task selection. Signed-off-by: Pavel Emelianov [EMAIL PROTECTED] --- diff

[Devel] [PATCH 8/8] Per-container pages reclamation

2007-05-30 Thread Pavel Emelianov
Implement try_to_free_pages_in_container() to free the pages in container that has run out of memory. The scan_control-isolate_pages() function is set to isolate_pages_in_container() that isolates the container pages only. The exported __isolate_lru_page() call makes things look simpler than in

[Devel] Re: [PATCH 01/10] Containers(V10): Basic container framework

2007-05-30 Thread Andrew Morton
On Wed, 30 May 2007 07:02:00 -0700 Paul Menage [EMAIL PROTECTED] wrote: People have hit unpleasant problems before now running iput() against partially-constructed inodes. What kinds of problems? Are there bits of state that I should fully construct even if I'm going to iput() it, or

[Devel] Re: [PATCH 11/13] Changes to show virtual ids to user

2007-05-30 Thread Cedric Le Goater
Hello ! The worst case I can see with pid == 0. Is that it would be a bug that we can fix later. For other cases it would seem to be a user space API thing that we get stuck with for all time. We cannot trust userspace application to expect some pid other than positive. All that we can is

[Devel] Re: [ckrm-tech] [RFC] [PATCH 0/3] Add group fairness to CFS

2007-05-30 Thread William Lee Irwin III
On Sat, May 26, 2007 at 08:41:12AM -0700, William Lee Irwin III wrote: The smpnice affair is better phrased in terms of task weighting. It's simple to honor nice in such an arrangement. First unravel the grouping hierarchy, then weight by nice. This looks like [...] conveniently collapse to

[Devel] Re: [PATCH 2/8] Add container pointer on struct page

2007-05-30 Thread Andrew Morton
On Wed, 30 May 2007 19:28:04 +0400 Pavel Emelianov [EMAIL PROTECTED] wrote: +#ifdef CONFIG_RSS_CONTAINER +#define page_container(page) (page-rss_container) +#else +#define page_container(page) (NULL) +#endif static inline C functions are nicer.

[Devel] Re: [PATCH 5/8] RSS accounting hooks over the code

2007-05-30 Thread Andrew Morton
On Wed, 30 May 2007 19:34:18 +0400 Pavel Emelianov [EMAIL PROTECTED] wrote: As described above, pages are charged to their first touchers. The first toucher is determined using pages' _mapcount manipulations in rmap calls. Page is charged in two stages: 1. preparation, in which the

[Devel] Re: [PATCH 4/8] RSS container core

2007-05-30 Thread Andrew Morton
On Wed, 30 May 2007 19:32:14 +0400 Pavel Emelianov [EMAIL PROTECTED] wrote: The core routines for tracking the page ownership, registration of RSS subsystem in the containers and the definition of rss_container as container subsystem combined with resource counter. ---

[Devel] Re: [PATCH 3/8] Add container pointer on mm_struct

2007-05-30 Thread Andrew Morton
On Wed, 30 May 2007 19:29:26 +0400 Pavel Emelianov [EMAIL PROTECTED] wrote: Naturally mm_struct determines the resource consumer in memory accounting. So each mm_struct should have a pointer on container it belongs to. When a new task is created its mm_struct is assigned to the container this

[Devel] Re: [PATCH 8/8] Per-container pages reclamation

2007-05-30 Thread Andrew Morton
On Wed, 30 May 2007 19:42:26 +0400 Pavel Emelianov [EMAIL PROTECTED] wrote: Implement try_to_free_pages_in_container() to free the pages in container that has run out of memory. The scan_control-isolate_pages() function is set to isolate_pages_in_container() that isolates the container

[Devel] Re: [ckrm-tech] [RFC] [PATCH 0/3] Add group fairness to CFS

2007-05-30 Thread William Lee Irwin III
On Wed, May 30, 2007 at 01:13:59PM -0700, William Lee Irwin III wrote: The step beyond was to show how nice numbers can be done with all that hierarchical task grouping so they have global effects instead of effects limited to the scope of the narrowest grouping hierarchy containing the task.