[Devel] Re: [PATCH 1/1] mm_owner: fix cgroup null dereference

2008-08-20 Thread KAMEZAWA Hiroyuki
On Wed, 20 Aug 2008 13:20:40 +0800 Li Zefan [EMAIL PROTECTED] wrote: Balbir Singh wote: * Jiri Slaby [EMAIL PROTECTED] [2008-08-14 22:16:53]: Hi, found this in mmotm, a fix for mm-owner-fix-race-between-swap-and-exit.patch Does the patch below fix your problem, it's against

[Devel] Re: [PATCH 09/15] sysfs: Implement sysfs tagged directory support.

2008-08-20 Thread Eric W. Biederman
Greg KH [EMAIL PROTECTED] writes: On Thu, Jul 03, 2008 at 06:16:08PM -0700, Eric W. Biederman wrote: The problem. When implementing a network namespace I need to be able to have multiple network devices with the same name. Currently this is a problem for /sys/class/net/*,

[Devel] Re: RFC: Attaching threads to cgroups is OK?

2008-08-20 Thread Hirokazu Takahashi
Hi Fernando! Hi Balbir, Kamezawa-san! On Tue, 2008-08-19 at 17:57 +0530, Balbir Singh wrote: Tsuruta-san, how about your bio-cgroup's tracking concerning this? If we want to use your tracking functions for each threads seperately, there seems to be a problem. ===cf.

[Devel] Re: RFC: Attaching threads to cgroups is OK?

2008-08-20 Thread Hirokazu Takahashi
Hi, Hi everyone, I have a question about cgroup's policy concerning the treatment of threads. Please consider that we want to attach an application which has some threads already to a certain cgroup. If we echo the pid of this application to the tasks file connected to this cgroup the

[Devel] Re: RFC: Attaching threads to cgroups is OK?

2008-08-20 Thread KAMEZAWA Hiroyuki
On Wed, 20 Aug 2008 16:12:47 +0900 (JST) Hirokazu Takahashi [EMAIL PROTECTED] wrote: - I think this kind of thread application should control its I/O requests inside of the application. I guess it seems to quite difficult to determine which thread is doing what kind of job in the

[Devel] Re: RFC: Attaching threads to cgroups is OK?

2008-08-20 Thread Hirokazu Takahashi
Hi, Tsuruta-san, how about your bio-cgroup's tracking concerning this? If we want to use your tracking functions for each threads seperately, there seems to be a problem. ===cf. mm_get_bio_cgroup()=== owner mm_struct task_struct bio_cgroup

[Devel] Re: [PATCH 1/6] Container Freezer: Fix freezer Kconfig

2008-08-20 Thread Pavel Machek
On Tue 2008-08-19 16:22:33, Matt Helsley wrote: kernel/power/Kconfig is not sourced from all architectures but the freezer code should be available to all architectures for the cgroup freezer subsystem. Sourcing a new kernel/Kconfig.freezer has the added advantage of keeping the config

[Devel] Re: [PATCH 2/6] Container Freezer: uninline thaw_process()

2008-08-20 Thread Pavel Machek
On Tue 2008-08-19 16:22:34, Matt Helsley wrote: Now that the cgroup freezer system also calls thaw_process() inlining these functions uses more space. Uninlining returns some space: Before: textdatabss dec hex filename 4260872 275532 290816 4827220 49a854

[Devel] Re: [Bugme-new] [Bug 11381] New: default shmmax

2008-08-20 Thread Andrew Morton
(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface). On Wed, 20 Aug 2008 05:58:57 -0700 (PDT) [EMAIL PROTECTED] wrote: http://bugzilla.kernel.org/show_bug.cgi?id=11381 Summary: default shmmax Product: Other

Re: [Devel] [RFC][PATCH 1/4] checkpoint-restart: general infrastructure

2008-08-20 Thread Dave Hansen
On Mon, 2008-08-18 at 13:26 +0400, Pavel Emelyanov wrote: diff -puN /dev/null ckpt/ckpt_hdr.h --- /dev/null 2007-04-11 11:48:27.0 -0700 +++ linux-2.6.git-dave/ckpt/ckpt_hdr.h 2008-08-07 15:37:22.0 -0700 @@ -0,0 +1,69 @@ +/* + * Generic container

[Devel] Re: [Bugme-new] [Bug 11381] New: default shmmax

2008-08-20 Thread adobriyan
On Wed, Aug 20, 2008 at 12:00:43PM -0700, Andrew Morton wrote: (switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface). On Wed, 20 Aug 2008 05:58:57 -0700 (PDT) [EMAIL PROTECTED] wrote: http://bugzilla.kernel.org/show_bug.cgi?id=11381

[Devel] Re: [Bugme-new] [Bug 11381] New: default shmmax

2008-08-20 Thread adobriyan
On Wed, Aug 20, 2008 at 11:12:57PM +0400, wrote: On Wed, Aug 20, 2008 at 12:00:43PM -0700, Andrew Morton wrote: (switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface). On Wed, 20 Aug 2008 05:58:57 -0700 (PDT) [EMAIL PROTECTED] wrote:

[Devel] Re: [Bugme-new] [Bug 11381] New: default shmmax

2008-08-20 Thread Alan Cox
It would be useful to get distro input on this. Do they override the kernel default at boot time? If so, what do they do? Red Hat provide a sysctl tuning config file and I believe things like the Oracle install docs cover this. There is btw no earthly reason why a postgres package can't

[Devel] [RFC v2][PATCH 0/9] kernel-based checkpoint-restart

2008-08-20 Thread Dave Hansen
These patches are from Oren Laadan. I've refactored them a bit to make them a wee bit more reviewable by: 1. Removing code that we're not yet using 2. Separating out i386 code into a separate patch 3. Simplyifying the filename handling This should also be at least build-bisetable. TODO: -

[Devel] [RFC v2][PATCH 3/9] checkpoint/restart: x86 support

2008-08-20 Thread Dave Hansen
The original version of Oren's patch contained a good hunk of #ifdefs. I've extracted all of those and created a bit of an API for new architectures to follow. Leaving Oren's sign-off because this is all still his code, even though he hasn't seen it mangled like this before. Signed-off-by:

[Devel] [RFC v2][PATCH 2/9] Remove CAP_SYS_ADMIN for checkpoint/restart

2008-08-20 Thread Dave Hansen
We need to do this so that we think about the security concerns as we add each and every bit of c/r functionality. There's nothing that we need privileges for, yet. Let's keep it that way as long as possible. --- oren-cr.git-dave/checkpoint/sys.c |6 -- 1 file changed, 6 deletions(-)

[Devel] [RFC v2][PATCH 6/9] Simplify filename handling for now

2008-08-20 Thread Dave Hansen
The filename handling is a bit clunky, as one of the reviewers noted. It can allocate memory in one of two places and that makes things look a bit weird. We don't *really* need the filename sitting around in its own buffer, so let's just combine the writing and the filename generation. It

[Devel] [RFC v2][PATCH 1/9] checkpoint-restart: general infrastructure

2008-08-20 Thread Dave Hansen
This patch adds those interfaces, as well as all of the helpers needed to easily manage the file format. The code is roughly broken out as follows: ckpt/sys.c - user/kernel data transfer, as well as setting up of the checkpoint/restart context (a per-checkpoint data

[Devel] [RFC v2][PATCH 7/9] remove temporary buffer structures

2008-08-20 Thread Dave Hansen
These will eventually help the efficiency of the restore and aid in creating incremental checkpoints. But, for now, they do not give us much, and introduce some unnecessary abstraction. Kill them for now, but we can always add them back later. We use stack allocations for some of these, and

[Devel] [RFC v2][PATCH 5/9] Create trivial sys_checkpoint and sys_restore system calls

2008-08-20 Thread Dave Hansen
From: Oren Laadan [EMAIL PROTECTED] Create trivial sys_checkpoint and sys_restore system calls. They will enable to checkpoint and restart an entire container, to and from a checkpoint image file. First create a template for both syscalls: they take a file descriptor (for the image file) and

[Devel] [RFC v2][PATCH 9/9] remove -cksum field

2008-08-20 Thread Dave Hansen
This is unused so far. We'll add it back later. --- oren-cr.git-dave/checkpoint/checkpoint.c |2 -- oren-cr.git-dave/checkpoint/ckpt_hdr.h |1 - oren-cr.git-dave/checkpoint/restart.c|3 +-- 3 files changed, 1 insertion(+), 5 deletions(-) diff -puN

[Devel] [RFC v2][PATCH 4/9] checkpoint/restart: memory management

2008-08-20 Thread Dave Hansen
For each vma, there is a 'struct cr_vma'; if the vma is file-mapped, it will be followed by the file name. The cr_vma-npages will tell how many pages were dumped for this vma. Then it will be followed by the actual data: first a dump of the addresses of all dumped pages (npages entries)

[Devel] [RFC v2][PATCH 8/9] Remove some BUG_ON()s that need some proper error handling instead.

2008-08-20 Thread Dave Hansen
--- oren-cr.git-dave/checkpoint/checkpoint.c | 12 ++-- oren-cr.git-dave/checkpoint/restart.c| 15 +++ 2 files changed, 25 insertions(+), 2 deletions(-) diff -puN checkpoint/checkpoint.c~0002-Remove-some-BUG_ON-s-that-need-some-proper-error-ha

[Devel] Re: [RFC v2][PATCH 8/9] Remove some BUG_ON()s that need some proper error handling instead.

2008-08-20 Thread Dave Hansen
On Wed, 2008-08-20 at 12:26 -0700, Dave Hansen wrote: diff -puN checkpoint/checkpoint.c~0002-Remove-some-BUG_ON-s-that-need-some-proper-error-ha checkpoint/checkpoint.c --- oren-cr.git/checkpoint/checkpoint.c~0002-Remove-some-BUG_ON-s-that-need-some-proper-error-ha 2008-08-20

[Devel] Re: [RFC][PATCH 1/4] checkpoint-restart: general infrastructure

2008-08-20 Thread Oren Laadan
Pavel Machek wrote: Hi! I have to wonder if this is just a symptom of us trying to do this the wrong way. We're trying to talk the kernel into writing internal gunk into a FD. You're right, it is like a splice where one end of the pipe is in the kernel. Any thoughts on a better way to

[Devel] Re: checkpoint/restart ABI

2008-08-20 Thread Oren Laadan
Jeremy Fitzhardinge wrote: Dave Hansen wrote: I'm not sure what you mean by closed files. Either the app has a fd, it doesn't, or it is in sys_open() somewhere. We have to get the app into a quiescent state before we can checkpoint, so we basically just say that we won't checkpoint things

[Devel] Re: checkpoint/restart ABI

2008-08-20 Thread Oren Laadan
Dave Hansen wrote: On Tue, 2008-08-12 at 09:32 -0700, Jeremy Fitzhardinge wrote: Inter-machine networking stuff is hard because its outside the checkpointed set, so the checkpoint is observable. Migration is easier, in principle, because you might be able to shift the connection endpoint

[Devel] Re: checkpoint/restart ABI

2008-08-20 Thread Dave Hansen
On Wed, 2008-08-20 at 17:54 -0400, Oren Laadan wrote: Me, personally, I think I'd probably re-link the thing, mark it as such, ship it across like a normal file, then unlink it after the restore. I don't know what we'd choose when actually implementing it. Re-linking works well when

[Devel] [RFC][PATCH 0/8][v2]: Enable multiple mounts of devpts

2008-08-20 Thread sukadev
I have been trying to address comments from the last patchset and here is another snapshot of the patches. Appreciate any feedback on the single/multi-mount semantics (patch 8/8). See below for changelog and pending stuff --- Enable multiple mounts of devpts filesystem so each container can

[Devel] [RFC][PATCH 1/8]: /dev/tty tweak in init_dev()

2008-08-20 Thread sukadev
From: Sukadev Bhattiprolu [EMAIL PROTECTED] Subject: [RFC][PATCH 1/8]: /dev/tty tweak in init_dev() When opening /dev/tty, __tty_open() finds the tty using get_current_tty(). When __tty_open() calls init_dev() it passes in this tty in '*ret_tty'. init_dev() ignores this and asks devpts again for

[Devel] [RFC][PATCH 2/8]: Add inode parameter devpts interfaces

2008-08-20 Thread sukadev
From: Sukadev Bhattiprolu [EMAIL PROTECTED] Subject: [RFC][PATCH 2/8]: Add inode parameter devpts interfaces Pass-in an 'inode' parameter to devpts interfaces. The parameter itself will be used in subsequent patches to identify the instance of devpts mounted. --- drivers/char/pty.c|

[Devel] [RFC][PATCH 3/8]: Remove devpts_root global

2008-08-20 Thread sukadev
From: Sukadev Bhattiprolu [EMAIL PROTECTED] Subject: [RFC][PATCH 3/8]: Remove devpts_root global Remove the 'devpts_root' global variable and find the root dentry using the super_block. The super-block can be found from the device inode, using the new wrapper, pts_sb_from_inode(). ---

[Devel] [RFC][PATCH 4/8]: Per-mount allocated_ptys

2008-08-20 Thread sukadev
From: Sukadev Bhattiprolu [EMAIL PROTECTED] Subject: [RFC][PATCH 4/8]: Per-mount allocated_ptys To enable multiple mounts of devpts, 'allocated_ptys' must be a per-mount variable rather than a global variable. Move 'allocated_ptys' into the super_block's s_fs_info. Changelog[v2]:

[Devel] [RFC][PATCH 7/8]: Auto-create ptmx node when mounting devpts

2008-08-20 Thread sukadev
From: Sukadev Bhattiprolu [EMAIL PROTECTED] Subject: [RFC][PATCH 7/8]: Auto-create ptmx node when mounting devpts /dev/ptmx is closely tied to the devpts filesystem. An open of /dev/ptmx, allocates the next pty index and the associated device shows up in the devpts fs as /dev/pts/n. Wih

[Devel] [RFC][PATCH 8/8]: Enable multiple mounts of devpts

2008-08-20 Thread sukadev
From: Sukadev Bhattiprolu [EMAIL PROTECTED] Subject: [RFC][PATCH 8/8]: Enable multiple mounts of devpts To support containers, allow multiple instances of devpts filesystem. But to preserve backward compatibility, provide this support for multiple-mounts under the new mount option, '-o newmnt'.

[Devel] [RFC][PATCH 5/8]: Per-mount 'config' object

2008-08-20 Thread sukadev
From: Sukadev Bhattiprolu [EMAIL PROTECTED] Subject: [RFC][PATCH 5/8]: Per-mount 'config' object With support for multiple mounts of devpts, the 'config' structure really represents per-mount options rather than config parameters. Rename 'config' structure to 'pts_mount_opts' and store it in the

[Devel] [RFC][PATCH 6/8]: Extract option parsing to new function

2008-08-20 Thread sukadev
From: Sukadev Bhattiprolu [EMAIL PROTECTED] Subject: [RFC][PATCH 6/8]: Extract option parsing to new function Move code to parse mount options into a separate function so it can (later) be shared between mount and remount operations. --- fs/devpts/inode.c | 12 +--- 1 file changed, 9

[Devel] Re: [RFC][PATCH 0/8][v2]: Enable multiple mounts of devpts

2008-08-20 Thread H. Peter Anvin
[EMAIL PROTECTED] wrote: TODO: - Remove even initial kernel mount of devpts ? (If we do, how do we preserve single-mount semantics) ? Doesn't make sense unless we decide to drop single-mount semantics in the (far) future. As long as we have an instance that services

[Devel] [RFC v2][PATCH 1/9] kernel based checkpoint-restart

2008-08-20 Thread Oren Laadan
These patches implement checkpoint-restart [CR v2]. This version adds save and restore of open files state (regular files and directories) which makes it more usable. Other changes address the feedback given for the previous version. It is also refactored (along Dave's posting) for easier

[Devel] [RFC v2][PATCH 1/9] Create trivial sys_checkpoint/sys_restart syscalls

2008-08-20 Thread Oren Laadan
Create trivial sys_checkpoint and sys_restore system calls. They will enable to checkpoint and restart an entire container, to and from a checkpoint image file. First create a template for both syscalls: they take a file descriptor (for the image file) and flags as arguments. For sys_checkpoint

[Devel] [RFC v2][PATCH 4/9] Memory management - dump state

2008-08-20 Thread Oren Laadan
For each VMA, there is a 'struct cr_vma'; if the VMA is file-mapped, it will be followed by the file name. The cr_vma-npages will tell how many pages were dumped for this VMA. Then it will be followed by the actual data: first a dump of the addresses of all dumped pages (npages entries)

[Devel] [RFC v2][PATCH 5/9] Memory managemnet - restore state

2008-08-20 Thread Oren Laadan
Restoring the memory address space begins with nuking the existing one of the current process, and then reading the VMA state and contents. Call do_mmap_pgoffset() for each VMA and then read in the data. Signed-off-by: Oren Laadan [EMAIL PROTECTED] --- checkpoint/Makefile|2 +-

[Devel] [RFC v2][PATCH 6/9] Checkpoint/restart: initial documentation

2008-08-20 Thread Oren Laadan
Covers application checkpoint/restart, overall design, interfaces and checkpoint image format. Signed-off-by: Oren Laadan [EMAIL PROTECTED] --- Documentation/checkpoint.txt | 177 ++ 1 files changed, 177 insertions(+), 0 deletions(-) create mode

[Devel] [RFC v2][PATCH 2/9] General infrastructure for checkpoint restart

2008-08-20 Thread Oren Laadan
Add those interfaces, as well as helpers needed to easily manage the file format. The code is roughly broken out as follows: ckpt/sys.c - user/kernel data transfer, as well as setup of the checkpoint/restart context (a per-checkpoint data structure for housekeeping) ckpt/checkpoint.c - output

[Devel] [RFC v2][PATCH 3/9] x86 support for checkpoint/restart

2008-08-20 Thread Oren Laadan
(Following Dave Hansen's refactoring of the original post) Add logic to save and restore architecture specific state, including thread-specific state, CPU registers and FPU state. Currently only x86-32 is supported. Compiling on x86-64 will trigger an explicit error. Signed-off-by: Oren Laadan

[Devel] Re: RFC: Attaching threads to cgroups is OK?

2008-08-20 Thread Fernando Luis Vázquez Cao
On Wed, 2008-08-20 at 20:48 +0900, Hirokazu Takahashi wrote: Hi, Tsuruta-san, how about your bio-cgroup's tracking concerning this? If we want to use your tracking functions for each threads seperately, there seems to be a problem. ===cf. mm_get_bio_cgroup()===

[Devel] [RFC v2][PATCH 9/9] File descriprtors (restore)

2008-08-20 Thread Oren Laadan
Restore open file descriptors: for each FD read 'struct cr_hdr_fd_ent' and lookup tag in the hash table; if not found (first occurence), read in 'struct cr_hdr_fd_data', create a new FD and register in the hash. Otherwise attach the file pointer from the hash as an FD. This patch only handles

[Devel] [RFC v2][PATCH 7/9] Infrastructure for shared objects

2008-08-20 Thread Oren Laadan
Infrastructure to handle objects that may be shared and referenced by multiple tasks or other objects, e..g open files, memory address space etc. The state of shared objects is saved once. On the first encounter, the state is dumped and the object is assigned a unique identifier and also stored

[Devel] [RFC v2][PATCH 8/9] File descriprtors - dump state

2008-08-20 Thread Oren Laadan
Dump the files_struct of a task with 'struct cr_hdr_files', followed by all open file descriptors. Since FDs can be shared, they are assigned a tag and registered in the object hash. For each open FD there is a 'struct cr_hdr_fd_ent' with the FD, its tag and its close-on-exec property. If the FD

[Devel] Re: [RFC][PATCH 0/8][v2]: Enable multiple mounts of devpts

2008-08-20 Thread sukadev
H. Peter Anvin [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: TODO: - Remove even initial kernel mount of devpts ? (If we do, how do we preserve single-mount semantics) ? Doesn't make sense unless we decide to drop single-mount semantics in the (far) future. As long as we

[Devel] Re: [RFC][PATCH 0/8][v2]: Enable multiple mounts of devpts

2008-08-20 Thread H. Peter Anvin
[EMAIL PROTECTED] wrote: I don't like the name newmnt for the option; it is not just another mount, but a whole new instance of the pty space. I agree. Its mostly a place-holder for now. How about newns or newptsns ? I suggest newinstance, but newns works, too. I observe you didn't

[Devel] Re: RFC: Attaching threads to cgroups is OK?

2008-08-20 Thread Balbir Singh
Fernando Luis Vázquez Cao wrote: On Wed, 2008-08-20 at 20:48 +0900, Hirokazu Takahashi wrote: Hi, Tsuruta-san, how about your bio-cgroup's tracking concerning this? If we want to use your tracking functions for each threads seperately, there seems to be a problem. ===cf.

[Devel] Re: [RFC v2][PATCH 1/9] kernel based checkpoint-restart

2008-08-20 Thread Oren Laadan
Subject line should have been 's/PATCH 1/PATCH 0/' ... I left cr_debug() in for now to provide more info than pr_debug(); eventually that will be changed back to pr_debug() In the mini-conference we considered doing CR in a kernel module, but decided against because we needed a system call. It

[Devel] Re: RFC: Attaching threads to cgroups is OK?

2008-08-20 Thread Fernando Luis Vázquez Cao
Hi Balbir, On Thu, 2008-08-21 at 09:02 +0530, Balbir Singh wrote: Fernando Luis Vázquez Cao wrote: On Wed, 2008-08-20 at 20:48 +0900, Hirokazu Takahashi wrote: Hi, Tsuruta-san, how about your bio-cgroup's tracking concerning this? If we want to use your tracking functions for each

[Devel] Re: [RFC v2][PATCH 9/9] File descriprtors (restore)

2008-08-20 Thread Oren Laadan
Restore open file descriptors: for each FD read 'struct cr_hdr_fd_ent' and lookup tag in the hash table; if not found (first occurence), read in 'struct cr_hdr_fd_data', create a new FD and register in the hash. Otherwise attach the file pointer from the hash as an FD. This patch only

[Devel] Re: [RFC v2][PATCH 1/9] Create trivial sys_checkpoint/sys_restart syscalls

2008-08-20 Thread Oren Laadan
this, of course, should be the last patch, not the first. Oren Laadan wrote: Create trivial sys_checkpoint and sys_restore system calls. They will enable to checkpoint and restart an entire container, to and from a checkpoint image file. First create a template for both syscalls: they take