Re: [lxc-devel] [PATCH 0/5] Signal stuff v2 and some documentation
Daniel Lezcano daniel.lezc...@free.fr writes: On 07/15/2010 10:07 PM, Ferenc Wagner wrote: Daniel Lezcanodaniel.lezc...@free.fr writes: On 06/09/2010 07:56 PM, Ferenc Wagner wrote: here are basically the same patches, with some obvious errors corrected and some unrelated documentation added. It actually survived some targeted testing in the past days and seems to behave as expected, ie. # lxc-start -n s -- sh -c trap 'echo TERM' TERM; sleep 10 can be interrupted by Ctrl-C from the terminal (the sleep process does not ignore the SIGINT sent to the foreground process group by the OS), while a # pkill lxc-start does not terminate the sleep as the SIGTERM gets forwarded to the shell only, which reports it after the sleep expires. This forwarding mechanism makes it possible to plug lxc into our batch queueing system. is it your last version or can I investigate with this patchset ? Yes, this is the version I've been using since I posted it. I haven't ported it to latest git, but it shouldn't be hard. It seems to do what I intended, but obviously interferes with the console handling, but that should be rethought anyway, as I see it. Ok, thanks. I will take the 2 first patches, so signal forwarding is done but without [tc]setpgrp for the moment. Fine, that's enough for our immediate purposes. I just figured out, in your use case, you are using 'lxc-start -n foo prog'. Yes. You are getting ride of the child reaping (the kernel reparents orphan processes to the container's init). Sorry, I'm afraid I don't understand the you are getting ride of the child reaping part. Could you please rephrase that? The purpose of lxc-init is to reap childs, mount /proc, /dev/shm, forward signals to process 2 and support daemons. This isn't a particularly coherent set in my opinion, and neither is it documented, so I always regarded lxc-execute as a quickdirty way to launch applications in a container. Reaping orphaned children is the usual task of init, so that's fine. Mounting this and that is purely convenience, lxc-start can do so just as well (and in a configurable way). Forwarding signals isn't something a usual init would do, and I wonder what purpose that serves. And what does supporting daemons mean exactly? Does lxc-start exit as soon as it's child exits, leaving possible other processes in the container running, and thus leaking a cgroup/namespace? It wouldn't seem like useful behaviour. Maybe you already noticed that, but maybe you should use the 'lxc-execute -n foo prog' (which spawns lxc-init). The above are just my general concerns with lxc-execute and co. My main reason for not using it is that SGE (the job scheduler I use) has a so called shepherd process for playing the role of init for each started job, so I meant lxc-init would be largely superfluous. In this case, it would be more convenient to do [tc]setpgrp in lxc-init, so we solve the problem with the console. I agree with the general idea of treating full-system and application containers differently in this respect. But I don't agree with tying this to lxc-execute, which isn't flexible enough for general use (for example, what if I don't want /proc in the container?). I still feel like console handling in lxc-start should be reworked. Basically, I feel like the container console from the user space PoV should be an alias for a terminal device, just like on a real system. /dev/console isn't virtualized by the kernel, so it shouldn't be accessible from a container, although bind mounting it to some tty is an option in case some program uses it explicitly. That was the first implementation but the '/sbin/init' process calls TIOCSCTTY, borrowing the tty to the current terminal. Interesting. What does /sbin/init do exactly? It hasn't got a controlling terminal on my systems: $ ps j 1 PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND 0 1 1 1 ? -1 Ss 0 0:17 init [2] In any case, the console presented by lxc-start should always be detachable, preferable even detached by default. Yep, I will send a matrix with a lxc-execute vs lxc-start vs start() common function vs console and hopefully we can find a nice way to fix this mess. Looking forward to it! :) -- Thanks, Feri. -- This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first ___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH 0/5] Signal stuff v2 and some documentation
Daniel Lezcano daniel.lezc...@free.fr writes: On 06/09/2010 07:56 PM, Ferenc Wagner wrote: here are basically the same patches, with some obvious errors corrected and some unrelated documentation added. It actually survived some targeted testing in the past days and seems to behave as expected, ie. # lxc-start -n s -- sh -c trap 'echo TERM' TERM; sleep 10 can be interrupted by Ctrl-C from the terminal (the sleep process does not ignore the SIGINT sent to the foreground process group by the OS), while a # pkill lxc-start does not terminate the sleep as the SIGTERM gets forwarded to the shell only, which reports it after the sleep expires. This forwarding mechanism makes it possible to plug lxc into our batch queueing system. is it your last version or can I investigate with this patchset ? Yes, this is the version I've been using since I posted it. I haven't ported it to latest git, but it shouldn't be hard. It seems to do what I intended, but obviously interferes with the console handling, but that should be rethought anyway, as I see it. Basically, I feel like the container console from the user space PoV should be an alias for a terminal device, just like on a real system. /dev/console isn't virtualized by the kernel, so it shouldn't be accessible from a container, although bind mounting it to some tty is an option in case some program uses it explicitly. In any case, the console presented by lxc-start should always be detachable, preferable even detached by default. -- Regards, Feri. -- This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first ___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH 0/5] Signal stuff v2 and some documentation
On 07/15/2010 10:07 PM, Ferenc Wagner wrote: Daniel Lezcanodaniel.lezc...@free.fr writes: On 06/09/2010 07:56 PM, Ferenc Wagner wrote: here are basically the same patches, with some obvious errors corrected and some unrelated documentation added. It actually survived some targeted testing in the past days and seems to behave as expected, ie. # lxc-start -n s -- sh -c trap 'echo TERM' TERM; sleep 10 can be interrupted by Ctrl-C from the terminal (the sleep process does not ignore the SIGINT sent to the foreground process group by the OS), while a # pkill lxc-start does not terminate the sleep as the SIGTERM gets forwarded to the shell only, which reports it after the sleep expires. This forwarding mechanism makes it possible to plug lxc into our batch queueing system. is it your last version or can I investigate with this patchset ? Yes, this is the version I've been using since I posted it. I haven't ported it to latest git, but it shouldn't be hard. It seems to do what I intended, but obviously interferes with the console handling, but that should be rethought anyway, as I see it. Ok, thanks. I will take the 2 first patches, so signal forwarding is done but without [tc]setpgrp for the moment. I have a couple a patches on top of yours where when lxc-init receives a SIGTERM, it does like the usual 'init' process by sending a kill(-1, SIGTERM) followed by a kill(-1, SIGKILL) if all the processes do not exit after a small amount of time. I just figured out, in your use case, you are using 'lxc-start -n foo prog'. You are getting ride of the child reaping (the kernel reparents orphan processes to the container's init). The purpose of lxc-init is to reap childs, mount /proc, /dev/shm, forward signals to process 2 and support daemons. Maybe you already noticed that, but maybe you should use the 'lxc-execute -n foo prog' (which spawns lxc-init). In this case, it would be more convenient to do [tc]setpgrp in lxc-init, so we solve the problem with the console. Basically, I feel like the container console from the user space PoV should be an alias for a terminal device, just like on a real system. /dev/console isn't virtualized by the kernel, so it shouldn't be accessible from a container, although bind mounting it to some tty is an option in case some program uses it explicitly. That was the first implementation but the '/sbin/init' process calls TIOCSCTTY, borrowing the tty to the current terminal. In any case, the console presented by lxc-start should always be detachable, preferable even detached by default. Yep, I will send a matrix with a lxc-execute vs lxc-start vs start() common function vs console and hopefully we can find a nice way to fix this mess. Thanks Ferenc, -- Daniel -- This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first ___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH 0/5] Signal stuff v2 and some documentation
On 06/09/2010 07:56 PM, Ferenc Wagner wrote: Hi, here are basically the same patches, with some obvious errors corrected and some unrelated documentation added. It actually survived some targeted testing in the past days and seems to behave as expected, ie. # lxc-start -n s -- sh -c trap 'echo TERM' TERM; sleep 10 can be interrupted by Ctrl-C from the terminal (the sleep process does not ignore the SIGINT sent to the foreground process group by the OS), while a # pkill lxc-start does not terminate the sleep as the SIGTERM gets forwarded to the shell only, which reports it after the sleep expires. This forwarding mechanism makes it possible to plug lxc into our batch queueing system. Hi Ferenc, is it your last version or can I investigate with this patchset ? Thanks -- Daniel -- This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first ___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH 0/5] Signal stuff v2 and some documentation
Daniel Lezcano daniel.lezc...@free.fr writes: On 06/15/2010 02:13 PM, Ferenc Wagner wrote: Daniel Lezcanodaniel.lezc...@free.fr writes: On 06/10/2010 11:47 PM, Ferenc Wagner wrote: If you provide me with an example (and some description of lxc.console), I can give it some testing and concretize this pure guesswork. lxc-create -n ubuntu -f ~/mynetwork.conf -t ubuntu lxc-start -n ubuntu -s lxc.console=$(tty) -o $(tty) -l DEBUG I'm not there yet, but found something interesting. If lxc-checkconfig reports full green, clone(NEWNS|NEWUTS|NEWIPC|NEWPID|NEWNET) in lxc-start shouldn't fail. Who's wrong here? $ lxc-checkconfig Kernel config /proc/config.gz not found, looking in other places... Found kernel config file /boot/config-2.6.26-2-686 2.6.26 ? Mmmh, You need at least a 2.6.29 for a system container (better to have a 2.6.32). Yeah, it runs with 2.6.32. Btw. what happened in 2.6.29, which made it particularly suitable for running system containers? Bah ! Looks like the lxc-checkconfig is buggy (fix in attachment). With your fix it indeed misses a couple of things: Network namespace: missing Multiple /dev/pts instances: missing Cgroup memory controller: missing Macvlan: missing Thanks for the fix! Now let's see why lxc-start gets suspended when I try to type at the console... Interestingly, it stays in S state until I kill the container. I'm afraid the console functionality (is there any documentation for it?) may make lxc-start unsuitable for pushing into the background. After all, it is an interactive foreground process in that case, a real proxy towards some getty (if I understand this console thingie right). Maybe this should be handled differently to application containers. But then I'm not sure how Ctrl-C and similar should be forwarded to a getty... -- Cheers, Feri. -- ThinkGeek and WIRED's GeekDad team up for the Ultimate GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky parental unit. See the prize list and enter to win: http://p.sf.net/sfu/thinkgeek-promo ___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH 0/5] Signal stuff v2 and some documentation
On 06/15/2010 04:47 PM, Ferenc Wagner wrote: Daniel Lezcanodaniel.lezc...@free.fr writes: On 06/15/2010 02:13 PM, Ferenc Wagner wrote: Daniel Lezcanodaniel.lezc...@free.fr writes: On 06/10/2010 11:47 PM, Ferenc Wagner wrote: If you provide me with an example (and some description of lxc.console), I can give it some testing and concretize this pure guesswork. lxc-create -n ubuntu -f ~/mynetwork.conf -t ubuntu lxc-start -n ubuntu -s lxc.console=$(tty) -o $(tty) -l DEBUG I'm not there yet, but found something interesting. If lxc-checkconfig reports full green, clone(NEWNS|NEWUTS|NEWIPC|NEWPID|NEWNET) in lxc-start shouldn't fail. Who's wrong here? $ lxc-checkconfig Kernel config /proc/config.gz not found, looking in other places... Found kernel config file /boot/config-2.6.26-2-686 2.6.26 ? Mmmh, You need at least a 2.6.29 for a system container (better to have a 2.6.32). Yeah, it runs with 2.6.32. Btw. what happened in 2.6.29, which made it particularly suitable for running system containers? The network virtualization was merged upstream. Bah ! Looks like the lxc-checkconfig is buggy (fix in attachment). With your fix it indeed misses a couple of things: Network namespace: missing Better to have it for a system container, otherwise the guest system will reconfigure your host network :/ Multiple /dev/pts instances: missing Better to have it but not mandatory until you remove the lxc.pts option. Cgroup memory controller: missing Not mandatory. Macvlan: missing Better to have, it is more flexible to configure the network. but not mandatory. Thanks for the fix! Now let's see why lxc-start gets suspended when I try to type at the console... I think it happens exactly what you described in the previous email, that is if a background process tries to read/write to the tty, then a SIGTTIN / SIGTTOU / is sent to it, where the default action is to stop the process. Interestingly, it stays in S state until I kill the container. I'm afraid the console functionality (is there any documentation for it?) may make lxc-start unsuitable for pushing into the background. After all, it is an interactive foreground process in that case, a real proxy towards some getty (if I understand this console thingie right). Maybe this should be handled differently to application containers. But then I'm not sure how Ctrl-C and similar should be forwarded to a getty... argh. yes, chicken-egg problem. -- ThinkGeek and WIRED's GeekDad team up for the Ultimate GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky parental unit. See the prize list and enter to win: http://p.sf.net/sfu/thinkgeek-promo ___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH 0/5] Signal stuff v2 and some documentation
On 06/15/2010 02:13 PM, Ferenc Wagner wrote:
Daniel Lezcanodaniel.lezc...@free.fr writes:
On 06/10/2010 11:47 PM, Ferenc Wagner wrote:
If you provide me with an example (and some description of
lxc.console), I can give it some testing and concretize this pure
guesswork.
lxc-create -n ubuntu -f ~/mynetwork.conf -t ubuntu
lxc-start -n ubuntu -s lxc.console=$(tty) -o $(tty) -l DEBUG
I'm not there yet, but found something interesting. If lxc-checkconfig
reports full green, clone(NEWNS|NEWUTS|NEWIPC|NEWPID|NEWNET) in
lxc-start shouldn't fail. Who's wrong here?
Cheers,
Feri.
$ lxc-checkconfig
Kernel config /proc/config.gz not found, looking in other places...
Found kernel config file /boot/config-2.6.26-2-686
2.6.26 ? Mmmh, You need at least a 2.6.29 for a system container (better
to have a 2.6.32).
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled
Multiple /dev/pts instances: enabled
--- Control groups ---
Cgroup: enabled
Cgroup namespace: enabled
Cgroup device: enabled
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled
--- Misc ---
Veth pair device: enabled
Macvlan: enabled
Vlan: enabled
File capabilities: enabled
Note : Before booting a new kernel, you can check its configuration
usage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig
$ sudo lxc-start -n debian -s lxc.console=$(tty)
lxc-start: failed to clone(0x6c02): Invalid argument
lxc-start: Invalid argument - failed to fork into a new namespace
lxc-start: failed to spawn 'debian'
lxc-start: No such file or directory - failed to remove cgroup '/mnt/debian'
Bah ! Looks like the lxc-checkconfig is buggy (fix in attachment).
---
src/lxc/lxc-checkconfig.in |4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
Index: lxc/src/lxc/lxc-checkconfig.in
===
--- lxc.orig/src/lxc/lxc-checkconfig.in
+++ lxc/src/lxc/lxc-checkconfig.in
@@ -11,8 +11,6 @@ SETCOLOR_NORMAL=echo -en \\033[0;39m
is_set() {
$GREP -q $1=[y|m] $CONFIG
-RES=$?
-
return $?
}
@@ -22,7 +20,7 @@ is_enabled() {
is_set $1
RES=$?
-if [ $RES = 0 ]; then
+if [ $RES -eq 0 ]; then
$SETCOLOR_SUCCESS echo -e enabled $SETCOLOR_NORMAL
else
if [ ! -z $mandatory -a $mandatory = yes ]; then
--
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit. See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo___
Lxc-devel mailing list
Lxc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH 0/5] Signal stuff v2 and some documentation
Hi, Apologies if this is the wrong list. Interestingly, it stays in S state until I kill the container. I'm afraid the console functionality (is there any documentation for it?) may make lxc-start unsuitable for pushing into the background. After all, it is an interactive foreground process in that case, a real proxy towards some getty (if I understand this console thingie right). Maybe this should be handled differently to application containers. But then I'm not sure how Ctrl-C and similar should be forwarded to a getty... argh. yes, chicken-egg problem. The lxc.console=$(tty) thing was something I was thinking about. There are a couple of things I've noticed and was pondering how to fix; 1) The expectation I have from xen, kvm etc is that you can detach from the console like lxc-console allows. i.e. lxc-start -C -n test behaves like lxc-start -n test ; lxc-console - at the moment you have an interactive foreground process. 2) If you have a getty on the console, when you start without -s lxc.console=$(tty) it puts the system messages and the getty on the host system console. That gets confusing when logging in on a lights out console. Was this what lxcd was for? Should it be that lxc-start always goes into the background, and holds onto the console, which you can connect to via lxcd by specifying a flag to lxc-console? lxc-start -s lxc.console gets replaced by lxc-start -C which is equivalent to lxc-start ; lxc-console Happy to devote some time to fixing this... Andy Andrew Phillips Head of Systems www.lmax.com Office: +44 203 1922509 Mobile: +44 (0)7595 242 900 LMAX | Level 2, Yellow Building | 1 Nicholas Road | London | W11 4AN The information in this e-mail and any attachment is confidential and is intended only for the named recipient(s). The e-mail may not be disclosed or used by any person other than the addressee, nor may it be copied in any way. If you are not a named recipient please notify the sender immediately and delete any copies of this message. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Any view or opinions presented are solely those of the author and do not necessarily represent those of the company. -- ThinkGeek and WIRED's GeekDad team up for the Ultimate GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky parental unit. See the prize list and enter to win: http://p.sf.net/sfu/thinkgeek-promo ___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
