Re: [Lxc-users] On clean reboot of Ubuntu 10.04 containers
Trent W. Buck writes:
This post describes my attempts to get clean shutdown of Ubuntu 10.04
containers. The goal here is that a shutdown -h now of the dom0
should not result in a potentially inconsistent domU postgres database,
cf. a naive lxc-stop.
As this is largely working for me, I have turned my attention to getting
clean rebooting of containers. That is, a user should be able to ssh
into a container, issue a sudo reboot, and have sshd hang up.
Reconnecting over ssh should indicate the container rebooted (e.g. new
processes and tmpfses).
I am unreliably informed that there is no native way to handle this in
lxc, and that OpenVZ handles this by having clunky cron job that checks
every minute for containers that need rebooting. So I set about
implementing something similar.
I changed my halt/reboot/poweroff wrapper thusly:
chroot $template_dir tee /dev/null /sbin/reboot -EOF
#!/bin/bash
while getopts nwdfiph opt
do [[ f = \$opt ]] exec kill -SEGV 1
done
+ [[ '-w' = \$* ]] || # /etc/init.d/umountnfs.sh ALWAYS calls halt -w.
+ touch /force\${0##*/}
exec -a $0 \$0.distrib \$@
EOF
And I have my container's event-spoofing /etc/init/lxc.conf remove them
on startup:
rm -f /forcereboot /forcehalt /forcepoweroff
Then, an /etc/cron.d/lxc of:
# Every minute, start all containers that contain /forcereboot.
* * * * * root LXC_START_ONLY=reboot /etc/init.d/lxc start /dev/null
And an /etc/init.d/lxc start rule of:
case $1 in
start)
log_daemon_msg Starting $DESC
shopt -s nullglob
for name in $ROOT_DIR/*/force${LXC_START_ONLY:-boot}
do name=${name%/force*}
name=${name#$ROOT_DIR/}
# Skip it if it doesn't have an RFC952 hostname (e.g. no
fooserv.~1~'s)
[[ $name =~ ^[[:alnum:]]([[:alnum:]-]{0,61}[[:alnum:]])?$ ]] ||
continue
lxc-start -n $name -f $CONF_DIR/$name.conf -d
log_progress_msg $name
done
log_end_msg 0
;;
...
esac
Comments?
PS: I saw that telinit u should make upstart re-exec itself, which I
hoped meant it would issue a startup event, allowing me to reboot at
the init level, without lxc-start noticing or caring -- no such luck.
--
What happens now with your Lotus Notes apps - do you make another costly
upgrade, or settle for being marooned without product support? Time to move
off Lotus Notes and onto the cloud with Force.com, apps are easier to build,
use, and manage than apps on traditional platforms. Sign up for the Lotus
Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] limiting RAM usage and disk space usage
Hi Gordon, On 12/03/2010 05:59 PM, Gordon Henderson wrote: On Fri, 3 Dec 2010, Matt Rechenburg wrote: Hi Lxc team, actually I would vote against a loop mount. I would vote to allow the local systems administrator the choice of what suits them best. fully agree with you on that And since there's no reason to explicitly block loopback mounts, then don't do it. Much easier and more flexible is to use lvm and mount a logical volume at the containers root directory before the container starts. Much easier and flexible (for me) would be to use files mounted via loop. This will automatically limit the available disk space for the container and gives you the flexibility to e.g. snapshot and resize the containers disk. Files mounted via loop will automatically limit the avalable disk space for the container and gives me the flexibility to e.g. snapshot and resize the containers disk. In my setup I have several master images located on logical volumes and when I need a new lxc container I just snapshot one of the master images and deploy the snapshot. btw: this way Lxc is integrated into openQRM (root on lvm). In my setup ... Well, it's different from yours, and I don't use LVM. My Linux kernel does not use LVM, I do not need the overhead of yet another layer of software for my servers to go through before the data gets to/from the disks, so I've no interst in LVM. Do not force LXC into one system or another - let us all have the choice to use what we want to use. many thanks + keep up the great work, Matt Rechenburg Project Manager openQRM So you have a vested interest in LVM - just because others don't, then don't exclude us from using other mechanisms. oh, please do not get me wrong. I am fully with you keeping the lxc commands generic and adapt specific features via script wrappers. Just meant it should not be limited to loop-mounts only. The sysadmin should have the choice. greetz + best, Matt Rechenburg Project Manager openQRM There's always more than one way to do something. Gordon -- Increase Visibility of Your 3D Game App Earn a Chance To Win $500! Tap into the largest installed PC base get more eyes on your game by optimizing for Intel(R) Graphics Technology. Get started today with the Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs. http://p.sf.net/sfu/intelisp-dev2dev ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users -- http://openqrm.com - The open-source Cloud Computing platform http://openqrm-enterprise.com - Professional services and support for openQRM -- What happens now with your Lotus Notes apps - do you make another costly upgrade, or settle for being marooned without product support? Time to move off Lotus Notes and onto the cloud with Force.com, apps are easier to build, use, and manage than apps on traditional platforms. Sign up for the Lotus Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] 0.7.2 - 0.7.3 changes
Cuma 12 Kasım 2010 günü (saat 12:05:29) Daniel Lezcano şunları yazmıştı: On 11/09/2010 09:33 PM, Brian K. White wrote: If I understood correctly, you are using the git head which is 0.7.3 + 1 patch, right ? After your container is started, what gives the ouput of lxc-ps --lxc ? We're having the same problems too. Here's the configuration file: lxc.utsname = c2-farm32 lxc.network.type = veth lxc.network.flags = up lxc.network.link = br0 lxc.network.ipv4 = 192.168.3.140/24 lxc.network.hwaddr = DE:AD:BE:EF:24:94 lxc.network.name = eth0 lxc.mount = /etc/lxc/c2-farm32/fstab lxc.rootfs = /etc/lxc/c2-farm32/rootfs lxc.arch = i686 /etc/lxc/c2-farm32/fstab: -- none /etc/lxc/c2-farm32/rootfs/dev/pts devpts defaults 0 0 none /etc/lxc/c2-farm32/rootfs/procproc defaults 0 0 none /etc/lxc/c2-farm32/rootfs/sys sysfs defaults 0 0 none /etc/lxc/c2-farm32/rootfs/dev/shm tmpfs defaults 0 0 With 0.7.2, no problem at all but with the 0.7.3 tarball, no /proc is available thus initscripts fail at various places. Is the behaviour change was intentional or is this a bug? --- Ozan Çağlayan TUBITAK/UEKAE - Pardus Linux http://www.pardus.org.tr/eng -- What happens now with your Lotus Notes apps - do you make another costly upgrade, or settle for being marooned without product support? Time to move off Lotus Notes and onto the cloud with Force.com, apps are easier to build, use, and manage than apps on traditional platforms. Sign up for the Lotus Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] On clean shutdown of Ubuntu 10.04 containers
Quoting Trent W. Buck (t...@cybersource.com.au): Unfortunately, lxc 0.7's utmp detect requires /var/run to NOT be a tmpfs. The shipped lxc-ubuntu script works around this by deleting the ifstate file and not mounting a tmpfs on /var/run, but to me that is simply waiting for something else to assume /var/run is empty. It also doesn't cope with a mountall upgrade rewriting /lib/init/fstab. Note that how to handle clean shutdown is one of the open topics. So if you're interested in helping to properly fix that, please feel free to join our development call this coming thursday morning, as per announcement last week :) IIRC Daniel L. had a kernel patch to help fix this workaround? He was experimenting with it at UDS-N. thanks, -serge -- What happens now with your Lotus Notes apps - do you make another costly upgrade, or settle for being marooned without product support? Time to move off Lotus Notes and onto the cloud with Force.com, apps are easier to build, use, and manage than apps on traditional platforms. Sign up for the Lotus Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] On clean shutdown of Ubuntu 10.04 containers
On 12/6/2010 2:42 AM, Trent W. Buck wrote: This post describes my attempts to get clean shutdown of Ubuntu 10.04 containers. The goal here is that a shutdown -h now of the dom0 should not result in a potentially inconsistent domU postgres database, cf. a naive lxc-stop. As at Ubuntu 10.04 with lxc 0.7.2, lxc-start detects that a container has halted by 1) seeing a reboot event incontainer/var/run/utmp; or 2) seeingcontainer's PID 1 terminate. Ubuntu 10.04 simply REQUIRES /var/run to be a tmpfs; this is hard-coded into mountall's (upstart's) /lib/init/fstab. Without it, the most immediate issue is that /var/run/ifstate isn't reaped on reboot, ifup(8) thinks lo (at least) is already configured, and the boot process hangs waiting for the network. Unfortunately, lxc 0.7's utmp detect requires /var/run to NOT be a tmpfs. The shipped lxc-ubuntu script works around this by deleting the ifstate file and not mounting a tmpfs on /var/run, but to me that is simply waiting for something else to assume /var/run is empty. It also doesn't cope with a mountall upgrade rewriting /lib/init/fstab. More or less by accident, I discovered that I can tell lxc-start that the container is ready to halt by crashing upstart: container# kill -SEGV 1 Likewise I can spoof a ctrl-alt-delete event in the container with: dom0# pkill -INT lxc-start I automate the former signalling at the end of shutdowns thusly: chroot $template_dir dpkg-divert --quiet --rename /sbin/reboot chroot $template_dir tee/dev/null /sbin/reboot-EOF #!/bin/bash while getopts nwdfiph opt do [[ f = \$opt ]] exec kill -SEGV 1 done exec -a $0 \$0.distrib \$@ EOF chroot $template_dir chmod +x /sbin/reboot chroot $template_dir ln -s reboot.distrib /sbin/halt.distrib chroot $template_dir ln -s reboot.distrib /sbin/poweroff.distrib I use the latter in my customized /etc/init.d/lxc stop rule. Note that the lxc-wait's SHOULD be parallelized, but this is not possible as at lxc 0.7.2 :-( Sure it is. I parallelize the shutdowns (in any version, including 0.7.2) by doing all the lxc-stop in parallel without looking or waiting, then in a separate following step do a loop that waits for no containers running. Here is my openSUSE init.d/lxc: https://build.opensuse.org/package/files?package=lxcproject=home:aljex And the packages: http://download.opensuse.org/repositories/home:/aljex/*/lxc-0.7.2*.rpm It makes assumptions that are wrong for ubuntu and is more limited than you may want in terms of what it even tries to handle. But that's beside the point of parallel shutdowns. * cgroup handling includes a particular stack of override logic for possible cgroup mount points that makes sense to me. - start with built-in default /var/run/lxc/cgroup, and name it lxc so as not to conflict with any other cgroup setup by default. - if you defined something in $LXC_CONF, prefer it over default - if kernel is providing /sys/fs/cgroup automatically, prefer that over either default or $LXC_CONF - if a cgroup named lxc is already mounted, prefer that over all else * assumes lxc 0.7.2 because the script is part of a lxc-0.7.2 rpm - removes the shutdown/reboot watchdog functions that were needed in 0.6.5 but are built in to 0.7.2 now. * only starts containers that are defined by $LXC_ETC/*/config * only shuts down containers that it started * the stop function greps for /sbin/init in container inittab instead of trying to allow for any random container pid #1 * no provision for application/service containers, just whole systems started with /sbin/init * starts containers in screen - I have not figured out what it would take to get nice behavior out of lxc-console yet and screen is both easy and standard. The $LXC_CONF (/etc/lxc/lxc.conf) referenced at the top does not exist usually so everything that happens is visible right in the script. I'm using this in production. So far so good. typical usage: nj10:~ # rclxc status Checking for LXC containers... running nj10:~ # rclxc list Listing LXC containers... 'vps001' is RUNNING 'vps002' is RUNNING 'vps003' is RUNNING 'vps004' is RUNNING 'vps005' is RUNNING 'vps006' is RUNNING 'vps007' is RUNNING 'vps008' is RUNNING 'vps009' is RUNNING 'vps011' is RUNNING 'vps012' is RUNNING 'vps013' is RUNNING nj10:~ # rclxc stop vps008 Shutting down LXC containers... done nj10:~ # rclxc list Listing LXC containers... 'vps001' is RUNNING 'vps002' is RUNNING 'vps003' is RUNNING 'vps004' is RUNNING 'vps005' is RUNNING 'vps006' is RUNNING 'vps007' is RUNNING 'vps008' is STOPPED 'vps009' is RUNNING 'vps011' is RUNNING 'vps012' is RUNNING 'vps013' is RUNNING nj10:~ # rclxc status Checking for LXC containers... running nj10:~ # rclxc stop Shutting down LXC containers...
Re: [Lxc-users] On clean reboot of Ubuntu 10.04 containers
On 12/6/2010 3:01 AM, Trent W. Buck wrote: Trent W. Buck writes: This post describes my attempts to get clean shutdown of Ubuntu 10.04 containers. The goal here is that a shutdown -h now of the dom0 should not result in a potentially inconsistent domU postgres database, cf. a naive lxc-stop. In my previous note about parallel shutdowns, that same system also works for this too. User may ssh in to the container as root and issue shutdown-r now or shutdown -h now and it works as expected from their point of view. No cron job on the host. In lxc 0.6.5 you would have a watchdog process per container that uses inotify to be alerted the instant the containers runlevel file and/or cgroup tasks list file changed. I had that as just a shell function right in the init script. In 0.7.2 this is handled by lxc internally and is rather more reliable, since it was possible to break or kill the separate watchdog processes. I think you are working harder than necessary for some things although it appears you have a legitimate problem with the upstart and tmpfs issue. Whether the fault is lxc's or ubuntu's in that case I can't say because ideally neither should have such a hard coded assumption. -- bkw -- What happens now with your Lotus Notes apps - do you make another costly upgrade, or settle for being marooned without product support? Time to move off Lotus Notes and onto the cloud with Force.com, apps are easier to build, use, and manage than apps on traditional platforms. Sign up for the Lotus Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] On clean shutdown of Ubuntu 10.04 containers
On Mon, 2010-12-06 at 12:38 -0500, Brian K. White wrote: On 12/6/2010 2:42 AM, Trent W. Buck wrote: This post describes my attempts to get clean shutdown of Ubuntu 10.04 containers. The goal here is that a shutdown -h now of the dom0 should not result in a potentially inconsistent domU postgres database, cf. a naive lxc-stop. As at Ubuntu 10.04 with lxc 0.7.2, lxc-start detects that a container has halted by 1) seeing a reboot event incontainer/var/run/utmp; or 2) seeingcontainer's PID 1 terminate. Ubuntu 10.04 simply REQUIRES /var/run to be a tmpfs; this is hard-coded into mountall's (upstart's) /lib/init/fstab. Without it, the most immediate issue is that /var/run/ifstate isn't reaped on reboot, ifup(8) thinks lo (at least) is already configured, and the boot process hangs waiting for the network. Unfortunately, lxc 0.7's utmp detect requires /var/run to NOT be a tmpfs. The shipped lxc-ubuntu script works around this by deleting the ifstate file and not mounting a tmpfs on /var/run, but to me that is simply waiting for something else to assume /var/run is empty. It also doesn't cope with a mountall upgrade rewriting /lib/init/fstab. More or less by accident, I discovered that I can tell lxc-start that the container is ready to halt by crashing upstart: container# kill -SEGV 1 Likewise I can spoof a ctrl-alt-delete event in the container with: dom0# pkill -INT lxc-start I automate the former signalling at the end of shutdowns thusly: chroot $template_dir dpkg-divert --quiet --rename /sbin/reboot chroot $template_dir tee/dev/null /sbin/reboot-EOF #!/bin/bash while getopts nwdfiph opt do [[ f = \$opt ]] exec kill -SEGV 1 done exec -a $0 \$0.distrib \$@ EOF chroot $template_dir chmod +x /sbin/reboot chroot $template_dir ln -s reboot.distrib /sbin/halt.distrib chroot $template_dir ln -s reboot.distrib /sbin/poweroff.distrib I use the latter in my customized /etc/init.d/lxc stop rule. Note that the lxc-wait's SHOULD be parallelized, but this is not possible as at lxc 0.7.2 :-( Sure it is. I parallelize the shutdowns (in any version, including 0.7.2) by doing all the lxc-stop in parallel without looking or waiting, then in a separate following step do a loop that waits for no containers running. Here is my openSUSE init.d/lxc: https://build.opensuse.org/package/files?package=lxcproject=home:aljex And the packages: http://download.opensuse.org/repositories/home:/aljex/*/lxc-0.7.2*.rpm It makes assumptions that are wrong for ubuntu and is more limited than you may want in terms of what it even tries to handle. But that's beside the point of parallel shutdowns. Also wrong for Fedora. * cgroup handling includes a particular stack of override logic for possible cgroup mount points that makes sense to me. - start with built-in default /var/run/lxc/cgroup, and name it lxc so as not to conflict with any other cgroup setup by default. - if you defined something in $LXC_CONF, prefer it over default - if kernel is providing /sys/fs/cgroup automatically, prefer that over either default or $LXC_CONF - if a cgroup named lxc is already mounted, prefer that over all else I'm not quite sure if I would put those last two in that order. Especially after the last little discussion over on LKML over the per tty cgroups in the kernel vs in user space, I think I would let the kernel defined /sys/fs/cgroup trump all else if it exists. Something that's been mounted may not have been mounted with all the options you may want, but I'm not sure how much difference that's going to make. I would think the kernel definition would be preferable. Is there something specific you had in mind that would lead you to want to override that? * assumes lxc 0.7.2 because the script is part of a lxc-0.7.2 rpm - removes the shutdown/reboot watchdog functions that were needed in 0.6.5 but are built in to 0.7.2 now. * only starts containers that are defined by $LXC_ETC/*/config Yeah, that's something where I wish we had an onboot and/or disabled config file like OpenVZ does. So you can have some configured but that don't autoboot when you boot the system. As that stands, you would have to rename or remove the config file. :-P * only shuts down containers that it started I don't quite see that as happening literally as described. Looks like it's going to shut down any container for which it can find a powerfail init, even if it was started by some other means, say manually. It doesn't seem to be actually tracking what ones it started. Granted, during normal operation, you're going to try to start everything with a config but it looks like it will shut down manually started containers as well, even if they are not listed with configs and would not even show up in your status commands. I don't even
Re: [Lxc-users] On clean shutdown of Ubuntu 10.04 containers
On 12/6/2010 3:34 PM, Michael H. Warfield wrote: On Mon, 2010-12-06 at 12:38 -0500, Brian K. White wrote: On 12/6/2010 2:42 AM, Trent W. Buck wrote: This post describes my attempts to get clean shutdown of Ubuntu 10.04 containers. The goal here is that a shutdown -h now of the dom0 should not result in a potentially inconsistent domU postgres database, cf. a naive lxc-stop. As at Ubuntu 10.04 with lxc 0.7.2, lxc-start detects that a container has halted by 1) seeing a reboot event incontainer/var/run/utmp; or 2) seeingcontainer's PID 1 terminate. Ubuntu 10.04 simply REQUIRES /var/run to be a tmpfs; this is hard-coded into mountall's (upstart's) /lib/init/fstab. Without it, the most immediate issue is that /var/run/ifstate isn't reaped on reboot, ifup(8) thinks lo (at least) is already configured, and the boot process hangs waiting for the network. Unfortunately, lxc 0.7's utmp detect requires /var/run to NOT be a tmpfs. The shipped lxc-ubuntu script works around this by deleting the ifstate file and not mounting a tmpfs on /var/run, but to me that is simply waiting for something else to assume /var/run is empty. It also doesn't cope with a mountall upgrade rewriting /lib/init/fstab. More or less by accident, I discovered that I can tell lxc-start that the container is ready to halt by crashing upstart: container# kill -SEGV 1 Likewise I can spoof a ctrl-alt-delete event in the container with: dom0# pkill -INT lxc-start I automate the former signalling at the end of shutdowns thusly: chroot $template_dir dpkg-divert --quiet --rename /sbin/reboot chroot $template_dir tee/dev/null /sbin/reboot-EOF #!/bin/bash while getopts nwdfiph opt do [[ f = \$opt ]] exec kill -SEGV 1 done exec -a $0 \$0.distrib \$@ EOF chroot $template_dir chmod +x /sbin/reboot chroot $template_dir ln -s reboot.distrib /sbin/halt.distrib chroot $template_dir ln -s reboot.distrib /sbin/poweroff.distrib I use the latter in my customized /etc/init.d/lxc stop rule. Note that the lxc-wait's SHOULD be parallelized, but this is not possible as at lxc 0.7.2 :-( Sure it is. I parallelize the shutdowns (in any version, including 0.7.2) by doing all the lxc-stop in parallel without looking or waiting, then in a separate following step do a loop that waits for no containers running. Here is my openSUSE init.d/lxc: https://build.opensuse.org/package/files?package=lxcproject=home:aljex And the packages: http://download.opensuse.org/repositories/home:/aljex/*/lxc-0.7.2*.rpm It makes assumptions that are wrong for ubuntu and is more limited than you may want in terms of what it even tries to handle. But that's beside the point of parallel shutdowns. Also wrong for Fedora. * cgroup handling includes a particular stack of override logic for possible cgroup mount points that makes sense to me. - start with built-in default /var/run/lxc/cgroup, and name it lxc so as not to conflict with any other cgroup setup by default. - if you defined something in $LXC_CONF, prefer it over default - if kernel is providing /sys/fs/cgroup automatically, prefer that over either default or $LXC_CONF - if a cgroup named lxc is already mounted, prefer that over all else I'm not quite sure if I would put those last two in that order. Especially after the last little discussion over on LKML over the per tty cgroups in the kernel vs in user space, I think I would let the kernel defined /sys/fs/cgroup trump all else if it exists. Something that's been mounted may not have been mounted with all the options you may want, but I'm not sure how much difference that's going to make. I would think the kernel definition would be preferable. Is there something specific you had in mind that would lead you to want to override that? * assumes lxc 0.7.2 because the script is part of a lxc-0.7.2 rpm - removes the shutdown/reboot watchdog functions that were needed in 0.6.5 but are built in to 0.7.2 now. * only starts containers that are defined by $LXC_ETC/*/config Yeah, that's something where I wish we had an onboot and/or disabled config file like OpenVZ does. So you can have some configured but that don't autoboot when you boot the system. As that stands, you would have to rename or remove the config file. :-P * only shuts down containers that it started I don't quite see that as happening literally as described. Looks like it's going to shut down any container for which it can find a powerfail init, even if it was started by some other means, say manually. It doesn't seem to be actually tracking what ones it started. Granted, during normal operation, you're going to try to start everything with a config but it looks like it will shut down manually started containers as well, even if they are not listed with configs and would not even show up in your status
Re: [Lxc-users] On clean shutdown of Ubuntu 10.04 containers
Serge E. Hallyn serge.hal...@canonical.com writes: Quoting Trent W. Buck (t...@cybersource.com.au): Unfortunately, lxc 0.7's utmp detect requires /var/run to NOT be a tmpfs. The shipped lxc-ubuntu script works around this by deleting the ifstate file and not mounting a tmpfs on /var/run, but to me that is simply waiting for something else to assume /var/run is empty. It also doesn't cope with a mountall upgrade rewriting /lib/init/fstab. Note that how to handle clean shutdown is one of the open topics. So if you're interested in helping to properly fix that, please feel free to join our development call this coming thursday morning, as per announcement last week :) IIRC Daniel L. had a kernel patch to help fix this workaround? He was experimenting with it at UDS-N. Sure, but I feel safer patching container's userland than the production server's kernel, so for me the right fix will need to wait until April 2012 (the next Ubuntu LTS). -- What happens now with your Lotus Notes apps - do you make another costly upgrade, or settle for being marooned without product support? Time to move off Lotus Notes and onto the cloud with Force.com, apps are easier to build, use, and manage than apps on traditional platforms. Sign up for the Lotus Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] On clean shutdown of Ubuntu 10.04 containers
Brian K. White br...@aljex.com writes: On 12/6/2010 2:42 AM, Trent W. Buck wrote: I use the latter in my customized /etc/init.d/lxc stop rule. Note that the lxc-wait's SHOULD be parallelized, but this is not possible as at lxc 0.7.2 :-( Sure it is. Sorry, I meant lxc-wait(8) cannot be parallelized. I parallelize the shutdowns (in any version, including 0.7.2) by doing all the lxc-stop in parallel without looking or waiting, Correct me if I'm wrong, but lxc-stop does NOT do a clean shutdown -- it just kill -9's all the processes in the container. then in a separate following step do a loop that waits for no containers running. I suppose I could do that. Here is my openSUSE init.d/lxc: https://build.opensuse.org/package/files?package=lxcproject=home:aljex And the packages: http://download.opensuse.org/repositories/home:/aljex/*/lxc-0.7.2*.rpm Thanks, I'll take a look at that. -- What happens now with your Lotus Notes apps - do you make another costly upgrade, or settle for being marooned without product support? Time to move off Lotus Notes and onto the cloud with Force.com, apps are easier to build, use, and manage than apps on traditional platforms. Sign up for the Lotus Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] On clean shutdown of Ubuntu 10.04 containers
Michael H. Warfield m...@wittsend.com writes: Yeah, that's something where I wish we had an onboot and/or disabled config file like OpenVZ does. So you can have some configured but that don't autoboot when you boot the system. As that stands, you would have to rename or remove the config file. :-P As a workaround I have my init script look for /forceboot inside the container's rootfs. I was going to have an entry like upstart.autostart = [yes|no] in the lxc.conf file, but lxc-start doesn't like unrecognized entries. * the stop function greps for /sbin/init in container inittab instead of trying to allow for any random container pid #1 Yeah, that little trick won't work with Ubuntu or Fedora using upstart. AFAIK init inside the container *MUST* be pid 1. pid 1 is handled specially by the kernel, such silently discarding all kill -9's sent to it. * starts containers in screen - I have not figured out what it would take to get nice behavior out of lxc-console yet and screen is both easy and standard. I usually start them with logging enabled and redirected with the -o option. To each his own. I set lxc.console = /var/log/lxc/$utsname.log, which captures the container's /dev/console. lxc-console attaches to gettys on the container's /dev/tty[1-6]. If something is wrong in the start script, I can run lxc-start manually, replacing -d with -s lxc.console=/dev/stderr to get an interactive /dev/console. -- What happens now with your Lotus Notes apps - do you make another costly upgrade, or settle for being marooned without product support? Time to move off Lotus Notes and onto the cloud with Force.com, apps are easier to build, use, and manage than apps on traditional platforms. Sign up for the Lotus Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] On clean shutdown of Ubuntu 10.04 containers
Michael H. Warfield m...@wittsend.com writes: Ubuntu 10.04 simply REQUIRES /var/run to be a tmpfs; this is hard-coded into mountall's (upstart's) /lib/init/fstab. Are you absolutely SURE about this? I was under the impression this was under control of the /etc/default/rcS file and the RAMRUN option. I set both that and RAMLOCK to no and didn't think I was having any problems with it but I'm not sure if that was specifically a 10.04 container I was testing with. I'll have to reverify to see if they've changed that. That should really be consider a bug, if true. Nothing should require something be on tmpfs. RAMRUN and RAMLOCK are Debian options. The code that handles them in Debian sysvinit is replaced by mountall(8), and the references to them in the rcS manpage are removed as at 10.04. Further, in Debian RAMRUN and RAMLOCK are undefined by default, and setting them to yes is what causes the sysvinit boot process to mount them. #ubuntu-server agreed with that analysis, but I haven't personally tried setting RAMRUN=no to confirm it (have to scounge a non-production lucid box...). I *do* know that adding/removing the entry from /lib/init/fstab, which is a mountallism, resulted in my containers having/not having a tmpfs mount on /var/run. -- What happens now with your Lotus Notes apps - do you make another costly upgrade, or settle for being marooned without product support? Time to move off Lotus Notes and onto the cloud with Force.com, apps are easier to build, use, and manage than apps on traditional platforms. Sign up for the Lotus Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] On clean reboot of Ubuntu 10.04 containers
Brian K. White br...@aljex.com writes: On 12/6/2010 3:01 AM, Trent W. Buck wrote: Trent W. Buck writes: This post describes my attempts to get clean shutdown of Ubuntu 10.04 containers. The goal here is that a shutdown -h now of the dom0 should not result in a potentially inconsistent domU postgres database, cf. a naive lxc-stop. In my previous note about parallel shutdowns, that same system also works for this too. User may ssh in to the container as root and issue shutdown-r now or shutdown -h now and it works as expected from their point of view. No cron job on the host. In lxc 0.6.5 you would have a watchdog process per container that uses inotify to be alerted the instant the containers runlevel file and/or cgroup tasks list file changed. I had that as just a shell function right in the init script. In 0.7.2 this is handled by lxc internally and is rather more reliable, since it was possible to break or kill the separate watchdog processes. I think that handling relies on /var/run/utmp not being hidden inside a tmpfs, which is why the shipped lxc-ubuntu template goes out of its way to avoid this. I know that shutdown -h now *does* work correctly if /var/run isn't a tmpfs; it hadn't occurred to me that this might also handle shutdown -r now correctly -- by the time I came to care about rebooting containers, I'd already decided that having a tmpfs was more important than lxc-start's autodetection. I might revise that opinion after trying to handle rebooting manually -- particularly since I've decided to administratively prohibit sys_admin inside containers. -- What happens now with your Lotus Notes apps - do you make another costly upgrade, or settle for being marooned without product support? Time to move off Lotus Notes and onto the cloud with Force.com, apps are easier to build, use, and manage than apps on traditional platforms. Sign up for the Lotus Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] On clean reboot of Ubuntu 10.04 containers
I rewrote my init script, too, and now I'm pretty happy. Example
interactions below. In the second case, I've dialled the timeout down
to 2s to simulate hung containers.
r...@omega:~# /etc/init.d/lxc restart
Stopping LXC containers: krb-client failed!
Killing LXC containers: kdc ldap.
Starting LXC containers: kdc krb-client ldap.
r...@omega:~# /etc/init.d/lxc restart
Stopping LXC containers: krb-client kdc ldap.
Starting LXC containers: kdc krb-client ldap.
r...@omega:~# /etc/init.d/lxc status
'kdc' is RUNNING
'krb-client' is RUNNING
'ldap' is RUNNING
r...@omega:~#
This uses -INT to signal to the container, so it assumes that containers
are upstart-based and have s/shutdown -r/shutdown -r/ in
/etc/init/control-alt-delete.conf (which is what -INT triggers).
I tried -PWR as was suggested upthread from SuSE, but that appears to be
ignored by upstart (sigh).
#!/bin/bash
# lxc containers starter script
# based on skeleton from Debian GNU/Linux
### BEGIN INIT INFO
# Provides: lxc
# Required-Start:$syslog $remote_fs
# Required-Stop: $syslog $remote_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Linux Containers
# Description: Linux Containers
### END INIT INFO
NAME=lxc
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
DESC=LXC containers
SCRIPTNAME=/etc/init.d/lxc
. /lib/lsb/init-functions
if [ -f /etc/default/$NAME ] ; then
. /etc/default/$NAME
fi
if [ x$RUN != xyes ] ; then
log_success_msg $NAME init script disabled; edit /etc/default/$NAME
exit 0
fi
shopt -s nullglob
case $1 in
start)
log_daemon_msg Starting $DESC
for name in $ROOT_DIR/*/forceboot
do name=${name%/forceboot}
name=${name#$ROOT_DIR/}
# Skip it if it doesn't have an RFC952 hostname (e.g. no fooserv.~1~'s)
[[ $name =~ ^[[:alnum:]]([[:alnum:]-]{0,61}[[:alnum:]])?$ ]] || continue
lxc-start -n $name -f $CONF_DIR/$name.conf -d
log_progress_msg $name
done
log_end_msg 0
;;
stop)
log_daemon_msg Stopping $DESC
pkill -INT lxc-start # Sends a ctrl-alt-del to containers' upstarts.
i=300 # Begin a spinlock, capped at five minutes.
while
then=$now now=$(lxc-ls | sort)
# Print the lines that have disappeared since the previous iteration.
for name in $(comm -23 (cat $then) (cat $now))
do log_progress_msg $name
done
[[ -n $now ]] ((i--))
do sleep 1
done
if [[ -z $now ]]
then log_end_msg 0
else log_end_msg 1
log_daemon_msg Killing $DESC
for name in $now
do lxc-stop -n $name
log_progress_msg $name
done
log_end_msg 0
fi
;;
restart|force-reload)
$0 stop
$0 start
;;
status)
for name in $ROOT_DIR/*/forceboot
do name=${name%/forceboot}
name=${name#$ROOT_DIR/}
# Skip it if it doesn't have an RFC952 hostname (e.g. no fooserv.~1~'s)
[[ $name =~ ^[[:alnum:]]([[:alnum:]-]{0,61}[[:alnum:]])?$ ]] || continue
lxc-info -n $name
done
;;
*)
log_success_msg Usage: $SCRIPTNAME {start|stop|force-reload|restart|status}
exit 1
;;
esac
exit 0
--
What happens now with your Lotus Notes apps - do you make another costly
upgrade, or settle for being marooned without product support? Time to move
off Lotus Notes and onto the cloud with Force.com, apps are easier to build,
use, and manage than apps on traditional platforms. Sign up for the Lotus
Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
