Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
On Mon, 2010-01-25 at 23:19 -0500, Brian K. White wrote:
> From lxc-check/lxc-waiton-init:
> > # We use to be able to use lxc-ls but it now prints active VM's twice
> Only if you ran lxc-create and then lxc-start.
Very true and that's been discussed a bit on the -devel list.
> If you just run...
> lxc-start -f config -n name
> ...with no prior lxc-create, then lxc-ls only shows the container name once.
Also very true. But, if you haven't done that then none of my scripts
will work at all, IAC, because they are all predicated upon being able
to find the hot configuration in ${LXC_VAR}/${VM} particularly to find
the rootfs and dereferences through the fstab. End of discussion. If
you don't do that or have some other standard location for the config
files, I don't see how any of it will work, so it's not even really
worth worrying about on my part and I'm not really interested in
supporting those cases. In that case, please feel free to use my
scripts as an example of how you might roll your own for your own way of
managing things. Which is all they are really offered up for, anyways.
I do try to comment the daylights out of them so others have some
insight into my thinking when I wrote them that way and why.
> Obviously assumes 0.6.5+ or whatever version first allowed skipping
> lxc-create.
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
From lxc-check/lxc-waiton-init: > # We use to be able to use lxc-ls but it now prints active VM's twice Only if you ran lxc-create and then lxc-start. If you just run... lxc-start -f config -n name ...with no prior lxc-create, then lxc-ls only shows the container name once. Obviously assumes 0.6.5+ or whatever version first allowed skipping lxc-create. -- bkw -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
this is great stuff, thanks for sharing your script and to Daniel for
the inotifywait trick; i've gotten it to work in my setup.
looking forward to a Just Works (kernel)? solution :-) but this is
working splendidly, thanks again.
On Mon, Jan 25, 2010 at 7:50 PM, Michael H. Warfield wrote:
> On Tue, 2010-01-26 at 00:42 +0100, Daniel Lezcano wrote:
>> I trick I just found:
>>
>> while $(true); do
>> inotifywait /var/lib/lxc/debian/rootfs/var/run/utmp;
>> if [ "$(wc -l /cgroup/debian/tasks | awk '{ print $1 }')" = "1" ]; then
>> lxc-stop -n debian
>> fi;
>> done
>
>> This command can stay always there and it will trigger a lxc-stop when
>> the container remains with 1 process.
>> No polling and immediate :)
>> At the first glance, it seems to work well, but of course not compatible
>> with upstart.
>
> Oh BABY! I like. I like mucho!
>
> I've just cobbled together an "lcx-waiton-init" script that incorporates
> that. After starting a container, running that script will monitor the
> container and, if it reboots it, restart the container and continue
> monitoring it, with a new log file (my peculiarity). If it halts, it
> shuts the container down and terminates. It adds a double quiet option
> and a timeout option to the inotifywait just in case. I'm going to
> integrate that into my own startup wrapper scripts so it runs
> automatically when a container is started.
>
> Mui Bien! Gracias!
>
> Attached.
>
> Mike
> --
> Michael H. Warfield (AI4NB) | (770) 985-6132 | ...@wittsend.com
> /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
> NIC whois: MHW9 | An optimist believes we live in the best of all
> PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
>
> --
> The Planet: dedicated and managed hosting, cloud storage, colocation
> Stay online with enterprise data centers and the best network in the business
> Choose flexible plans and management services without long-term contracts
> Personal 24x7 support from experience hosting pros just a phone call away.
> http://p.sf.net/sfu/theplanet-com
> ___
> Lxc-users mailing list
> Lxc-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/lxc-users
>
>
--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
On Tue, 2010-01-26 at 00:42 +0100, Daniel Lezcano wrote:
> I trick I just found:
>
> while $(true); do
> inotifywait /var/lib/lxc/debian/rootfs/var/run/utmp;
> if [ "$(wc -l /cgroup/debian/tasks | awk '{ print $1 }')" = "1" ]; then
> lxc-stop -n debian
> fi;
> done
> This command can stay always there and it will trigger a lxc-stop when
> the container remains with 1 process.
> No polling and immediate :)
> At the first glance, it seems to work well, but of course not compatible
> with upstart.
Oh BABY! I like. I like mucho!
I've just cobbled together an "lcx-waiton-init" script that incorporates
that. After starting a container, running that script will monitor the
container and, if it reboots it, restart the container and continue
monitoring it, with a new log file (my peculiarity). If it halts, it
shuts the container down and terminates. It adds a double quiet option
and a timeout option to the inotifywait just in case. I'm going to
integrate that into my own startup wrapper scripts so it runs
automatically when a container is started.
Mui Bien! Gracias!
Attached.
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
lxc-waiton-init
Description: application/shellscript
signature.asc
Description: This is a digitally signed message part
--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
On Mon, 2010-01-25 at 20:50 -0500, Michael H. Warfield wrote: > Mui Bien! Gracias! Or... I really should have responded... Merci beaucop. Studying Spanish. Haven't had French since high school (and my Russian is real rusty). But I'm trying... > Attached. > Mike Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
On Tue, 2010-01-26 at 01:37 +0100, Daniel Lezcano wrote: > Ah, ok didn't know upstart kept using utmp for compatibility. Interesting. Too much depends on that entire stuff. Thou shalt NOT break Posix compliance. Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
On Tue, 2010-01-26 at 01:50 +0100, Daniel Lezcano wrote:
> Michael H. Warfield wrote:
> > On Tue, 2010-01-26 at 00:42 +0100, Daniel Lezcano wrote:
> >
> >
> >> I trick I just found:
> >>
> >> while $(true); do
> >> inotifywait /var/lib/lxc/debian/rootfs/var/run/utmp;
> >> if [ "$(wc -l /cgroup/debian/tasks | awk '{ print $1 }')" = "1" ]; then
> >> lxc-stop -n debian
> >> fi;
> >> done
> >>
> >
> > Seems to be a problem with that approach. I'm not seeing anythign
> > in /var/lib/lxc/${VM}/rootfs any more. Use to with 0.6.4 and earlier
> > but not now. That seems to have "gone away". That might have been a
> > way to get around that tmpfs problem but it's not there and I have to
> > resort back to the real rootfs.
> >
> With a version > 0.6.4 the lxc-debian script write the rootfs in the
> /var/lib/lxc/debian/rootfs.
> With a version <= 0.6.4 the rootfs is created in the current directory.
> Note if you have the source code from git, I incremented the version
> right before the tag 0.6.5, so you may have a pre-0.6.5 but with the
> 0.6.4 version information.
Ah! If you look at that lxc-check script I had attached, it tries to
find the rootfs by looking up the config and then, if necessary,
dereferencing through any mount points for that rootfs that may be in
the VM fstab (part of my OpenVZ ve -> lxc conversion and compatiblity)
so I guess I've got that covered since I go for the ${ROOTFS}. Try and
keep things as generic as possible.
Thanks for that clarification!
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
On Tue, 2010-01-26 at 00:42 +0100, Daniel Lezcano wrote:
> Michael H. Warfield wrote:
> > On Mon, 2010-01-25 at 21:50 +0100, Daniel Lezcano wrote:
> >
> >
> >>> apologies for the length, but how is everyone else handling this?
> >>> this is the last thing i need to solve before i actually start running
> >>> all my services on this setup.
> >>>
> >>>
> >> I was wondering if the kernel shouldn't send a signal to the init's
> >> parent when sys_reboot is called.
> >>
> >
> > Which still leaves open the question of telling the difference between a
> > halt and a reboot. My trick of using the final runlevel
> > in /var/run/utmp ran a foul over a gotcha in the Debian containers that
> > they seem to default to mounting tmpfs over /var/run and /var/lock so
> > you loose that information. I had to disable "RAMRUN" and "RAMLOCK"
> > in /etc/default/rcS in the debian images to get around that but I'm not
> > sure I'm happy doing that. The alternative examining /var/log/wtmp
> > didn't work out as reliable. OpenVZ has a similar problem and it writes
> > a "reboot" file that can be read but that seems inelegant as well. I
> > don't think anything works if someone does a "reboot -f" but I need to
> > test that condition yet.
> >
> > To also answer the OP's question. Here's what I use. I have a script
> > that runs periodically in the host. If the number of processes in a
> > running container is 1, then I check for the runlevel in
> > ${rootfs}/var/run/utmp. If that's "? 0" then it's a halt and I run
> > lxc-stop. If it's "? 6" then it's a reboot and I stop the container and
> > then restart it. I run it every 5 minutes or so or manually. It runs
> > fast and could be run more often. Just sucks polling things like that,
> > though. That script, lxc-check, is attached.
> >
> I trick I just found:
> while $(true); do
> inotifywait /var/lib/lxc/debian/rootfs/var/run/utmp;
> if [ "$(wc -l /cgroup/debian/tasks | awk '{ print $1 }')" = "1" ]; then
> lxc-stop -n debian
> fi;
> done
Oh I LIKE THAT!
Just a couple of comments...
1) Still have the problem with the Debian default of mounting tmpfs
on /var/run.
2) Probably still need some sort of deadman's switch in case things die
and didn't update utmp.
3) That "if" check is overly complicated. You don't really need the
awk. (nit => pick)
4) Still need to separate halt and reboot. If utmp is valid for
detecting changes then utmp is valid for doing a runlevel query and we
can tee off from that.
But I like it. I can incorporate that into my start-up wrapper in a
heartbeat.
> This command can stay always there and it will trigger a lxc-stop when
> the container remains with 1 process.
> No polling and immediate :)
> At the first glance, it seems to work well, but of course not compatible
> with upstart.
Why isn't it compatible with upstart? I thought utmp was being
maintained properly even with upstart. I know upstart doesn't have the
same concept of runlevels and maintains them purely for "compatibility"
but I thought the whole utmp/wtmp was part of that. Wouldn't this also
work equally well with .../var/log/wtmp instead (other than the fact
that you're going to wake up on every damn wtmp record)? I'll have to
test.
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
On Tue, 2010-01-26 at 00:42 +0100, Daniel Lezcano wrote:
> I trick I just found:
>
> while $(true); do
> inotifywait /var/lib/lxc/debian/rootfs/var/run/utmp;
> if [ "$(wc -l /cgroup/debian/tasks | awk '{ print $1 }')" = "1" ]; then
> lxc-stop -n debian
> fi;
> done
Seems to be a problem with that approach. I'm not seeing anythign
in /var/lib/lxc/${VM}/rootfs any more. Use to with 0.6.4 and earlier
but not now. That seems to have "gone away". That might have been a
way to get around that tmpfs problem but it's not there and I have to
resort back to the real rootfs.
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
Michael H. Warfield wrote:
> On Tue, 2010-01-26 at 00:42 +0100, Daniel Lezcano wrote:
>
>
>> I trick I just found:
>>
>> while $(true); do
>> inotifywait /var/lib/lxc/debian/rootfs/var/run/utmp;
>> if [ "$(wc -l /cgroup/debian/tasks | awk '{ print $1 }')" = "1" ]; then
>> lxc-stop -n debian
>> fi;
>> done
>>
>
> Seems to be a problem with that approach. I'm not seeing anythign
> in /var/lib/lxc/${VM}/rootfs any more. Use to with 0.6.4 and earlier
> but not now. That seems to have "gone away". That might have been a
> way to get around that tmpfs problem but it's not there and I have to
> resort back to the real rootfs.
>
With a version > 0.6.4 the lxc-debian script write the rootfs in the
/var/lib/lxc/debian/rootfs.
With a version <= 0.6.4 the rootfs is created in the current directory.
Note if you have the source code from git, I incremented the version
right before the tag 0.6.5, so you may have a pre-0.6.5 but with the
0.6.4 version information.
--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
Michael H. Warfield wrote:
> On Tue, 2010-01-26 at 00:42 +0100, Daniel Lezcano wrote:
>
>> Michael H. Warfield wrote:
>>
>>> On Mon, 2010-01-25 at 21:50 +0100, Daniel Lezcano wrote:
>>>
>>>
>>>
> apologies for the length, but how is everyone else handling this?
> this is the last thing i need to solve before i actually start running
> all my services on this setup.
>
>
>
I was wondering if the kernel shouldn't send a signal to the init's
parent when sys_reboot is called.
>>> Which still leaves open the question of telling the difference between a
>>> halt and a reboot. My trick of using the final runlevel
>>> in /var/run/utmp ran a foul over a gotcha in the Debian containers that
>>> they seem to default to mounting tmpfs over /var/run and /var/lock so
>>> you loose that information. I had to disable "RAMRUN" and "RAMLOCK"
>>> in /etc/default/rcS in the debian images to get around that but I'm not
>>> sure I'm happy doing that. The alternative examining /var/log/wtmp
>>> didn't work out as reliable. OpenVZ has a similar problem and it writes
>>> a "reboot" file that can be read but that seems inelegant as well. I
>>> don't think anything works if someone does a "reboot -f" but I need to
>>> test that condition yet.
>>>
>>> To also answer the OP's question. Here's what I use. I have a script
>>> that runs periodically in the host. If the number of processes in a
>>> running container is 1, then I check for the runlevel in
>>> ${rootfs}/var/run/utmp. If that's "? 0" then it's a halt and I run
>>> lxc-stop. If it's "? 6" then it's a reboot and I stop the container and
>>> then restart it. I run it every 5 minutes or so or manually. It runs
>>> fast and could be run more often. Just sucks polling things like that,
>>> though. That script, lxc-check, is attached.
>>>
>>>
>
>
>> I trick I just found:
>>
>
>
>> while $(true); do
>> inotifywait /var/lib/lxc/debian/rootfs/var/run/utmp;
>> if [ "$(wc -l /cgroup/debian/tasks | awk '{ print $1 }')" = "1" ]; then
>> lxc-stop -n debian
>> fi;
>> done
>>
>
> Oh I LIKE THAT!
>
> Just a couple of comments...
>
> 1) Still have the problem with the Debian default of mounting tmpfs
> on /var/run.
>
> 2) Probably still need some sort of deadman's switch in case things die
> and didn't update utmp.
>
> 3) That "if" check is overly complicated. You don't really need the
> awk. (nit => pick)
>
> 4) Still need to separate halt and reboot. If utmp is valid for
> detecting changes then utmp is valid for doing a runlevel query and we
> can tee off from that.
>
It's a quick and dirty lines of script. For example, I don't know when
the init process write to utmp at startup. I guess it already forked a
process otherwise this script will kill the container when it starts.
But I suppose it's simple to check the runlevel at this point.
> But I like it. I can incorporate that into my start-up wrapper in a
> heartbeat.
>
>
>> This command can stay always there and it will trigger a lxc-stop when
>> the container remains with 1 process.
>> No polling and immediate :)
>> At the first glance, it seems to work well, but of course not compatible
>> with upstart.
>>
>
> Why isn't it compatible with upstart? I thought utmp was being
> maintained properly even with upstart. I know upstart doesn't have the
> same concept of runlevels and maintains them purely for "compatibility"
> but I thought the whole utmp/wtmp was part of that. Wouldn't this also
> work equally well with .../var/log/wtmp instead (other than the fact
> that you're going to wake up on every damn wtmp record)? I'll have to
> test.
>
Ah, ok didn't know upstart kept using utmp for compatibility. Interesting.
--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
Michael H. Warfield wrote: > On Mon, 2010-01-25 at 23:39 +0100, Daniel Lezcano wrote: > >> Michael H. Warfield wrote: >> >>> On Mon, 2010-01-25 at 21:50 +0100, Daniel Lezcano wrote: >>> >>> >>> > apologies for the length, but how is everyone else handling this? > this is the last thing i need to solve before i actually start running > all my services on this setup. > > > I was wondering if the kernel shouldn't send a signal to the init's parent when sys_reboot is called. >>> Which still leaves open the question of telling the difference between a >>> halt and a reboot. >>> >> Well, with the correct information in siginfo, that should do the trick: >> > > >> si_num = SIGINFO ? SIGHUP ? >> si_code = SI_KERNEL >> si_int = the "cmd" passed to the reboot (2) function. >> > > I concur that sounds like a good option. But that's a kernel mod and > will require a kernel patch and getting that through the process. Once > that's agreed on that's the route to go, we've got to get the containers > guys involved and get that pushed through. And is this going to work > without any modifications to init itself (per the discussion over on the > -devel list wrt modifications to init and the difficulty and pain of > pulling teeth). What's the next step? > Send a patch with this hack even if it is not the right approach, let's receive some flaming and discuss with containers@/lkml@ about this problem. As I have, one foot in userspace with lxc and another foot in the container kernel development, if we reach a consensus, that should not be a big deal to push upstream a patch, especially if this is a blocker for the container technology. The objective is to have a kernel patch making possible to support the "shutdown / halt / reboot / etc ... " without modifying the init command and compatible with sysv init and upstart. The patch I propose is to send a signal to the parent process of the pid namespace, in our case it's lxc-start. Handling this signal is quite easy as we have just "kill -9" the init process and, in case of a reboot, return to the starting code without exiting lxc-start. -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
Michael H. Warfield wrote:
> On Mon, 2010-01-25 at 21:50 +0100, Daniel Lezcano wrote:
>
>
>>> apologies for the length, but how is everyone else handling this?
>>> this is the last thing i need to solve before i actually start running
>>> all my services on this setup.
>>>
>>>
>> I was wondering if the kernel shouldn't send a signal to the init's
>> parent when sys_reboot is called.
>>
>
> Which still leaves open the question of telling the difference between a
> halt and a reboot. My trick of using the final runlevel
> in /var/run/utmp ran a foul over a gotcha in the Debian containers that
> they seem to default to mounting tmpfs over /var/run and /var/lock so
> you loose that information. I had to disable "RAMRUN" and "RAMLOCK"
> in /etc/default/rcS in the debian images to get around that but I'm not
> sure I'm happy doing that. The alternative examining /var/log/wtmp
> didn't work out as reliable. OpenVZ has a similar problem and it writes
> a "reboot" file that can be read but that seems inelegant as well. I
> don't think anything works if someone does a "reboot -f" but I need to
> test that condition yet.
>
> To also answer the OP's question. Here's what I use. I have a script
> that runs periodically in the host. If the number of processes in a
> running container is 1, then I check for the runlevel in
> ${rootfs}/var/run/utmp. If that's "? 0" then it's a halt and I run
> lxc-stop. If it's "? 6" then it's a reboot and I stop the container and
> then restart it. I run it every 5 minutes or so or manually. It runs
> fast and could be run more often. Just sucks polling things like that,
> though. That script, lxc-check, is attached.
>
I trick I just found:
while $(true); do
inotifywait /var/lib/lxc/debian/rootfs/var/run/utmp;
if [ "$(wc -l /cgroup/debian/tasks | awk '{ print $1 }')" = "1" ]; then
lxc-stop -n debian
fi;
done
This command can stay always there and it will trigger a lxc-stop when
the container remains with 1 process.
No polling and immediate :)
At the first glance, it seems to work well, but of course not compatible
with upstart.
linux-owop:~ # lxc-start -n debian && reset; echo
" exited ##"
SELinux: Could not open policy file <=
/etc/selinux/targeted/policy/policy.24:
No such file or directory
INIT: version 2.86 booting
Activating swap...done.
Cleaning up ifupdown
Loading kernel modules...FATAL: Could not load
/lib/modules/2.6.32-mcr-3.18/modu
les.dep: No such file or directory
Checking file systems...fsck 1.41.3 (12-Oct-2008)
done.
Setting kernel variables (/etc/sysctl.conf)...done.
Mounting local filesystems...done.
Activating swapfile swap...done.
Setting up networking
Configuring network interfaces...Internet Systems Consortium DHCP Client
V3.1.1
Copyright 2004-2008 Internet Systems Consortium.
All rights reserved.
For info, please visit http://www.isc.org/sw/dhcp/
Listening on LPF/eth0/d6:c8:78:f7:09:12
Sending on LPF/eth0/d6:c8:78:f7:09:12
Sending on Socket/fallback
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 7
DHCPOFFER from 172.20.0.1
DHCPREQUEST on eth0 to 255.255.255.255 port 67
DHCPACK from 172.20.0.1
bound to 172.20.0.10 -- renewal in 34492 seconds.
done.
INIT: Entering runlevel: 3
Starting OpenBSD Secure Shell server: sshd.
Debian GNU/Linux 5.0 debian console
debian login: root
Last login: Mon Jan 25 23:28:34 UTC 2010 on console
Linux debian 2.6.32-mcr-3.18 #19 Mon Jan 25 11:19:47 CET 2010 x86_64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
debian:~# poweroff
Broadcast message from r...@debian (console) (Mon Jan 25 23:32:07 2010):
The system is going down for system halt NOW!
INIT: Switching to runlevel: 0
INIT: Sending processes the TERM signal
debian:~# Asking all remaining processes to terminate...done.
Killing all remaining processes...failed.
Deconfiguring network interfaces...There is already a pid file
/var/run/dhclient.eth0.pid with pid 187
removed stale PID file
Internet Systems Consortium DHCP Client V3.1.1
Copyright 2004-2008 Internet Systems Consortium.
All rights reserved.
For info, please visit http://www.isc.org/sw/dhcp/
Listening on LPF/eth0/d6:c8:78:f7:09:12
Sending on LPF/eth0/d6:c8:78:f7:09:12
Sending on Socket/fallback
DHCPRELEASE on eth0 to 172.20.0.1 port 67
done.
Cleaning up ifupdown
mount: / is busy
Will now halt.
INIT: no more processes left in this runlevel
exited ##
linux-owop:~ #
--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
On Mon, 2010-01-25 at 23:39 +0100, Daniel Lezcano wrote: > Michael H. Warfield wrote: > > On Mon, 2010-01-25 at 21:50 +0100, Daniel Lezcano wrote: > > > > > >>> apologies for the length, but how is everyone else handling this? > >>> this is the last thing i need to solve before i actually start running > >>> all my services on this setup. > >>> > >>> > >> I was wondering if the kernel shouldn't send a signal to the init's > >> parent when sys_reboot is called. > >> > > > > Which still leaves open the question of telling the difference between a > > halt and a reboot. > Well, with the correct information in siginfo, that should do the trick: > si_num = SIGINFO ? SIGHUP ? > si_code = SI_KERNEL > si_int = the "cmd" passed to the reboot (2) function. I concur that sounds like a good option. But that's a kernel mod and will require a kernel patch and getting that through the process. Once that's agreed on that's the route to go, we've got to get the containers guys involved and get that pushed through. And is this going to work without any modifications to init itself (per the discussion over on the -devel list wrt modifications to init and the difficulty and pain of pulling teeth). What's the next step? Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
Michael H. Warfield wrote: > On Mon, 2010-01-25 at 21:50 +0100, Daniel Lezcano wrote: > > >>> apologies for the length, but how is everyone else handling this? >>> this is the last thing i need to solve before i actually start running >>> all my services on this setup. >>> >>> >> I was wondering if the kernel shouldn't send a signal to the init's >> parent when sys_reboot is called. >> > > Which still leaves open the question of telling the difference between a > halt and a reboot. Well, with the correct information in siginfo, that should do the trick: si_num = SIGINFO ? SIGHUP ? si_code = SI_KERNEL si_int = the "cmd" passed to the reboot (2) function. -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
And here I am replying to myself... Again... :-/ On Mon, 2010-01-25 at 16:18 -0500, Michael H. Warfield wrote: > On Mon, 2010-01-25 at 21:50 +0100, Daniel Lezcano wrote: > > > > apologies for the length, but how is everyone else handling this? > > > this is the last thing i need to solve before i actually start running > > > all my services on this setup. > > > > > I was wondering if the kernel shouldn't send a signal to the init's > > parent when sys_reboot is called. > Which still leaves open the question of telling the difference between a > halt and a reboot. My trick of using the final runlevel > in /var/run/utmp ran a foul over a gotcha in the Debian containers that > they seem to default to mounting tmpfs over /var/run and /var/lock so > you loose that information. I had to disable "RAMRUN" and "RAMLOCK" > in /etc/default/rcS in the debian images to get around that but I'm not > sure I'm happy doing that. The alternative examining /var/log/wtmp > didn't work out as reliable. OpenVZ has a similar problem and it writes > a "reboot" file that can be read but that seems inelegant as well. I > don't think anything works if someone does a "reboot -f" but I need to > test that condition yet. And the answer is... "But the dog did nothing in the night." "Exactly." Neither "halt -f" nor "reboot -f" did a darn thing. The containers continue to run as if nothing had happened. I guess I should have expected that. Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
On Mon, 2010-01-25 at 21:50 +0100, Daniel Lezcano wrote:
> > apologies for the length, but how is everyone else handling this?
> > this is the last thing i need to solve before i actually start running
> > all my services on this setup.
> >
> I was wondering if the kernel shouldn't send a signal to the init's
> parent when sys_reboot is called.
Which still leaves open the question of telling the difference between a
halt and a reboot. My trick of using the final runlevel
in /var/run/utmp ran a foul over a gotcha in the Debian containers that
they seem to default to mounting tmpfs over /var/run and /var/lock so
you loose that information. I had to disable "RAMRUN" and "RAMLOCK"
in /etc/default/rcS in the debian images to get around that but I'm not
sure I'm happy doing that. The alternative examining /var/log/wtmp
didn't work out as reliable. OpenVZ has a similar problem and it writes
a "reboot" file that can be read but that seems inelegant as well. I
don't think anything works if someone does a "reboot -f" but I need to
test that condition yet.
To also answer the OP's question. Here's what I use. I have a script
that runs periodically in the host. If the number of processes in a
running container is 1, then I check for the runlevel in
${rootfs}/var/run/utmp. If that's "? 0" then it's a halt and I run
lxc-stop. If it's "? 6" then it's a reboot and I stop the container and
then restart it. I run it every 5 minutes or so or manually. It runs
fast and could be run more often. Just sucks polling things like that,
though. That script, lxc-check, is attached.
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
lxc-check
Description: application/shellscript
signature.asc
Description: This is a digitally signed message part
--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
ah ok. i think i remember reading that message about the shutdown
issues. also in my last message i mixed up SIGINT and SIGPWR; with
the inittab im using, SIGPWR to container will enter runlevel 0, and
SIGINT will enter runlevel 6 then immediately back to 3 (reboot).
i managed to create a workaround that will suffice for the time being
(wish i'd done this sooner):
ps --no-headers -C init -o pid | \
while read x; do
[ ${x} -eq 1 ] && continue
[ $(ps --no-headers --ppid ${x} | wc -l) -eq 0 ] && kill -9 ${x}
done
this will scan for any process named init, except the real init, and
check how many children it has. if the child count is zero, then it's
sent a SIGKILL and container life ends properly.
you could also adds some checks to /proc/${x}/cgroup and check for
conatiner names.
thanks
On Mon, Jan 25, 2010 at 2:50 PM, Daniel Lezcano wrote:
> Tony Risinger wrote:
>>
>> hello,
>>
>> over the past several weeks i have been working intensively on setting
>> up my personal servers with LXC based containers. at this point, i am
>> extremely pleased with my setup, and will soon be sharing all that i
>> have accomplished in the form of several scripts and implemented
>> configuration ideas. i am using LXC in conjunction with BTRFS; all of
>> my containers are populated inside BTRFS subvolumes/snapshots, and i
>> have to say this is _very_ slick. i can create several containers in
>> a matter of seconds, all sharing the base install (COW of course).
>> this allows me to create a "nano" template, fork it at the filesystem
>> level, update it to a "base" template via a package manager, and
>> repeat this as many times as i wish (LAMP/etc). i then fork an
>> appropriate template into a usable domain, again at the FS level, and
>> run an LXC container inside it.
>>
>
> Right, I tried this kind of configuration and the true is the btrfs is
> *very* useful for consolidating the rootfs of several containers.
>
>> anyways, that is all working extremely well, all my build/run/manage
>> scripts are complete. i am however experiencing one nuisance that
>> works against the elegance of it all... how to convince init to die
>> once it enters runlevel 0 and all other processes are dead.
>>
>
> You can't. This is something we are trying to solve right now.
>
> Dietmar Maurer posted a some month ago a process forker running in the
> container which has a socket unix opened with the supervisor process
> (lxc-start).
> The process allows to enter the container or execute command inside the
> container from the outside via commands send with the socket.
> During the discussion about the shutdown problem, Dietmar pointed when the
> container shutdowns, all the processes exit the container including the
> "forker". So we should be able to detect that with the af_unix socket
> closing and then kill the container from lxc-start.
>
> I don't know if it's the right way to handle the shutdown (that would be
> nice to kill/start again the container at reboot) but I think Dietmar's idea
> is a reasonable workaround and acceptable to be kept long enough before we
> found something better.
>
>> i swear i have tried/considered about everything... playing with
>> /etc/powersave, /etc/initscript, powerfail/SIGPWR, replacing
>> /sbin/init with /bin/true and calling init U, named pipes from host to
>> container/read only bind mounts of a folder with a named pipe to
>> trigger something in the host, kill -9 1 in inittab itself, writing a
>> custom init in bash, maybe using something other than init like
>> upstart (?), and probably several other things that i've forgotten
>> but they all feel kludgey and complicated.
>>
>> init simply refuses to die unless its issued a SIGKILL from the host.
>> and thats super inconvenient :-(. i know pid 1 has special properties
>> but i hoped there would be a nice way to address the fact that its not
>> _really_ pid one... it just thinks it is.
>>
>> this is what i have for an /etc/inittab right now in the containers,
>> mostly pretty good i think:
>>
>> id:3:initdefault:
>> rc::sysinit:/etc/rc.sysinit
>> rs:S1:wait:/etc/rc.single
>> rm:2345:wait:/etc/rc.multi
>> rh:06:wait:/etc/rc.shutdown
>> su:S:wait:/sbin/sulogin -p
>> c1:2345:respawn:/sbin/agetty -n -l /bin/autologin -8 38400 tty1 linux
>> rb:6:once:/sbin/init 3
>> kl:0:once:/bin/touch /dev/hostctl
>> p6::ctrlaltdel:/sbin/init 6
>> p0::powerfail:/sbin/init 0
>>
>> this lets me reboot the container from the inside correctly, or from
>> the host with a SIGPWR, or "shutdown" with a SIGINT from the host.
>> the autologin binary lets the host login as root no matter what. this
>> next line is my latest/final attempt at managing these "zombie"
>> containers once they enter runlevel 0:
>>
>> kl:0:once:/bin/touch /dev/hostctl
>>
>> on the host i basically run a "timeout 5m cat
>> /vps/dom//rootfs/dev/hostctl" for each dom, and monitor the
>> return code from that process. the cat command will block until
>> "touched" by init at the end of
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
Tony Risinger wrote: > hello, > > over the past several weeks i have been working intensively on setting > up my personal servers with LXC based containers. at this point, i am > extremely pleased with my setup, and will soon be sharing all that i > have accomplished in the form of several scripts and implemented > configuration ideas. i am using LXC in conjunction with BTRFS; all of > my containers are populated inside BTRFS subvolumes/snapshots, and i > have to say this is _very_ slick. i can create several containers in > a matter of seconds, all sharing the base install (COW of course). > this allows me to create a "nano" template, fork it at the filesystem > level, update it to a "base" template via a package manager, and > repeat this as many times as i wish (LAMP/etc). i then fork an > appropriate template into a usable domain, again at the FS level, and > run an LXC container inside it. > Right, I tried this kind of configuration and the true is the btrfs is *very* useful for consolidating the rootfs of several containers. > anyways, that is all working extremely well, all my build/run/manage > scripts are complete. i am however experiencing one nuisance that > works against the elegance of it all... how to convince init to die > once it enters runlevel 0 and all other processes are dead. > You can't. This is something we are trying to solve right now. Dietmar Maurer posted a some month ago a process forker running in the container which has a socket unix opened with the supervisor process (lxc-start). The process allows to enter the container or execute command inside the container from the outside via commands send with the socket. During the discussion about the shutdown problem, Dietmar pointed when the container shutdowns, all the processes exit the container including the "forker". So we should be able to detect that with the af_unix socket closing and then kill the container from lxc-start. I don't know if it's the right way to handle the shutdown (that would be nice to kill/start again the container at reboot) but I think Dietmar's idea is a reasonable workaround and acceptable to be kept long enough before we found something better. > i swear i have tried/considered about everything... playing with > /etc/powersave, /etc/initscript, powerfail/SIGPWR, replacing > /sbin/init with /bin/true and calling init U, named pipes from host to > container/read only bind mounts of a folder with a named pipe to > trigger something in the host, kill -9 1 in inittab itself, writing a > custom init in bash, maybe using something other than init like > upstart (?), and probably several other things that i've forgotten > but they all feel kludgey and complicated. > > init simply refuses to die unless its issued a SIGKILL from the host. > and thats super inconvenient :-(. i know pid 1 has special properties > but i hoped there would be a nice way to address the fact that its not > _really_ pid one... it just thinks it is. > > this is what i have for an /etc/inittab right now in the containers, > mostly pretty good i think: > > id:3:initdefault: > rc::sysinit:/etc/rc.sysinit > rs:S1:wait:/etc/rc.single > rm:2345:wait:/etc/rc.multi > rh:06:wait:/etc/rc.shutdown > su:S:wait:/sbin/sulogin -p > c1:2345:respawn:/sbin/agetty -n -l /bin/autologin -8 38400 tty1 linux > rb:6:once:/sbin/init 3 > kl:0:once:/bin/touch /dev/hostctl > p6::ctrlaltdel:/sbin/init 6 > p0::powerfail:/sbin/init 0 > > this lets me reboot the container from the inside correctly, or from > the host with a SIGPWR, or "shutdown" with a SIGINT from the host. > the autologin binary lets the host login as root no matter what. this > next line is my latest/final attempt at managing these "zombie" > containers once they enter runlevel 0: > > kl:0:once:/bin/touch /dev/hostctl > > on the host i basically run a "timeout 5m cat > /vps/dom//rootfs/dev/hostctl" for each dom, and monitor the > return code from that process. the cat command will block until > "touched" by init at the end of its life. at that point i mercilessly > SIGKILL the container init. the other option i'm considering is a > cronjob that loops thru "running" containers, does an lxc-ps on them, > and if only one process is running assume its init and SIGKILL the > pesky bugger, this is probably the easier way. > > apologies for the length, but how is everyone else handling this? > this is the last thing i need to solve before i actually start running > all my services on this setup. > I was wondering if the kernel shouldn't send a signal to the init's parent when sys_reboot is called. -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away.
[Lxc-users] How to make a container init DIE after finishing runlevel 0
hello, over the past several weeks i have been working intensively on setting up my personal servers with LXC based containers. at this point, i am extremely pleased with my setup, and will soon be sharing all that i have accomplished in the form of several scripts and implemented configuration ideas. i am using LXC in conjunction with BTRFS; all of my containers are populated inside BTRFS subvolumes/snapshots, and i have to say this is _very_ slick. i can create several containers in a matter of seconds, all sharing the base install (COW of course). this allows me to create a "nano" template, fork it at the filesystem level, update it to a "base" template via a package manager, and repeat this as many times as i wish (LAMP/etc). i then fork an appropriate template into a usable domain, again at the FS level, and run an LXC container inside it. anyways, that is all working extremely well, all my build/run/manage scripts are complete. i am however experiencing one nuisance that works against the elegance of it all... how to convince init to die once it enters runlevel 0 and all other processes are dead. i swear i have tried/considered about everything... playing with /etc/powersave, /etc/initscript, powerfail/SIGPWR, replacing /sbin/init with /bin/true and calling init U, named pipes from host to container/read only bind mounts of a folder with a named pipe to trigger something in the host, kill -9 1 in inittab itself, writing a custom init in bash, maybe using something other than init like upstart (?), and probably several other things that i've forgotten but they all feel kludgey and complicated. init simply refuses to die unless its issued a SIGKILL from the host. and thats super inconvenient :-(. i know pid 1 has special properties but i hoped there would be a nice way to address the fact that its not _really_ pid one... it just thinks it is. this is what i have for an /etc/inittab right now in the containers, mostly pretty good i think: id:3:initdefault: rc::sysinit:/etc/rc.sysinit rs:S1:wait:/etc/rc.single rm:2345:wait:/etc/rc.multi rh:06:wait:/etc/rc.shutdown su:S:wait:/sbin/sulogin -p c1:2345:respawn:/sbin/agetty -n -l /bin/autologin -8 38400 tty1 linux rb:6:once:/sbin/init 3 kl:0:once:/bin/touch /dev/hostctl p6::ctrlaltdel:/sbin/init 6 p0::powerfail:/sbin/init 0 this lets me reboot the container from the inside correctly, or from the host with a SIGPWR, or "shutdown" with a SIGINT from the host. the autologin binary lets the host login as root no matter what. this next line is my latest/final attempt at managing these "zombie" containers once they enter runlevel 0: kl:0:once:/bin/touch /dev/hostctl on the host i basically run a "timeout 5m cat /vps/dom//rootfs/dev/hostctl" for each dom, and monitor the return code from that process. the cat command will block until "touched" by init at the end of its life. at that point i mercilessly SIGKILL the container init. the other option i'm considering is a cronjob that loops thru "running" containers, does an lxc-ps on them, and if only one process is running assume its init and SIGKILL the pesky bugger, this is probably the easier way. apologies for the length, but how is everyone else handling this? this is the last thing i need to solve before i actually start running all my services on this setup. thanks -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
