Re: [Lxc-users] [lxc-devel] Request for inclusion into mainline LXC utils
On Mon, 2010-01-25 at 01:01 -0500, Michael H. Warfield wrote: : - Snip... FOUND IT! [r...@alcove ~]# cat /proc/sys/net/ipv6/conf/all/accept_ra 1 r...@ubuntu:~# cat /proc/sys/net/ipv6/conf/all/accept_ra 0 That's what was killing me and blocking autoconf in Debian. I set that to 1 for all and for eth0 and it all magically starts working. Leaves unresolved why this is required in the Debian containers and NOT in the Fedora containers but someone else can worry about that while I integrate this into my container hacks. This is what I had to add to the container /etc/sysctl.conf to make this all work: net.ipv6.conf.all.forwarding=0 net.ipv6.conf.all.accept_ra=1 net.ipv6.conf.default.accept_ra=1 net.ipv6.conf.eth0.accept_ra=1 Had to add all of them. Leave any one of them out and it fails (which probably means, if there is an eth1 or eth2, they need to be there as well... Gag...) Which begs a question (not begs the question which is a logical conundrum of a different sort)... WHY is this necessary in Debian containers and not at all in Fedora containers? And found the answer to this one too and it raises a serious (to me) issue for the lxc developers... The problem is rooted in the differences between network setup between Debian and Fedora / RedHat et al. The later initializes the IPv6 subsystem based on the excellent and comprehensive IPv6 init scripts by Peter Bieringer and company. Debian basically does nothing, which is OK but somewhat suboptimal (doesn't set up local routes or block private address 6to4 routes amongst others which can cause problems if you are not connected to a real IPv6 network) in most cases. In the Fedora case, if you have NETWORKING_IPV6=yes and IPV6FORWARDING=no, it simply does the right thing and sets those up properly, even though it's somewhat redundant in that it's explicitly setting some values to their default values. In the Debian case, it depends on those values defaulting to a reasonable host leaf node, as if it were not a router, and expects you to manually set it correctly if it is a router. That is very true in most cases. It's not true in this case. It's a corner case of running a virtualized leaf node on a host system that is acting as an IPv6 router, which shouldn't be that common. The Debian container is booting up with sysctl values which are not normally default for a real, non-virtualized, system. It means that a Debian container will behave differently if it is migrated between hosts which are and are not routers. That's where the question comes for the lxc developers. Should it? Or should LXC have something in it to set these sysctl values or configure these sysctl values back to reasonable defaults so it behaves the same? Apparently, the container virtualization is working properly and we can change them in the containers. Should there be something in the lxc configurations for setting sysctl values as part of the lxc container initialization? Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
On Tue, 2010-01-26 at 00:42 +0100, Daniel Lezcano wrote:
I trick I just found:
while $(true); do
inotifywait /var/lib/lxc/debian/rootfs/var/run/utmp;
if [ $(wc -l /cgroup/debian/tasks | awk '{ print $1 }') = 1 ]; then
lxc-stop -n debian
fi;
done
Seems to be a problem with that approach. I'm not seeing anythign
in /var/lib/lxc/${VM}/rootfs any more. Use to with 0.6.4 and earlier
but not now. That seems to have gone away. That might have been a
way to get around that tmpfs problem but it's not there and I have to
resort back to the real rootfs.
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
On Tue, 2010-01-26 at 01:50 +0100, Daniel Lezcano wrote:
Michael H. Warfield wrote:
On Tue, 2010-01-26 at 00:42 +0100, Daniel Lezcano wrote:
I trick I just found:
while $(true); do
inotifywait /var/lib/lxc/debian/rootfs/var/run/utmp;
if [ $(wc -l /cgroup/debian/tasks | awk '{ print $1 }') = 1 ]; then
lxc-stop -n debian
fi;
done
Seems to be a problem with that approach. I'm not seeing anythign
in /var/lib/lxc/${VM}/rootfs any more. Use to with 0.6.4 and earlier
but not now. That seems to have gone away. That might have been a
way to get around that tmpfs problem but it's not there and I have to
resort back to the real rootfs.
With a version 0.6.4 the lxc-debian script write the rootfs in the
/var/lib/lxc/debian/rootfs.
With a version = 0.6.4 the rootfs is created in the current directory.
Note if you have the source code from git, I incremented the version
right before the tag 0.6.5, so you may have a pre-0.6.5 but with the
0.6.4 version information.
Ah! If you look at that lxc-check script I had attached, it tries to
find the rootfs by looking up the config and then, if necessary,
dereferencing through any mount points for that rootfs that may be in
the VM fstab (part of my OpenVZ ve - lxc conversion and compatiblity)
so I guess I've got that covered since I go for the ${ROOTFS}. Try and
keep things as generic as possible.
Thanks for that clarification!
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
On Tue, 2010-01-26 at 01:37 +0100, Daniel Lezcano wrote: Ah, ok didn't know upstart kept using utmp for compatibility. Interesting. Too much depends on that entire utmp.h stuff. Thou shalt NOT break Posix compliance. Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How to make a container init DIE after finishing runlevel 0
On Mon, 2010-01-25 at 20:50 -0500, Michael H. Warfield wrote: Mui Bien! Gracias! Or... I really should have responded... Merci beaucop. Studying Spanish. Haven't had French since high school (and my Russian is real rusty). But I'm trying... Attached. Mike Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Poor network performance between container and host
On Sat, 2010-01-30 at 14:16 +0100, Dominik Schulz wrote: Hi, I'm currently evaluating LXC as an alternative to other contextualisation technologies, and I have the problem that network connections between the container and the host on which it is running are very slow (about 35 kbit/s) wheras connections to other system can exploit the full connection speed. Is this a known issue or is someting wrong with my configuration? The relevant snippet of my lxc config: lxc.network.type = veth lxc.network.flags = up lxc.network.link = br0 lxc.network.name = eth0 lxc.network.mtu = 1500 My Bridge (br0) has an MTU of 1480, could that be the problem? That certainly could be a contributing problem with large packets from the container to the host being fragmented but PMTU discovery should address that. You might also try macvlan instead of veth. Some people prefer it although I've had problems with it on some kernel revs with host to guest connections. Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Executing a command inside a running container?
On Sun, 2010-01-31 at 13:38 -0600, Tony Risinger wrote:
On Sat, Jan 30, 2010 at 12:11 PM, Michael H. Warfield m...@wittsend.com
wrote:
On Sat, 2010-01-30 at 14:20 +0100, Dominik Schulz wrote:
Hi,
I'm fairly new to LXC and I am looking for a way to execute a command
inside a
running container (a full blow one with its own rootfs and full isolation).
lxc-execute doesn't seem to do the trick and lxc-console requires
credential
to login. I'm looking for a way to execute command w/o having to login via
ssh
or the lxc-console, so I can execute command directly inside the
containers to
shut them down properly.
This is being worked on and looked at now through the use of some sort
of daemon in the running container but is not currently possible. More
over, cgroups in the kernel does not currently support joining an active
cgroup, and may never support it, so it would have to be done through
the mediation of some sort of daemon in the running container and
communicating with the host. There's been discussion on the -devel list
over the optimal way to do it and there are some patches running around
but AFAICT nothing has really been committed or committed to at this
time.
if your using the standard init program, and you are only trying to
control stutdown/reboot, i use something like this in my container
inittab:
p6::ctrlaltdel:/sbin/init 6
p0::powerfail:/sbin/init 0
ctrlaltdel responds to a SIGINT, and powerfail responds to SIGPWR.
this lets you send a:
kill -INT init pid
to reboot, and:
kill -PWR init pid
Ah that's a nice trick. I'll have to incorporate that into my scripts.
to shutdown. you will also need the help of a monitor process to
overcome the fact that the init process will hang around, even though
all other processes in the container are dead. an earlier thread
detailed a nice trick using inotifywait and the utmp file in the
container, see this thread for ideas:
http://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg00040.html
specifically this post on:
http://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg00049.html
you basically monitor the utmp file in the container for a change,
then run runlevel utmp file to get the runlevel in the container,
and act appropriately. here is an excerpt from my scripts (originally
based off one provided in the previous thread), it has variables
specific to my setup but it's fairly self explanatory:
while true; do
# time of 5 minutes on it JUST IN CASE...
vps_utmp=${VPS_DOM}/${vps}/rootfs/var/run/utmp
inotifywait -qqt 300 ${vps_utmp}
if [ $(wc -l ${VPS_VAR}/cgroup/${vps}/tasks) -eq 1 ]; then
runlevel=$(runlevel ${vps_utmp})
case $runlevel in
N*)
# nothing for new boot state
;;
??0)
# halted... kill vps.
lxc-stop -n ${vps}
break
;;
??6)
# rebooting... kill vps and start again...
lxc-stop -n ${vps}
lxc-wait -n ${vps} -s STOPPED
lxc-start -d -n ${vps} -o ${VPS_LOG}/${vps}.log
# loop again.
;;
*)
# make sure vps is still running
state=$(lxc-info -n ${vps} | sed -e 's/.* is //')
[ $state = RUNNING ] || break
;;
esac
fi
done
Yeah, that's some of my stuff. I've since rolled that into an lxc-run
script that does some initialization and then forks off a subprocess
that calls lxc_start as a daemon and then drops into a variation of that
loop. I'll probably be posting that script in the next day or so. I'm
almost done cleaning up some of the OpenVZ migration hacks that are
incorporated into it.
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] mac addresses
On Fri, 2010-02-12 at 11:37 -0500, Brian K. White wrote: The reason I'm trying to use the new mac numbering scheme in case it's not apparent is, an OUI consumes 3 of the available 6 bytes in a mac, leaving only 3 to make a unique number out of. Yet IP addresses have 4 bytes and I want a _simple_ rule or bit of script that will always result in a safely unique, statically assigned mac, without having to maintain a db of them. Something I could put into documentation and it would always work, not just usually work in the simple cases. So I figured, 02:00 for the first two bytes, and directly convert the IP into the remaining 4 bytes. Every admin already has to make his IP's unique at least within the same network, so those macs should always work. My initial rule of using the OUI that Novell uses for Xen guests (I'm using openSUSE) and then convering just the last byte of the IP into the last byte of the mac, and using the second-to-last byte of the mac to seperate different networks if a host or container has multiple nics on multiple networks, that rule falls down in all but the simplest cases. OUI's consume 3 byte (minus 2 bits) but that really only applies if the locally administered bit is not set. You're setting it so you can do what ever you like with the remaining (which you obviously recognize by using the 4 bytes). Just want to clarify that. So my question is, is 02:x:x:x:x:x in some way non-routable just because it sets the locally-administered bit? I use that all the time without any problems. It may be something in the way their switch is set up that limits the number of mac addresses on that port. I have seen that (we've got it at work administered by our IT department where we have a Cisco switching fabric that manages 3 buildings with 3 network ports to each station) where one network is limited by default to 5 mac addresses per port and another, more restrictive, network is limited to just one. If it's something like a Cisco switching fabric, it's pretty straight forward to set up limits like that. I think ours are all within a 24 hour period and resets after that to allow you to change machines, do maintenance, and other such. Is there some sort of packet I can send that will trigger Verizons switches routers to update to the new mac for more than 5 seconds? That's a question for Verizon. If I traceroute from a remote box in to one of these containers, it just gets to a particular Verizon router that I have no control over and goes no further. pa2:~ # traceroute nj12 traceroute to nj12 (71.187.206.76), 30 hops max, 40 byte packets 1 gw-238-225.quonix.net (208.72.238.225) 0.349 ms 0.224 ms 0.198 ms 2 ge-11-1-2.mpr3.phl2.us.above.net (209.249.122.165) 0.280 ms 0.300 ms 0.199 ms 3 xe-4-0-0.mpr1.lga5.us.above.net (64.125.31.34) 2.533 ms 2.437 ms 2.431 ms 4 xe-0-1-0.er1.lga5.us.above.net (64.125.27.61) 2.264 ms 2.312 ms 2.325 ms 5 0.ge-3-2-0.BR3.NYC4.ALTER.NET (204.255.168.25) 2.291 ms 2.294 ms 2.321 ms 6 0.ge-4-2-0.NY5030-BB-RTR2.verizon-gni.net (152.63.10.54) 2.652 ms 2.683 ms 6.821 ms 7 so-6-3-0-0.NWRK-BB-RTR2.verizon-gni.net (130.81.19.97) 4.155 ms 4.303 ms 3.696 ms 8 P15-0-0.NWRKNJ-LCR-04.verizon-gni.net (130.81.29.195) 4.904 ms 4.806 ms 4.697 ms 9 P12-0-0.NWRKNJ-LCR-06.verizon-gni.net (130.81.27.7) 5.657 ms 5.434 ms 5.447 ms 10 P14-0.NWRKNJ-LCR-08.verizon-gni.net (130.81.30.95) 5.658 ms 5.551 ms 5.573 ms If I traceroute from the same remote box to the host that the containers is on, it take a very different-looking path and reaches the host. Though possibly the differences are just load-balancing hardware? pa2:~ # traceroute nj10 traceroute to nj10 (71.187.206.74), 30 hops max, 40 byte packets 1 gw-238-225.quonix.net (208.72.238.225) 0.325 ms 0.187 ms 0.196 ms 2 ge-11-1-2.mpr3.phl2.us.above.net (209.249.122.165) 0.282 ms 0.179 ms 0.201 ms 3 xe-4-0-0.mpr1.lga5.us.above.net (64.125.31.34) 2.535 ms 2.439 ms 2.430 ms 4 xe-0-1-0.er1.lga5.us.above.net (64.125.27.61) 2.259 ms 2.288 ms 2.317 ms 5 0.ge-3-2-0.BR3.NYC4.ALTER.NET (204.255.168.25) 2.327 ms 2.314 ms 2.307 ms 6 0.ge-8-1-0.NY325-BB-RTR1.verizon-gni.net (152.63.18.38) 2.690 ms 2.756 ms 2.646 ms 7 so-4-0-0-0.NWRK-BB-RTR1.verizon-gni.net (130.81.17.7) 7.708 ms 20.080 ms 7.678 ms 8 P15-0-0.NWRKNJ-LCR-03.verizon-gni.net (130.81.29.193) 8.419 ms 8.098 ms 8.031 ms 9 P12-0-0.NWRKNJ-LCR-05.verizon-gni.net (130.81.27.2) 9.065 ms 9.111 ms 9.059 ms 10 P14-0.NWRKNJ-LCR-07.verizon-gni.net (130.81.30.9) 7.729 ms 7.671 ms 7.595 ms 11 * * * 12 static-71-187-206-74.nwrknj.fios.verizon.net (71.187.206.74) 17.883 ms 12.380 ms 14.285 ms pa2:~ # Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw
Re: [Lxc-users] Dynamic devices...
On Sun, 2010-03-07 at 15:48 +0100, Daniel Lezcano wrote: : - Big snip... At this point, the udev events for the network devices should be implemented to send events only for the container where the network devices belong. And then generalize the udev events for all the containers. When udev per container will be implemented, I will be curious to see a good lxc.cgroup.devices configuration :) But you've got a huge problem there anyways that I've recognized right from the very get go. Those are based on major and minor numbers and the whole precept of udev (and devfs before it) is that those should no longer be needed to be fixed. There are several drivers that are both in the primary kernel source tree, such as my own ip2 multiport serial interface, and outside of the tree, such as x10dev, that use HUGE numbers of devices that really just do not NEED a fixed major number. The Intelliport II driver (ip2) has three entire major numbers assigned to it from the dark ancient days of the 1.x kernel! In a fixed allocation scheme, it needed them! It supports up to 256 high speed serial ports (all the minors that were available in a major) and needed both raw and cooked interfaces (there go two majors) and a couple of per driver and per board control interfaces (and there goes the third major). Even though I have never run into a single installation that was fully decked out with 4 boards (64 interfaces per board - I have seen ONE that had THREE and that was scary). Now days, I could reduce that to 1 major (move the controls to sysfs and the raw/cooked to an ioctl and/or sysfs) and float that. The majors are valuable resources. I've actually tried to give back one of those majors (the raw/cooked) and it really can't be done for legacy reasons even as late as this, now. But how will this LXC config with majors and minors play with drivers that may have different major numbers and, heavens, how do you deal with all these USB devices and potentially varying minor numbers. You can't, really. Not with the current configuration scheme, you just can't. Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Dynamic devices...
On Sat, 2010-03-06 at 23:21 -0600, C Anthony Risinger wrote:
What does udev listen to while under container (or host)? Is it
possible to generate events from a host process to each containerized
udevd, and let udevd do it's job? I'm not familiar; a spoofed
kernel event generator of sorts
You know, I was thinking about that, exactly. I'm not totally sure, but
it just seems like that would almost work. Then you COULD have udev
rules in the container that did things that were different from the host
and you would not be constantly fighting with driving a stake through
udevd's heart in the containers (the containers hang stone cold dead in
system initialization).
Daniel has it right, though... This is part of the network namespace
and needs to be really handled at the kernel level. And this is a
problem shared by both LXC and OpenVZ (and probably Vservers and most
certainly libvirt and their lxc driver). Then we really really could
virtualize even udev effectively. I'm not totally sure how effective
spoofing the udev events from the host to the container would be. I
would say, tricky, to say the least. You would have to keep from
screwing up any other netlink kinds of things that are overloaded on
that interface. You'd have to be operating on the udevd level in the
host and understand how to separate out those events for the various
containers and then ship them off to the emulated netlink interfaces.
Doable? Probably. For some value of doable.
On top of that, as Daniel pointed out, you have the issue of the static
major/minor device acls in the LXC configs. I'm not sure how you would
deal with that, but it needs to be dealt with with or without the
hotplug udev issues. In my case, the usb sharing device has a fixed
major and minor because none of the servers have any other hid devices.
But, what if I plugged in a USB keyboard or mouse while I was on-site
(which does happen once in a lllooonnnggg while) and THEN switched the
console? Ooopppsss... No workie. Minors are now different. So I have
to allocate ALL of that major. Same thing with those USB tty's only
worse and I have to deal with that right now. Two of the systems have
static serial adapters going to X10 controllers that control power to
the various servers. The other two do not. Guess what... The minor
numbers are now different between those systems. I can't assign just
ttyUSB5 to Onyx and ttyUSB6 to Opal and expect to be able to migrate
those containers between the various host engines that have with and
without. I have to allow the entire major and trust those admins
(which, fortunately, I can) to NOT go screwing with each others consoles
(which they would need the root passwords to the servers, which they
don't anyways). But there are certainly worse scenarios which are easy
to envision where you would want tighter restrictions.
I now have a udev hotplug script, /lib/udev/lxc-hotplug, that seems to
be really close (I know, I owe a lot of people some posting of some of
my big hairball scripts already) to what I want and is working for me
now. I'm storing a lot more stuff under /var/lib/lxc/${VM} (like an
entire dynamic udev dev tree) to make it work without committing heinous
random acts of terrorism (like, if you don't create it DON'T REMOVE
IT!!!), but it seems to work pretty nicely and it mirrors the host
behavior into configured selected containers. But it still DOESN'T
allow for custom udev events and scripting (which is fine by me) in the
containers. Maybe, once I clean up a lot of the debugging cruft and
minimizing the mikeyism's, I can post what I've done. May be of some
use to others or as yet another fine example of what not to do and what
to avoid. :-)
Mike
On Mar 6, 2010, at 11:49 AM, Michael H. Warfield m...@wittsend.com
wrote:
Hey all,
This is sort of a jump ball for both the OpenVZ camp and the LXC camp.
Maybe some food for thought as well. I think it's something that
needs
to be thought out.
I know udev does not work in containers. Neither OpenVZ nor LXC. Ok,
fine. So devices are static. Problem is that I've got a situation
where I need dynamic devices, specifically some USB devices.
I have an 8 port serial to USB converter used for controlling the
serial
consoles of a cluster of remote servers. So ttyS0 on each server is
hooked to a port on the controller and configured for serial console
(they also have serial BIOS console redirection as well - very VERY
nice
for remote management where you don't have a remote IP KVM).
That converter then plugs into a USB sharing device which shares the
converter between 4 of the servers. So... Any one of the four
servers
can take control of the converter and can then access all of the 8
serial consoles for all of the servers. Nice redundancy.
I want to grant access to that subsystem to selected containers so
certain administrators can be given their very own container
Re: [Lxc-users] lxc-start on openvz tempate
On Thu, 2010-06-24 at 08:55 +0200, Daniel Lezcano wrote: : - snip Console output is not defaulting to the current tty, so you have to specify lxc-start -n fsn -s lxc.console=$(tty) That will be fixed for 0.7.1, I will release probably tomorrow. Will that also be including the ro bind mount options as discussed in some other threads? I would like to deploy and test that and test to see if remounting still propagates ro/rw changes between containers. Thanks -- Daniel Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- ThinkGeek and WIRED's GeekDad team up for the Ultimate GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky parental unit. See the prize list and enter to win: http://p.sf.net/sfu/thinkgeek-promo___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] failed to create pty #0
On Mon, 2010-09-20 at 05:29 -0400, l...@jelmail.com wrote: Hi Daniel, I have tracked down this issue somewhat. It seems to be caused by shutting down a container (not by lxc-stop) and is caused by the rc.shutdown script present in Arch Linux. I've seen this problem too even when lxc-stop is used and the container is a Fedora container (mostly F12's). If I shut down the container and stop it with lxc-stop then restart the container, I get that failed to create pty #0) when sshing into the container. I have to restart the host system once that's happened. I don't know what specifically causes the problem because I haven't had time to investigate but I do know that it's fixed by removing everything from rc.shutdown onwards from the line containing stat_busy “Saving System Clock” as suggested on lxc.teegra.net (I had done this on a prior container but missed this step on a new one which is why the problem only started happening recently). I'm going to have to see if there's something similar in the Fedora shutdown scripts. Interesting. I hadn't tried using lxc-stop without shutting down the contained OS, so I hadn't narrowed it down that far. Interesting. So something in that shutdown file has the capacity to disable the host's ability to start further containers and also disable the ability to ssh into already running ones (thankfully, lxc-console still worked). John Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] How do I remove udev from upgrade
On Sat, 2010-10-30 at 08:45 +1100, Mark C. Hawkins wrote: I've got a Lucid (10.04) guest running in a production environment that I don't really want to rebuild After running apt-get upgrade I get errors from the udev package. From reading other posts I see that udev is not used and that holding the package back prevents the problem when doing an upgrade, but does anyone have a solution to get rid of the error message if the upgrade was already attempted? Removing udev doesn't work as there are too many packages that depend on udev. is there anyway to remove udev without breaking the other packages or some way to remove the error? Simplest thing I found on Fedora is simply add an exit 0 at the top of /etc/udev/udev.conf. When that gets sourced, the startup script simply exists and udev never starts. I'm presuming that the same thing should work on Ubuntu. Only downsides are that you are carrying and upgrading a package you will never use and it's an esthetically unappealing hack. Oh well... Regards, Mark Hawkins Partner Medalist (M) +61 411 471 684 (P) +613 9702 8713 (F) +613 9005 2990 mhawk...@medalist.com.au I am a thing that thinks, that is to say, a thing that doubts, affrims, denies, understands a few things, is ignorant of many things, wills, refrains from willing, and also imagines and senses. - Rene Descartes Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Nokia and ATT present the 2010 Calling All Innovators-North America contest Create new apps games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] LXC and IPv6
On Thu, 2010-11-18 at 19:38 +, Gordon Henderson wrote: Anyone tried LXC with IPv6? Any reason it shouldn't just work? Works fine for me and I've been using it (IPv6) for years. Cheers, Gordon Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Beautiful is writing same markup. Internet Explorer 9 supports standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 L3. Spend less time writing and rewriting code and more time creating great experiences on the web. Be a part of the beta today http://p.sf.net/sfu/msIE9-sfdev2dev___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] limiting RAM usage and disk space usage
On Thu, 2010-12-02 at 18:29 -0600, Serge E. Hallyn wrote: Quoting Gordon Henderson (gor...@drogon.net): On Thu, 2 Dec 2010, Serge E. Hallyn wrote: Quoting Gordon Henderson (gor...@drogon.net): On Mon, 29 Nov 2010, Trent W. Buck wrote: Siju George sgeorge...@gmail.com writes: 1) how do I limit the RAM usage of a container? In lxc.conf(5): lxc.cgroup.memory.limit_in_bytes = 256M lxc.cgroup.memory.memsw.limit_in_bytes = 1G 2) how do I limit the disk usage of a container ? Ensure the rootfs is a dedicated filesystem (e.g. an LVM LV), and limit its size accordingly. Finally, a use for LVM And here was me about to embark on using filesystems in files loopback mounted... That should also work, just a bit slower, right? I don't know, really. I use loopback mounted filesystems for other things and I don't really notice much slowdowns compared to the native filesystem, but I've never benchmarked it. It is some years since I looked at LVM and I wasn't impressed then so stopped thinking about it and used something else (I was looking at snapshot abilities at the time) I build systems for a purpose so rarely have the need to do the only thing LVM might be handy for - ie. resizing a volume so it just doesn't fgure in my systems, but maybe this is a valid use now. It seems to me that a little extension to lxc-create which creates the container inside a loopback fs would be pretty neat. Wouldn't simply adding the rootfs to the fstab conf with the file system image and the -o loop option do it? I would think it would. I haven't tried it in lxc but I do it all the time just for general mounting and day to day work. -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Increase Visibility of Your 3D Game App Earn a Chance To Win $500! Tap into the largest installed PC base get more eyes on your game by optimizing for Intel(R) Graphics Technology. Get started today with the Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs. http://p.sf.net/sfu/intelisp-dev2dev___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] limiting RAM usage and disk space usage
Sorry, I should have added an example... Below... On Thu, 2010-12-02 at 22:18 -0500, Michael H. Warfield wrote: On Thu, 2010-12-02 at 18:29 -0600, Serge E. Hallyn wrote: Quoting Gordon Henderson (gor...@drogon.net): On Thu, 2 Dec 2010, Serge E. Hallyn wrote: Quoting Gordon Henderson (gor...@drogon.net): On Mon, 29 Nov 2010, Trent W. Buck wrote: Siju George sgeorge...@gmail.com writes: 1) how do I limit the RAM usage of a container? In lxc.conf(5): lxc.cgroup.memory.limit_in_bytes = 256M lxc.cgroup.memory.memsw.limit_in_bytes = 1G 2) how do I limit the disk usage of a container ? Ensure the rootfs is a dedicated filesystem (e.g. an LVM LV), and limit its size accordingly. Finally, a use for LVM And here was me about to embark on using filesystems in files loopback mounted... That should also work, just a bit slower, right? I don't know, really. I use loopback mounted filesystems for other things and I don't really notice much slowdowns compared to the native filesystem, but I've never benchmarked it. It is some years since I looked at LVM and I wasn't impressed then so stopped thinking about it and used something else (I was looking at snapshot abilities at the time) I build systems for a purpose so rarely have the need to do the only thing LVM might be handy for - ie. resizing a volume so it just doesn't fgure in my systems, but maybe this is a valid use now. It seems to me that a little extension to lxc-create which creates the container inside a loopback fs would be pretty neat. Wouldn't simply adding the rootfs to the fstab conf with the file system image and the -o loop option do it? I would think it would. I haven't tried it in lxc but I do it all the time just for general mounting and day to day work. So this is an excerpt from one of my LXC VM's fstab (I normally use an explicit bind mount for them): /srv/lxc/private/1004 /srv/lxc/rootfsnone bind 0 0 Now... Wouldn't this work as well? Untested: /srv/lxc/private/disk.image /srv/lxc/rootfsnone loop 0 0 Seems to me that if the bind option works the loop option should as well. -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Increase Visibility of Your 3D Game App Earn a Chance To Win $500! Tap into the largest installed PC base get more eyes on your game by optimizing for Intel(R) Graphics Technology. Get started today with the Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs. http://p.sf.net/sfu/intelisp-dev2dev___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] On clean shutdown of Ubuntu 10.04 containers
see a check in that routine for the config file. * the stop function greps for /sbin/init in container inittab instead of trying to allow for any random container pid #1 Yeah, that little trick won't work with Ubuntu or Fedora using upstart. That gives you the peculiar situation here that if you have a modern Fedora or Ubuntu container, your script even on an OpenSUSE system would start them but could not stop them while it would not start a container without a config but would stop a running SUSE one regardless of a config. * no provision for application/service containers, just whole systems started with /sbin/init That only makes sense. * starts containers in screen - I have not figured out what it would take to get nice behavior out of lxc-console yet and screen is both easy and standard. I usually start them with logging enabled and redirected with the -o option. To each his own. The $LXC_CONF (/etc/lxc/lxc.conf) referenced at the top does not exist usually so everything that happens is visible right in the script. That's a real good practice. I'm using this in production. So far so good. typical usage: nj10:~ # rclxc status Of what is this rclxc command of which you speaketh? Oh, I see from the script that rclxc is a symlink to /etc/init.d/lxc. But what's the rc_* things in there? Something you sourced out of rc.status? That's something SUSE? Nice thoughts in there. Could be adaptable. :-)=) Checking for LXC containers... running : - Snip Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- What happens now with your Lotus Notes apps - do you make another costly upgrade, or settle for being marooned without product support? Time to move off Lotus Notes and onto the cloud with Force.com, apps are easier to build, use, and manage than apps on traditional platforms. Sign up for the Lotus Notes Migration Kit to learn more. http://p.sf.net/sfu/salesforce-d2d___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] failed to create pty #0
Hey guys!
I know, I know... This is like 9 months old. But I finally got caught
between a rock and a hard place on Fedora 15 and was researching that
problem (which others have pointed good pointers to on this list - thank
you very much - got me going again) and I kept constantly running into
this same damn problem. Still. So I finally had to drill into it.
On Mon, 2010-09-20 at 09:03 -0400, Michael H. Warfield wrote:
On Mon, 2010-09-20 at 05:29 -0400, l...@jelmail.com wrote:
Hi Daniel,
I have tracked down this issue somewhat. It seems to be caused by shutting
down a container (not by lxc-stop) and is caused by the rc.shutdown script
present in Arch Linux.
I've seen this problem too even when lxc-stop is used and the container
is a Fedora container (mostly F12's). If I shut down the container and
stop it with lxc-stop then restart the container, I get that failed to
create pty #0) when sshing into the container. I have to restart the
host system once that's happened.
I don't know what specifically causes the problem because I haven't had
time to investigate but I do know that it's fixed by removing everything
from rc.shutdown onwards from the line containing stat_busy “Saving System
Clock” as suggested on lxc.teegra.net (I had done this on a prior
container but missed this step on a new one which is why the problem only
started happening recently).
I'm going to have to see if there's something similar in the Fedora
shutdown scripts.
Interesting. I hadn't tried using lxc-stop without shutting down the
contained OS, so I hadn't narrowed it down that far. Interesting.
I narrowed this down to a specific set of commands in the Fedora halt
script. These are the buggers that are causing the problem...
# Remount read only anything that's left mounted.
echo $Remounting remaining filesystems readonly
mount | awk '{ print $1,$3 }' | while read dev dir; do
fstab-decode mount -n -o ro,remount $dev $dir
done
Comment those lines out. Problem goes away.
Oh, I gotta bad feeling here. We've been fighting this whole bloody
remount thing propagating back into the host and the random acts of
terrorism that lie therein for a long time.
Let's see...
mount | awk '{ print $1,$3 }'
rootfs /
/dev/sdb1 /
/dev/sda8 /srv/shared
none /dev/pts
none /proc
none /sys
none /dev/shm
devpts /dev/console
devpts /dev/tty1
devpts /dev/tty2
devpts /dev/tty3
devpts /dev/tty4
devpts /dev/tty5
devpts /dev/tty6
/proc/bus/usb /proc/bus/usb
none /proc/sys/fs/binfmt_misc
Yup... Ok... That doesn't take much guessing. The container is
remounting the /dev/pts as ro and kiss it good by in the host. Sigh.
I just got done testing this on an F15 host / F14 client w/ LXC 0.7.4.2.
2.6.38.6-27.fc15 kernel. Probably not a lot we can do from user space.
That's some isolation we really need down in kernel land somewhere.
Yes, I can hear it now. Old country doctor's advice. Well, then, don't
do that. But the fact is the container can do something horrible that
propagates back into the host. Yes! Now that I know what,
specifically, is causing this, I can correct it in the guest. But a
rogue guest can do bad things. This is not good. The container should
NEVER have that kind of power to affect the host.
Regards,
Mike
So something in that shutdown file has the capacity to disable the host's
ability to start further containers and also disable the ability to ssh
into already running ones (thankfully, lxc-console still worked).
John
Regards,
Mike
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
___ Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today.
http://p.sf.net/sfu/quest-sfdev2dev___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [PATCH] ignore non-lxc configuration line
On Fri, 2011-05-13 at 22:32 +0200, Daniel Lezcano wrote:
From: Daniel Lezcano daniel.lezc...@free.fr
We ignore the line of in the configuration file not beginning by lxc.
So we can mix the configuration file with another information used for
another component through the lxc library.
Wow...
I seem to recall requesting this sort of thing ages ago. Maybe even
before we created the -users list and only had the -dev list and was
shot down. I have s wanted this feature. That can implement many
of the OpenVZ compatibility things we need the high level scripts to
perform and keep them in one file. Many thanks. I as SO glad to see
this!
Regards,
Mike
Signed-off-by: Daniel Lezcano dlezc...@fr.ibm.com
---
src/lxc/confile.c | 12
1 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/src/lxc/confile.c b/src/lxc/confile.c
index 791f04f..d632404 100644
--- a/src/lxc/confile.c
+++ b/src/lxc/confile.c
@@ -799,7 +799,7 @@ static int parse_line(char *buffer, void *data)
char *dot;
char *key;
char *value;
- int ret = -1;
+ int ret = 0;
if (lxc_is_line_empty(buffer))
return 0;
@@ -815,10 +815,14 @@ static int parse_line(char *buffer, void *data)
}
line += lxc_char_left_gc(line, strlen(line));
- if (line[0] == '#') {
- ret = 0;
+
+ /* martian option - ignoring it, the commented lines beginning by '#'
+ * fall in this case
+ */
+ if (strncmp(line, lxc., 4))
goto out;
- }
+
+ ret = -1;
dot = strstr(line, =);
if (!dot) {
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today.
http://p.sf.net/sfu/quest-sfdev2dev___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [PATCH] ignore non-lxc configuration line
On Wed, 2011-06-01 at 20:10 -0400, Michael H. Warfield wrote:
On Fri, 2011-05-13 at 22:32 +0200, Daniel Lezcano wrote:
From: Daniel Lezcano daniel.lezc...@free.fr
We ignore the line of in the configuration file not beginning by lxc.
So we can mix the configuration file with another information used for
another component through the lxc library.
Wow...
I seem to recall requesting this sort of thing ages ago. Maybe even
before we created the -users list and only had the -dev list and was
shot down. I have s wanted this feature. That can implement many
of the OpenVZ compatibility things we need the high level scripts to
perform and keep them in one file. Many thanks. I as SO glad to see
this!
I see that this has not, apparently, made it into a release bundle yet.
Any idea when it will be out?
Regards,
Mike
Signed-off-by: Daniel Lezcano dlezc...@fr.ibm.com
---
src/lxc/confile.c | 12
1 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/src/lxc/confile.c b/src/lxc/confile.c
index 791f04f..d632404 100644
--- a/src/lxc/confile.c
+++ b/src/lxc/confile.c
@@ -799,7 +799,7 @@ static int parse_line(char *buffer, void *data)
char *dot;
char *key;
char *value;
- int ret = -1;
+ int ret = 0;
if (lxc_is_line_empty(buffer))
return 0;
@@ -815,10 +815,14 @@ static int parse_line(char *buffer, void *data)
}
line += lxc_char_left_gc(line, strlen(line));
- if (line[0] == '#') {
- ret = 0;
+
+ /* martian option - ignoring it, the commented lines beginning by '#'
+* fall in this case
+*/
+ if (strncmp(line, lxc., 4))
goto out;
- }
+
+ ret = -1;
dot = strstr(line, =);
if (!dot) {
Regards,
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Discover what all the cheering's about.
Get your free trial download today.
http://p.sf.net/sfu/quest-dev2dev2 ___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxc on Fedora 15
On Tue, 2011-05-31 at 14:00 -0500, Serge Hallyn wrote:
Quoting Ramez Hanna (rha...@informatiq.org):
On Tue, May 31, 2011 at 5:38 PM, Serge Hallyn
serge.hal...@canonical.comwrote:
Quoting Daniel Lezcano (daniel.lezc...@free.fr):
On 05/31/2011 01:44 PM, Ramez Hanna wrote:
On Tue, May 31, 2011 at 2:07 PM, Daniel Lezcanodaniel.lezc...@free.fr
wrote:
On 05/31/2011 12:33 PM, Ramez Hanna wrote:
it seems that lxc cannot handle cgroups when capabilities are not
all
in
the
same mount
it fails now because it cannot write the devices.deny in the cgroup
if i comment out all the lxc.cgroup.devices lines in the config of
the
container then i can actually start it
I would think that the way lxc identifies the cgroup mount might be
the
part
that needs patching
Thanks for investigating.
The main problem is lxc is cgroup agnostic, so we should find a
solution
where we don't break that.
Maybe one solution would be to collect all the mount points found for
the
cgroup and try to find the right path when writing or reading from
one
cgroup file.
that is what i had in mind, tried looking into the code but my C
skills
are
next to zero
Does systemd run lxc within a cgroup which is not the root cgroup ?
the lxc-start command would run under $user/master/
(/sys/fs/cgroup/systemd/$user/$master)
and the container itself would run under $container_name
(/sys/fs/cgroup/systemd/$container_name)
so it would run the container in the root cgroup
ouch ! I have to install systemd on a test machine to check how systemd
plays with the cgroup.
I don't think the cgroup created by lxc should escape the cgroup the
command is assigned to.
Another similar - and easier to setup - thing we need to address is
running
on a system with libcgroup installed.
For both, I assume it'll basically come down to:
1. figure out the path of the cgroup we are in for each cgroup we care
about
2. create new child cgroup for ourselves in each of the above paths whic
is unique
3. track those through the lifetime of the container
So it just slightly complicates what's being done now.
-serge
how does libcgroup change things? does it also mount cgroup on different
points ?
Yes, in whatever way you ask it to.
I noticed this a couple of clicks back. Maybe even F13 where I had
libcgroup installed and it was mounting things, initially, in /cgroup
(or some such) before the kernel dudes created the mountpoint
in /sys/fs/cgroup. I got burned by it, even back then, and had to
disable libcgroup and do the manual mount stuff in fstab. That was back
months ago when we were having the controversy over whether cgroups
should be mounted under /cgroup or /dev/cgroup or /var/lib/cgroup
or /var/run/cgroup. I thought I raised the whole issue that these
things were in a hierarchy and not a flat mount even back then. Now
it's under the /sys/fs/cgroup mount point and we need to deal with this,
now. I've had to disable the devices.{allow|deny} options on several of
my host machines at this point. Is anyone working on a solution?
Daniel mentioned getting systemd running on a system but it's more
fundamental than that. Like you say, even setting up and enabling
libcgroup is going to be problematical and we need to play nicey nicey
with the other kids in the sandbox.
Regards,
Mike
-serge
--
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today.
http://p.sf.net/sfu/quest-sfdev2dev
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [PATCH 2/2] cgroups: support cgroups mounted in multiple places
*filename, char *value, size_t len); /* diff --git a/src/lxc/state.c b/src/lxc/state.c index 6720011..b435eba 100644 --- a/src/lxc/state.c +++ b/src/lxc/state.c @@ -71,7 +71,7 @@ static int freezer_state(const char *name) FILE *file; int err; - err = lxc_cgroup_path_get(nsgroup, name); + err = lxc_cgroup_path_get(nsgroup, freezer, name); if (err) return -1; -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense.. http://p.sf.net/sfu/splunk-d2d-c1___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [PATCH 2/2] cgroups: support cgroups mounted in multiple places
On Sat, 2011-06-25 at 20:05 +0200, Daniel Lezcano wrote: On 06/24/2011 07:54 PM, Serge Hallyn wrote: I.e. with systemd or libcgroup. To do this, instead of looking for one cgroup called 'lxc' or otherwise taking the first cgroup we find, we actually create a container in every mounted cgroup fs. Right now it's done under the root of each fs. We may want to put that under lxc, or, better yet, make that configurable. Note the use of clone_children seems not quite right, but that's not for this patch to fix. In particular, if clone_children is not in the mntopts, we reject it. Yet later we try to set it ourselves. I believe we should simply, if ns cgroup is not composed, always try to set clone_children to 1. As it stands, with libcgroup installed, I had to do cd /sys/fs/cgroup for d in `/bin/ls`; do echo 1 $d/cgroup.clone_children done But after that, 'lxc-start -n l1' worked like a charm. It also continues to work with a single mount of cgroups under /cgroup. Signed-off-by: Serge Hallyn serge.hal...@ubuntu.com --- Ok, I installed f15 on a kvm machine. But WTF are all these daemons !! 1 root 20 0 48232 15m 1772 R 66.1 1.5 4:53.82 systemd 19247 dlezcano 20 0 790m 40m 11m R 62.2 4.0 0:46.93 gnome-settings- 19255 root 20 0 104m 27m 3252 R 35.9 2.8 0:44.83 packagekitd 1173 root 20 0 196m 65m 2416 R 30.6 6.5 2:43.70 udisks-daemon ... eating all the cpu and making impossible to use the fedora ? Don't get me started, yeah... systemd and udisks-daemon should not be consuming that much cpu. That is weird. Gnome-settings and packagekitd, I don't know. Why do we need a bloody daemon to check for packages and updates? I just don't know. OTOH... I'm not experiencing that on Forest (my hard-iron F15 host engine). My load average is staying low. So I'm not sure what's going on for your. systemd is looking all the time at /proc/1/mountinfo and reading the content, nice for an init process ! And tomorrow the init process will be firefox ... Hmmm... Something fishy there for sure. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense.. http://p.sf.net/sfu/splunk-d2d-c1___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [PATCH 2/2] cgroups: support cgroups mounted in multiple places
On Sun, 2011-06-26 at 18:27 +0200, Daniel Lezcano wrote: On 06/26/2011 05:52 PM, Michael H. Warfield wrote: On Sun, 2011-06-26 at 17:27 +0200, Daniel Lezcano wrote: On 06/26/2011 05:06 PM, Serge E. Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): cd /sys/fs/cgroup for d in `/bin/ls`; do echo 1 $d/cgroup.clone_children done Doing this step alone broke lxc totally for me, with or without the patch below. This was on Fedora 15 testing with lxc 0.7.4.2 as well as Do you have the ns cgroup mounted? The above is only for without ns cgroup. Yes, the ns_cgroup and clone_children are mutually exclusive. If you try to create a namespace with a ns cgroup mounted and the clone_children flag is set. The 'clone' will return EINVAL. I noticed with the patch applied because there was a bug in it which was setting this flag even if the cgroup was ns. It is possible, you ran lxc with the patch, the clone_children was set, and then you remove the patch and try to run lxc again. As the clone_children is still set, that will make impossible to clone a new namespace. With the patch I sent previously on top of Serge's patch, that should be fixed. Ok... Got that an threw it into the mix. I'm still seeing the segfaults on lxc-info. Seems to fault even if I specify the name of a non-existent machine as well. Oh, seems I missed this info :s If you launch lxc-info -n foo, that segfaults ? On what architecture does it fault ? Fedora 15, x86_64. 0.7.4.2 is fine. Just rebuilt a fresh set of packages from GIT sources alone. They're fine and lxc-info from git works. Git patched with Serge's patch. Segfault. Git patched with both patches. Segfault. Here's what gdb has to say about it: [root@forest lxc]# gdb lxc-info core.13546 GNU gdb (GDB) Fedora (7.2.90.20110525-39.fc15) Copyright (C) 2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type show copying and show warranty for details. This GDB was configured as x86_64-redhat-linux-gnu. For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/... Reading symbols from /home/mhw/rpmbuild/BUILD/lxc-0.7.4.2git2/src/lxc/lxc-info...done. [New LWP 13546] Core was generated by `./lxc-info -n Alcove'. Program terminated with signal 11, Segmentation fault. #0 lxc_cgroup_path_get (path=0x7fff89b35928, subsystem=0x7fba52a4fdc1 freezer, name=0x7fff89b3987d Alcove) at cgroup.c:326 326*s = '\0'; (gdb) trace Tracepoint 1 at 0x7fba52a41e74: file cgroup.c, line 326. (gdb) backtrace #0 lxc_cgroup_path_get (path=0x7fff89b35928, subsystem=0x7fba52a4fdc1 freezer, name=0x7fff89b3987d Alcove) at cgroup.c:326 #1 0x7fba52a4a337 in freezer_state (name=optimized out) at state.c:74 #2 0x7fba52a4a42d in lxc_getstate (name=0x7fff89b3987d Alcove) at state.c:130 #3 0x0040068d in main (argc=optimized out, argv=optimized out) at lxc_info.c:63 Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [PATCH 2/2] cgroups: support cgroups mounted in multiple places
Looking at the sources and Serge's patch...
On Sun, 2011-06-26 at 13:33 -0400, Michael H. Warfield wrote:
On Sun, 2011-06-26 at 18:27 +0200, Daniel Lezcano wrote:
On 06/26/2011 05:52 PM, Michael H. Warfield wrote:
On Sun, 2011-06-26 at 17:27 +0200, Daniel Lezcano wrote:
On 06/26/2011 05:06 PM, Serge E. Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
cd /sys/fs/cgroup
for d in `/bin/ls`; do
echo 1 $d/cgroup.clone_children
done
Doing this step alone broke lxc totally for me, with or without the
patch below. This was on Fedora 15 testing with lxc 0.7.4.2 as well as
Do you have the ns cgroup mounted? The above is only for without
ns cgroup.
Yes, the ns_cgroup and clone_children are mutually exclusive.
If you try to create a namespace with a ns cgroup mounted and the
clone_children flag is set. The 'clone' will return EINVAL.
I noticed with the patch applied because there was a bug in it which was
setting this flag even if the cgroup was ns.
It is possible, you ran lxc with the patch, the clone_children was set,
and then you remove the patch and try to run lxc again. As the
clone_children is still set, that will make impossible to clone a new
namespace.
With the patch I sent previously on top of Serge's patch, that should be
fixed.
Ok... Got that an threw it into the mix.
I'm still seeing the segfaults on lxc-info. Seems to fault even if I
specify the name of a non-existent machine as well.
Oh, seems I missed this info :s
If you launch lxc-info -n foo, that segfaults ?
On what architecture does it fault ?
Fedora 15, x86_64.
0.7.4.2 is fine.
Just rebuilt a fresh set of packages from GIT sources alone. They're
fine and lxc-info from git works.
Git patched with Serge's patch. Segfault.
Git patched with both patches. Segfault.
Here's what gdb has to say about it:
[root@forest lxc]# gdb lxc-info core.13546
GNU gdb (GDB) Fedora (7.2.90.20110525-39.fc15)
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type show copying
and show warranty for details.
This GDB was configured as x86_64-redhat-linux-gnu.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from
/home/mhw/rpmbuild/BUILD/lxc-0.7.4.2git2/src/lxc/lxc-info...done.
[New LWP 13546]
Core was generated by `./lxc-info -n Alcove'.
Program terminated with signal 11, Segmentation fault.
#0 lxc_cgroup_path_get (path=0x7fff89b35928, subsystem=0x7fba52a4fdc1
freezer, name=0x7fff89b3987d Alcove) at cgroup.c:326
326 *s = '\0';
(gdb) trace
Tracepoint 1 at 0x7fba52a41e74: file cgroup.c, line 326.
(gdb) backtrace
#0 lxc_cgroup_path_get (path=0x7fff89b35928, subsystem=0x7fba52a4fdc1
freezer, name=0x7fff89b3987d Alcove) at cgroup.c:326
#1 0x7fba52a4a337 in freezer_state (name=optimized out) at state.c:74
#2 0x7fba52a4a42d in lxc_getstate (name=0x7fff89b3987d Alcove) at
state.c:130
#3 0x0040068d in main (argc=optimized out, argv=optimized out)
at lxc_info.c:63
Ok... That looks like this piece from that patch:
+/*
+ * lxc_cgroup_path_get: put into *path the pathname for
+ * %subsystem and cgroup %name. If %subsystem is NULL, then
+ * the first mounted cgroup will be used (for nr_tasks)
+ */
+int lxc_cgroup_path_get(char **path, const char *subsystem, const char
*name)
+{
+ static charbuf[MAXPATHLEN];
+ static charretbuf[MAXPATHLEN];
+
+ /* what lxc_cgroup_set calls subsystem is actually the filename,
i.e.
+ 'devices.allow'. So for our purposee we trim it */
+ if (subsystem) {
+ snprintf(retbuf, MAXPATHLEN, %s, subsystem);
+ char *s = index(retbuf, '.');
+ *s = '\0';
+ DEBUG(%s: called for subsys %s name %s\n, __func__,
retbuf, name);
}
Now wait a minute. Is that a typo here:
char *s = index(retbuf, '.');
If you're doing, in effect, a dirname here should that be this:
char *s = index(retbuf, '/');
IAC... That *s = '\0'; should include a NULL check.
Adding the NULL check and lxc-info works.
Looks like that subsystem name in the call to that routine is not what
Serge thought it was. I threw a print above the snprintf about just for
giggles to print out the subsystem name being passed to it and this is
what I got back...
[mhw@forest SPECS]$ sudo lxc-info -n Alcove
subsystem name: freezer
'Alcove' is RUNNING
No wonder s was null. No dot and no /.
Regards,
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we
Re: [Lxc-users] [PATCH 2/2] cgroups: support cgroups mounted in multiple places
On Sun, 2011-06-26 at 13:56 -0400, Michael H. Warfield wrote:
Looking at the sources and Serge's patch...
On Sun, 2011-06-26 at 13:33 -0400, Michael H. Warfield wrote:
On Sun, 2011-06-26 at 18:27 +0200, Daniel Lezcano wrote:
On 06/26/2011 05:52 PM, Michael H. Warfield wrote:
On Sun, 2011-06-26 at 17:27 +0200, Daniel Lezcano wrote:
On 06/26/2011 05:06 PM, Serge E. Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
cd /sys/fs/cgroup
for d in `/bin/ls`; do
echo 1 $d/cgroup.clone_children
done
Doing this step alone broke lxc totally for me, with or without the
patch below. This was on Fedora 15 testing with lxc 0.7.4.2 as well
as
Do you have the ns cgroup mounted? The above is only for without
ns cgroup.
Yes, the ns_cgroup and clone_children are mutually exclusive.
If you try to create a namespace with a ns cgroup mounted and the
clone_children flag is set. The 'clone' will return EINVAL.
I noticed with the patch applied because there was a bug in it which
was
setting this flag even if the cgroup was ns.
It is possible, you ran lxc with the patch, the clone_children was set,
and then you remove the patch and try to run lxc again. As the
clone_children is still set, that will make impossible to clone a new
namespace.
With the patch I sent previously on top of Serge's patch, that should
be
fixed.
Ok... Got that an threw it into the mix.
I'm still seeing the segfaults on lxc-info. Seems to fault even if I
specify the name of a non-existent machine as well.
Oh, seems I missed this info :s
If you launch lxc-info -n foo, that segfaults ?
On what architecture does it fault ?
Fedora 15, x86_64.
0.7.4.2 is fine.
Just rebuilt a fresh set of packages from GIT sources alone. They're
fine and lxc-info from git works.
Git patched with Serge's patch. Segfault.
Git patched with both patches. Segfault.
Here's what gdb has to say about it:
[root@forest lxc]# gdb lxc-info core.13546
GNU gdb (GDB) Fedora (7.2.90.20110525-39.fc15)
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type show copying
and show warranty for details.
This GDB was configured as x86_64-redhat-linux-gnu.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from
/home/mhw/rpmbuild/BUILD/lxc-0.7.4.2git2/src/lxc/lxc-info...done.
[New LWP 13546]
Core was generated by `./lxc-info -n Alcove'.
Program terminated with signal 11, Segmentation fault.
#0 lxc_cgroup_path_get (path=0x7fff89b35928, subsystem=0x7fba52a4fdc1
freezer, name=0x7fff89b3987d Alcove) at cgroup.c:326
326*s = '\0';
(gdb) trace
Tracepoint 1 at 0x7fba52a41e74: file cgroup.c, line 326.
(gdb) backtrace
#0 lxc_cgroup_path_get (path=0x7fff89b35928, subsystem=0x7fba52a4fdc1
freezer, name=0x7fff89b3987d Alcove) at cgroup.c:326
#1 0x7fba52a4a337 in freezer_state (name=optimized out) at state.c:74
#2 0x7fba52a4a42d in lxc_getstate (name=0x7fff89b3987d Alcove) at
state.c:130
#3 0x0040068d in main (argc=optimized out, argv=optimized out)
at lxc_info.c:63
Ok... That looks like this piece from that patch:
+/*
+ * lxc_cgroup_path_get: put into *path the pathname for
+ * %subsystem and cgroup %name. If %subsystem is NULL, then
+ * the first mounted cgroup will be used (for nr_tasks)
+ */
+int lxc_cgroup_path_get(char **path, const char *subsystem, const char
*name)
+{
+ static charbuf[MAXPATHLEN];
+ static charretbuf[MAXPATHLEN];
+
+ /* what lxc_cgroup_set calls subsystem is actually the filename,
i.e.
+ 'devices.allow'. So for our purposee we trim it */
+ if (subsystem) {
+ snprintf(retbuf, MAXPATHLEN, %s, subsystem);
+ char *s = index(retbuf, '.');
+ *s = '\0';
+ DEBUG(%s: called for subsys %s name %s\n, __func__,
retbuf, name);
}
Now wait a minute. Is that a typo here:
char *s = index(retbuf, '.');
If you're doing, in effect, a dirname here should that be this:
char *s = index(retbuf, '/');
IAC... That *s = '\0'; should include a NULL check.
Adding the NULL check and lxc-info works.
Looks like that subsystem name in the call to that routine is not what
Serge thought it was. I threw a print above the snprintf about just for
giggles to print out the subsystem name being passed to it and this is
what I got back...
[mhw@forest SPECS]$ sudo lxc-info -n Alcove
subsystem name: freezer
'Alcove' is RUNNING
No wonder s was null. No dot and no /.
I
Re: [Lxc-users] lxc-start at boot
On Sun, 2011-06-26 at 17:46 +0300, Mihamina Rakotomandimby wrote: Hi all, I usually launch my containers with 'sudo lxc-start -n name'. The problem with this is the fact it doesn't get my back to a shell prompt. To have what I want, I have to workarround, with screen or end the command line with ''. More globally, if the containers configuration files are all stored in /etc/lxc/, with the basename same as the container name, How to make /etc/lxc/ scanned for '*.conf' file and then all VMs started? If it's necessary, I'm running Debian Squeeze. I've got a fairly complex script I use to fire up containers modeled largely on the OpenVZ model. The problem here is that you may want containers to start at boot and ones you DON'T want to start at boot. You may also have experimental containers you want set up and configured but will only started if you force them. You'll also discover that a little load leveling helps if you think you are going to fire up 3 dozen VM's all at once. You'll find out rapidly that it's not a good idea to just start everything blindly for a number of reasons. I have some parameters I add to the configuration files (commented out if running stock code) like this: lxc-boot.onboot = yes lxc-boot.disabled = no These correspond directly with the equivalent OpenVZ features. Then I have a script (which still needs a lot of cleanup) which will go through all the configuration files and start VM's. If it's started with a -b option (boot time) it will only start VM's that are set to: .onboot = yes. If you run it manually, you can start one or more VM's but only if they are NOT disabled. A -f option will force firing up disabled VM's but only one at a time. When it starts a container, it dumps the output to a log file and forks it off as it's own process. The script use to use the iowait trick to restart containers and, now that's part of the base binaries, that's what needs cleaning up now. But this is all high level management stuff that needs to be looked at and decided upon. Maybe, one of these days, I'll finish cleaning it up and send it to Daniel. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [PATCH 2/2] cgroups: support cgroups mounted in multiple places
On Sun, 2011-06-26 at 14:00 -0500, Serge E. Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
Thanks, Michael, good catch.
Now wait a minute. Is that a typo here:
No it's not, but:
char *s = index(retbuf, '.');
If you're doing, in effect, a dirname here should that be this:
char *s = index(retbuf, '/');
IAC... That *s = '\0'; should include a NULL check.
Adding the NULL check and lxc-info works.
Looks like that subsystem name in the call to that routine is not what
Serge thought it was. I threw a print above the snprintf about just for
giggles to print out the subsystem name being passed to it and this is
what I got back...
[mhw@forest SPECS]$ sudo lxc-info -n Alcove
subsystem name: freezer
'Alcove' is RUNNING
No wonder s was null. No dot and no /.
I applied this patch and it got lxc-info working. But it was a quick
hack just to address the NULL pointer. Is it the correct fix?
No, it's not.
For the calls to this function that come from cgroup.c itself, '.' is the
right thing. The problem is that lxc_cgroup_set() and lxc_cgroup_get()
pass in things like 'devices.allow'. I was going to make the index
conditional, but all the callers of this function pass in either a filename
(with a '.' in it) or NULL.
I failed to notice these:
src/lxc/freezer.c: ret = lxc_cgroup_path_get(nsgroup, freezer, name);
src/lxc/state.c:err = lxc_cgroup_path_get(nsgroup, freezer, name);
:)
These are what you are running into.
So the thing to do is leave it searching for index(s, '.') but do nothing
if s is NULL.
And that's what I believe results with my little hack. Only truncate if
there was a hit and s was non-null. I see now from your comments that
the check on '.' was correct. I was uncertain about the inputs and
outputs in this routine. Checking for the NILL condition may not be
the solution, in this case, but it is still a best common practice to
check pointers like that. Never can tell what may crop up in the
future.
Really it would be cleaner to have lxc_cgroup_{sg}et() do the index, so
that lxc_cgorup_path_get() always gets a subsystem or NULL. I'm not doing
that patch right now, though, trivial as it ought to be.
I hear you.
thanks,
-serge
Regards,
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [PATCH 2/2] cgroups: support cgroups mounted in multiple places
On Sun, 2011-06-26 at 18:49 -0500, Serge E. Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
On Sun, 2011-06-26 at 14:00 -0500, Serge E. Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
Thanks, Michael, good catch.
Now wait a minute. Is that a typo here:
No it's not, but:
char *s = index(retbuf, '.');
If you're doing, in effect, a dirname here should that be this:
char *s = index(retbuf, '/');
IAC... That *s = '\0'; should include a NULL check.
Adding the NULL check and lxc-info works.
Looks like that subsystem name in the call to that routine is not what
Serge thought it was. I threw a print above the snprintf about just
for
giggles to print out the subsystem name being passed to it and this is
what I got back...
[mhw@forest SPECS]$ sudo lxc-info -n Alcove
subsystem name: freezer
'Alcove' is RUNNING
No wonder s was null. No dot and no /.
I applied this patch and it got lxc-info working. But it was a quick
hack just to address the NULL pointer. Is it the correct fix?
No, it's not.
For the calls to this function that come from cgroup.c itself, '.' is the
right thing. The problem is that lxc_cgroup_set() and lxc_cgroup_get()
pass in things like 'devices.allow'. I was going to make the index
conditional, but all the callers of this function pass in either a
filename
(with a '.' in it) or NULL.
I failed to notice these:
src/lxc/freezer.c: ret = lxc_cgroup_path_get(nsgroup, freezer,
name);
src/lxc/state.c:err = lxc_cgroup_path_get(nsgroup, freezer,
name);
:)
These are what you are running into.
So the thing to do is leave it searching for index(s, '.') but do nothing
if s is NULL.
And that's what I believe results with my little hack. Only truncate if
Oops, sorry, I didn't look closely enough and assumed your patch was
switching to checking for '/'.
there was a hit and s was non-null. I see now from your comments that
the check on '.' was correct. I was uncertain about the inputs and
outputs in this routine. Checking for the NILL condition may not be
the solution, in this case, but it is still a best common practice to
check pointers like that. Never can tell what may crop up in the
future.
Really it would be cleaner to have lxc_cgroup_{sg}et() do the index, so
that lxc_cgorup_path_get() always gets a subsystem or NULL. I'm not doing
that patch right now, though, trivial as it ought to be.
I hear you.
So
Acked-by: Serge Hallyn serge.hal...@ubuntu.com
to your patch to fix my bug, and let's leave it at that for now until it
gets more testing?
Cool. I now have it running in 3 environments. Two are Fedora 12 (I
know, I know, it's on my todo list) i686 systems with single cgroup
mounts while the other is my Fedora 15 x86_64 platform. All are running
quite smoothly and lxc-info is working.
Daniel: Did you pick up my little patch from a couple of messages back?
That message did not including the -devel list. Should I also send it
there?
Thanks again for testing and looking into it!
-serge
Regards,
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] read only rootfs
On Mon, 2011-06-27 at 17:20 +0100, Justin Cormack wrote: On Mon, 2011-06-27 at 18:05 +0200, Samuel Maftoul wrote: I tried several ways to have the rootfs mounted RO. First I removed the lxc.rootfs from my config file and the tried: - lxc-start -n vm0 -o /tmp/lxc-vm0.log -l DEBUG -s lxc.mount.entry=/ /var/lib/lxc/vm0/rootfs none ro,bind 0 0 Then I tried: - echo / /var/lib/lxc/vm0/rootfs none ro,bind 0 0 /var/lib/lxc/vm0/fstab ; lxc-start -n vm0 -o /tmp/lxc-vm0.log -l DEBUG -s lxc.mount = /var/lib/lxc/vm0/fstab Finally I tried to boot with lxc.rootfs pointing to the same content, but on it's block device, mounted read-only The system starts, I have a console, but in the logs I get: lxc_conf - ignoring mount point '/var/lib/lxc/vm0/rootfs/lib' lxc_conf - ignoring mount point '/var/lib/lxc/vm0/rootfs/usr/lib' and of course, If I ls these directories, I have nothing inside. Bind mounting the root fs is fine, but it will not bind mount file systems under this, so you will need to add these to your fstab too. It looks like you have /lib and /usr/lib mounted on separate file systems and need to bind mount these too? Bind mounts work but, iirc, there was (in the past) a problem that if the container did a remount, the remount would propagate to the parent device. That caused all sorts of headaches (and I know, I was suppose to retest that scenario ages ago and I haven't) like when a container remounted its rootfs ro during a shutdown it made partitions ro to the host. Very bad. This was also at the heart of the problem with shutdowns causing ptty failures for any subsequent connections an container starts (it made that fs ro). If you try to do this, you may have to prohibit mounts inside the containers to prohibit the remount problems. It would probably be a good idea to test it and see if the container can remount an ro mount point as rw and what the impact would be. Justin Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] read only rootfs
On Mon, 2011-06-27 at 12:33 -0500, C Anthony Risinger wrote: On Mon, Jun 27, 2011 at 12:06 PM, Michael H. Warfield m...@wittsend.com wrote: On Mon, 2011-06-27 at 17:20 +0100, Justin Cormack wrote: On Mon, 2011-06-27 at 18:05 +0200, Samuel Maftoul wrote: I tried several ways to have the rootfs mounted RO. First I removed the lxc.rootfs from my config file and the tried: - lxc-start -n vm0 -o /tmp/lxc-vm0.log -l DEBUG -s lxc.mount.entry=/ /var/lib/lxc/vm0/rootfs none ro,bind 0 0 Then I tried: - echo / /var/lib/lxc/vm0/rootfs none ro,bind 0 0 /var/lib/lxc/vm0/fstab ; lxc-start -n vm0 -o /tmp/lxc-vm0.log -l DEBUG -s lxc.mount = /var/lib/lxc/vm0/fstab Finally I tried to boot with lxc.rootfs pointing to the same content, but on it's block device, mounted read-only The system starts, I have a console, but in the logs I get: lxc_conf - ignoring mount point '/var/lib/lxc/vm0/rootfs/lib' lxc_conf - ignoring mount point '/var/lib/lxc/vm0/rootfs/usr/lib' and of course, If I ls these directories, I have nothing inside. Bind mounting the root fs is fine, but it will not bind mount file systems under this, so you will need to add these to your fstab too. It looks like you have /lib and /usr/lib mounted on separate file systems and need to bind mount these too? Bind mounts work but, iirc, there was (in the past) a problem that if the container did a remount, the remount would propagate to the parent device. That caused all sorts of headaches (and I know, I was suppose to retest that scenario ages ago and I haven't) like when a container remounted its rootfs ro during a shutdown it made partitions ro to the host. Very bad. This was also at the heart of the problem with shutdowns causing ptty failures for any subsequent connections an container starts (it made that fs ro). If you try to do this, you may have to prohibit mounts inside the containers to prohibit the remount problems. It would probably be a good idea to test it and see if the container can remount an ro mount point as rw and what the impact would be. does this happen when the container rootfs is marked as a slave/private mount? slaves et al should not propagate changes to the master/host. That's exactly the thing that needs to be tested. I don't know at this point but I do know at one point it did not work properly and it did propagate. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [PATCH 2/2] cgroups: support cgroups mounted in multiple places
On Fri, 2011-07-01 at 10:12 +0200, Daniel Lezcano wrote: On 06/27/2011 03:53 PM, Michael H. Warfield wrote: Cool. I now have it running in 3 environments. Two are Fedora 12 (I know, I know, it's on my todo list) i686 systems with single cgroup mounts while the other is my Fedora 15 x86_64 platform. All are running quite smoothly and lxc-info is working. That's cool :) Daniel: Did you pick up my little patch from a couple of messages back? That message did not including the -devel list. Should I also send it there? Thank Michael for investigating and fixing the bug. I have to look at Serge patch to check if it does not already fix the problem, otherwise I will fix it with your patch on top of it. I looked at his revised patch and it does incorporate my fix. I'll test it out a little later today but it looks good to me. Only different between what he did and what I did was in style. I tend to be a little more pedantic about braces and use them even when it's not necessary around a single line. Thanks -- Daniel Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Latest test results - Was: cgroups: support cgroups mounted in multiple places (v3)
On Sat, 2011-07-02 at 15:13 -0500, Serge Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
...
F15 systemd: Passed.
F12 single mount: Passed.
F13 single mount: Passed.
F14 single mount: Passed.
F14 libcgroup:Failed.
I had the default /etc/cgconfig.conf file and here are the results:
[root@berserker-base ~]# cat /etc/cgconfig.conf
#
# Copyright IBM Corporation. 2007
#
# Authors: Balbir Singh bal...@linux.vnet.ibm.com
# This program is free software; you can redistribute it and/or modify it
# under the terms of version 2.1 of the GNU Lesser General Public License
# as published by the Free Software Foundation.
#
# This program is distributed in the hope that it would be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# See man cgconfig.conf for further details.
#
# By default, mount all separately controllers
# to /cgroup/controller name
mount {
cpuset = /cgroup/cpuset;
cpu = /cgroup/cpu;
cpuacct = /cgroup/cpuacct;
memory = /cgroup/memory;
devices = /cgroup/devices;
freezer = /cgroup/freezer;
net_cls = /cgroup/net_cls;
ns = /cgroup/ns;
blkio = /cgroup/blkio;
}
[root@berserker-base ~]# uname -a
Linux berserker-base.wittsend.com 2.6.35.13-92.fc14.i686 #1 SMP Sat May 21
17:39:42 UTC 2011 i686 i686 i386 GNU/Linux
[root@berserker-base ~]# mount -t cgroup
cgroup on /cgroup/cpuset type cgroup (rw,relatime,cpuset)
cgroup on /cgroup/cpu type cgroup (rw,relatime,cpu)
cgroup on /cgroup/cpuacct type cgroup (rw,relatime,cpuacct)
cgroup on /cgroup/memory type cgroup (rw,relatime,memory)
cgroup on /cgroup/devices type cgroup (rw,relatime,devices)
cgroup on /cgroup/freezer type cgroup (rw,relatime,freezer)
cgroup on /cgroup/net_cls type cgroup (rw,relatime,net_cls)
cgroup on /cgroup/ns type cgroup (rw,relatime,ns)
cgroup on /cgroup/blkio type cgroup (rw,relatime,blkio)
[root@berserker-base ~]# lxc-start -n Ashaman
lxc-start: no ns_cgroup option specified
Just a thought - does F14's kernel not support clone_children? See output
of ls /cgroup/cpuset and see if /cgroup/cpuset/cgroup.clone_children exists.
If not, then yeah there's nothing that can be done without the ns cgroup.
1) The ns cgroup does exist and is mounted.
2) It works with the single mount point and clone children is not
present.
To answer your question...
[mhw@berserker-base ~]$ ls /cgroup/cpuset/
cgroup.event_controlcpuset.memory_spread_page
cgroup.procscpuset.memory_spread_slab
cpuset.cpu_exclusivecpuset.mems
cpuset.cpus cpuset.sched_load_balance
cpuset.mem_exclusivecpuset.sched_relax_domain_level
cpuset.mem_hardwall libvirt
cpuset.memory_migrate notify_on_release
cpuset.memory_pressure release_agent
cpuset.memory_pressure_enabled tasks
I guess not...
Thanks for testing!
If that's all I can do, I'll do the best I can. I'm currently up to my
eyeballs in another OpenSource project, some XAUTH coding in Openswan.
I need about 6 clones of me, I swear. :-P
-serge
Regards,
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Latest test results - Was: cgroups: support cgroups mounted in multiple places (v3)
On Sat, 2011-07-02 at 23:04 +, Serge E. Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
On Sat, 2011-07-02 at 15:13 -0500, Serge Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
...
F15 systemd: Passed.
F12 single mount: Passed.
F13 single mount: Passed.
F14 single mount: Passed.
F14 libcgroup:Failed.
I had the default /etc/cgconfig.conf file and here are the results:
[root@berserker-base ~]# cat /etc/cgconfig.conf
#
# Copyright IBM Corporation. 2007
#
# Authors: Balbir Singh bal...@linux.vnet.ibm.com
# This program is free software; you can redistribute it and/or modify
it
# under the terms of version 2.1 of the GNU Lesser General Public
License
# as published by the Free Software Foundation.
#
# This program is distributed in the hope that it would be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# See man cgconfig.conf for further details.
#
# By default, mount all separately controllers
# to /cgroup/controller name
mount {
cpuset = /cgroup/cpuset;
cpu = /cgroup/cpu;
cpuacct = /cgroup/cpuacct;
memory = /cgroup/memory;
devices = /cgroup/devices;
freezer = /cgroup/freezer;
net_cls = /cgroup/net_cls;
ns = /cgroup/ns;
blkio = /cgroup/blkio;
}
[root@berserker-base ~]# uname -a
Linux berserker-base.wittsend.com 2.6.35.13-92.fc14.i686 #1 SMP Sat May
21 17:39:42 UTC 2011 i686 i686 i386 GNU/Linux
[root@berserker-base ~]# mount -t cgroup
cgroup on /cgroup/cpuset type cgroup (rw,relatime,cpuset)
cgroup on /cgroup/cpu type cgroup (rw,relatime,cpu)
cgroup on /cgroup/cpuacct type cgroup (rw,relatime,cpuacct)
cgroup on /cgroup/memory type cgroup (rw,relatime,memory)
cgroup on /cgroup/devices type cgroup (rw,relatime,devices)
cgroup on /cgroup/freezer type cgroup (rw,relatime,freezer)
cgroup on /cgroup/net_cls type cgroup (rw,relatime,net_cls)
cgroup on /cgroup/ns type cgroup (rw,relatime,ns)
cgroup on /cgroup/blkio type cgroup (rw,relatime,blkio)
[root@berserker-base ~]# lxc-start -n Ashaman
lxc-start: no ns_cgroup option specified
Just a thought - does F14's kernel not support clone_children? See output
of ls /cgroup/cpuset and see if /cgroup/cpuset/cgroup.clone_children
exists.
If not, then yeah there's nothing that can be done without the ns cgroup.
1) The ns cgroup does exist and is mounted.
Yes but it needs to be composed with the others :)
2) It works with the single mount point and clone children is not
present.
Ok, cool. So the results make perfect sense.
Sounds like we are in violent agreement then.
Acked-by: Michael H. Warfield m...@wittsend.com
thanks,
-serge
Regards,
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Latest test results - Was: cgroups: support cgroups mounted in multiple places (v3)
On Fri, 2011-07-15 at 17:25 +0300, Ramez Hanna wrote: tested f14 and debian squeeze containers on f15 host (systemd) lxc-ps returns nothing for running container Confirmed broken for F14 container on F15 host. Working on F14 container on F14 host. Another point on the curve. lxc-ps -A On F15 host shows all the processes but container names are missing. On F14 host shows all the processes with correct container names. Regards, Mike [root@lxc ~]# lxc-info --name=f14 'f14' is RUNNING [root@lxc ~]# lxc-ps --name=f14 CONTAINERPID TTY TIME CMD [root@lxc ~]# used lxc from master built a few days ago (haven't seen any change in master since then) On Thu, Jul 14, 2011 at 12:38 AM, Ramez Hanna rha...@informatiq.org wrote: here is an rpm build for latest master (bcbd102cba31a0054fe4204a39b5e8a411cde42f) http://download.opensuse.org/repositories/home:/ramezhanna/Fedora_15/ for those who want to test on f15 I will keep following master so keep an eye here for updates as well On Thu, Jul 7, 2011 at 4:08 PM, Michael H. Warfield m...@wittsend.comwrote: On Wed, 2011-07-06 at 14:06 +0300, Ramez Hanna wrote: where can i get that patched code? should i clone master? Yeah, that would be one way. Daniel checked it into git. So building from git is probably the best way. It hasn't popped out into a release yet. Maybe soon. His shot to call. On Mon, Jul 4, 2011 at 1:47 AM, Daniel Lezcano daniel.lezc...@free.fr wrote: On 07/02/2011 07:28 PM, Michael H. Warfield wrote: Hey all... So my testing has continued and I've now regression tested the v3 patch and extended my testing. Looks like, over all, everything came together nicely. I'd ack that... [ ... ] IAC... The v3 patch does no harm to existing, working, cases and certainly covers the systemd case with F15 and that multipoint mount on /sys/fs/cgroup. The lxc stuff is broken on F15 without it. That's an important step forward and needs to be pushed. Not sure what the deal is here above with the libcgroup cgconfig service enabled on F14 (maybe I'm doing something wrong) but that should not be a show stopper as a mount point in fstab deals with that situation nicely. I'd like to see this applied ASAP and a turn cranked on the revision handle as this is needed for F15 and beyond. Great ! Thanks Michael for testing. Applied. -- Daniel Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on Lean Startup Secrets Revealed. This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Latest test results - Was: cgroups: support cgroups mounted in multiple places (v3)
On Fri, 2011-07-15 at 17:46 +0300, Ramez Hanna wrote: On Fri, Jul 15, 2011 at 5:38 PM, Michael H. Warfield m...@wittsend.comwrote: On Fri, 2011-07-15 at 17:25 +0300, Ramez Hanna wrote: tested f14 and debian squeeze containers on f15 host (systemd) lxc-ps returns nothing for running container Confirmed broken for F14 container on F15 host. Working on F14 container on F14 host. I wonder what would be the difference between f14 and f15 in regards to the layout of cgroups that makes it work on f14 (libcgroup) and not f15(systemd) On F15 systemd has cgroups mounted with multiple mount points. On F14 you probably don't have libgroup enabled and are using a single mount point. Run the command mount -t cgroup on each and compare. ITMT... Try the patch on the -devel list with the following subject: PATCH: lxc-ps works with systemd Seems to fix the problem. Run the patch against the git master. Regards, Mike Another point on the curve. lxc-ps -A On F15 host shows all the processes but container names are missing. I confirm that On F14 host shows all the processes with correct container names. Regards, Mike [root@lxc ~]# lxc-info --name=f14 'f14' is RUNNING [root@lxc ~]# lxc-ps --name=f14 CONTAINERPID TTY TIME CMD [root@lxc ~]# used lxc from master built a few days ago (haven't seen any change in master since then) On Thu, Jul 14, 2011 at 12:38 AM, Ramez Hanna rha...@informatiq.org wrote: here is an rpm build for latest master (bcbd102cba31a0054fe4204a39b5e8a411cde42f) http://download.opensuse.org/repositories/home:/ramezhanna/Fedora_15/ for those who want to test on f15 I will keep following master so keep an eye here for updates as well On Thu, Jul 7, 2011 at 4:08 PM, Michael H. Warfield m...@wittsend.com wrote: On Wed, 2011-07-06 at 14:06 +0300, Ramez Hanna wrote: where can i get that patched code? should i clone master? Yeah, that would be one way. Daniel checked it into git. So building from git is probably the best way. It hasn't popped out into a release yet. Maybe soon. His shot to call. On Mon, Jul 4, 2011 at 1:47 AM, Daniel Lezcano daniel.lezc...@free.fr wrote: On 07/02/2011 07:28 PM, Michael H. Warfield wrote: Hey all... So my testing has continued and I've now regression tested the v3 patch and extended my testing. Looks like, over all, everything came together nicely. I'd ack that... [ ... ] IAC... The v3 patch does no harm to existing, working, cases and certainly covers the systemd case with F15 and that multipoint mount on /sys/fs/cgroup. The lxc stuff is broken on F15 without it. That's an important step forward and needs to be pushed. Not sure what the deal is here above with the libcgroup cgconfig service enabled on F14 (maybe I'm doing something wrong) but that should not be a show stopper as a mount point in fstab deals with that situation nicely. I'd like to see this applied ASAP and a turn cranked on the revision handle as this is needed for F15 and beyond. Great ! Thanks Michael for testing. Applied. -- Daniel Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on Lean Startup Secrets Revealed. This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] what's the difference in lxc-attach
On Fri, 2011-07-15 at 17:50 +0300, Ramez Hanna wrote: how can i check if lxc-attach is not working because of the kernel or because of other bug? On Thu, Apr 7, 2011 at 10:09 AM, Cedric Le Goater legoa...@free.fr wrote: On 04/07/2011 07:46 AM, Ramez Hanna wrote: from a post that i found earlier in the archive subject entering a container by Daniel Lezcano i cannot see the differece between lxc-attach and lxc-execute could someone explain? lxc-execute creates a container and exec's a command/application inside it (see manual). lxc-attach enters a *running* container and exec's a command inside it (manual soon to come). This ability of creating an exogenous process inside a container requires a kernel patchset. Has that patch set even made it into a release? If so, what version is it in and what version are you running. It does not work on my F15 system with a 2.6.38 kernel. If it has not made it into a released kernel, have you built a custom kernel with it? C. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on Lean Startup Secrets Revealed. This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] what's the difference in lxc-attach
On Fri, 2011-07-15 at 18:36 +0300, Ramez Hanna wrote: On Fri, Jul 15, 2011 at 6:04 PM, Michael H. Warfield m...@wittsend.comwrote: On Fri, 2011-07-15 at 17:50 +0300, Ramez Hanna wrote: how can i check if lxc-attach is not working because of the kernel or because of other bug? On Thu, Apr 7, 2011 at 10:09 AM, Cedric Le Goater legoa...@free.fr wrote: On 04/07/2011 07:46 AM, Ramez Hanna wrote: from a post that i found earlier in the archive subject entering a container by Daniel Lezcano i cannot see the differece between lxc-attach and lxc-execute could someone explain? lxc-execute creates a container and exec's a command/application inside it (see manual). lxc-attach enters a *running* container and exec's a command inside it (manual soon to come). This ability of creating an exogenous process inside a container requires a kernel patchset. Has that patch set even made it into a release? If so, what version is it in and what version are you running. It does not work on my F15 system with a 2.6.38 kernel. If it has not made it into a released kernel, have you built a custom kernel with it? I don't know about that patch, so hence my question if there is anyway to know from the host if that capability is available From what I can tell, based on some threads from back in March, the patchset has not been merged into the upstream kernel at this time and is almost certainly NOT in 2.6.38.*. I'm currently running Fedora 15 2.6.38.8-32.fc15.x86_64 which does not have the patch and lxc-attach gives this error: [root@forest Alcove]# lxc-attach --name Alcove lxc-attach: Does this kernel version support 'attach' ? lxc-attach: failed to enter the namespace That's probably about the best answer you're going to get. From what I can tell, the last patchset is here: http://lxc.sourceforge.net/patches/linux/2.6.38/ If you want it, you're probably going to have to build yourself a custom kernel with it patched in. Some of the patches have been merged into the upstream kernel but it's not clear to me if we'll have to wait for 3.0 to be released to see them but I suspect that to be the case. We're currently sitting at 3.0-rc7 on that one. 2.6.39.3 is released and stable nut I have no clue what's in there. 2.6.38 is currently at 2.6.38.8, which is what we see in F15 so it is what it is. C. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on Lean Startup Secrets Revealed. This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] read only rootfs
On Mon, 2011-07-04 at 22:16 +0200, Matto Fransen wrote: Hi, On Mon, Jun 27, 2011 at 06:05:13PM +0200, Samuel Maftoul wrote: I'm searching for a solution to have a read only rootfs inside an LXC container. I have a webserver running this way :) I created a container with the busybox template, this container works. As soon as I try to mount it read only I have this message in the logs: Create a rootfs outside the container. In the config of your container you add lines like: lxc.mount.entry=/path/to/rootfs/lib /var/lib/lxc/container/rootfs/lib none ro,bind 0 0 and so on for all the dir's you want to mount readonly Also create some system directories: # system mounts lxc.mount.entry=proc /var/lib/lxc/container/rootfs/proc proc none defaults 0 0 lxc.mount.entry=shmfs /var/lib/lxc/container/rootfs/dev/shm tmpfs mode=0644 0 0 lxc.mount.entry=sysfs /var/lib/lxc/container/rootfs/sys sysfs defaults 0 0 I find the bind mount is providing protection for propagating mount point option changes from the guest to the host or to other guests. That's good. Unfortunately, I also still find that if there's a -o remount,ro in the halt/reboot script, it still sets /dev/pts to ro and that still propagates to the host and to the other containers triggering random acts of terrorism like unable to create pty/0 in the containers and inability to start new containers in the host. Not sure if we can apply a bind to that or not. I've been commenting those remounts out of the halt and reboot scripts but I recently upgraded some containers from F12 to F13 and then to F14 and that installed new scripts. Since I was commenting out those lines automatically when I started the containers, I didn't catch it and it clobbered the host. Fortunately, I caught it and just did a mount -o remount,rw /dev/pts in the host and everything was back to normal. Guess I'll do a preshutdown script that runs in the containers to catch that and prevent it but it doesn't change the fact a the container can have a serious impact on the host. And add the following line to the config of your container: lxc.cap.drop=sys_admin Doing this also triggers random acts of terrorism in my containers. The sys_admin cap is far to broad and affects far to many other things like setting the host names. It would also prevent using the loopback device and setting crypto keys on loopback devices and prevent mounting other file systems like iso images and encrypted file systems or nfs file systems from within the container. I can only assume that autofs would be totally broken. This last line prevents that one can jumo out of the readonly bind mounts from inside the container :) We need a better way than that. This needs kernel work in the container logic but, if something is mounted ro outside the container, the container should treat it as ro hardware and disallow rw remounting. That's one thing. The kernel should also prohibit, totally, the propagation of remount options from inside a container to the outer host or to other containers. That is tantamount to a security vulnerability and clearly a violation of container isolation. It amounts to a semi-local Denial of Service (DoD) vulnerability (semi-local because it's local to that machine and its constellation of guests but it propagates between guests). I don't see any clean fix for this that doesn't involve creating other problems (like dropping sys_admin) which violates the ability to virtualize the machines properly. Cheers, Matto -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on Lean Startup Secrets Revealed. This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc
Re: [Lxc-users] what's the difference in lxc-attach
On Fri, 2011-07-15 at 19:41 +0300, Ramez Hanna wrote: On Fri, Jul 15, 2011 at 7:28 PM, Michael H. Warfield m...@wittsend.comwrote: On Fri, 2011-07-15 at 18:36 +0300, Ramez Hanna wrote: On Fri, Jul 15, 2011 at 6:04 PM, Michael H. Warfield m...@wittsend.com wrote: On Fri, 2011-07-15 at 17:50 +0300, Ramez Hanna wrote: how can i check if lxc-attach is not working because of the kernel or because of other bug? On Thu, Apr 7, 2011 at 10:09 AM, Cedric Le Goater legoa...@free.fr wrote: On 04/07/2011 07:46 AM, Ramez Hanna wrote: from a post that i found earlier in the archive subject entering a container by Daniel Lezcano i cannot see the differece between lxc-attach and lxc-execute could someone explain? lxc-execute creates a container and exec's a command/application inside it (see manual). lxc-attach enters a *running* container and exec's a command inside it (manual soon to come). This ability of creating an exogenous process inside a container requires a kernel patchset. Has that patch set even made it into a release? If so, what version is it in and what version are you running. It does not work on my F15 system with a 2.6.38 kernel. If it has not made it into a released kernel, have you built a custom kernel with it? I don't know about that patch, so hence my question if there is anyway to know from the host if that capability is available From what I can tell, based on some threads from back in March, the patchset has not been merged into the upstream kernel at this time and is almost certainly NOT in 2.6.38.*. I'm currently running Fedora 15 2.6.38.8-32.fc15.x86_64 which does not have the patch and lxc-attach gives this error: [root@forest Alcove]# lxc-attach --name Alcove lxc-attach: Does this kernel version support 'attach' ? lxc-attach: failed to enter the namespace That's probably about the best answer you're going to get. From what I can tell, the last patchset is here: http://lxc.sourceforge.net/patches/linux/2.6.38/ If you want it, you're probably going to have to build yourself a custom kernel with it patched in. Some of the patches have been merged into the upstream kernel but it's not clear to me if we'll have to wait for 3.0 to be released to see them but I suspect that to be the case. We're currently sitting at 3.0-rc7 on that one. 2.6.39.3 is released and stable nut I have no clue what's in there. 2.6.38 is currently at 2.6.38.8, which is what we see in F15 so it is what it is. C. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! thanks a lot for the detailed answer by the way have you been succesfull in starting a f15 container on your f15? I have been debuggin for 2 hours now when i start f15 container it screws my host by interfering with my hosts's systemd which somehow doesn't make sense and when i use systemd-nspawn i get a bunch of errors and the system doesn't finish starting here is a paste of systemd log from systemd-nspawn session http://pastie.org/2218625 I haven't tried it yet. Will see what I can do. Couple of quick questions. 1) You say it screws your host if you don't uses nospawn. What happens? 2) Have you disabled the sys_admin cap by dropping it in that container? I find that causes me all sorts of grief. 3) Was this a fresh template build or did you upgrade an F14 machine to F15 (I was going to use yum --releasever=15 distro-sync in one of my running F14 containers). Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on Lean Startup Secrets Revealed. This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] what's the difference in lxc-attach
On Sat, 2011-07-16 at 23:59 +0300, Ramez Hanna wrote:
On Sat, Jul 16, 2011 at 8:27 PM, Michael H. Warfield m...@wittsend.comwrote:
On Fri, 2011-07-15 at 20:17 +0300, Ramez Hanna wrote:
Big Snip
thanks a lot for the detailed answer
by the way have you been succesfull in starting a f15 container on
your
f15?
I now have an F15 container working.
I have been debuggin for 2 hours now
when i start f15 container it screws my host by interfering with my
hosts's
systemd which somehow doesn't make sense
and when i use systemd-nspawn i get a bunch of errors and the system
doesn't
finish starting
here is a paste of systemd log from systemd-nspawn session
http://pastie.org/2218625
I haven't tried it yet. Will see what I can do.
Couple of quick questions.
1) You say it screws your host if you don't uses nospawn. What
happens?
host console is not useable, random issues around missing characters when
i
type
unable to login on other terminals because i cannot type
and i see so many systemd logs on the console
I have a very strong suspicion that systemd is not going to be
compatible with running in a container because it wants to set up and
managed cgroups in the container which it can not do.
When I try to start it with systemd, the first process doesn't even seem
to come up (number of tasks is 0) and then the host can not remove the
container even after I've done an lxc-stop on it. But that's when I'm
logged in and running lxc-start from an ssh terminal Window. If I start
it from a real ttyX console then I get all sorts of startup messages
from the container and the consoles are hosed up like the console in the
container has gotten crosswise with the console in the host. Things try
to initialize but all sorts of things time out and eventually I have to
reset the host with an Magic SysRq sequence.
Gave up on systemd.
2) Have you disabled the sys_admin cap by dropping it in that
container?
I find that causes me all sorts of grief.
i will try that
Don't. It wouldn't do any good and causes lots of other problems (for
me at least).
3) Was this a fresh template build or did you upgrade an F14 machine to
F15 (I was going to use yum --releasever=15 distro-sync in one of my
running F14 containers).
yes fresh install
Here's what I've done and now gotten an F15 container to work.
I started out with an F14 container and upgraded it to F15 using the
yum --releasever=15 distro-sync method. I was able to reproduce your
problems above and thought there may be some conflicts over cgroups so I
decided to disable systemd.
If it's not present (it wasn't for me) install upstart into the
container from the host using yum --installroot={your VM root}
upstart.
Next cd to {your VM root}/sbin and rm init (which is symlinked
to ../bin/systemd) and symlink it to upstart (which is in sbin).
This got me almost there. The machine was starting but I was having
your funky console problem and I realized (largely because I'm working
on other related problems) that it was the ptmx device causing this. It
was mapping incorrectly.
So, cd to {your VM root}/dev and rm ptmx if it's a hard device and not a
symlink. Then symlink pts/ptmx to ptmx. If you started with some sort
of template, this may already be done and you may not run into this
problem at all.
Now you should be able to fire your F15 container up.
Also find the lines in /etc/init.d/halt that remount file systems ro or
you'll screw your /dev/pts fs in the host when you shut that container
down or reboot it (and, no, newinstance is not helping with that
problem).
Regards,
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 |
http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of
all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
it is very clear to me that systemd is interfering with the host's systemd
your solution of running f15 is not much different than running a f14
container (as systemd is the major diff)
systemd-nspawn can start systemd inside a light weight container
i think the problem is related to the fact that when lxc starts teh cgroup
is on the root of the tree
while it should have been under the user's tree
I'm not so sure I understand what you mean by that last line. What
user's tree are you referring to?
maybe serge can say somethiing about this
Maybe, maybe not.
The cgroup mounts are where systemd is putting them, not where lxc is
putting them. As it is, lxc is not starting the cgroup anywhere, it's
just using them where they are found. And systemd-nspawn has nothing to
do with lxc.
Seems to me
Re: [Lxc-users] [PATCH] Re: read only rootfs
On Mon, 2011-07-18 at 07:31 -0500, Serge E. Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
Unfortunately, I also still find that if there's a -o remount,ro in the
halt/reboot script, it still sets /dev/pts to ro and that still
propagates to the host and to the other containers triggering random
Wow.
Did a quick grep; is there any reason why lxc-start doesn't turn on
MS_SLAVE for the client's root? Something like:
From 7fbc3ec940403605c53b253d8630c3f47fad154c Mon Sep 17 00:00:00 2001
From: Serge Hallyn serge.hal...@ubuntu.com
Date: Mon, 18 Jul 2011 07:29:57 -0500
Subject: [PATCH 1/1] (untested) turn container rootfs into MS_SLAVE
Signed-off-by: Serge Hallyn serge.hal...@ubuntu.com
---
src/lxc/conf.c |5 +
1 files changed, 5 insertions(+), 0 deletions(-)
diff --git a/src/lxc/conf.c b/src/lxc/conf.c
index 2eb598b..d36fe47 100644
--- a/src/lxc/conf.c
+++ b/src/lxc/conf.c
@@ -732,6 +732,11 @@ static int setup_rootfs(const struct lxc_rootfs *rootfs)
return -1;
}
+ if (mount(rootfs-path, rootfs-path, none, MS_SLAVE, 0)) {
+ ERROR(failed to turn child rootfs into slave);
+ return -1;
+ }
+
DEBUG(mounted '%s' on '%s', rootfs-path, rootfs-mount);
return 0;
--
1.7.4.1
The kernel should also prohibit, totally, the propagation of remount
The kernel doesn't know about containers, so it's up to userspace :)
Acceptable and that's reasonable. But, for security and integrity
reasons, it has to be the host / parent user space not the guest / child
user space which can make that decision. And it has to be the default.
Anything less is a security vulnerability.
-serge
Regards,
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
Magic Quadrant for Content-Aware Data Loss Prevention
Research study explores the data loss prevention market. Includes in-depth
analysis on the changes within the DLP market, and the criteria used to
evaluate the strengths and weaknesses of these DLP solutions.
http://www.accelacomm.com/jaw/sfnl/114/51385063/___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [PATCH] Re: read only rootfs
On Mon, 2011-07-18 at 07:31 -0500, Serge E. Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
Unfortunately, I also still find that if there's a -o remount,ro in the
halt/reboot script, it still sets /dev/pts to ro and that still
propagates to the host and to the other containers triggering random
Wow.
Did a quick grep; is there any reason why lxc-start doesn't turn on
MS_SLAVE for the client's root? Something like:
From 7fbc3ec940403605c53b253d8630c3f47fad154c Mon Sep 17 00:00:00 2001
From: Serge Hallyn serge.hal...@ubuntu.com
Date: Mon, 18 Jul 2011 07:29:57 -0500
Subject: [PATCH 1/1] (untested) turn container rootfs into MS_SLAVE
Signed-off-by: Serge Hallyn serge.hal...@ubuntu.com
---
src/lxc/conf.c |5 +
1 files changed, 5 insertions(+), 0 deletions(-)
diff --git a/src/lxc/conf.c b/src/lxc/conf.c
index 2eb598b..d36fe47 100644
--- a/src/lxc/conf.c
+++ b/src/lxc/conf.c
@@ -732,6 +732,11 @@ static int setup_rootfs(const struct lxc_rootfs *rootfs)
return -1;
}
+ if (mount(rootfs-path, rootfs-path, none, MS_SLAVE, 0)) {
+ ERROR(failed to turn child rootfs into slave);
+ return -1;
+ }
+
DEBUG(mounted '%s' on '%s', rootfs-path, rootfs-mount);
return 0;
--
1.7.4.1
Problem...
lxc-start 1311083210.678 ERRORlxc_conf - failed to turn child rootfs
into slave
lxc-start 1311083210.678 ERRORlxc_conf - failed to setup rootfs for
'Alcove'
lxc-start 1311083210.678 ERRORlxc_start - failed to setup the
container
lxc-start 1311083210.679 ERRORlxc_sync - invalid sequence number 1.
expected 2
lxc-start 1311083210.679 ERRORlxc_start - failed to spawn 'Alcove'
This may be do to the way I have my rootfs set up. Mine are bind mounts
from a private area to the common mount point. I've noticed that inside
the containers, I see my root file system show up twice like this (over
on one of my production machines):
[mhw@Berserker ~]$ df
Filesystem 1K-blocks Used Available Use% Mounted on
rootfs 480719056 404550544 51749312 89% /
/dev/sdb1480719056 404550544 51749312 89% /
/dev/sdc1480719056202816 456097040 1% /export
/dev/sda7693727244 624715768 33772100 95% /srv/shared
none 1030612 0 1030612 0% /dev/shm
[mhw@berserker-base ~]$ cat /srv/lxc/config/Berserker.fstab
/srv/lxc/private/Berserker /srv/lxc/rootfs none bind 0 0
/export /srv/lxc/rootfs/exportnone bind 0 0
/home/shared /srv/lxc/rootfs/srv/sharednone bind 0 0
Maybe not such a good idea (mine)?
Regards,
Mike
The kernel should also prohibit, totally, the propagation of remount
The kernel doesn't know about containers, so it's up to userspace :)
-serge
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
Magic Quadrant for Content-Aware Data Loss Prevention
Research study explores the data loss prevention market. Includes in-depth
analysis on the changes within the DLP market, and the criteria used to
evaluate the strengths and weaknesses of these DLP solutions.
http://www.accelacomm.com/jaw/sfnl/114/51385063/___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [PATCH] Re: read only rootfs
On Tue, 2011-07-19 at 09:55 -0400, Michael H. Warfield wrote:
On Mon, 2011-07-18 at 07:31 -0500, Serge E. Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
Unfortunately, I also still find that if there's a -o remount,ro in the
halt/reboot script, it still sets /dev/pts to ro and that still
propagates to the host and to the other containers triggering random
Wow.
Did a quick grep; is there any reason why lxc-start doesn't turn on
MS_SLAVE for the client's root? Something like:
From 7fbc3ec940403605c53b253d8630c3f47fad154c Mon Sep 17 00:00:00 2001
From: Serge Hallyn serge.hal...@ubuntu.com
Date: Mon, 18 Jul 2011 07:29:57 -0500
Subject: [PATCH 1/1] (untested) turn container rootfs into MS_SLAVE
Signed-off-by: Serge Hallyn serge.hal...@ubuntu.com
---
src/lxc/conf.c |5 +
1 files changed, 5 insertions(+), 0 deletions(-)
diff --git a/src/lxc/conf.c b/src/lxc/conf.c
index 2eb598b..d36fe47 100644
--- a/src/lxc/conf.c
+++ b/src/lxc/conf.c
@@ -732,6 +732,11 @@ static int setup_rootfs(const struct lxc_rootfs
*rootfs)
return -1;
}
+ if (mount(rootfs-path, rootfs-path, none, MS_SLAVE, 0)) {
+ ERROR(failed to turn child rootfs into slave);
+ return -1;
+ }
+
DEBUG(mounted '%s' on '%s', rootfs-path, rootfs-mount);
return 0;
--
1.7.4.1
Problem...
lxc-start 1311083210.678 ERRORlxc_conf - failed to turn child
rootfs into slave
lxc-start 1311083210.678 ERRORlxc_conf - failed to setup rootfs for
'Alcove'
lxc-start 1311083210.678 ERRORlxc_start - failed to setup the
container
lxc-start 1311083210.679 ERRORlxc_sync - invalid sequence number 1.
expected 2
lxc-start 1311083210.679 ERRORlxc_start - failed to spawn 'Alcove'
This may be do to the way I have my rootfs set up. Mine are bind mounts
from a private area to the common mount point. I've noticed that inside
the containers, I see my root file system show up twice like this (over
on one of my production machines):
Or, maybe not. Tried specifying the private directory as the rootfs
mount point directly and still got the same error, no bind mount in
fstab involved. :-P
Mike
Snip
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
Magic Quadrant for Content-Aware Data Loss Prevention
Research study explores the data loss prevention market. Includes in-depth
analysis on the changes within the DLP market, and the criteria used to
evaluate the strengths and weaknesses of these DLP solutions.
http://www.accelacomm.com/jaw/sfnl/114/51385063/___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [PATCH] Re: read only rootfs
On Mon, 2011-07-18 at 07:31 -0500, Serge E. Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
Unfortunately, I also still find that if there's a -o remount,ro in the
halt/reboot script, it still sets /dev/pts to ro and that still
propagates to the host and to the other containers triggering random
Wow.
Did a quick grep; is there any reason why lxc-start doesn't turn on
MS_SLAVE for the client's root? Something like:
From 7fbc3ec940403605c53b253d8630c3f47fad154c Mon Sep 17 00:00:00 2001
From: Serge Hallyn serge.hal...@ubuntu.com
Date: Mon, 18 Jul 2011 07:29:57 -0500
Subject: [PATCH 1/1] (untested) turn container rootfs into MS_SLAVE
Signed-off-by: Serge Hallyn serge.hal...@ubuntu.com
---
src/lxc/conf.c |5 +
1 files changed, 5 insertions(+), 0 deletions(-)
diff --git a/src/lxc/conf.c b/src/lxc/conf.c
index 2eb598b..d36fe47 100644
--- a/src/lxc/conf.c
+++ b/src/lxc/conf.c
@@ -732,6 +732,11 @@ static int setup_rootfs(const struct lxc_rootfs *rootfs)
return -1;
}
+ if (mount(rootfs-path, rootfs-path, none, MS_SLAVE, 0)) {
+ ERROR(failed to turn child rootfs into slave);
+ return -1;
+ }
+
DEBUG(mounted '%s' on '%s', rootfs-path, rootfs-mount);
return 0;
--
1.7.4.1
I poked at this some more.
First, I stuck a perror in that if check right after the mount to find
out what it was griping about...
[root@forest SPECS]# lxc-start --name Alcove
setup_rootfs, MS_SLAVE: Invalid argument
lxc-start: failed to turn child rootfs into slave
lxc-start: failed to setup rootfs for 'Alcove'
lxc-start: failed to setup the container
lxc-start: invalid sequence number 1. expected 2
lxc-start: failed to spawn 'Alcove'
Yuck. Invalid argument? Then I started thinking... Wasn't something
missing in that command. Wouldn't you need those flags to be
MS_REMOUNT | MS_SLAVE since you're remounting? Tried that. No joy.
Same error.
Sooo... I looked up at mount_rootfs called from the stanza above it.
Tried adding the MS_SLAVE to the mount command proper in
mount_rootfs_dir and commenting out the remount block entirely. The
file system mounts properly but adding MS_SLAVE didn't solve the
problem. :-P That should have been the right spot.
Then I noticed a discrepancy. In mount_rootfs_dir mount it called with
path and target while back in setup_rootfs the remount is calling it
with path and path. Ah ha... So I changed that mount to read like
this:
if (mount(rootfs-path, rootfs-mount, none, MS_REMOUNT|MS_SLAVE, 0)) {
Again. Fixed the code but not the problem.
I think the problem is that you are only doing this on the rootfs and
that flag does not automagically propagate to the submounts. That's
sort of implied by the way the man page for the mount command reads. It
doesn't mention a slave option on individual file systems but has a
--make-rslave function which recursively makes that mountpoint and all
its submountpoints into slaves. Tried running mount
--make-slave /dev/pts in the container and didn't get an error but still
didn't fix the problem but I figured that would be too easy and it
needed to be done back in lxc proper.
At that point, I went through src/lxc/conf.c and added MS_SLAVE to every
mount call. Yes, overly aggressive but just for a shotgun test. It
ran. But still failed the test. If I remount /dev/pts as ro in the
container, I still get it ro in the host. Then remount rw in the
container and it's back to rw in the host. Is this a kernel bug or what
am I missing here?
Strange too (and I just noticed this) is that I have two mount instances
of devpts on /dev/pts:
[root@alcove mhw]# mount -t devpts
devpts on /dev/console type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/tty1 type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/tty2 type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/tty3 type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/tty4 type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/tty5 type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/tty6 type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/pts type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/pts type devpts (rw,relatime,mode=600,ptmxmode=666)
I don't have any devpts mounts in either the container /etc/fstab, the
fstab lxc is referencing, or in any lxc.mount.entry in the config file.
Why are there two of them there and is that indicative of a problem?
Regards,
Mike
The kernel should also prohibit, totally, the propagation of remount
The kernel doesn't know about containers, so it's up to userspace :)
-serge
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist
Re: [Lxc-users] read only rootfs
On Tue, 2011-07-19 at 13:34 -0500, Serge E. Hallyn wrote: Quoting C Anthony Risinger (anth...@xtfx.me): there it would seem. however, while i could *maybe* see the rootfs being an unconditional slave, i would NOT want to see any lxc default/enforcement preventing container - host propagation on a globally recursive scale. im of the opinion that the implementor should decide the best tactic ... especially in light of the fact the one distro may not even have the same problems as say ubutnu/fedora/etc because they keep mount points private by default. Good point. (I don't see it on ubuntu either fwiw) Perhaps there should be a toggle in the per-container config file? Quick question. Is there any way to test for these flags (SHARED, PRIVATE, SLAVE)? I don't see them showing up anywhere from mount, in proc mounts or mountstats. How do you check to see if they are set? Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Magic Quadrant for Content-Aware Data Loss Prevention Research study explores the data loss prevention market. Includes in-depth analysis on the changes within the DLP market, and the criteria used to evaluate the strengths and weaknesses of these DLP solutions. http://www.accelacomm.com/jaw/sfnl/114/51385063/___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] read only rootfs
On Tue, 2011-07-19 at 16:50 -0400, Michael H. Warfield wrote: On Tue, 2011-07-19 at 15:32 -0500, Serge E. Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Tue, 2011-07-19 at 13:34 -0500, Serge E. Hallyn wrote: Quoting C Anthony Risinger (anth...@xtfx.me): there it would seem. however, while i could *maybe* see the rootfs being an unconditional slave, i would NOT want to see any lxc default/enforcement preventing container - host propagation on a globally recursive scale. im of the opinion that the implementor should decide the best tactic ... especially in light of the fact the one distro may not even have the same problems as say ubutnu/fedora/etc because they keep mount points private by default. Good point. (I don't see it on ubuntu either fwiw) Perhaps there should be a toggle in the per-container config file? Quick question. Is there any way to test for these flags (SHARED, PRIVATE, SLAVE)? I don't see them showing up anywhere from mount, in proc mounts or mountstats. How do you check to see if they are set? /proc/self/mountinfo is supposed to tell that. i.e. if you do a --make-shared on /mnt, it'll show 'shared' next to the /mnt entry. (I say 'is supposed to' bc --make-rslave just shows nothing, but maybe that's bc the way i did it it wasn't a slave to anything, so it was actually private) Ok... This may be telling us something. What? Oh, meant to include the info on / on each, sorry... On the host Forest: 22 1 8:5 / / rw,relatime - ext4 /dev/sda5 rw,barrier=1,data=ordered [root@forest ~]# cat /proc/self/mountinfo | grep export 50 22 8:18 / /export rw,relatime - ext4 /dev/sdb2 rw,barrier=1,data=ordered [root@forest ~]# mount --make-shared /export [root@forest ~]# cat /proc/self/mountinfo | grep export 50 22 8:18 / /export rw,relatime shared:1 - ext4 /dev/sdb2 rw,barrier=1,data=ordered [root@forest ~]# mount --make-slave /export [root@forest ~]# cat /proc/self/mountinfo | grep export 50 22 8:18 / /export rw,relatime - ext4 /dev/sdb2 rw,barrier=1,data=ordered [root@forest ~]# mount --make-private /export [root@forest ~]# cat /proc/self/mountinfo | grep export 50 22 8:18 / /export rw,relatime - ext4 /dev/sdb2 rw,barrier=1,data=ordered So, shared looks like it worked and the other two didn't? Does something have to be done to enable them? You say maybe that's bc the way i did it it wasn't a slave to anything meaning we're missing a step. What's the missing step. On the guest Alcove (with your patch to add MS_REC | MS_SLAVE): 105 55 8:17 /lxc/private/Alcove / rw,relatime - ext4 /dev/sdb1 rw,barrier=1,data=ordered [root@alcove mhw]# cat /proc/self/mountinfo | grep devpts 107 105 0:10 /6 /dev/console rw,relatime - devpts devpts rw,mode=600,ptmxmode=666 108 105 0:10 /0 /dev/tty1 rw,relatime - devpts devpts rw,mode=600,ptmxmode=666 109 105 0:10 /1 /dev/tty2 rw,relatime - devpts devpts rw,mode=600,ptmxmode=666 110 105 0:10 /2 /dev/tty3 rw,relatime - devpts devpts rw,mode=600,ptmxmode=666 111 105 0:10 /3 /dev/tty4 rw,relatime - devpts devpts rw,mode=600,ptmxmode=666 112 105 0:10 /4 /dev/tty5 rw,relatime - devpts devpts rw,mode=600,ptmxmode=666 113 105 0:10 /5 /dev/tty6 rw,relatime - devpts devpts rw,mode=600,ptmxmode=666 56 105 0:44 / /dev/pts rw,relatime - devpts devpts rw,mode=600,ptmxmode=666 64 56 0:10 / /dev/pts rw,relatime - devpts devpts rw,mode=600,ptmxmode=666 I'd say that's not good. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Magic Quadrant for Content-Aware Data Loss Prevention Research study explores the data loss prevention market. Includes in-depth analysis on the changes within the DLP market, and the criteria used to evaluate the strengths and weaknesses of these DLP solutions. http://www.accelacomm.com/jaw/sfnl/114/51385063/___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] read only rootfs
On Wed, 2011-07-20 at 07:10 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): [root@forest ~]# lxc-start --name Plover lxc-start: Invalid argument - pivot_root syscall failed sort of unrelated, but Rob Landley had mentioned he wanted to fix chroot to prevent the chdir-based chroot escape, allowing lxc to use chroot in place of pivot_root. As you see above, pivot_root has some very stringent constaints regarding the ms_shared state of the mounpoints *and* their parents, so this would be a very good thing. Interesting. Very interesting. Especially since I remember being involved in the whole transition from chroot over to pivot_root way back when. Explains a couples of other anomalies I observed. And would prevent the above. I'm going to have to refresh my memory of the reasons for switching from chroot over to pivot_root in the first place. There was more than one and the chdir-based chroot escape was only one. As far as the main topic of this thread, I feel I can't really do it justice without trying harder to reproduce, which I can't do today. I'm going to try and find time tomorrow or friday to do so (if you haven't gotten to the bottom of this before that). We also might want to point dhansen at the mail archive of this thread and get his input. Ok... I've still got some testing and experimenting to do based on those hints C Anthony gave me. ITMT, I'm not sure who you are referring to as dhansen (so I can't point them anywhere) but, the more the merrier if it helps this thing along. -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- 10 Tips for Better Web Security Learn 10 ways to better secure your business today. Topics covered include: Web security, SSL, hacker attacks Denial of Service (DoS), private keys, security Microsoft Exchange, secure Instant Messaging, and much more. http://www.accelacomm.com/jaw/sfnl/114/51426210/___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxc-console over ssh
On Wed, 2011-07-27 at 21:12 +0100, John wrote: Hi, quick one that is puzzling me. Do the lxc commands work over ssh ? I am trying to open a console on a remote host. I try this: $ ssh remote_host lxc-console -n vps_on_remote_host But I get an error: lxc-console: '0' is not a tty lxc-console: failed to setup tios That's because ssh did not allocate a pty for your command. Try this: ssh -t remote_host lxc_console -n vps_on_remote Other commands work fine like this $ ssh remote_host lxc-ls Doesn't need a pty. works fine. $ ssh remote_host lxc-start -n vps_on_remote_host also works fine but the output (boot messages) don't appear in the ssh session. The ssh session appears to hang until the container is closed down. What am I missing? In case you're wondering... I can't ssh vps_on_remote_host because it isn't set up. The remote_host is in another room making the ssh more convenient. Thanks, John Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Mitigating LXC Container Evasion?
On Sun, 2011-07-31 at 17:59 +0200, Robert Kawecki wrote: Dnia 2011-07-30, sob o godzinie 21:10 -0400, Matthew Franz pisze: Had seen some previous discussions before, but are there any ways to mitigate this design vulnerability? http://blog.bofh.it/debian/id_413 Are there any workarounds? Thanks, - mdf The blog post explicitly mounts /sys... Why would you want your container to even have the capability to mount anything? Let's see... Where shall we start. If you're containerizing a machine or full system there's a laundry list of reasons you would want to. Maybe you want to mount an image, like a CD image or maybe an encrypted file system? Then, there's a variety of fuse file systems you might want access to for a variety of reasons. Maybe you want to run a service, like bind's named in a bind mounted chrooted environment (default for running name servers on Fedora for some time now: /var/named on /var/named/chroot/var/named type none (rw,bind) /var/named/etc/named.conf on /var/named/chroot/etc/named.conf type none (rw,bind) /etc/named.rfc1912.zones on /var/named/chroot/etc/named.rfc1912.zones type none (rw,bind) /etc/rndc.conf on /var/named/chroot/etc/rndc.conf type none (rw,bind) /etc/rndc.key on /var/named/chroot/etc/rndc.key type none (rw,bind) /usr/lib/bind on /var/named/chroot/usr/lib/bind type none (rw,bind) /etc/named.iscdlv.key on /var/named/chroot/etc/named.iscdlv.key type none (rw,bind) /etc/named.root.key on /var/named/chroot/etc/named.root.key type none (rw,bind) That gets done by service named start. If possible, drop CAP_SYS_ADMIN. That's been proposed as a workaround for some of the remount problems we have to where a partition is suppose to be mounted ro but the container can remount it rw. I've actually tried that trick. Yeah, that was an epic failure. Generally not possible in a generalized system container. Way too many things are impacted by CAP_SYS_ADMIN. Disabling that, you can't even set your own hostname in the container. You can't set crypto keys either. That's a sledge hammer approach that won't work in many if not most system containers. A simply application container? It might work in, depending on the application. Check out man capabilities and scroll down to CAP_SYS_ADMIN and look at that laundry list under it (and I'm not totally sure that list is comprehensive in the man page either) and honestly say there are not reasons for a container wanting to do at least one or two of them (even given that every container I have sets its own hostname). Gotta be a better answer than that. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Mitigating LXC Container Evasion?
On Sun, 2011-07-31 at 23:02 +0200, Olivier Mauras wrote: That's where MAC system comes handy. Was just reading up on that from your earlier post. Very nice. I see I have some reading a research to do. I posted a URL to an IBM paper in a reply to your earlier post. -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. Michael H. Warfield m...@wittsend.com wrote: On Sun, 2011-07-31 at 17:59 +0200, Robert Kawecki wrote: Dnia 2011-07-30, sob o godzinie 21:10 -0400, Matthew Franz pisze: Had seen some previous discussions before, but are there any ways to mitigate this design vulnerability? http://blog.bofh.it/debian/id_413 Are there any workarounds? Thanks, - mdf The blog post explicitly mounts /sys... Why would you want your container to even have the capability to mount anything? Let's see... Where shall we start. If you're containerizing a machine or full system there's a laundry list of reasons you would want to. Maybe you want to mount an image, like a CD image or maybe an encrypted file system? Then, there's a variety of fuse file systems you might want access to for a variety of reasons. Maybe you want to run a service, like bind's named in a bind mounted chrooted environment (default for running name servers on Fedora for some time now: /var/named on /var/named/chroot/var/named type none (rw,bind) /var/named/etc/named.conf on /var/named/chroot/etc/named.conf type none (rw,bind) /etc/named.rfc1912.zones on /var/named/chroot/etc/named.rfc1912.zones type none (rw,bind) /etc/rndc.conf on /var/named/chroot/etc/rndc.conf type none (rw,bind) /etc/rndc.key on /var/named/chroot/etc/rndc.key type none (rw,bind) /usr/lib/bind on /var/named/chroot/usr/lib/bind type none (rw,bind) /etc/named.iscdlv.key on /var/named/chroot/etc/named.iscdlv.key type none (rw,bind) /etc/named.root.key on /var/named/chroot/etc/named.root.key type none (rw,bind) That gets done by service named start. If possible, drop CAP_SYS_ADMIN. That's been proposed as a workaround for some of the remount problems we have to where a partition is suppose to be mounted ro but the container can remount it rw. I've actually tried that trick. Yeah, that was an epic failure. Generally not possible in a generalized system container. Way too many things are impacted by CAP_SYS_ADMIN. Disabling that, you can't even set your own hostname in the container. You can't set crypto keys either. That's a sledge hammer approach that won't work in many if not most system containers. A simply application container? It might work in, depending on the application. Check out man capabilities and scroll down to CAP_SYS_ADMIN and look at that laundry list under it (and I'm not totally sure that list is comprehensive in the man page either) and honestly say there are not reasons for a container wanting to do at least one or two of them (even given that every container I have sets its own hostname). Gotta be a better answer than that. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it! _ Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey_ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists
Re: [Lxc-users] Fedora 15 on Fedora 15 LXC with Libvirt
Ah... Serge? Reference back to the thread on [Lxc-users] what's the
difference in lxc-attach please? We were discussing systemd back in
that thread too.
On Tue, 2011-08-02 at 19:22 +0300, Iliyan ILF Stoyanov wrote:
Hi,
I don't think this template will work as it references two files that do
not exist in F15 i.e.
${rootfs_path}/etc/rc.sysinit
and
${rootfs_path}/etc/rc.d/rc.sysinit
this is because of the fact that system.d doesn't use such scripts for
init.
You mean systemd and not system.d but I got your point.
I have an F15 container running on an F15 host but the only way I was
able to do this was by installing upstart and relinking init to upstart.
Systemd seems to get a great deal of heartburn over trying to
mount /sys/fs/cgroup in the container and things catch fire and burn all
over the place. I do have it working with Upstart. Until a bunch of us
can sort out the details of what systemd is doing and expect and should
behave under what circumstances, F15 systemd in a container is going to
be a crap shoot.
Regards,
Mike
I can always just use Scientific Linux for my setup, however we made the
decision a long time ago that all our dev servers will stick to Fedora,
so that we are better prepared for the quirks that might come up in
RHEL/CentOS/SL when some technology gets included in the TUV releases.
It is not a must to have LXC running on Fedora 15 with Fedora 15 guests,
but it would be nice to crack the hard nut that system.d is proving to
be. By the way, I see that your email is at canonical and it seems a
most of the people here are running LXC on Ubuntu, is LXC actually
sponsored by canonical or is primarily developed around Debian/Ubuntu,
because if that is the case, it might explain certain incompatibilities
with the way Red Hat/Fedora are set up.
BR,
ilf
On Tue, 2011-08-02 at 09:02 -0500, Serge E. Hallyn wrote:
Quoting Iliyan ILF Stoyanov (i...@ilf.me):
On the other side, would someone be so kind to point me in the right
direction (either documentation, source or anything else available) that
I can follow so that I set up lxc container just with the LXC tools. I
Ramez Hanna has posted a new fedora template. I think this was the
latest:
http://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg01707.html
It looks like it was for F13 and F14 containers, but it should be a
starting point. See the usage() section. I've not used it myself,
but the author should be on this list.
-serge
--
BlackBerryreg; DevCon Americas, Oct. 18-20, San Francisco, CA
The must-attend event for mobile developers. Connect with experts.
Get tools for creating Super Apps. See the latest technologies.
Sessions, hands-on labs, demos much more. Register early save!
http://p.sf.net/sfu/rim-blackberry-1
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
BlackBerryreg; DevCon Americas, Oct. 18-20, San Francisco, CA
The must-attend event for mobile developers. Connect with experts.
Get tools for creating Super Apps. See the latest technologies.
Sessions, hands-on labs, demos much more. Register early save!
http://p.sf.net/sfu/rim-blackberry-1___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Mitigating LXC Container Evasion?
On Wed, 2011-08-03 at 17:41 -0300, Andre Nathan wrote: Hi Olivier On Wed, 2011-08-03 at 19:48 +0200, Mauras Olivier wrote: You're true it won't work out of the box, sorry i forgot the network part. echo 0.0.0.0/0 @ /smack/netlabel Apparently this doesn't support IPv6... do you happen to know of a workaround? That's v4 syntax. Does it not work at all? Did you try this: echo ::/0 @ /smack/netlabel Not having tried this myself at all, I'm just asking. If it doesn't work, that needs to be fixed but it's a SMACK bug. Thanks again, Andre Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- BlackBerryreg; DevCon Americas, Oct. 18-20, San Francisco, CA The must-attend event for mobile developers. Connect with experts. Get tools for creating Super Apps. See the latest technologies. Sessions, hands-on labs, demos much more. Register early save! http://p.sf.net/sfu/rim-blackberry-1___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Mitigating LXC Container Evasion?
On Wed, 2011-08-03 at 21:01 -0700, Casey Schaufler wrote: On 8/3/2011 4:24 PM, Serge E. Hallyn wrote: Quoting Andre Nathan (an...@digirati.com.br): Hi Mike On Wed, 2011-08-03 at 17:52 -0400, Michael H. Warfield wrote: That's v4 syntax. Does it not work at all? Did you try this: echo ::/0 @ /smack/netlabel Not having tried this myself at all, I'm just asking. If it doesn't work, that needs to be fixed but it's a SMACK bug. Olivier's IPv4 example works fine, but with IPv6 I get an error: # echo ::/0 @ /smack/netlabel -bash: echo: write error: Invalid argument Looking at linux-2.6/security/smack/smackfs.c, nothing but 'a.b.c.d label' or 'a.b.c.d/mask label' is allowed. Now, smack_lsm.c does suggest that it wants to work with IPV6, but I haven't looked closely enough to tell how it will try to match the labels. Casey, is Smack netlabel supposed to work with IPV6? IPv6 support is a pending work item for Smack. The whole IPSEC thing makes it much more difficult than IPv4. ??? Whoa... Hold da phone a minute! I'm a contributor and developer to Openswan (I'm the author of some code for some Cisco ASA compatibility) and other VPN projects. That does not compute to me. How does IPsec make IPv6 more difficult? Are you saying you do not support IPsec on IPv4 but support is required on IPv6 or is there something else in v6 that I'm missing here. IPv6 does complicate things when you get into IKE v2 world where you can directly tunnel a v6 network over v4 endpoints which IKE v1 did not provide for. Is this the problem? The cross protocol encapsulations? Openswan supports 3 stacks, Netkey (the kernel native), KLIPS (the original FreeS/WAN stack), and Mast. My personal primary focus has been on the Netkey stack which is managed through the ip xfrm commands and functions. To the user space, IPv6 and IPv4 are agnostic. How does v6 in SMACK space become more difficult for v6? It shouldn't be... thanks, -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- BlackBerryreg; DevCon Americas, Oct. 18-20, San Francisco, CA The must-attend event for mobile developers. Connect with experts. Get tools for creating Super Apps. See the latest technologies. Sessions, hands-on labs, demos much more. Register early save! http://p.sf.net/sfu/rim-blackberry-1___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Mitigating LXC Container Evasion?
On Wed, 2011-08-03 at 22:21 -0700, Casey Schaufler wrote: Smack does not use IPsec on IPv4. Smack uses CIPSO. CIPSO is implemented completely within the kernel. It has no user space component. There is no CIPSO equivalent for IPv6 due to the expectation that all IPv6 implementations will use IPsec and IPsec will address all security issues known to man and then some. Oh, one other point... due to the expectation that all IPv6 implementations will use IPsec and IPsec will address all security issues known to man and then some. Who's assumption? Certainly not that of the IETF. Sounds like some non-sense promulgated by some anti-IPv6 camps and sounds somewhat denigrating and disparaging. It's demonstrably false. We still have MD5 signatures on tcp packets used by BGP on IPv6 (I'm also a contributor to quagga in that very area) even though it was originally expected that AH would replace MD5 signatures for BGP authentication. That expectation went bye-bye many years ago. We still have Kerberos. I don't see anyone going back to telnet instead of ssh over IPv6. We still have SSL over IPv6. The very statement is facetious on its face and can't possibly be taken seriously. If SMACK does not support IPv6 then SMACK is broken. Fix it. IPv6 is a reality. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- BlackBerryreg; DevCon Americas, Oct. 18-20, San Francisco, CA The must-attend event for mobile developers. Connect with experts. Get tools for creating Super Apps. See the latest technologies. Sessions, hands-on labs, demos much more. Register early save! http://p.sf.net/sfu/rim-blackberry-1___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Mitigating LXC Container Evasion?
On Thu, 2011-08-04 at 09:11 -0700, Casey Schaufler wrote: On 8/4/2011 6:52 AM, Michael H. Warfield wrote: On Wed, 2011-08-03 at 22:21 -0700, Casey Schaufler wrote: Smack does not use IPsec on IPv4. Smack uses CIPSO. CIPSO is implemented completely within the kernel. It has no user space component. There is no CIPSO equivalent for IPv6 due to the expectation that all IPv6 implementations will use IPsec and IPsec will address all security issues known to man and then some. Oh, one other point... due to the expectation that all IPv6 implementations will use IPsec and IPsec will address all security issues known to man and then some. Who's assumption? Certainly not that of the IETF. Sounds like some non-sense promulgated by some anti-IPv6 camps and sounds somewhat denigrating and disparaging. Sorry about that. I was a founding member of TSIG* and we had a very uncomfortable set of interactions with IETF regarding CIPSO and SAMP**. We were very forcefully told to let the IETF provide for us, as we clearly didn't know what we were doing. IPsec was the solution presented, it didn't provide the security attribute transmission we required, and the systems that we needed the solution for had been dismantled long before IPsec was ready for deployment. Yes, there is some bitterness. The Unix trusted systems community never recovered from the lack of a standard that we could use to have the various vendor's systems talk to each other. Gotcha. Yeah, I've been involved with several WGs at the IETF and was one of the original founders of the IDWG WG representing Internet Security Systems at the IETF. It's been described as the highest density of assholes per square meter on the face of the earth. I had an area director pull me into one of the Emergency Preparedness WG meetings one meeting just to sit in an critique the noise that was going on in that one (some of it centered around some disagreements between the ITU and the IETF and what should be provided for emergency responders). The discussion lead by the emergency responders could best be described as: We like toast. Make the Internet make toast. No comprehension. No clue. I understand fully from both sides of those arguments. I'm equally sure they thought the same thing about us. I also understand that CIPSO was a draft for a common implementation of IPSO, RFC 1108, which seems to be largely DoD oriented. I saw the WG finished up business and the last draft expired back in 94 with no RFC (no biggie - XAUTH never got an RFC and I'm up to my behind in the Openswan XAUTH code). There's always been a certain level of tension between the purists in the IETF and others such as the military crowd or ITU or certain commercial interests. I took part in some of the discussions over making IPsec support mandatory in IPv6 back in the bad ole days of ITAR when crypto was a tightly regulated export restricted munition. Yeah, back then, IPsec was presented as a be all and end all and they had dreams of end-to-end encryption for all. And here we are. Reality has had to set in. Sigh. Par for da course. --- * Trusted Systems Interoperability Group ** Security Attribute Modulation Protocol It's demonstrably false. We still have MD5 signatures on tcp packets used by BGP on IPv6 (I'm also a contributor to quagga in that very area) even though it was originally expected that AH would replace MD5 signatures for BGP authentication. That expectation went bye-bye many years ago. We still have Kerberos. I don't see anyone going back to telnet instead of ssh over IPv6. We still have SSL over IPv6. The very statement is facetious on its face and can't possibly be taken seriously. You are of course correct. My comment was sarcastic and inappropriate. NP. I've been rightfully accused of worse myself. If SMACK does not support IPv6 then SMACK is broken. Fix it. That is and has always been the plan. It's really a matter of getting the hands onto it. It's a big project and will require more work than I can plan on getting done in the short term. Well the short course should be just to get the CIPSO tags into IPv6, but that's just IP option 134, right? You really don't need to mess with IPsec one way or the other. I know, I know, there's the whole layer of API and management and what not around it, so it's obviously not so simple as simply adding the AF to those modules. But it should just parallel the v4 code and don't do anything special wrt the IPsec logic. IPv6 is a reality. I never said otherwise. I believe you. Cool. Regards, Mike Likewise, Casey Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc
Re: [Lxc-users] [lxc-devel] mount ro in guest change host filesystem to ro
On Sun, 2011-09-04 at 21:53 +0200, axel.schoe...@gmx.de wrote: Hi, in my opinion it's never a bad idea to drop the sys_admin cap. except you really need it. It's been my personal experience that it's ALWAYS a bad experience to drop sys_admin cap when you are doing a full system container. You can NOT even set your own host name for crying out loud! You also can not mount file images or set crypto keys. If I was one of those container owners, I would be asking what the shit is this crap... Seriously... Not an option. I' ve searched for some help because i'm using ubuntu only for some study (normally gentoo). I found a little help here: http://qemu-buch.de/de/index.php/QEMU-KVM- Buch/_Anhang/_Weitere_Virtualisierer_und_Emulatoren/_LXC#Die_LXC- Konfigurationsdatei . My guest is using these settings: lxc.cap.drop = sys_module mknod sys_admin My fstab for a ubuntu host look like this: # cat /var/lib/lxc/guest.temp01/fstab proc/var/lib/lxc/guest.temp01/rootfs/proc proc nodev,noexec,nosuid 0 0 sysfs /var/lib/lxc/guest.temp01/rootfs/syssysfs defaults 0 0 none/var/lib/lxc/guest.temp01/rootfs/dev/shmtmpfs mode=0644 0 0 none/var/lib/lxc/guest.temp01/rootfs/dev/pts devpts defaults 0 0 none/var/lib/lxc/guest.temp01/rootfs/var/runtmpfs defaults 0 0 none/var/lib/lxc/guest.temp01/rootfs/sys/fs/fuse/connections fusectl optional0 0 none/var/lib/lxc/guest.temp01/rootfs/sys/kernel/debug debugfs optional0 0 none/var/lib/lxc/guest.temp01/rootfs/sys/kernel/security securityfs optional0 0 Inside the container the lib/init/fstab has to be modified like this: # /lib/init/fstab: static file system information. # # These are the filesystems that are always mounted on boot, you can # override any of these by copying the appropriate line from this file into # /etc/fstab and tweaking it as you see fit. See fstab(5). # # file system mount point type options dump pass /dev/root / rootfs defaults 0 1 #none/proc proc nodev,noexec,nosuid 0 0 none/proc/sys/fs/binfmt_misc binfmt_misc nodev,noexec,nosuid,optional 0 0 none/sys sysfs nodev,noexec,nosuid 0 0 #none/sys/fs/fuse/connections fusectl optional 0 0 #none/sys/kernel/debug debugfs optional 0 0 #none/sys/kernel/security securityfs optional 0 0 none/spu spufs gid=spu,optional 0 0 #none/dev devtmpfs,tmpfs mode=0755 0 0 #none/dev/pts devpts noexec,nosuid,gid=tty,mode=0620 0 0 none/dev/shm tmpfs nosuid,nodev 0 0 none/tmp nonedefaults 0 0 none/var/run tmpfs mode=0755,nosuid,showthrough 0 0 #none/var/lock tmpfs nodev,noexec,nosuid,showthrough 0 0 none/lib/init/rw tmpfs mode=0755,nosuid,optional 0 0 Regards, Axel Schöner On Friday, 2. September 2011 11:51:55 Michael H. Warfield wrote: On Fri, 2011-09-02 at 08:35 +0400, Michael Tokarev wrote: On 02.09.2011 00:46, Daniel Lezcano wrote: On 09/01/2011 09:30 PM, Nico wrote: Hi, I just wanted to give it a try again with lxc after one year, this is so bad same bugs are always here : * you can do a mount -o romount,ro / inside container (reported since first times ... :( ), and host filesystem is remounted ro !! Argh ! I still don't understand how that can happen with a CLONE_NEWNS and a pivot_root. Do you have particular mount options on your host's rootfs ? In order for guest remount to NOT influence host mount, you have to give -o bind option to mount inside guest. If you don't specify MS_BIND with MS_REMOUNT, the remount applies to _host_ mountpoint, not guest. Last time I recall playing with this was a couple of months ago and was not the rootfs that was causing me headaches with random acts
Re: [Lxc-users] stopping a container
On Tue, 2011-09-06 at 17:34 -0400, Brian K. White wrote: On 9/5/2011 12:34 PM, Michael H. Warfield wrote: On Mon, 2011-09-05 at 09:24 +0200, Papp Tamas wrote: On 09/05/2011 08:38 AM, Jäkel, Guido wrote: What is the right way to stop a container? Dear Papp, Like with the thread paradigm in computing langugages, the right way is that the thread decides to stop. Therefore your container have to leave. Depending on your Linxus flavor inside the container, you e.g. may send a signal to it's init process to proper shut down. This mechanism is historical intended to be used by an USV poser supply. In the moment, I'm using an old-style sytem v init and I may just send a SIGINT to reboot and a SIGPWR to halt it (must be enabled in the inittab). Another (planned) way is to use lxc-execute, but this is still not working. Ulli Hornbacher therefore wrote it's own workaround: A little daemon executes all command pushed in by a command running at the host -- disregarding to all aspects of security. If you're running a sshd inside the container -- and in the most case you will I think -- you may use this (with a deposited key) to directly send commands to it. hi, I don't like the ssh way. I think, halting a container automatically through an ssh connection is a joke, wich should not be used in any way. Another way that I have used is to send the init process a kill signal. I think it was the power fail sig but that should do it. That definitely worked with the sysv init but I see to recall I had some problem with upstart (and systemd won't currently run in a container anyways - so you can forget that pig). That's not another way. That's exactly the way already stated first above. Oh, yeah, guess it's better I read up and read the entire quoted thread and not stop at the comment about ssh. :-P My bad. -- bkw Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Malware Security Report: Protecting Your Business, Customers, and the Bottom Line. Protect your business and customers by understanding the threat from malware and how it can impact your online business. http://www.accelacomm.com/jaw/sfnl/114/51427462/___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] seeing a network pause when starting and stopping LXCs - how do I stop this?
On Thu, 2011-12-08 at 19:47 +1300, Gary Ballantyne wrote: On 08/12/11 19:39, Daniel Lezcano wrote: On 12/08/2011 12:38 AM, Joseph Heck wrote: I've been seeing a pause in the whole networking stack when starting and stopping LXC - it seems to be somewhat intermittent, but happens reasonably consistently the first time I start up the LXC. I'm using ubuntu 11.10, which is using LXC 0.7.5 I'm starting the container with lxc-start -d -n $CONTAINERNAME That could be the bridge configuration. Did you do 'brctl setfd br0 0' ? FWIW, I see the same issue following [1], which has 'brctl setfd br0 0'. [1] http://www.activestate.com/blog/2011/10/virtualization-ec2-cloud-using-lxc That just sets the bridge delay to zero (which could have an impact) but does not change the MAC address policy of assigning the lowest MAC address to the bridge itself. If the bridge mac address changes then any OTHER nodes or routers on the network have to fail out their ARP caches and re-ARP for the new MAC address, resulting in the delay. The only way to work around this problem is to assign very high MAC addresses to the nodes on the bridge to keep them from changing the MAC address of the bridge itself. Setting the locally administered (0200 in first word) bit in the MAC address helps but may not be sufficient (warning - DO NOT set the MULTICAST bit (0100 in first word)!). This was a design decision made many years ago in the Linux kernel for reasons I did not agree with and do not agree with but it is what it is and I doubt it will ever change. I think there's a patch going into the lxc-tools to avoid this problem. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Cloud Services Checklist: Pricing and Packaging Optimization This white paper is intended to serve as a reference, checklist and point of discussion for anyone considering optimizing the pricing and packaging model of a cloud services business. Read Now! http://www.accelacomm.com/jaw/sfnl/114/51491232/___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] seeing a network pause when starting and stopping LXCs - how do I stop this ?
On Mon, 2011-12-12 at 08:43 +0100, Ulli Horlacher wrote: On Sun 2011-12-11 (19:48), Derek Simkowiak wrote: The problem is not related to the setfd option. It is caused by the bridge acquired a new MAC address. Libvirt already has a fix for this, and there is a patch in the works for the LXC tools. I am wonder why I do not have this problem: I really often start new containers and I do not have this patch, but no network freeze at all. Look to your MAC addresses. If the MAC address of the new container is greater than the MAC address of the bridge, you will never see this problem. If it's less than the bridge MAC address, the MAC address will change and, if you have any active network traffic, you WILL see this problem. The problem is old and related to a design decision wrt the bridges in the Linux kernel. I don't really agree with where the decision went but I understand the problem and their concerns (IIRC). The problem is the bridge NEEDS a MAC address but has no intrinsic address. So, they started out by assigning the MAC address of the first device assigned to the bridge. That worked but had a problem. The bridge is an independent autonomous object within the kernel and must be independent of the connected devices. Sooo, what happens to the MAC address if you delete that device from the bridge? It can't stay the same. Now, my memory is very fuzzy after about a decade (and Google searches are NOT helping my research) and anyone is welcome to correct me if I recall incorrect but... The arbitrary decision was made to always use the lowest MAC address on the bridge. The trouble is that they did that when they added devices. The problem really only occurs when you delete devices and, even then, only when the MAC of the bridge matches the MAC of the device being deleted and there is no other device with that MAC address (multihomed SPARCs) on that bridge. I would have chosen differently but this was not my shot to call and I didn't have any strong arguments to even insert into the discussion. They chose to always use the lowest MAC address. The choice was arbitrary but a choice had to be made. No choice was a non-starter and all choices had some downside to them. There are actually TWO solutions to this problem. The first one people have already glommed onto. You have to set your MAC address of your containers to be greater than the MAC address of the bridge. Problems there are that 1) we don't have our own vendor code and 2) we have to make sure we're higher than ANY vendor code and 3) the locally administered bit is NOT the highest order bit in the MAC address (that really would have made it too easy to fix but that's and even more ancient bad choice). The good point is that we CAN auto assign anything we WANT as long as that locally administered bit is set and we can then set the vendor code as high as we want. THAT works. Use FF:FF:F2:: and you are gold. Play with the remaining bits all you want. There is another. When creating the bridge, assign it a MAC address that's very LOW. You can do that with ifconfig or ip. It doesn't HAVE to have a MAC address that matches ANY interface attached to it. That's a misconception. It merely has to HAVE a MAC address. The problem with this solution is that technically that MAC address is then locally administered so the locally administered bit SHOULD be set (00:02::) but then anything of the vendor codes from 00:00:00:: through 00:00:ff:: would be less than that (00:01:: is the multicast bit). Grrr... But it's only 256 old vendors. Using 00:00:00:: for the vendor code works and is sooo unlikely to conflict with Xerox Corporation (vendor code 00:00:00) or their Ethernet cards (I know I'll get HATE MAIL from purists on THAT comment) that it's really NOT worth worrying about. You do have to be forewarned that tinkering with MAC addresses at runtime can be even MORE disasterous when dealing with IPv6 and autoconf addresses. Assigning a fixed MAC address to a bridge should only be done at boot time and not changed after (one of the arguments against that earlier decision). Which points out another PITA associated with that early decision regarding MAC addresses. It makes a MESS out of IPv6! Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Systems Optimization Self Assessment Improve efficiency and utilization of IT resources. Drive out cost and improve service delivery. Take 5 minutes to use this Systems Optimization Self Assessment. http://www.accelacomm.com/jaw/sdnl/114/51450054
Re: [Lxc-users] seeing a network pause when starting and stopping LXCs - how do I stop this ?
I always hate replying to my own posts but I have stumbled onto some interesting clarification as I've continued to play with this... Below in-line. On Tue, 2011-12-13 at 01:30 -0500, Michael H. Warfield wrote: On Mon, 2011-12-12 at 08:43 +0100, Ulli Horlacher wrote: On Sun 2011-12-11 (19:48), Derek Simkowiak wrote: The problem is not related to the setfd option. It is caused by the bridge acquired a new MAC address. Libvirt already has a fix for this, and there is a patch in the works for the LXC tools. I am wonder why I do not have this problem: I really often start new containers and I do not have this patch, but no network freeze at all. Look to your MAC addresses. If the MAC address of the new container is greater than the MAC address of the bridge, you will never see this problem. If it's less than the bridge MAC address, the MAC address will change and, if you have any active network traffic, you WILL see this problem. The problem is old and related to a design decision wrt the bridges in the Linux kernel. I don't really agree with where the decision went but I understand the problem and their concerns (IIRC). The problem is the bridge NEEDS a MAC address but has no intrinsic address. So, they started out by assigning the MAC address of the first device assigned to the bridge. That worked but had a problem. The bridge is an independent autonomous object within the kernel and must be independent of the connected devices. Sooo, what happens to the MAC address if you delete that device from the bridge? It can't stay the same. Now, my memory is very fuzzy after about a decade (and Google searches are NOT helping my research) and anyone is welcome to correct me if I recall incorrect but... The arbitrary decision was made to always use the lowest MAC address on the bridge. The trouble is that they did that when they added devices. The problem really only occurs when you delete devices and, even then, only when the MAC of the bridge matches the MAC of the device being deleted and there is no other device with that MAC address (multihomed SPARCs) on that bridge. I would have chosen differently but this was not my shot to call and I didn't have any strong arguments to even insert into the discussion. They chose to always use the lowest MAC address. The choice was arbitrary but a choice had to be made. No choice was a non-starter and all choices had some downside to them. There are actually TWO solutions to this problem. The first one people have already glommed onto. You have to set your MAC address of your containers to be greater than the MAC address of the bridge. Problems there are that 1) we don't have our own vendor code and 2) we have to make sure we're higher than ANY vendor code and 3) the locally administered bit is NOT the highest order bit in the MAC address (that really would have made it too easy to fix but that's and even more ancient bad choice). The good point is that we CAN auto assign anything we WANT as long as that locally administered bit is set and we can then set the vendor code as high as we want. THAT works. Use FF:FF:F2:: and you are gold. Play with the remaining bits all you want. There is another. When creating the bridge, assign it a MAC address that's very LOW. You can do that with ifconfig or ip. It doesn't HAVE to have a MAC address that matches ANY interface attached to it. That's a misconception. It merely has to HAVE a MAC address. The problem with this solution is that technically that MAC address is then locally administered so the locally administered bit SHOULD be set (00:02::) but then anything of the vendor codes from 00:00:00:: through 00:00:ff:: would be less than that (00:01:: is the multicast bit). Grrr... But it's only 256 old vendors. Using 00:00:00:: for the vendor code works and is sooo unlikely to conflict with Xerox Corporation (vendor code 00:00:00) or their Ethernet cards (I know I'll get HATE MAIL from purists on THAT comment) that it's really NOT worth worrying about. According to this post, I may be mistaken that it doesn't have to be an address of a connected interface. http://backreference.org/2010/07/28/linux-bridge-mac-addresses-and-dynamic-ports/ In my testing I've certainly found that it does NOT have to be a MAC of a connected interface for IPv6 to work. My results for IPv4 have been mixed where, in one case, it did not and, in another case, it did. IPv6 worked in both cases but there are some peculiar filtering paths down in the kernel bridging logic that could well be coming into play here. It may also have to do with the MAC address detection logic and spanning tree algorithms. So be cautious there. But there was this remark: -- Fortunately, there is a way to ensure that the bridge's MAC address is fixed and never changes, thus entirely avoiding the problem. Thanks to this thread http://www.spinics.net
Re: [Lxc-users] LXC and IPv6 Launch day (June 6th, 2012.)
On Wed, 2012-01-18 at 12:36 -0800, István Király - LaKing wrote: I know this is not the developer list, but I would like to know if we will reach a stabile LXC 1.0 by June 6th, when IPv6 should launch. I don't understand the relationship (other than the numbers). IPv6 works great for me with LXC (bridged adapters). Couldn't get macvlan to work but macvlan was borked along several vectors last time I tried it. Something you're having problems with? And I really would like to get lxc working on fedora ... by then, maybe on F17? Huh? Working fine for me on F13, F14, and F15 (F15 and systemd are a PITA but that's more the fact that systemd is a PITA). Haven't tried F16 yet. F15 host works for me but F15 guest was a pain. Suppose to be working now but I've been too lazy to retest. :-P Can you be more specific about what you are looking for and what your criterion is for working? Greetings from Hungary Cool... Was there in Budapest (mostly on Buda side of the river) years ago. Lovely, lovely. Didn't get out in the countryside nearly as much as I would have liked. Cheers from the States! lak...@d250.hu D250 Laboratories www.D250.hu Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxc macvlan bridge problem
On Mon, 2012-09-24 at 14:48 +0800, 宣铭艺 wrote: Hi guys I have a host machine and want to create 3 containers on it. Containers should can ping each other,but can't communicate with the host. All containers shoud surf the Internet. Should I use macvlan bridge mode?And how to set the host's and containers' network environment Geeze, this sounds like an OLD problem. What distribution and what version? What version of the kernel? What version of lxc (which really shouldn't matter - it's a kernel problem). I honestly don't know if it's been ever fixed but macvlan use to always have this problem and I finally threw up my hands and went straight bridging. I have not looked back or tested macvlan on any containers since. Regards, Mike the goal blow My Host: eth0 192.168.1.23 gateway 192.168.1.1 It connects to the Internet lxcbr0?? bridge? need it? Containers: c1: 192.168.2.45 ? how to configure c2: 192.168.2.46 ? how to configure c3: 192.168.2.47 ? how to configure Thanks -- 樱宝宝: http://www.xuanmingyi.com -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- How fast is your code? 3 out of 4 devs don\\\'t know how their code performs in production. Find out how slow your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219672;13503038;z? http://info.appdynamics.com/FreeJavaPerformanceDownload.html___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] systemd inside LXC
: lxc.utsname = test2 lxc.rootfs = /srv/lxc/test Add: lxc.network.type = empty If you don't have any lxc.network.type sections, then the container shares network with the host, and so the container talks to the host's systemd. (same with upstart) When I start the container in LXC, all that happens is that my X session dies (this is running in another container). The X session re-starts but the keyboard does not work. I have to connect using another machine to kill the test container and re-start my desktop container. I can't see anything starting inside the test container. I'd be grateful for any help and/or pointers in the right direction so I can complete this transition to systemd. Many thanks, John -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
[Lxc-users] Unable to run systemd in an LXC / cgroup container.
Hello, This is being directed to the systemd-devel community but I'm cc'ing the lxc-users community and the Fedora community on this for their input as well. I know it's not always good to cross post between multiple lists but this is of interest to all three communities who may have valuable input. I'm new to this particular list, just having joined after tracking a problem down to some systemd internals... Several people over the last year or two on the lxc-users list have been discussions trying to run certain distros (notably Fedora 16 and above, recent Arch Linux and possibly others) in LXC containers, virualizing entire servers this way. This is very similar to Virtuoso / OpenVZ only it's using the native Linux cgroups for the containers (primary reason I dumped OpenVZ was to avoid their custom patched kernels). These recent distros have switched to systemd for the main init process and this has proven to be disastrous for those of us using LXC and trying to install or update our containers. To put it bluntly, it doesn't work and causes all sorts of problems on the host. To summarize the problem... The LXC startup binary sets up various things for /dev and /dev/pts for the container to run properly and this works perfectly fine for SystemV start-up scripts and/or Upstart. Unfortunately, systemd has mounts of devtmpfs on /dev and devpts on /dev/pts which then break things horribly. This is because the kernel currently lacks namespaces for devices and won't for some time to come (in design). When devtmpfs gets mounted over top of /dev in the container, it then hijacks the hosts console tty and several other devices which had been set up through bind mounts by LXC and should have been LEFT ALONE. Yes! I recognize that this problem with devtmpfs and lack of namespaces is a potential security problem anyways that could (and does) cause serious container-to-host problems. We're just not going to get that fixed right away in the linux cgroups and namespaces. How do we work around this problem in systemd where it has hard coded mounts in the binary that we can't override or configure? Or is it there and I'm just missing it trying to examine the sources? That's how I found where the problem lay. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] systemd inside LXC
On Sun, 2012-10-21 at 14:49 -0500, Serge Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
Serge,
I'm going to top post here simply because this is going to go off in a
different direction and bringing in an old thread but it is related...
Back on February 14 you responded to a message about Fedora 16 in a
container, which is something I've been trying to do and I had run into
that posters problems. You responded with this:
Subject: Re: [Lxc-users] fedora 16 under lxc
On Tue, 2012-02-14 at 09:23 -0600, Serge Hallyn wrote:
Quoting Ramez Hanna (rha...@informatiq.org):
now all my efforts have not succeedd to get getty on tty1 to start
unmasking udev did something different
it created all the /dev devices
and made getty start but it started on the hosts's tty not on the
container's
could someone shed some light here?
Blind guess:
lxc-start creates some ptys and bind mounts them onto the guest's
/dev/{console,tty{1,2,3,4}}. It sounds like fedora's init is mounting
over the /dev set up by lxc causing a new /dev/tty to be created as
chardev 4:{1-4}. Devices namespaces would help this. We're hoping to
discuss design for those at next UDS, but those will come after user
namespaces. In the mean time, you'll need to make sure that the guest
does not mount over /dev, and does not remount /dev/pts.
-serge
That got me thinking and started into looking deeper into systemd, which
Fedora 16 and above uses and why it may be related here. I've made
Fedora 16 work in the past by installing upstart and disabling systemd
but that really becomes impractical in Fedora 17 because they're
including so few of the compatibility scripts. Yes, you are right, the
Fedora's init (systemd) is mounting something on /dev like this:
devtmpfs on /dev type devtmpfs
(rw,nosuid,seclabel,size=1784160k,nr_inodes=446040,mode=755)
This is very bad for the reasons you pointed out in Feb. Looking at the
source code for systemd, this is hard coded into the binary and is not
configurable.
systemd-37/src/mount-setup.c:
--
/* The first three entries we might need before SELinux is up. The
* other ones we can delay until SELinux is loaded. */
#define N_EARLY_MOUNT 3
static const MountPoint mount_table[] = {
{ proc, /proc, proc, NULL,
MS_NOSUID|MS_NOEXEC|MS_NODEV, true },
{ sysfs,/sys, sysfs,NULL,
MS_NOSUID|MS_NOEXEC|MS_NODEV, true },
{ devtmpfs, /dev, devtmpfs, mode=755,
MS_NOSUID,true },
{ tmpfs,/dev/shm, tmpfs,mode=1777,
MS_NOSUID|MS_NODEV, true },
{ devpts, /dev/pts, devpts, mode=620,gid=
STRINGIFY(TTY_GID), MS_NOSUID|MS_NOEXEC, false },
{ tmpfs,/run, tmpfs,mode=755,
MS_NOSUID|MS_NODEV, true },
{ tmpfs,/sys/fs/cgroup, tmpfs,mode=755,
MS_NOSUID|MS_NOEXEC|MS_NODEV, false },
{ cgroup, /sys/fs/cgroup/systemd, cgroup,
none,name=systemd, MS_NOSUID|MS_NOEXEC|MS_NODEV, false },
};
--
Short of building a custom systemd, I don't know how to fix that problem
and I suspect this OP is going to run into this same thing (container
taking over host's console) and might explain some of what he's seeing.
Several of these look like they could cause problems (like /dev/pts in
there). I've really reached an impasse at getting systemd (at least
Fedora 16 and 17) to work in a container without screwing up the host.
Prohibiting mounts entirely in the container might work but I suspect
(having read some systemd error messages) systemd is going to have some
serious heartburn there.
Thoughts?
IIRC, simply having apparmor(/selinux) refuse the mount of /dev by the
container should work, i.e. systemd was not going to fail as a result.
I'm not sure how that would work or if that would work in the case where
you didn't have selinux in the host kernel or you were crossing apparmor
in the container and selinux in the host or vice-versa.
In any case, I'm hitting the systemd-devel list looking to raise their
awareness of the problem and including this list and the fedora list.
If someone wants to mention it on the Arch Linux list, please do, I
don't participate over there.
-serge
Thanks
Regards,
Mike
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
--
Michael H. Warfield
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Mon, 2012-10-22 at 02:53 +0200, Kay Sievers wrote: On Sun, Oct 21, 2012 at 11:25 PM, Michael H. Warfield m...@wittsend.com wrote: This is being directed to the systemd-devel community but I'm cc'ing the lxc-users community and the Fedora community on this for their input as well. I know it's not always good to cross post between multiple lists but this is of interest to all three communities who may have valuable input. I'm new to this particular list, just having joined after tracking a problem down to some systemd internals... Several people over the last year or two on the lxc-users list have been discussions trying to run certain distros (notably Fedora 16 and above, recent Arch Linux and possibly others) in LXC containers, virualizing entire servers this way. This is very similar to Virtuoso / OpenVZ only it's using the native Linux cgroups for the containers (primary reason I dumped OpenVZ was to avoid their custom patched kernels). These recent distros have switched to systemd for the main init process and this has proven to be disastrous for those of us using LXC and trying to install or update our containers. To put it bluntly, it doesn't work and causes all sorts of problems on the host. To summarize the problem... The LXC startup binary sets up various things for /dev and /dev/pts for the container to run properly and this works perfectly fine for SystemV start-up scripts and/or Upstart. Unfortunately, systemd has mounts of devtmpfs on /dev and devpts on /dev/pts which then break things horribly. This is because the kernel currently lacks namespaces for devices and won't for some time to come (in design). When devtmpfs gets mounted over top of /dev in the container, it then hijacks the hosts console tty and several other devices which had been set up through bind mounts by LXC and should have been LEFT ALONE. Yes! I recognize that this problem with devtmpfs and lack of namespaces is a potential security problem anyways that could (and does) cause serious container-to-host problems. We're just not going to get that fixed right away in the linux cgroups and namespaces. How do we work around this problem in systemd where it has hard coded mounts in the binary that we can't override or configure? Or is it there and I'm just missing it trying to examine the sources? That's how I found where the problem lay. As a first step, this probably explains most of it: http://www.freedesktop.org/wiki/Software/systemd/ContainerInterface A very long ways, yeah. That looks like it could be just what we've been looking for. Just gotta figure out how to set that environment variable but that's up to a couple of others to comment on in the lxc-users list. Then we'll see where we go from there. Many thanks! Kay Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] systemd inside LXC
On Sun, 2012-10-21 at 14:49 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): Serge, ... Short of building a custom systemd, I don't know how to fix that problem and I suspect this OP is going to run into this same thing (container taking over host's console) and might explain some of what he's seeing. Several of these look like they could cause problems (like /dev/pts in there). I've really reached an impasse at getting systemd (at least Fedora 16 and 17) to work in a container without screwing up the host. Prohibiting mounts entirely in the container might work but I suspect (having read some systemd error messages) systemd is going to have some serious heartburn there. Thoughts? IIRC, simply having apparmor(/selinux) refuse the mount of /dev by the container should work, i.e. systemd was not going to fail as a result. Hopefully, you've seen the message from Kay Sievers cc'ed to this list from my post to the systemd-devel list. Looks like they have a mechanism in place to do this... http://www.freedesktop.org/wiki/Software/systemd/ContainerInterface First step appears to be to set a container=LXC (or some other short string) before invoking init in the container. Is there a mechanism to do this? Might look over the rest of their recommendation and see if there's anything else we need to do. Looks like there might be some additional mounts (some read-only) in there that need to be handled in lxc-start as well. -serge -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] systemd inside LXC
On Mon, 2012-10-22 at 15:14 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): Serge, On Mon, 2012-10-22 at 09:12 -0500, Serge Hallyn wrote: Quoting Serge Hallyn (serge.hal...@canonical.com): Quoting Michael H. Warfield (m...@wittsend.com): On Sun, 2012-10-21 at 14:49 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): Serge, ... Short of building a custom systemd, I don't know how to fix that problem and I suspect this OP is going to run into this same thing (container taking over host's console) and might explain some of what he's seeing. Several of these look like they could cause problems (like /dev/pts in there). I've really reached an impasse at getting systemd (at least Fedora 16 and 17) to work in a container without screwing up the host. Prohibiting mounts entirely in the container might work but I suspect (having read some systemd error messages) systemd is going to have some serious heartburn there. Thoughts? IIRC, simply having apparmor(/selinux) refuse the mount of /dev by the container should work, i.e. systemd was not going to fail as a result. Hopefully, you've seen the message from Kay Sievers cc'ed to this list from my post to the systemd-devel list. Looks like they have a mechanism in place to do this... http://www.freedesktop.org/wiki/Software/systemd/ContainerInterface Saw the email, haven't yet read the page, thanks. So based on that page, what we do (set 'container=lxc') should already be sufficient. Thanks to the dude asking a libvirt-lxc question on the list, I was let to a page that let to a page that led to some discussion you were having back in March with Ramez Hanna on this very subject, Re: [Lxc-users] f16 update... http://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg03263.html thanks, I knew we'd been over some of this, but couldn't find my logs of it. This would look to be the kludge to make a workaround for this problem, I'm just not sure how to make it happen. Given you already found the answer that the device for /dev has to be different than the device for the parent, what should we do. I tried this in the config... lxc.mount.entry=tmpfs /var/lib/lxc/private/Alcove/dev tmpfs defaults 0 0 How about just a devtmpfs? We actually now do this by default (as of very recently) in ubuntu by adding devtmpfsdev devtmpfs defaults 0 0 NO! That's the problem! That leads to the container connecting to the hosts console and other devices and committing random acts of terrorism. That's exactly the case we are trying to avoid and which is causing the problems to begin with. That's what systemd is doing if it doesn't find /dev mounted to begin with. to /var/lib/lxc/$container/fstab Or, are you on an older kernel which doesn't have devtmpfs? Maybe I got that entry wrong but it didn't do anything (and would have resulted in other failures later as I found out). A similar mount entry for a shared filesystem worked just fine. Trying an empty /dev fails because it's missing EVERYTHING so starting out with a tmpfs without populating it is not the answer either. The correct answer would be to mount a tmpfs file system and then populate it before firing up systemd in the container. I don't see how to do that. A bind mount isn't going to work either, for the reasons you stated in that thread, it ends up on the same device. Having another mount on another real device would be a workaround but a really bad kludge that would complicate things immensely. -serge -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Mon, 2012-10-22 at 22:50 +0200, Lennart Poettering wrote: On Mon, 22.10.12 11:48, Michael H. Warfield (m...@wittsend.com) wrote: To summarize the problem... The LXC startup binary sets up various things for /dev and /dev/pts for the container to run properly and this works perfectly fine for SystemV start-up scripts and/or Upstart. Unfortunately, systemd has mounts of devtmpfs on /dev and devpts on /dev/pts which then break things horribly. This is because the kernel currently lacks namespaces for devices and won't for some time to come (in design). When devtmpfs gets mounted over top of /dev in the container, it then hijacks the hosts console tty and several other devices which had been set up through bind mounts by LXC and should have been LEFT ALONE. Please initialize a minimal tmpfs on /dev. systemd will then work fine. My containers have a reasonable /dev that work with Upstart just fine but they are not on tmpfs. Is mounting tmpfs on /dev and recreating that minimal /dev required? Well, it can be any kind of mount really. Just needs to be a mount. And the idea is to use tmpfs for this. What /dev are you currently using? It's probably not a good idea to reuse the hosts' /dev, since it contains so many device nodes that should not be accessible/visible to the container. Got it. And that explains the problems we're seeing but also what I'm seeing in some libvirt-lxc related pages, which is a separate and distinct project in spite of the similarities in the name... http://wiki.1tux.org/wiki/Lxc/Installation#Additional_notes Unfortunately, in our case, merely getting a mount in there is a complication in that it also has to be populated but, at least, we understand the problem set now. systemd will make use of pre-existing mounts if they exist, and only mount something new if they don't exist. So you're saying that, if we have something mounted on /dev, that's what prevents systemd from mounting devtmpfs on /dev? Yes. But, I have systemd running on my host system (F17) and containers with sysvinit or upstart inits are all starting just fine. That sounds like it should impact all containers as pivot_root() is issued before systemd in the container is started. Or am I missing something here? That sounds like a problem for Serge and others to investigate further. I'll see about trying that workaround though. The shared issue is F18, and it's about running LXC on a systemd system, not about running systemd inside of LXC. Whew! I'll deal with F18 when I need to deal with F18. That explains why my F17 hosts are running and gives Serge and others a chance to address this, forewarned. Thanks for that info. Lennart -- Lennart Poettering - Red Hat, Inc. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] systemd inside LXC
On Mon, 2012-10-22 at 16:21 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Mon, 2012-10-22 at 15:14 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): Serge, On Mon, 2012-10-22 at 09:12 -0500, Serge Hallyn wrote: Quoting Serge Hallyn (serge.hal...@canonical.com): Quoting Michael H. Warfield (m...@wittsend.com): On Sun, 2012-10-21 at 14:49 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): Serge, ... Short of building a custom systemd, I don't know how to fix that problem and I suspect this OP is going to run into this same thing (container taking over host's console) and might explain some of what he's seeing. Several of these look like they could cause problems (like /dev/pts in there). I've really reached an impasse at getting systemd (at least Fedora 16 and 17) to work in a container without screwing up the host. Prohibiting mounts entirely in the container might work but I suspect (having read some systemd error messages) systemd is going to have some serious heartburn there. Thoughts? IIRC, simply having apparmor(/selinux) refuse the mount of /dev by the container should work, i.e. systemd was not going to fail as a result. Hopefully, you've seen the message from Kay Sievers cc'ed to this list from my post to the systemd-devel list. Looks like they have a mechanism in place to do this... http://www.freedesktop.org/wiki/Software/systemd/ContainerInterface Saw the email, haven't yet read the page, thanks. So based on that page, what we do (set 'container=lxc') should already be sufficient. Thanks to the dude asking a libvirt-lxc question on the list, I was let to a page that let to a page that led to some discussion you were having back in March with Ramez Hanna on this very subject, Re: [Lxc-users] f16 update... http://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg03263.html thanks, I knew we'd been over some of this, but couldn't find my logs of it. This would look to be the kludge to make a workaround for this problem, I'm just not sure how to make it happen. Given you already found the answer that the device for /dev has to be different than the device for the parent, what should we do. I tried this in the config... lxc.mount.entry=tmpfs /var/lib/lxc/private/Alcove/dev tmpfs defaults 0 0 How about just a devtmpfs? We actually now do this by default (as of very recently) in ubuntu by adding devtmpfsdev devtmpfs defaults 0 0 NO! That's the problem! That leads to the container connecting to the hosts console and other devices and committing random acts of terrorism. No, it shouldn't, because lxc sets up the console after doing the mounts. Maybe it shouldn't but that appears to be what is happening and even you remarked that maybe the problem was something doing a remount of /dev after entering the container... I see your point though. If you did that mount after LXC set up the console, then systemd wouldn't set it up and would drop into its more restricted mode. That MIGHT help but you still have the entire dev space of the host exposed to the guest which is what you were talking about before wrt namespaces on devices. It might help. Would it be the answer? Given that we've restricted access to those nodes in the config, maybe yes. I'm just not so sure. Will give it a shot though. Strange, though, my earlier effort at tmpfs on dev had no effect. Will give it a shot. -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] systemd inside LXC
On Mon, 2012-10-22 at 16:21 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Mon, 2012-10-22 at 15:14 -0500, Serge Hallyn wrote: Trimming some overhead we've seen enough of... How about just a devtmpfs? We actually now do this by default (as of very recently) in ubuntu by adding devtmpfsdev devtmpfs defaults 0 0 NO! That's the problem! That leads to the container connecting to the hosts console and other devices and committing random acts of terrorism. No, it shouldn't, because lxc sets up the console after doing the mounts. Damn, dude! That worked! That kludge rang da bell. Of course, I also discovered the boneheaded typo I had in attempting the tmpfs mount in the process. :-P I now have a container running systemd up and running with Fedora 17 in it. I'm not sure I'm totally happy with it. Because of doing the devtmpfs thing, the guest can immediately see things like removable drives coming and going and might, presumably, be able to mount them. Not thrilled with that from a security standpoint. Would also mean the guests could access things like my permanent forensic CDs that are in the CD drives. I guess that can be restricted in the config but still makes me a bit uncomfortable that the guest has complete visibility into the hosts dev system. Another gotcha, albeit a much more minor one... When systemd drops into this mode, you no longer have vty consoles available so lxc-console won't work. That's actually on their page. I remember seeing this: -- If systemd detects it is run in a container it will spawn a single shell on /dev/console, and not care about VTs or multiple gettys on VTs -- Suboptimal but a small price to pay I suppose. -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] systemd inside LXC
On Mon, 2012-10-22 at 18:05 -0400, Michael H. Warfield wrote: On Mon, 2012-10-22 at 16:21 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Mon, 2012-10-22 at 15:14 -0500, Serge Hallyn wrote: Trimming some overhead we've seen enough of... How about just a devtmpfs? We actually now do this by default (as of very recently) in ubuntu by adding devtmpfsdev devtmpfs defaults 0 0 NO! That's the problem! That leads to the container connecting to the hosts console and other devices and committing random acts of terrorism. No, it shouldn't, because lxc sets up the console after doing the mounts. Damn, dude! That worked! That kludge rang da bell. Of course, I also discovered the boneheaded typo I had in attempting the tmpfs mount in the process. :-P I now have a container running systemd up and running with Fedora 17 in it. I'm not sure I'm totally happy with it. Because of doing the devtmpfs thing, the guest can immediately see things like removable drives coming and going and might, presumably, be able to mount them. Not thrilled with that from a security standpoint. Would also mean the guests could access things like my permanent forensic CDs that are in the CD drives. I guess that can be restricted in the config but still makes me a bit uncomfortable that the guest has complete visibility into the hosts dev system. Another downside. Container does not shut down cleanly... init 0 inside the container... In lxc-start - Unmounting file systems. Could not remount as read-only /: Device or resource busy Not all file systems unmounted, 1 left. Detaching loop devices. Could not delete loopback /dev/loop7: Operation not permitted Could not delete loopback /dev/loop6: Operation not permitted Could not delete loopback /dev/loop5: Operation not permitted Could not delete loopback /dev/loop4: Operation not permitted Could not delete loopback /dev/loop3: Operation not permitted Could not delete loopback /dev/loop2: Operation not permitted Could not delete loopback /dev/loop1: Operation not permitted Could not delete loopback /dev/loop0: Operation not permitted Not all loop devices detached, 8 left. Cannot finalize remaining file systems and devices, giving up. Exiting container. lxc-start: Device or resource busy - failed to remove cgroup '/sys/fs/cgroup/systemd/Alcove' Not good. The tasks file is empty but... Can't get rid of it. Operation not permitted. Sigh... Getting closer though. Much closer. Another gotcha, albeit a much more minor one... When systemd drops into this mode, you no longer have vty consoles available so lxc-console won't work. That's actually on their page. I remember seeing this: -- If systemd detects it is run in a container it will spawn a single shell on /dev/console, and not care about VTs or multiple gettys on VTs -- Suboptimal but a small price to pay I suppose. -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] systemd inside LXC
On Mon, 2012-10-22 at 18:37 -0400, Michael H. Warfield wrote:
On Mon, 2012-10-22 at 18:05 -0400, Michael H. Warfield wrote:
On Mon, 2012-10-22 at 16:21 -0500, Serge Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
On Mon, 2012-10-22 at 15:14 -0500, Serge Hallyn wrote:
Trimming some overhead we've seen enough of...
How about just a devtmpfs? We actually now do this by default (as of
very
recently) in ubuntu by adding
devtmpfsdev devtmpfs defaults 0 0
NO! That's the problem! That leads to the container connecting to the
hosts console and other devices and committing random acts of terrorism.
No, it shouldn't, because lxc sets up the console after doing the mounts.
Damn, dude! That worked! That kludge rang da bell. Of course, I also
discovered the boneheaded typo I had in attempting the tmpfs mount in
the process. :-P I now have a container running systemd up and running
with Fedora 17 in it.
I'm not sure I'm totally happy with it.
Because of doing the devtmpfs thing, the guest can immediately see
things like removable drives coming and going and might, presumably, be
able to mount them. Not thrilled with that from a security standpoint.
Would also mean the guests could access things like my permanent
forensic CDs that are in the CD drives. I guess that can be restricted
in the config but still makes me a bit uncomfortable that the guest has
complete visibility into the hosts dev system.
Another downside. Container does not shut down cleanly...
And another...
Container seems to hang if lxc-start is run in disconnected mode
(lxc-start -d -o {log}). Starts up fine with a console that's connected
to pty's but not to a log it seems...
init 0 inside the container...
In lxc-start -
Unmounting file systems.
Could not remount as read-only /: Device or resource busy
Not all file systems unmounted, 1 left.
Detaching loop devices.
Could not delete loopback /dev/loop7: Operation not permitted
Could not delete loopback /dev/loop6: Operation not permitted
Could not delete loopback /dev/loop5: Operation not permitted
Could not delete loopback /dev/loop4: Operation not permitted
Could not delete loopback /dev/loop3: Operation not permitted
Could not delete loopback /dev/loop2: Operation not permitted
Could not delete loopback /dev/loop1: Operation not permitted
Could not delete loopback /dev/loop0: Operation not permitted
Not all loop devices detached, 8 left.
Cannot finalize remaining file systems and devices, giving up.
Exiting container.
lxc-start: Device or resource busy - failed to remove cgroup
'/sys/fs/cgroup/systemd/Alcove'
Not good. The tasks file is empty but... Can't get rid of it.
Operation not permitted.
Sigh...
Getting closer though. Much closer.
Another gotcha, albeit a much more minor one... When systemd drops into
this mode, you no longer have vty consoles available so lxc-console
won't work. That's actually on their page.
I remember seeing this:
--
If systemd detects it is run in a container it will spawn a single shell
on /dev/console, and not care about VTs or multiple gettys on VTs
--
Suboptimal but a small price to pay I suppose.
-serge
Regards,
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it!
signature.asc
Description: This is a digitally signed message part
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
Sorry for taking a few days to get back on this. I was delivering a
guest lecture up at Fordham University last Tuesday so I was out of
pocket a couple of days or I would have responded sooner...
On Mon, 2012-10-22 at 16:59 -0400, Michael H. Warfield wrote:
On Mon, 2012-10-22 at 22:50 +0200, Lennart Poettering wrote:
On Mon, 22.10.12 11:48, Michael H. Warfield (m...@wittsend.com) wrote:
To summarize the problem... The LXC startup binary sets up various
things for /dev and /dev/pts for the container to run properly and
this
works perfectly fine for SystemV start-up scripts and/or Upstart.
Unfortunately, systemd has mounts of devtmpfs on /dev and devpts
on /dev/pts which then break things horribly. This is because the
kernel currently lacks namespaces for devices and won't for some time
to
come (in design). When devtmpfs gets mounted over top of /dev in the
container, it then hijacks the hosts console tty and several other
devices which had been set up through bind mounts by LXC and should
have
been LEFT ALONE.
Please initialize a minimal tmpfs on /dev. systemd will then work fine.
My containers have a reasonable /dev that work with Upstart just fine
but they are not on tmpfs. Is mounting tmpfs on /dev and recreating
that minimal /dev required?
Well, it can be any kind of mount really. Just needs to be a mount. And
the idea is to use tmpfs for this.
What /dev are you currently using? It's probably not a good idea to
reuse the hosts' /dev, since it contains so many device nodes that
should not be accessible/visible to the container.
Got it. And that explains the problems we're seeing but also what I'm
seeing in some libvirt-lxc related pages, which is a separate and
distinct project in spite of the similarities in the name...
http://wiki.1tux.org/wiki/Lxc/Installation#Additional_notes
Unfortunately, in our case, merely getting a mount in there is a
complication in that it also has to be populated but, at least, we
understand the problem set now.
Ok... Serge and I were corresponding on the lxc-users list and he had a
suggestion that worked but I consider to be a bit of a sub-optimal
workaround. Ironically, it was to mount devtmpfs on /dev. We don't
(currently) have a method to auto-populate a tmpfs mount with the needed
devices and this provided it. It does have a problem that makes me
uncomfortable in that the container now has visibility into the
hosts /dev system. I'm a security expert and I'm not comfortable with
that solution even with the controls we have. We can control access
but still, not happy with that.
I now have a container that starts with systemd running more or less
properly. We do have some problems with the convention that has been
set up, however.
When running in this mode, you run on the console and you don't spawn
getty's on the tty's. There seems to be a problem with this.
In this mode, if I manually start the container in a terminal window,
that eventually results in a login prompt there. Under sysvinit and
upstart I don't get that and can detach.
If I run lxc-console (which attaches to one of the vtys) it gives me
nothing. Under sysvinit and upstart I get vty login prompts because
they have started getty on those vtys. This is important in case
network access has not started for one reason or another and the
container was started detached in the background.
If I start lxc-start in detached mode (-d -o {logfile}) lxc-start
redirects the system console to the log file and goes daemon. In this
case, the systemd container hangs and never starts.
I SUSPECT the hang condition is something to do with systemd trying to
start and interactive console on /dev/console, which sysvinit and
upstart do not do. Maybe we have to do something different with the
redirects in this case, but it's not working consistent with the other
packages. We should also start appropriate gettys on those vtys if they
are configured. Maybe start the getty's if the tty? exists up to a
configured limit (and don't restart if they immediately fail) and
obviously don't start them if they don't. It then gives up control over
that process. Also don't start a login on /dev/console if you DO start
a getty? That would make your behavior congruent with that of the other
two systems.
I've got some more problems relating to shutting down containers, some
of which may be related to mounting tmpfs on /run to which /var/run is
symlinked to. We're doing halt / restart detection by monitoring utmp
in that directory but it looks like utmp isn't even in that directory
anymore and mounting tmpfs on it was always problematical. We may have
to have a more generic method to detect when a container has shut down
or is restarting in that case. I'm also finding we end up with dangling
resources where we can't remove to cgroup directories after a halt and
that creates a serious problem I have
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Thu, 2012-10-25 at 11:19 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): Sorry for taking a few days to get back on this. I was delivering a guest lecture up at Fordham University last Tuesday so I was out of pocket a couple of days or I would have responded sooner... On Mon, 2012-10-22 at 16:59 -0400, Michael H. Warfield wrote: On Mon, 2012-10-22 at 22:50 +0200, Lennart Poettering wrote: On Mon, 22.10.12 11:48, Michael H. Warfield (m...@wittsend.com) wrote: To summarize the problem... The LXC startup binary sets up various things for /dev and /dev/pts for the container to run properly and this works perfectly fine for SystemV start-up scripts and/or Upstart. Unfortunately, systemd has mounts of devtmpfs on /dev and devpts on /dev/pts which then break things horribly. This is because the kernel currently lacks namespaces for devices and won't for some time to come (in design). When devtmpfs gets mounted over top of /dev in the container, it then hijacks the hosts console tty and several other devices which had been set up through bind mounts by LXC and should have been LEFT ALONE. Please initialize a minimal tmpfs on /dev. systemd will then work fine. My containers have a reasonable /dev that work with Upstart just fine but they are not on tmpfs. Is mounting tmpfs on /dev and recreating that minimal /dev required? Well, it can be any kind of mount really. Just needs to be a mount. And the idea is to use tmpfs for this. What /dev are you currently using? It's probably not a good idea to reuse the hosts' /dev, since it contains so many device nodes that should not be accessible/visible to the container. Got it. And that explains the problems we're seeing but also what I'm seeing in some libvirt-lxc related pages, which is a separate and distinct project in spite of the similarities in the name... http://wiki.1tux.org/wiki/Lxc/Installation#Additional_notes Unfortunately, in our case, merely getting a mount in there is a complication in that it also has to be populated but, at least, we understand the problem set now. Ok... Serge and I were corresponding on the lxc-users list and he had a suggestion that worked but I consider to be a bit of a sub-optimal workaround. Ironically, it was to mount devtmpfs on /dev. We don't Oh, sorry - I take back that suggestion :) Well, it worked (sort of) and reinforced what the problem was and where the solution lay so there's no need to be sorry for it. We learned and we know why it's not the right solution. This is good. We made a lot of progress on this just in the last week. This is very good. Note that we have mount hooks, so templates could install a mount hook to mount a tmpfs onto /dev and populate it. Ah, now that is interesting. I haven't looked at that before. I need to explore that further. Or, if everyone is going to need it, we could just add a 'lxc.populatedevs = 1' option which does that without needing a hook. Eventually, with Fedora (and later RHEL / CentOS / SL), Arch Linux, and others going to systemd, I think this is going to be needed sooner than later. devtmpfs should not be used in containers :) Concur! -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
Hey Serge, On Thu, 2012-10-25 at 11:19 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): Sorry for taking a few days to get back on this. I was delivering a guest lecture up at Fordham University last Tuesday so I was out of pocket a couple of days or I would have responded sooner... On Mon, 2012-10-22 at 16:59 -0400, Michael H. Warfield wrote: On Mon, 2012-10-22 at 22:50 +0200, Lennart Poettering wrote: On Mon, 22.10.12 11:48, Michael H. Warfield (m...@wittsend.com) wrote: To summarize the problem... The LXC startup binary sets up various things for /dev and /dev/pts for the container to run properly and this works perfectly fine for SystemV start-up scripts and/or Upstart. Unfortunately, systemd has mounts of devtmpfs on /dev and devpts on /dev/pts which then break things horribly. This is because the kernel currently lacks namespaces for devices and won't for some time to come (in design). When devtmpfs gets mounted over top of /dev in the container, it then hijacks the hosts console tty and several other devices which had been set up through bind mounts by LXC and should have been LEFT ALONE. Please initialize a minimal tmpfs on /dev. systemd will then work fine. My containers have a reasonable /dev that work with Upstart just fine but they are not on tmpfs. Is mounting tmpfs on /dev and recreating that minimal /dev required? Well, it can be any kind of mount really. Just needs to be a mount. And the idea is to use tmpfs for this. What /dev are you currently using? It's probably not a good idea to reuse the hosts' /dev, since it contains so many device nodes that should not be accessible/visible to the container. Got it. And that explains the problems we're seeing but also what I'm seeing in some libvirt-lxc related pages, which is a separate and distinct project in spite of the similarities in the name... http://wiki.1tux.org/wiki/Lxc/Installation#Additional_notes Unfortunately, in our case, merely getting a mount in there is a complication in that it also has to be populated but, at least, we understand the problem set now. Ok... Serge and I were corresponding on the lxc-users list and he had a suggestion that worked but I consider to be a bit of a sub-optimal workaround. Ironically, it was to mount devtmpfs on /dev. We don't Oh, sorry - I take back that suggestion :) Note that we have mount hooks, so templates could install a mount hook to mount a tmpfs onto /dev and populate it. Ok... I've done some cursory search and turned up nothing but some comments about pre mount hooks. Where is the documentation about this feature and how I might use / implement it? Some examples would probably suffice. Is there a require release version of lxc-utils? Or, if everyone is going to need it, we could just add a 'lxc.populatedevs = 1' option which does that without needing a hook. devtmpfs should not be used in containers :) -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Thu, 2012-10-25 at 13:23 -0400, Michael H. Warfield wrote: Hey Serge, On Thu, 2012-10-25 at 11:19 -0500, Serge Hallyn wrote: ... Oh, sorry - I take back that suggestion :) Note that we have mount hooks, so templates could install a mount hook to mount a tmpfs onto /dev and populate it. Ok... I've done some cursory search and turned up nothing but some comments about pre mount hooks. Where is the documentation about this feature and how I might use / implement it? Some examples would probably suffice. Is there a require release version of lxc-utils? I think I found what I needed in the changelog here: http://www.mail-archive.com/lxc-devel@lists.sourceforge.net/msg01490.html I'll play with it and report back. Or, if everyone is going to need it, we could just add a 'lxc.populatedevs = 1' option which does that without needing a hook. devtmpfs should not be used in containers :) -serge Regards, Mike -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Thu, 2012-10-25 at 14:02 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Thu, 2012-10-25 at 13:23 -0400, Michael H. Warfield wrote: Hey Serge, On Thu, 2012-10-25 at 11:19 -0500, Serge Hallyn wrote: ... Oh, sorry - I take back that suggestion :) Note that we have mount hooks, so templates could install a mount hook to mount a tmpfs onto /dev and populate it. Ok... I've done some cursory search and turned up nothing but some comments about pre mount hooks. Where is the documentation about this feature and how I might use / implement it? Some examples would probably suffice. Is there a require release version of lxc-utils? I think I found what I needed in the changelog here: http://www.mail-archive.com/lxc-devel@lists.sourceforge.net/msg01490.html I'll play with it and report back. Also the Lifecycle management hooks section in https://help.ubuntu.com/12.10/serverguide/lxc.html This isn't working... Based on what was in both of those articles, I added this entry to another container (Plover) to test... lxc.hook.mount = /var/lib/lxc/Plover/mount When I run lxc-start -n Plover, I see this: [root@forest ~]# lxc-start -n Plover lxc-start: unknow key lxc.hook.mount lxc-start: failed to read configuration file I'm running the latest rc... [root@forest ~]# rpm -qa | grep lxc lxc-0.8.0.rc2-1.fc16.x86_64 lxc-libs-0.8.0.rc2-1.fc16.x86_64 lxc-doc-0.8.0.rc2-1.fc16.x86_64 Is it something in git that hasn't made it to a release yet? Note that I'm thinking that having lxc-start guess how to fill in /dev is wrong, because different distros and even different releases of the same distros have different expectations. For instance ubuntu lucid wants /dev/shm to be a directory, while precise+ wants a symlink. So somehow the template should get involved, be it by adding a hook, or simply specifying a configuration file which lxc uses internally to decide how to create /dev. I agree this needs to be by some sort of convention or template that we can adjust. Personally I'd prefer if /dev were always populated by the templates, and containers (i.e. userspace) didn't mount a fresh tmpfs for /dev. But that does complicate userspace, and we've seen it in debian/ubuntu as well (i.e. at certain package upgrades which rely on /dev being cleared after a reboot). -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Thu, 2012-10-25 at 23:38 +0200, Lennart Poettering wrote: On Thu, 25.10.12 11:59, Michael H. Warfield (m...@wittsend.com) wrote: I've got some more problems relating to shutting down containers, some of which may be related to mounting tmpfs on /run to which /var/run is symlinked to. We're doing halt / restart detection by monitoring utmp in that directory but it looks like utmp isn't even in that directory anymore and mounting tmpfs on it was always problematical. We may have to have a more generic method to detect when a container has shut down or is restarting in that case. I can't parse this. The system call reboot() is virtualized for containers just fine and the container managaer (i.e. LXC) can check for that easily. The problem we have had was with differentiating between reboot and halt to either shut the container down cold or restarted it. You say easily and yet we never came up with an easy solution and monitored utmp instead for the next runlevel change. What is your easy solution for that problem? Lennart -- Lennart Poettering - Red Hat, Inc. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Thu, 2012-10-25 at 20:30 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Thu, 2012-10-25 at 23:38 +0200, Lennart Poettering wrote: On Thu, 25.10.12 11:59, Michael H. Warfield (m...@wittsend.com) wrote: I've got some more problems relating to shutting down containers, some of which may be related to mounting tmpfs on /run to which /var/run is symlinked to. We're doing halt / restart detection by monitoring utmp in that directory but it looks like utmp isn't even in that directory anymore and mounting tmpfs on it was always problematical. We may have to have a more generic method to detect when a container has shut down or is restarting in that case. I can't parse this. The system call reboot() is virtualized for containers just fine and the container managaer (i.e. LXC) can check for that easily. The problem we have had was with differentiating between reboot and halt to either shut the container down cold or restarted it. You say easily and yet we never came up with an easy solution and monitored utmp instead for the next runlevel change. What is your easy solution for that problem? I think you're on older kernels, where we had to resort to that. Pretty recently Daniel Lezcano's patch was finally accepted upstream, which lets a container call reboot() and lets the parent of init tell whether it called reboot or shutdown by looking at wTERMSIG(status). Now THAT is wonderful news! I hadn't realized that had been accepted. So we no longer need to rely on the old utmp kludge? -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Thu, 2012-10-25 at 15:42 -0400, Michael H. Warfield wrote: On Thu, 2012-10-25 at 14:02 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Thu, 2012-10-25 at 13:23 -0400, Michael H. Warfield wrote: Hey Serge, On Thu, 2012-10-25 at 11:19 -0500, Serge Hallyn wrote: ... Oh, sorry - I take back that suggestion :) Note that we have mount hooks, so templates could install a mount hook to mount a tmpfs onto /dev and populate it. Ok... I've done some cursory search and turned up nothing but some comments about pre mount hooks. Where is the documentation about this feature and how I might use / implement it? Some examples would probably suffice. Is there a require release version of lxc-utils? I think I found what I needed in the changelog here: http://www.mail-archive.com/lxc-devel@lists.sourceforge.net/msg01490.html I'll play with it and report back. Also the Lifecycle management hooks section in https://help.ubuntu.com/12.10/serverguide/lxc.html This isn't working... Based on what was in both of those articles, I added this entry to another container (Plover) to test... lxc.hook.mount = /var/lib/lxc/Plover/mount When I run lxc-start -n Plover, I see this: [root@forest ~]# lxc-start -n Plover lxc-start: unknow key lxc.hook.mount lxc-start: failed to read configuration file I'm running the latest rc... [root@forest ~]# rpm -qa | grep lxc lxc-0.8.0.rc2-1.fc16.x86_64 lxc-libs-0.8.0.rc2-1.fc16.x86_64 lxc-doc-0.8.0.rc2-1.fc16.x86_64 Is it something in git that hasn't made it to a release yet? nm... I see it. It's in git and hasn't made it to a release. I'm working on a git build to test now. If this is something that solves some of this, we need to move things along here and get these things moved out. According to git, 0.8.0rc2 was 7 months ago? What's the show stoppers here? Note that I'm thinking that having lxc-start guess how to fill in /dev is wrong, because different distros and even different releases of the same distros have different expectations. For instance ubuntu lucid wants /dev/shm to be a directory, while precise+ wants a symlink. So somehow the template should get involved, be it by adding a hook, or simply specifying a configuration file which lxc uses internally to decide how to create /dev. I agree this needs to be by some sort of convention or template that we can adjust. Personally I'd prefer if /dev were always populated by the templates, and containers (i.e. userspace) didn't mount a fresh tmpfs for /dev. But that does complicate userspace, and we've seen it in debian/ubuntu as well (i.e. at certain package upgrades which rely on /dev being cleared after a reboot). -serge Regards, Mike Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
Adding in the lxc-devel list. On Thu, 2012-10-25 at 22:59 -0400, Michael H. Warfield wrote: On Thu, 2012-10-25 at 15:42 -0400, Michael H. Warfield wrote: On Thu, 2012-10-25 at 14:02 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Thu, 2012-10-25 at 13:23 -0400, Michael H. Warfield wrote: Hey Serge, On Thu, 2012-10-25 at 11:19 -0500, Serge Hallyn wrote: ... Oh, sorry - I take back that suggestion :) Note that we have mount hooks, so templates could install a mount hook to mount a tmpfs onto /dev and populate it. Ok... I've done some cursory search and turned up nothing but some comments about pre mount hooks. Where is the documentation about this feature and how I might use / implement it? Some examples would probably suffice. Is there a require release version of lxc-utils? I think I found what I needed in the changelog here: http://www.mail-archive.com/lxc-devel@lists.sourceforge.net/msg01490.html I'll play with it and report back. Also the Lifecycle management hooks section in https://help.ubuntu.com/12.10/serverguide/lxc.html This isn't working... Based on what was in both of those articles, I added this entry to another container (Plover) to test... lxc.hook.mount = /var/lib/lxc/Plover/mount When I run lxc-start -n Plover, I see this: [root@forest ~]# lxc-start -n Plover lxc-start: unknow key lxc.hook.mount lxc-start: failed to read configuration file I'm running the latest rc... [root@forest ~]# rpm -qa | grep lxc lxc-0.8.0.rc2-1.fc16.x86_64 lxc-libs-0.8.0.rc2-1.fc16.x86_64 lxc-doc-0.8.0.rc2-1.fc16.x86_64 Is it something in git that hasn't made it to a release yet? nm... I see it. It's in git and hasn't made it to a release. I'm working on a git build to test now. If this is something that solves some of this, we need to move things along here and get these things moved out. According to git, 0.8.0rc2 was 7 months ago? What's the show stoppers here? While the git repo says 7 months ago, the date stamp on the lxc-0.8.0-rc2 tarball is from July 10, so about 3-1/2 months ago. Sounds like we've accumulated some features (like the hooks) we are going to need like months ago to deal with this systemd debacle. How close are we to either 0.8.0rc3 or 0.8.0? Any blockers or are we just waiting on some more features? Note that I'm thinking that having lxc-start guess how to fill in /dev is wrong, because different distros and even different releases of the same distros have different expectations. For instance ubuntu lucid wants /dev/shm to be a directory, while precise+ wants a symlink. So somehow the template should get involved, be it by adding a hook, or simply specifying a configuration file which lxc uses internally to decide how to create /dev. I agree this needs to be by some sort of convention or template that we can adjust. Personally I'd prefer if /dev were always populated by the templates, and containers (i.e. userspace) didn't mount a fresh tmpfs for /dev. But that does complicate userspace, and we've seen it in debian/ubuntu as well (i.e. at certain package upgrades which rely on /dev being cleared after a reboot). -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Fri, 2012-10-26 at 12:11 -0400, Michael H. Warfield wrote: On Thu, 2012-10-25 at 23:38 +0200, Lennart Poettering wrote: On Thu, 25.10.12 11:59, Michael H. Warfield (m...@wittsend.com) wrote: I SUSPECT the hang condition is something to do with systemd trying to start and interactive console on /dev/console, which sysvinit and upstart do not do. Yes, this is documented, please see the link I already posted, and which I linked above a second time. This may have been my fault. I was using the -o option to lxc-start (output logfile) and failed to specify the -c (console output redirect) option. It seems to fire up nicely (albeit with other problems) with that additional option. Continuing my research. Confirming. Using the -c option for the console file works. Unfortunately, thanks to no getty's on the ttys so lxc-console does not work and no way to connect to that console redirect and the failure of the network to start, I'm still trying to figure out just what is face planting in a container I can not access. :-/=/ Punch out the punch list one PUNCH at at time here. I've got some more problems relating to shutting down containers, some of which may be related to mounting tmpfs on /run to which /var/run is symlinked to. We're doing halt / restart detection by monitoring utmp in that directory but it looks like utmp isn't even in that directory anymore and mounting tmpfs on it was always problematical. We may have to have a more generic method to detect when a container has shut down or is restarting in that case. I can't parse this. The system call reboot() is virtualized for containers just fine and the container managaer (i.e. LXC) can check for that easily. Apparently, in recent kernels, we can. Unfortunately, I'm still finding that I can not restart a container I have previously halted. I have no problem with sysvinit and upstart systems on this host, so it is a container problem peculiar to systemd containers. Continuing to research that problem. Lennart -- Lennart Poettering - Red Hat, Inc. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- WINDOWS 8 is here. Millions of people. Your app in 30 days. Visit The Windows 8 Center at Sourceforge for all your go to resources. http://windows8center.sourceforge.net/ join-generation-app-and-make-money-coding-fast/___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
/me erasing everything at this point and taking off the systemd crew, since this will have no relevance to them... Testing the hook feature out using git rev (finally got it built)... I added this line to my config... lxc.mount.entry=tmpfs /srv/lxc/private/Plover/dev.tmp tmpfs defaults 0 0 lxc.hook.mount = /var/lib/lxc/Plover/mount In /var/lib/lxc/Plover/mount I have this: -- rsync -avAH /srv/lxc/private/Plover/dev.template/. /srv/lxc/private/Plover/dev.tmp/ -- (This is just testing out the concepts. If I understand this correctly, lxc.hook.pre-mount runs BEFORE the mounting takes place and lxc.hook.mount takes place after the mount. Problem is, the result of that rsync is not showing up in the mounted tmpfs file system but is showing up in the underlying parent file system as if it were run pre-mount. Something not right here... Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- WINDOWS 8 is here. Millions of people. Your app in 30 days. Visit The Windows 8 Center at Sourceforge for all your go to resources. http://windows8center.sourceforge.net/ join-generation-app-and-make-money-coding-fast/___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Sat, 2012-10-27 at 13:40 -0400, Michael H. Warfield wrote: /me erasing everything at this point and taking off the systemd crew, since this will have no relevance to them... Testing the hook feature out using git rev (finally got it built)... I added this line to my config... lxc.mount.entry=tmpfs /srv/lxc/private/Plover/dev.tmp tmpfs defaults 0 0 lxc.hook.mount = /var/lib/lxc/Plover/mount In /var/lib/lxc/Plover/mount I have this: -- rsync -avAH /srv/lxc/private/Plover/dev.template/. /srv/lxc/private/Plover/dev.tmp/ -- (This is just testing out the concepts. If I understand this correctly, lxc.hook.pre-mount runs BEFORE the mounting takes place and lxc.hook.mount takes place after the mount. Problem is, the result of that rsync is not showing up in the mounted tmpfs file system but is showing up in the underlying parent file system as if it were run pre-mount. Something not right here... I changed it to lxc.hook.start = /srv/lxc/mount (where I put the script in the container) which then works but that then requires the template and the command to be in the container. Suboptimal to say the least. But it gives me a way to test this tmpfs thing out. I also noticed that the .start hook runs, it appears, after caps are dropped and I see a lot of bitching about mknod on certain devices. I had to thrown an exit 0 into that script so it would continue in spite of the errors but, now, I can refine my template based on what it could create. Regards, Mike Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- WINDOWS 8 is here. Millions of people. Your app in 30 days. Visit The Windows 8 Center at Sourceforge for all your go to resources. http://windows8center.sourceforge.net/ join-generation-app-and-make-money-coding-fast/___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Sat, 2012-10-27 at 13:51 -0400, Michael H. Warfield wrote: On Sat, 2012-10-27 at 13:40 -0400, Michael H. Warfield wrote: /me erasing everything at this point and taking off the systemd crew, since this will have no relevance to them... Testing the hook feature out using git rev (finally got it built)... I added this line to my config... lxc.mount.entry=tmpfs /srv/lxc/private/Plover/dev.tmp tmpfs defaults 0 0 lxc.hook.mount = /var/lib/lxc/Plover/mount In /var/lib/lxc/Plover/mount I have this: -- rsync -avAH /srv/lxc/private/Plover/dev.template/. /srv/lxc/private/Plover/dev.tmp/ -- (This is just testing out the concepts. If I understand this correctly, lxc.hook.pre-mount runs BEFORE the mounting takes place and lxc.hook.mount takes place after the mount. Problem is, the result of that rsync is not showing up in the mounted tmpfs file system but is showing up in the underlying parent file system as if it were run pre-mount. Something not right here... I changed it to lxc.hook.start = /srv/lxc/mount (where I put the script in the container) which then works but that then requires the template and the command to be in the container. Suboptimal to say the least. But it gives me a way to test this tmpfs thing out. I also noticed that the .start hook runs, it appears, after caps are dropped and I see a lot of bitching about mknod on certain devices. I had to thrown an exit 0 into that script so it would continue in spite of the errors but, now, I can refine my template based on what it could create. Crap. I've got a catch-22 here... This is going to take some work. Yes, we can create the /dev directory with tmpfs from a template. Problem is that /dev/pts does not exist at the time we need to mount the devpts on /dev/pts for the pty's so that hurls chunks and dies. We can't create the /dev/ directory contents prior to mounting in the pre-mount hook because we won't have tmpfs in place at the time. We have to get tmpfs mounted on /dev and then create /dev/pts and then mount the ptys in there. There has to be a mkdir in between those two mount actions. Simplest solution would seem to be to add some logic to the mount logic that says test if directory exists and, if not, create it. I'm not sure of the consequences of that, though. I don't see a way to make this happen with hooks. It's almost like we need and on-mount per mount hook. Regards, Mike Regards, Mike Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- WINDOWS 8 is here. Millions of people. Your app in 30 days. Visit The Windows 8 Center at Sourceforge for all your go to resources. http://windows8center.sourceforge.net/ join-generation-app-and-make-money-coding-fast/___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Sat, 2012-10-27 at 19:44 +0100, Colin Guthrie wrote: 'Twas brillig, and Michael H. Warfield at 26/10/12 18:18 did gyre and gimble: What the hell is this? /var/run is symlinked to /run and is mounted with a tmpfs. Yup, that's how /var/run and /run is being handled these days. It provides a consistent space to pass info from the initrd over to the main system and has various other uses also. Interesting. I hadn't considered that aspect of it before. Very interesting. If you want to ensure files are created in this folder, just drop a config file in to /usr/lib/tmpfiles.d/ in the package in question. See man systemd-tmpfiles for more info. NOW THAT is something else I needed to know about! Thank you very very much! Learned something new. This whole thing has been a massive learning experience getting this container kick started. Could be some packages are not fully upgraded to this concept in F17. As a non-fedora user, I can't really comment on that specifically. As it turns out, the kernel has had some of our patches applied that I wasn't aware of vis-a-vis reboot/halt and this should no longer be an issue. I'm still struggling with the tmpfs on /dev thing and have run into a catch-22 with regards to that. I can mount tmpfs on /dev just fine and can populate it just fine in a post mount hook but, then, we're trying to mount a devpts file system on /dev/pts before we've had a chance to populate it and it's then crashing on the mount. Sigh... I think that's going to now have to wait for Serge or Daniel to comment on. Col -- Colin Guthrie gmane(at)colin.guthr.ie http://colin.guthr.ie/ Day Job: Tribalogic Limited http://www.tribalogic.net/ Open Source: Mageia Contributor http://www.mageia.org/ PulseAudio Hacker http://www.pulseaudio.org/ Trac Hacker http://trac.edgewall.org/ Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- WINDOWS 8 is here. Millions of people. Your app in 30 days. Visit The Windows 8 Center at Sourceforge for all your go to resources. http://windows8center.sourceforge.net/ join-generation-app-and-make-money-coding-fast/___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Sun, 2012-10-28 at 18:52 +0100, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Sat, 2012-10-27 at 13:51 -0400, Michael H. Warfield wrote: On Sat, 2012-10-27 at 13:40 -0400, Michael H. Warfield wrote: /me erasing everything at this point and taking off the systemd crew, since this will have no relevance to them... Testing the hook feature out using git rev (finally got it built)... I added this line to my config... lxc.mount.entry=tmpfs /srv/lxc/private/Plover/dev.tmp tmpfs defaults 0 0 lxc.hook.mount = /var/lib/lxc/Plover/mount In /var/lib/lxc/Plover/mount I have this: -- rsync -avAH /srv/lxc/private/Plover/dev.template/. /srv/lxc/private/Plover/dev.tmp/ -- (This is just testing out the concepts. If I understand this correctly, lxc.hook.pre-mount runs BEFORE the mounting takes place and lxc.hook.mount takes place after the mount. Problem is, the result of that rsync is not showing up in the mounted tmpfs file system but is showing up in the underlying parent file system as if it were run pre-mount. Something not right here... I changed it to lxc.hook.start = /srv/lxc/mount (where I put the script in the container) which then works but that then requires the template and the command to be in the container. Suboptimal to say the least. But it gives me a way to test this tmpfs thing out. I also noticed that the .start hook runs, it appears, after caps are dropped and I see a lot of bitching about mknod on certain devices. I had to thrown an exit 0 into that script so it would continue in spite of the errors but, now, I can refine my template based on what it could create. Crap. I've got a catch-22 here... This is going to take some work. Hey, I've got a rather minimal patch (appended below) to add the support for mounting and populating a minimal /dev working. (A few hours were wasted due to my not knowing that upstart was going to issue mounted-dev even though /dev was mounted before upstart started - and the mounted-dev hook deletes and recreates all consoles. GAH) Yes, we can create the /dev directory with tmpfs from a template. Problem is that /dev/pts does not exist at the time we need to mount the devpts on /dev/pts for the pty's so that hurls chunks and dies. We can't create the /dev/ directory contents prior to mounting in the pre-mount hook because we won't have tmpfs in place at the time. We have to get tmpfs mounted on /dev and then create /dev/pts and then mount the ptys in there. There has to be a mkdir in between those two mount actions. Simplest solution would seem to be to add some logic to the mount logic that says test if directory exists and, if not, create it. I'm not sure of the consequences of that, though. I don't see a way to make this happen with hooks. It's almost like we need and on-mount per mount hook. Should be moot given my patch, which I intend to push this week, but why couldn't a lxc.hook.mount do the whole thing, mount /dev and and populate it? I wasn't thinking a lxc.hook.start, for the reasons you encountered, but I assume you tried lxc.hook.mount and it failed? See my other comment about lxc.hook.mount. I tried using it to populate /dev but it showed up in the parent of the mount undeneath the tmpfs mount. It was like it ran pre-mount. I tried it for several different combinations and couldn't get it to go. Would still have the problem with mounting /dev/pts which would take place before the mount hook at run to mount the file system and populate it. That actually MIGHT work (gotta think on this now) if I used lxc.hook.pre-mount and mounted tmpfs over /dev, and populated it but then I run into a problem where I was using a bind-mount for the rootfs. Might still work. I'll test your patch out first though. Patch below: snip Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- WINDOWS 8 is here. Millions of people. Your app in 30 days. Visit The Windows 8 Center at Sourceforge for all your go to resources. http://windows8center.sourceforge.net/ join-generation-app-and-make-money-coding-fast/___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Sun, 2012-10-28 at 18:52 +0100, Serge Hallyn wrote: Should be moot given my patch, which I intend to push this week, but why couldn't a lxc.hook.mount do the whole thing, mount /dev and and populate it? I wasn't thinking a lxc.hook.start, for the reasons you encountered, but I assume you tried lxc.hook.mount and it failed? Patch below: Patch failed against 0.8.0rc2 and git root. Even with loose patching for whitespace, had failures... This was against git: [mhw@forest lxc]$ patch -p1 -l ../lxc-autodev.patch patching file src/lxc/conf.c Hunk #1 succeeded at 616 (offset -3 lines). Hunk #2 succeeded at 633 (offset -3 lines). Hunk #3 succeeded at 839 (offset -3 lines). Hunk #4 succeeded at 2203 (offset -66 lines). patching file src/lxc/conf.h Hunk #1 FAILED at 227. 1 out of 1 hunk FAILED -- saving rejects to file src/lxc/conf.h.rej patching file src/lxc/confile.c Hunk #1 FAILED at 77. Hunk #2 FAILED at 118. Hunk #3 succeeded at 854 with fuzz 2 (offset 1 line). 2 out of 3 hunks FAILED -- saving rejects to file src/lxc/confile Version to patch to? I'm trying to manually integrate those failed hunks now. Shouldn't be too difficult for only three. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- WINDOWS 8 is here. Millions of people. Your app in 30 days. Visit The Windows 8 Center at Sourceforge for all your go to resources. http://windows8center.sourceforge.net/ join-generation-app-and-make-money-coding-fast/___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Sun, 2012-10-28 at 18:52 +0100, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Sat, 2012-10-27 at 13:51 -0400, Michael H. Warfield wrote: On Sat, 2012-10-27 at 13:40 -0400, Michael H. Warfield wrote: /me erasing everything at this point and taking off the systemd crew, since this will have no relevance to them... Testing the hook feature out using git rev (finally got it built)... I added this line to my config... lxc.mount.entry=tmpfs /srv/lxc/private/Plover/dev.tmp tmpfs defaults 0 0 lxc.hook.mount = /var/lib/lxc/Plover/mount In /var/lib/lxc/Plover/mount I have this: -- rsync -avAH /srv/lxc/private/Plover/dev.template/. /srv/lxc/private/Plover/dev.tmp/ -- (This is just testing out the concepts. If I understand this correctly, lxc.hook.pre-mount runs BEFORE the mounting takes place and lxc.hook.mount takes place after the mount. Problem is, the result of that rsync is not showing up in the mounted tmpfs file system but is showing up in the underlying parent file system as if it were run pre-mount. Something not right here... I changed it to lxc.hook.start = /srv/lxc/mount (where I put the script in the container) which then works but that then requires the template and the command to be in the container. Suboptimal to say the least. But it gives me a way to test this tmpfs thing out. I also noticed that the .start hook runs, it appears, after caps are dropped and I see a lot of bitching about mknod on certain devices. I had to thrown an exit 0 into that script so it would continue in spite of the errors but, now, I can refine my template based on what it could create. Crap. I've got a catch-22 here... This is going to take some work. Hey, I've got a rather minimal patch (appended below) to add the support for mounting and populating a minimal /dev working. (A few hours were wasted due to my not knowing that upstart was going to issue mounted-dev even though /dev was mounted before upstart started - and the mounted-dev hook deletes and recreates all consoles. GAH) I am happy to report that, after manually editing my git head branch to patch in the failed hunks, I was able to build it and test it and my Fedora 17 systemd based container fired right up after adding the lxc.autodev = 1 parameter to the config file. Yeah I did run into one gotcha, but one I can live with. I had been bind mounting the private root file system to another directory and then using that as the rootfs like this: -- lxc.rootfs = /srv/lxc/rootfs lxc.mount.entry=/srv/lxc/private/Alcove /srv/lxc/rootfs none bind.shared 0 0 lxc.mount.entry=/home/shared /srv/lxc/private/Alcove/srv/shared none ro,bind 0 0 lxc.autodev = 1 -- This did not work and I got the startup error that it can not mount to /dev because it doesn't exist. I believe I can see why... You're doing the autodev populate prior to any of the mounts being performed, so that private root file system is not bound to the directory at that time. Drop that bind mount for the rootfs and this then worked like a charm: -- lxc.rootfs = /srv/lxc/private/Alcove lxc.mount.entry=/home/shared /srv/lxc/private/Alcove/srv/shared none ro,bind 0 0 lxc.autodev = 1 -- I think that rootfs directory bind was an effort to more fully match the OpenVZ behavior but also trying to deal with some of the read-only problems were where having in the past with shutdowns. If it won't work, it won't work and I won't miss it. I did see some errors setting up that dev... -- [root@forest mhw]# lxc-start -n Alcove lxc-start: No such file or directory - failed to mount '/dev/pts/59'-'/usr/lib64/lxc/rootfs/dev/tty1' lxc-start: No such file or directory - failed to mount '/dev/pts/60'-'/usr/lib64/lxc/rootfs/dev/tty2' lxc-start: No such file or directory - failed to mount '/dev/pts/61'-'/usr/lib64/lxc/rootfs/dev/tty3' lxc-start: No such file or directory - failed to mount '/dev/pts/62'-'/usr/lib64/lxc/rootfs/dev/tty4' lxc-start: No such file or directory - failed to mount '/dev/pts/63'-'/usr/lib64/lxc/rootfs/dev/tty5' lxc-start: No such file or directory - failed to mount '/dev/pts/64'-'/usr/lib64/lxc/rootfs/dev/tty6' systemd 44 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP; fedora) Welcome to Fedora 17 (Beefy Miracle)! -- Not sure what that's all about but, since systemd isn't going to start getty's on the tty? interfaces anyways, it probably doesn't make much difference. Regards, Mike Yes, we can create the /dev directory with tmpfs from a template. Problem is that /dev/pts does not exist at the time we need to mount the devpts on /dev/pts for the pty's so that hurls chunks and dies. We can't create the /dev/ directory contents prior to mounting in the pre-mount hook because we won't have tmpfs in place at the time. We have to get
Re: [Lxc-users] [lxc-devel] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Sun, 2012-10-28 at 23:02 +0100, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Sun, 2012-10-28 at 18:52 +0100, Serge Hallyn wrote: : I've got a rather minimal patch (appended below) to add the support for mounting and populating a minimal /dev working. (A few hours were wasted due to my not knowing that upstart was going to issue mounted-dev even though /dev was mounted before upstart started - and the mounted-dev hook deletes and recreates all consoles. GAH) I am happy to report that, after manually editing my git head branch to Sorry, it was against the ubuntu quantal package. I've been in the air without onboard wifi, so working with what i had at hand. Oh, I figured it was a package mismatch. Wasn't too terribly difficult to patch those hunks in and kick out a diff against git. patch in the failed hunks, I was able to build it and test it and my Fedora 17 systemd based container fired right up after adding the lxc.autodev = 1 parameter to the config file. Yeah I did run into one gotcha, but one I can live with. I had been bind mounting the private root file system to another directory and then using that as the rootfs like this: -- lxc.rootfs = /srv/lxc/rootfs lxc.mount.entry=/srv/lxc/private/Alcove /srv/lxc/rootfs none bind.shared 0 0 lxc.mount.entry=/home/shared /srv/lxc/private/Alcove/srv/shared none ro,bind 0 0 lxc.autodev = 1 -- This did not work and I got the startup error that it can not mount to /dev because it doesn't exist. Hm, yeah. If you do need to play a game like this, you might be best off using a pre-mount hook for that. Yeah, I don't think I need to play a game like this anymore. I'd have to go back through some old old E-Mails to see why I did that before. I seem to recall we were playing with all sorts of bind mount options for some PRIVATE thing or another. It may not be necessary at all any longer. IAC, it's minor to switch it back. I seem to recall switching back and forth using bind mounts several times back when that got done that way. I may play with the pre-mount hook just for giggles and see how that might work as well. Any idea why I was experiencing the problem with the mount hook when trying to populate /dev? I know it wouldn't have worked because of the /dev/pts mount but I have more heartburn in that it looks like it ran too early and the mount on /dev had not even taken place at that time. I believe I can see why... You're doing the autodev populate prior to any of the mounts being performed, so that private root file system is not bound to the directory at that time. Drop that bind mount for the rootfs and this then worked like a charm: -- lxc.rootfs = /srv/lxc/private/Alcove lxc.mount.entry=/home/shared /srv/lxc/private/Alcove/srv/shared none ro,bind 0 0 lxc.autodev = 1 -- I think that rootfs directory bind was an effort to more fully match the OpenVZ behavior but also trying to deal with some of the read-only problems were where having in the past with shutdowns. If it won't work, it won't work and I won't miss it. I did see some errors setting up that dev... -- [root@forest mhw]# lxc-start -n Alcove lxc-start: No such file or directory - failed to mount '/dev/pts/59'-'/usr/lib64/lxc/rootfs/dev/tty1' lxc-start: No such file or directory - failed to mount '/dev/pts/60'-'/usr/lib64/lxc/rootfs/dev/tty2' lxc-start: No such file or directory - failed to mount '/dev/pts/61'-'/usr/lib64/lxc/rootfs/dev/tty3' lxc-start: No such file or directory - failed to mount '/dev/pts/62'-'/usr/lib64/lxc/rootfs/dev/tty4' lxc-start: No such file or directory - failed to mount '/dev/pts/63'-'/usr/lib64/lxc/rootfs/dev/tty5' lxc-start: No such file or directory - failed to mount '/dev/pts/64'-'/usr/lib64/lxc/rootfs/dev/tty6' systemd 44 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP; fedora) Welcome to Fedora 17 (Beefy Miracle)! -- Not sure what that's all about but, since systemd isn't going to start getty's on the tty? interfaces anyways, it probably doesn't make much difference. Oh, I see. Yeah, in the !lxc.ttydir case, when we created our own /dev we should create the tty files. I need to fix that. Cool. Once again... Looks like we got some real progress here with this one. I've still got more testing to do, undoing some of my changes in the container itself and making sure it all still works. Also looks like I can stop and restart one of these containers now without the hung cgroup directory. I suspected it was some stray devices behind that. This is good. Of course in your case since systemd isn't going to start getty's on them, you should not have the lxc.tty = 6 in your container config, which it looks like you still do? Yeah. I was taking it one step at a time. I wish they WOULD fire up
Re: [Lxc-users] [lxc-devel] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Sun, 2012-10-28 at 23:02 +0100, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): : I did see some errors setting up that dev... -- [root@forest mhw]# lxc-start -n Alcove lxc-start: No such file or directory - failed to mount '/dev/pts/59'-'/usr/lib64/lxc/rootfs/dev/tty1' lxc-start: No such file or directory - failed to mount '/dev/pts/60'-'/usr/lib64/lxc/rootfs/dev/tty2' lxc-start: No such file or directory - failed to mount '/dev/pts/61'-'/usr/lib64/lxc/rootfs/dev/tty3' lxc-start: No such file or directory - failed to mount '/dev/pts/62'-'/usr/lib64/lxc/rootfs/dev/tty4' lxc-start: No such file or directory - failed to mount '/dev/pts/63'-'/usr/lib64/lxc/rootfs/dev/tty5' lxc-start: No such file or directory - failed to mount '/dev/pts/64'-'/usr/lib64/lxc/rootfs/dev/tty6' systemd 44 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP; fedora) Welcome to Fedora 17 (Beefy Miracle)! -- Not sure what that's all about but, since systemd isn't going to start getty's on the tty? interfaces anyways, it probably doesn't make much difference. Oh, I see. Yeah, in the !lxc.ttydir case, when we created our own /dev we should create the tty files. I need to fix that. Well... I'm not sure I understand what you mean by that. The /dev/pts/* entries do does exist in the host system. But the bind mount fails saying they do not exist. Not sure I understand what the problem is here but I would like them connected even if systemd doesn't start getty's on them. I've used them for other purposes in the past. Of course in your case since systemd isn't going to start getty's on them, you should not have the lxc.tty = 6 in your container config, which it looks like you still do? Actually, I've decided this is worthy of debugging and there may be other ways to start a getty (or something else) on that tty. It really should work. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Tue, 2012-10-30 at 19:35 +0100, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Sun, 2012-10-28 at 23:02 +0100, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): : I did see some errors setting up that dev... -- [root@forest mhw]# lxc-start -n Alcove lxc-start: No such file or directory - failed to mount '/dev/pts/59'-'/usr/lib64/lxc/rootfs/dev/tty1' lxc-start: No such file or directory - failed to mount '/dev/pts/60'-'/usr/lib64/lxc/rootfs/dev/tty2' lxc-start: No such file or directory - failed to mount '/dev/pts/61'-'/usr/lib64/lxc/rootfs/dev/tty3' lxc-start: No such file or directory - failed to mount '/dev/pts/62'-'/usr/lib64/lxc/rootfs/dev/tty4' lxc-start: No such file or directory - failed to mount '/dev/pts/63'-'/usr/lib64/lxc/rootfs/dev/tty5' lxc-start: No such file or directory - failed to mount '/dev/pts/64'-'/usr/lib64/lxc/rootfs/dev/tty6' systemd 44 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP; fedora) Welcome to Fedora 17 (Beefy Miracle)! -- Not sure what that's all about but, since systemd isn't going to start getty's on the tty? interfaces anyways, it probably doesn't make much difference. Oh, I see. Yeah, in the !lxc.ttydir case, when we created our own /dev we should create the tty files. I need to fix that. Well... I'm not sure I understand what you mean by that. The /dev/pts/* entries do does exist in the host system. But the bind mount fails saying they do not exist. Not sure I understand what the In Ubuntu, we use lxc.ttydir = lxc. That means the actual /dev/ttyN in the container is a symlink to /dev/lxc/ttyN (to allow package upgrades which insist on removing /dev/ttyN to succeed - they will fail if /dev/ttyN is mounted over). When /dev was pre-populated, /dev/ttyN existed. But when we are populating it, it does not. So before we try tomount /dev/pts/NN from the host onto /dev/ttyN in the container, we have to create a file to bind mount over. I didn't put that in the patch. Yet. Got it! You and I both did the same thing. I'm thinking from the view of Fedora and you're thinking from the view of Ubuntu. Got it. We'll get this right yet. :-)=) problem is here but I would like them connected even if systemd doesn't start getty's on them. I've used them for other purposes in the past. Of course in your case since systemd isn't going to start getty's on them, you should not have the lxc.tty = 6 in your container config, which it looks like you still do? Actually, I've decided this is worthy of debugging and there may be other ways to start a getty (or something else) on that tty. It really should work. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Mon, 2012-10-29 at 10:18 +0100, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): ... Yeah, I don't think I need to play a game like this anymore. I'd have to go back through some old old E-Mails to see why I did that before. I seem to recall we were playing with all sorts of bind mount options for some PRIVATE thing or another. It may not be necessary at all any longer. IAC, it's minor to switch it back. I seem to recall switching back and forth using bind mounts several times back when that got done that way. I may play with the pre-mount hook just for giggles and see how that might work as well. Any idea why I was experiencing the problem with the mount hook when trying to populate /dev? I know it wouldn't have The only idea I have is that perhaps your root is MS_SHARED by default? Can you post the script you were using and the container config? Another point on the curve... The documentation says that pre-mount takes place before the mount occurs and mount takes place after the mount occurs. Only problem is that pre-mount is not being recognized. lxc-start 1351627853.032 ERRORlxc_confile - unknown key lxc.hook.pre-mount This is the same binaries from git that recognize lxc.hook.mount so I'm assuming the doco and the code don't match at this point. Even without my original bind mount, if I have a mount hook that does something in a newly mounted tmpfs directory, it doesn't show up in that directory it shows up in the parent directory as if it ran before the mounts took place. I could put the mount in the hook.mount script and then do it but it's seriously acting like the pre-mount hook isn't even there (parameter unknown) and the mount hook is running before the mounts are complete. Simple exerts from some test scripts doing, really, nothing but testing sequencing... Ok... Lets try this. I won't post entire configs but... For machine Alcove (my testbed)... -- lxc.tty = 6 lxc.pts = 64 lxc.rootfs = /srv/lxc/private/Alcove lxc.mount.entry=none/srv/lxc/private/Alcove/dev.tmp tmpfs defaults 0 0 lxc.mount.entry=/home/shared /srv/lxc/private/Alcove/srv/shared none ro,bind 0 0 lxc.autodev = 1 # lxc.hook.pre-mount = /var/lib/lxc/Alcove/pre-mount lxc.hook.mount = /var/lib/lxc/Alcove/mount lxc.mount.entry=shmfs /var/lib/lxc/private/Alcove/dev/shm tmpfs mode=0644 0 0 -- Now /var/lib/lxc/Alcove/mount: -- #!/bin/sh - touch /srv/lxc/private/Alcove/dev.tmp/mounted -- In that directory on the host fs I have this: [root@forest mhw]# touch /srv/lxc/private/Alcove/dev.tmp/no-mounted [root@forest mhw]# ls /srv/lxc/private/Alcove/dev.tmp/ no-mounted Now, when I start the container, the tmpfs should get mounted on /dev.tmp in the container (relative to the container rootfs) and should have the single file mounted in it while the parent file system back on the host should have the single file not-mounted in it. Let's see... lxc-start -n Alcove... In the container... [mhw@alcove ~]$ mount | grep tmpfs none on /dev type tmpfs (rw,relatime,seclabel,size=100k) none on /dev.tmp type tmpfs (rw,relatime,seclabel) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel) tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755) tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,seclabel,mode=755) tmpfs on /media type tmpfs (rw,nosuid,nodev,noexec,relatime,seclabel,mode=755) Looks like the mount took place. I have tmpfs on /dev.tmp [mhw@alcove ~]$ ls /dev.tmp/ [mhw@alcove ~]$ Opps... Where did the file end up? Let's look on the host... [mhw@forest ~]$ ls /srv/lxc/private/Alcove/dev.tmp/ mounted no-mounted Arg... Wrong answer. It ended up in the parent file system before tmpfs was mounted. But, the documentation says hook.mount runs after the mounts have completed. There's something wrong here or I am badly mistaken in my understanding... (Probably the later, I'll admit.) Regards, Mike worked because of the /dev/pts mount but I have more heartburn in that it looks like it ran too early and the mount on /dev had not even taken place at that time. I believe I can see why... You're doing the autodev populate prior to any of the mounts being performed, so that private root file system is not bound to the directory at that time. Drop that bind mount for the rootfs and this then worked like a charm: -- lxc.rootfs = /srv/lxc/private/Alcove lxc.mount.entry=/home/shared /srv/lxc/private/Alcove/srv/shared none ro,bind 0 0 lxc.autodev = 1 -- I think that rootfs directory bind was an effort to more fully match the OpenVZ behavior but also trying to deal with some of the read-only problems were where having in the past with shutdowns. If it won't work, it won't work and I won't miss it. I did see some errors setting up that dev... -- [root@forest mhw]# lxc-start -n Alcove lxc-start: No such file or directory
Re: [Lxc-users] [lxc-devel] [GIT] lxc branch, master, updated. 7f99e339363d9f005c9386f60a1d8c0953c85053
f1ccde27c038e7fb7e538913505248b36ddd9e65 Author: Serge Hallyn serge.ha...@ubuntu.com Date: Tue Aug 21 09:56:03 2012 -0500 ubuntu and debian templates: Clean up cache if cache build is interrupted Otherwise the next lxc-create may rsync a bad cache. Signed-off-by: Serge Hallyn serge.hal...@ubuntu.com commit 4a311c1241805dac5893918854fd40f77b2b6f49 Author: Serge Hallyn serge.hal...@ubuntu.com Date: Thu Aug 16 21:11:50 2012 -0500 Cleanup partial container if -h was passed to template If user calls 'lxc-create -t ubuntu -- -h' (as opposed to 'lxc-create -t ubuntu -h') then the ubuntu template will print its help then exit 0. Then lxc-create does not cleanup. So detect this in lxc-create. commit 4d5fb23ad827eda17b64676f527c3f168cd56ebd Author: Serge Hallyn serge@amd1.(none) Date: Fri Jul 20 10:38:15 2012 -0500 lxc-clone: fix handling of lxc.mount entries The 'lxc.mount =' entry can have more than one space, or tabs, before the =. We only need to disambiguate from 'lxc.mount.entry'. So just check for a space or tab after mount. Signed-off-by: Serge Hallyn serge.hal...@ubuntu.com commit 8b892c55b077d1716eb130e76f9c9725ecb0f73a Author: Serge Hallyn serge.hal...@ubuntu.com Date: Thu Jul 19 17:54:54 2012 -0500 lxc-clone: change uuid on xfs Otherwise after cloning an lvm+xfs container you can't run the original and clone at the same time. Based on a patch by Maurizio Sambati posted at https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1013549 Signed-off-by: Serge Hallyn serge.hal...@ubuntu.com --- Summary of changes: Makefile.am |2 +- configure.ac |8 +- doc/lxc-attach.sgml.in| 11 ++- doc/lxc-cgroup.sgml.in|7 +- doc/lxc-checkpoint.sgml.in|8 +- doc/lxc-console.sgml.in |8 +- doc/lxc-create.sgml.in| 12 ++-- doc/lxc-destroy.sgml.in |7 +- doc/lxc-execute.sgml.in | 10 ++-- doc/lxc-freeze.sgml.in|4 +- doc/lxc-kill.sgml.in |4 +- doc/lxc-ls.sgml.in|5 +- doc/lxc-monitor.sgml.in |4 +- doc/lxc-ps.sgml.in| 14 ++-- doc/lxc-restart.sgml.in | 11 ++-- doc/lxc-shutdown.sgml.in |7 +- doc/lxc-start.sgml.in | 16 +++-- doc/lxc-stop.sgml.in |4 +- doc/lxc-unfreeze.sgml.in |4 +- doc/lxc-wait.sgml.in |6 +- lxc.spec.in | 16 +++-- src/lxc/cgroup.c |9 ++- src/lxc/conf.c| 51 +- src/lxc/conf.h|6 -- src/lxc/lxc-clone.in | 34 ++ src/lxc/lxc-create.in | 14 - src/lxc/lxc-destroy.in| 17 -- src/lxc/lxc-ls.in |5 +- src/lxc/lxc-ps.in |4 +- src/lxc/lxc-setcap.in |3 - src/lxc/lxc-setuid.in |3 - src/lxc/lxc_start.c | 16 - src/lxc/namespace.h |4 + src/lxc/start.c | 15 +++- templates/lxc-altlinux.in |4 + templates/lxc-archlinux.in|4 + templates/lxc-busybox.in | 30 - templates/lxc-debian.in | 19 +- templates/lxc-fedora.in | 55 +++- templates/lxc-lenny.in|6 ++- templates/lxc-opensuse.in |3 + templates/lxc-sshd.in | 48 -- templates/lxc-ubuntu-cloud.in | 144 - templates/lxc-ubuntu.in | 43 +--- 44 files changed, 465 insertions(+), 240 deletions(-) hooks/post-receive -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ Lxc-devel mailing list lxc-de...@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key
Re: [Lxc-users] [lxc-devel] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Wed, 2012-10-31 at 18:15 +0100, Serge Hallyn wrote: Can you tell me the exact git tree and branch you are using? I'm using head. I'm not specifying a tree. The results you're getting don't make sense to me... Hoping I can find a simple answer. Me too. -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] [systemd-devel] Unable to run systemd in an LXC / cgroup container.
On Wed, 2012-10-31 at 18:30 +0100, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Wed, 2012-10-31 at 18:15 +0100, Serge Hallyn wrote: Can you tell me the exact git tree and branch you are using? I'm using head. I'm not specifying a tree. ? I'm not sure what you mean - are you using git://github.com/lxc/lxc, or the tree on lxc.sf.net? IOW, I'm not using a branch in the tree. I'm using the main trunk. Created my tree with - git clone git://github.com/lxc/lxc So the former, not the later. The results you're getting don't make sense to me... Hoping I can find a simple answer. Me too. -serge Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] [GIT] lxc branch, master, updated. 7f99e339363d9f005c9386f60a1d8c0953c85053
On Thu, 2012-11-01 at 21:20 +0100, Daniel Baumann wrote: On 11/01/2012 09:08 PM, Michael H. Warfield wrote: I know, I KNOW this is an 11th hour request. Can we please get Serge's autodev stuff into this release? Please? release early, release often? just release current git as 0.8.0 now, and the one with the autofoo as 0.8.1 soon after that? That would be ideal but we've been sitting at 0.8.0rc2 for something like 3-1/2 months now. I know Daniel (the other Daniel, the Daniel) has been incredibly busy. I have no objection to getting this out the door as 0.8.0 with a fast bump to 0.8.1 for the systemd stuff, but another several months is not good. Can we get this fast bump? We'll be staring Fedora 18 in the face by then. The working versions of Fedora are no longer in support and we've got more distros adopting systemd. -- Address:Daniel Baumann, Donnerbuehlweg 3, CH-3012 Bern Email: daniel.baum...@progress-technologies.net Internet: http://people.progress-technologies.net/~daniel.baumann/ Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [lxc-devel] [GIT] lxc branch, master, updated. 7f99e339363d9f005c9386f60a1d8c0953c85053
On Thu, 2012-11-01 at 22:44 +0100, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Thu, 2012-11-01 at 21:20 +0100, Daniel Baumann wrote: On 11/01/2012 09:08 PM, Michael H. Warfield wrote: I know, I KNOW this is an 11th hour request. Can we please get Serge's autodev stuff into this release? Please? release early, release often? just release current git as 0.8.0 now, and the one with the autofoo as 0.8.1 soon after that? That would be ideal but we've been sitting at 0.8.0rc2 for something like 3-1/2 months now. I know Daniel (the other Daniel, the Daniel) has been incredibly busy. I have no objection to getting this out the door as 0.8.0 with a fast bump to 0.8.1 for the systemd stuff, but another several months is not good. Can we get this fast bump? We'll be staring Fedora 18 in the face by then. The working versions of Fedora are no longer in support and we've got more distros adopting systemd. I think this will end up slated for 0.9.0 (which we're hoping will be soon), but in any case I went ahead and created a branch at git://github.com/hallyn/lxc called upstream.nov1.2012.autodev, with an autodev patch on top of Daniel's latest push. I quickly tried my hand at fixing the error you had with /dev/ttyN. I haven't tested that bit. I will not be able to be online at all from now until weekend or monday, so if it needs more tweaks please feel free to 'just fix it'. Problem. Works for the systemd containers but not for my older containers. I get this... [root@forest Plover]# cat 2012-10-30-18:17:46.log lxc-start 1351635466.998 ERRORlxc_conf - Operation not permitted - error 1 creating /usr/lib64/lxc/rootfs/dev/tty6 lxc-start 1351635466.999 ERRORlxc_conf - failed to setup the ttys for 'Plover' lxc-start 1351635466.999 ERRORlxc_start - failed to setup the container lxc-start 1351635466.999 ERRORlxc_sync - invalid sequence number 1. expected 2 lxc-start 1351635466.999 ERRORlxc_start - failed to spawn 'Plover' Alcove (the systemd container) was the first one started so it may be an ordinal thing or it may be a systemd thing. But it's a problem. (there are also some todos in the commit msg - if we're going to wait for 0.9.0 then I can handle those later, and port the patch on top of the 100 additional patches queued in github.com/lxc/lxc#staging) -serge Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- LogMeIn Central: Instant, anywhere, Remote PC access and management. Stay in control, update software, and manage PCs from one command center Diagnose problems and improve visibility into emerging IT issues Automate, monitor and manage. Do more in less time with Central http://p.sf.net/sfu/logmein12331_d2d___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
