,----[ Initial Thought/Message ] | God I wish we had a .32 OpenVZ kernel then this discussion wouldn't | even take place ... I appreciate all the excellent work done by all | OpenVZ folks! Kir, you rock! Well, here it is: `----
Hello folks! We have evaluated the situation once again and made the decision going forward with LXC. Yes, unfortunately that means ditching OpenVZ. This is nothing personal (although I am a bit sad since I have invested a lot of time) but purely logical. We want/need investment security therefore stuff needs to be in mainline. There is no argument against that, this is 2010 and not 1995 anymore i.e. out of tree is of no interest from a business point of view anymore. The next stable releases for Debian and Ubuntu are to be scheduled for March/April. At this moment no one knows if a new OpenVZ kernel will be available by then. This, we cannot have. We were looking at KVM and LXC. Linux-VServer and OpenVZ are not in mainline so they are not considered. KVM is to "fat" for what we want plus as it looks like after a few tests, switching from OpenVZ to LXC is quite feasible and the few LXC test systems we created during the last few days run smoothly on .32. As with OpenVZ, our hosts as well as containers will run Debian. There is quite good support for Debian already e.g. /usr/share/doc/lxc/examples/lxc-debian.gz for example. Michael> I use to use Linux-vserver years and years ago but when they Michael> broke IPv6 support moving from 1.x to 2.x I was forced to Michael> abandon Linux-vserver and switch a number of VM's over to Michael> OpenVZ. To this day IPv6 remains an "experimental patch" for Michael> Linux-vserver and I see that question come up on their list Michael> periodically, so I couldn't migrate back there, even if I Michael> wanted to. That being said, IPv6 support in the OpenVZ vnet Michael> device is nothing to brag about either and I have had to Michael> strictly use the veth devices. Before OpenVZ we/I used Linux-VServer too. It is excellent I think but then here is the problem: LVS is basically a one-man show by Herbert Poetzl. He's a great guy and I meet him a few times in Vienna (were I live too). What happens if Herbert is run over by a train (which of course hopefully does not happen but you get the idea)?! This is a problem, so we switched to OpenVZ. Michael> However... There is a new kid on the block, depending on your Michael> requirements. Linux Containers or LXC. It still has a few rough Michael> edges and some differences with OpenVZ but has the big Michael> advantage that it's all in the mainline kernel (2.6.29 and Michael> above), so no more patches (yeah!), it is supported under Michael> libvirt, and the utilities are in the major cutting edge Michael> distros like Fedora and Ubuntu. Michael, you are nothing but right here. Stuff must be in mainline, I cannot get tired of saying that enough these days. The energy spend sketching possible scenarios about what we are going to do if and when will resolve immediately once we use LXC. You just know what will be the case in X months for now ... that is an irreplaceable peace of mind. That is true for any Distros out there, host or container ... Michael> I found that with a couple of scripts, I could directly convert Michael> OpenVZ config files to LXC config files and start my old OpenVZ Michael> containers as a container under LXC with no further Michael> modification inside the container. Please provide your scripts to the public. I would love to see them, help improve things and maybe others will join in so nobody needs to be alone by switching to LXC. Dietmar, since we are both interested on making this work for Debian plus, we are in Austria, maybe we should work on this together a bit? Maybe even have a sprint? My email is suno.ano[at]sunoano.org just in case ... Here is what I found so far http://sysadmin-cookbook.rot13.org/#lxc , go down to ve2lxc. I have already started a very rough/ugly collection of bits and pieces of information for my personal matters which can be found at http://sunoano.name/ws/public_xhtml/linux_containers.html Michael> Other than a couple of initial test containers I was Michael> experimenting with, once I got my scripts settled down and Michael> tested, I migrated over 3 dozen VM's from OpenVZ to LXC in a Michael> single day with none of the containers experiencing more that a Michael> minute or so of down time (transfer time between hosts). Michael> Because there were no changes in the containers themselves, I Michael> could migrate them back, if I needed to, just as fast. I want this! Tell us more please. Details sir ;-) Michael> 1) /proc/mounts shows mounts outside of the container (ugly but Michael> not fatal). Fixed in git. Is this true for kernels >= .32 ? Michael> 2) Possible to break out of a container file system (related to #1 Michael> above). It's possible to break out of chrooted jails. Fixed in Michael> git by using pivot root. This is serious and if you have Michael> potential hostiles in a container, I wouldn't use LXC yet or Michael> use the utilities from git. Also, is this true for kernels >= .32 ? Michael> 3) Halt and Reboot of a container not working. You have to Michael> manually shut down and restart the container from the host. Michael> Being worked on right now. I use a script that detects when Michael> there's only one process running (init) in the container and Michael> the container runlevel is 0 or 6 to decide to shut it down Michael> or restart it. Ugly but works. Can you please provide the scipt/resolution you are using. This is still true for >= .32 yes? Hm, my containers started automatically when rebooting the host. I am on .32, Debian standard kernel in unstable: ,----[ uname -a ] | Linux wks 2.6.32-trunk-amd64 #1 SMP Sun Jan 10 22:40:40 UTC 2010 x86_64 GNU/Linux `---- Michael> 4) There still seems to be a lot of development work going on Michael> in the kernel wrt checkpoint and restore. Since I don't use Michael> that much, I haven't paid that much attention but LXC does Michael> support freezing and unfreezing containers. True. I can live without that for a while i.e. it is no problem if I do not have that feature available for the next half year or so. Michael> * LXC supports virtual consoles you can connect to and log into Michael> (lxc-console). Michael> * LXC does not (yet) support the equivalent of "vzctl enter" (under Michael> discussion - some possible patches). Personally I do not care since I setup sshd with each container. It sure is a nice to have thingy though. http://sunoano.name/ws/public_xhtml/ssh.html Michael> * Does not have the same level of fine grained resource control Michael> available with OpenVZ (something that is not a requirement for Michael> me) and the user_beancounters (some controls in the cgroups Michael> resources but not as many). I second that but then I think it is just a matter of time until resource control will improve to the current state of OpenVZ. Michael> * Handles the bridge management for the eth interfaces Michael> automatically, so no need for extra config files in the host. I hate bridges therefore I use lxc.network.type = macvlan which is the equivalent to OpenVZ's venet device ... basically a pipe-like connection between container and host. No bridge involved. Imho a bride just complicates a setup and introduces an additional layer of indirection. Michael> Primary disadvantage to LXC is that the utlities are at 0.6.4 Michael> from source forge and 0.6.3 from Fedora and really really under Michael> active development and change. Features and facilities are Michael> still subject to discussion and change and it's not fully Michael> mature on that level. I don't know how long that will take but Michael> I wouldn't use anything less that what's in their git repo Michael> right now or 0.6.5 or higher, when it comes out. I had no problems on Debian testing/sid; stuff is quite recent here ,----[ dpkg -l lxc* | grep ii ] | ii lxc 0.6.4-2 Linux containers userspace tools `---- Michael> Sooo... If you WANT to run a newer leading edge distro like Michael> Ubuntu or Fedora for your host system and you can deal with Michael> those differences, LXC MIGHT be an option. It is not just an option from my point of view, it is one of two ways (lxc, kvm) to go about the future of virtualization. Only things that ship with mainline are worth being considered. Michael> Exactly. Which is why the burning need to get to a mainstream Michael> kernel. Containers, namespaces, and cgroups in the kernel have Michael> matured to the point where they are eminently usable. I agree. It looks good to me, even after some stress tests on the network layer I did everything continued to fine. Michael> I can't speak for the developers here but, I would not be Michael> surprised if this were a real reason for some of the lack of Michael> recent progress on newer kernels. Why invest the effort at all Michael> if you are going to be able to take advantage of mainline Michael> features in the near future? Skip the transition period and get Michael> ready for the big jump. Better to organize and prepare for when Michael> it reaches that level of maturity. I would like to see OpenVZ Michael> running on a recent linux kernel just using the whole cgroup Michael> and namespaces facility, even if not all of the granular Michael> user_beancounters are fully supported (and may never be fully Michael> supported to that degree of granularity). Yes, it is paradox. Kir and the rest of the core developers around OpenVZ have contributed a ton of good stuff to cgroup, namespaces and the networking part, all of which in mainline now, but then OpenVZ itself is horribly outdated (.27 where .32 is available). On Debian I am on .26 on most server around here and yesterday updating the host failed because the current udev version does not work with .26 but only newer kernels. It now really is the time to act, even if it is a bit melancholic because OpenVZ is great, great but to old :-/ Michael> If I had the maturity and stability of the OpenVZ utilities Michael> running on the mainline kernel using namespaces and cgroups and Michael> no custom patch, that would be my ideal combination right now. I totally agree even though my experience was that the utilities work fine. Mainline is mainline is mainline ... :-) _______________________________________________ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
