Re: [Lxc-users] LXC a feature complete replacement of OpenVZ?
Hi, At first, thanks for all the great feedback and the quickly ongoing development to lxc. On May 13, 2010, at 11:22 PM, Daniel Lezcano wrote: On 05/13/2010 06:17 PM, Christian Haintz wrote: 6) is LXC production ready? yes and no :) If you plan to run several webserver (not a full system) or non-root applications, then yes IMHO it is ready for production. If you plan to run a full system and you have very aggressive users inside with root privilege then it may not be ready yet. If you setup a full system and you plan to have only the administrator of the host to be the administrator of the containers, and the users inside the container are never root, then IMHO it ready if you accept for example to have the iptables logs to go to the host system. In my opinion there is not a big different if i run some software which might have a security bug which people could exploit or if i have a root user who trys to escape the container. In both ways i need isolation which i can trust. For me this is the main reason doing things in isolation like lxc or openvz, because i don't need the overhead of kvm or xen but i still need isolation which jail a software or a system - root users inside or not. It looks to me like you already know a way how to escape from a container, don't you? And if so, is that a desired behavior or just a bug? The point i'd like to come: Is one goal of lxc to make it a container where nothing/nobody can escape or is this feature just nice-to-have but not a must have on the roadmap? Really, it depends of what you want to do ... I don't know OpenVZ very well, but AFAIK it is focused on system container while LXC can setup different level of isolation allowing to run an application sharing a filesystem or a network for example, as well as running a full system. But this flexibility is a drawback too because the administrator of the container needs a bit of knowledge on the system administration and the container technology. For me, all aspects of lxc are interesting, I am not only focused to full system virtualization. I am also thinking of jailing just some apps with some libs in containers (e.g. python). But in the end, for me it is about encapsulation with no escape :-) Regards, Christian -- Christian Haintz Student of Software Development and Business Management Graz, University of Technology -- ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] LXC a feature complete replacement of OpenVZ?
3) Is there support for tun/tap device The drivers are ready to be used in the container but not sysfs and that unfortunately prevent to create a tun/tap in a container. sysfs per namespace is on the way to be merged upstream. The workaround here is to create a TUN/TAP on the host and install it into the container using something like 'ip link set tap0 netns '. The app running in the container may not complain (from my experience) and can open and use the tap0. Another option (which cannot be done with OpenVZ) is creating a TUN/TAP on the host, opening the socket end from a host process, then pushing it into the namespace container. Once in the namespace you can rename it to anything you'd like. Processes in the namespace may read/write as a normal device, and you get all of the packets in the host system. This is useful for connecting together containers with simulators such as ns-3. -Jeff -- ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
[Lxc-users] LXC a feature complete replacement of OpenVZ?
Hi, At first LXC seams to be a great work from what we have read already. There are still a few open questions for us (we are currently running dozens of OpenVZ Hardwarenodes). 1) OpenVZ in the long-term seams to be a dead end. Will LXC be a feature complete replacement for OpenVZ in the 1.0 Version? As of the current version 2) is there IPTable support, any sort of control like the OpenVZ IPTable config. 3) Is there support for tun/tap device 4) is there support for correct memory info and disk space info (are df and top are showing the container ressources or the resources of the hardwarenode) 5) is there something compared to the fine grained controll about memory resources like vmguarpages/privmpages/oomguarpages in LXC? 6) is LXC production ready? Thanks in Advance, and we are looking forward to switch to Linux Containers when all Questions are answered with yes :-) Regards, Christian -- Christian Haintz Student of Software Development and Business Management Graz, University of Technology -- ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] LXC a feature complete replacement of OpenVZ?
On Thu, 13 May 2010, Christian Haintz wrote: Hi, At first LXC seams to be a great work from what we have read already. There are still a few open questions for us (we are currently running dozens of OpenVZ Hardwarenodes). I can't answer for the developers, but here's my answers/observations based on what I've seen and used ... 1) OpenVZ in the long-term seams to be a dead end. Will LXC be a feature complete replacement for OpenVZ in the 1.0 Version? I looked at OpenVZ and while it looked promising, didn't seem to be going anywhere. I also struggled to get their patches into a recent kernel and it looked like there was no Debian support for it. LXC was in the kernel as standard - I doubt it'll come out now... (and there is a back-ported lxc debian package that works fine under Lenny) As of the current version 2) is there IPTable support, any sort of control like the OpenVZ IPTable config. I run iptables - and in some cases different iptable setups in each container on a host (which also has it's own iptables). Seems to just work. Each container has an eth0 and the host has a br0 (as well as an eth0). Logging is at the kernel level though, so goes into the log-files on the server host rather than in the container - it may be possible to isolate that, but it's not something I'm too bothered with. My iptables are just shell-scripts that get called as part of the boot sequence - I really don't know what sort of control OpenVZ gives you. 3) Is there support for tun/tap device Doesn't look like it yet... http://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg00239.html 4) is there support for correct memory info and disk space info (are df and top are showing the container ressources or the resources of the hardwarenode) Something I'm looking at myself - top gives your own processes, but cpu usage is for the whole machine. 'df' I can get by manipulating /etc/mtab - then I get the size of the entire partition my host is running under. I'm not doing anything 'clever' like creating a file and loopback mounting it - all my containers in a host are currently on the same partition. I'm not looking to give fixed-size disks to each container though. YMMV. However gathering cpu stats for each container is something I am interested in - and was about to post to the list about it - I think there are files (on the host) under /cgroup/container-name/cpuacct.stat and a few others which might help me though, but I'm going to have to look them up... 5) is there something compared to the fine grained controll about memory resources like vmguarpages/privmpages/oomguarpages in LXC? Pass.. 6) is LXC production ready? Not sure who could make that definitive decision ;-) It sounds like the lack of tun/tap might be a show-stopper for you though. (come back next week ;-) However, I'm using it in production - got a dozen LAMPy type boxes running it so-far with several containers inside, and a small number of asterisk hosts. (I'm not mixing the LAMP and asterisk hosts though) My clients haven't noticed any changes which makes me happy. I don't think what I'm doing is very stressful to the systems though, but so-far I'm very happy with it. I did test it to my own satisfaction before I committed myself to it on servers 300 miles away though. One test was to create 20 containers on an old 1.8GHz celeron box, each running asterisk with one connected to the next and so on - then place a call into the first. It manged 3 loops playing media before it had any problems - and those were due to kernel context/network switching rather than anything to do with the LXC setup. (I suspect there is more network overhead though due to the bridge and vlan nature of the underlying plumbing) So right now, I'm happy with LXC - I've no need for other virtualisation as I'm purely running Linux, so don't need to host Win, different kernels, etc. And for me, it's a management tool - I can now take a container and move it to different hardware (not yet a proper live migration, but the final rsync is currently only a few minutes and I can live with that) I have also saved myself a headache or two by moving old servers with OS's I couldn't upgrade into new hardware - so I have one server running Debian Lenny, kernel 2.6.33.1 hosting an old Debian Woody server inside a container running the customers custom application which they developed 6 years ago... They're happy as they got new hardware and I'm happy as I didn't have to worry about migrating their code to a new version of Debian on new hardware... And I can also take that entire image now and move it to another server if I needed to load-balance, upgrade, cater for h/w failure, etc. I'm using kernel 2.6.33.x (which I custom compile for the server hardware) and Debian Lenny FWIW. I'm trying to not sound like a complete fanboi, but until the start of this year, I had no interest in virtualisation at all, but once
Re: [Lxc-users] LXC a feature complete replacement of OpenVZ?
On 05/13/2010 06:17 PM, Christian Haintz wrote: Hi, At first LXC seams to be a great work from what we have read already. There are still a few open questions for us (we are currently running dozens of OpenVZ Hardwarenodes). 1) OpenVZ in the long-term seams to be a dead end. Will LXC be a feature complete replacement for OpenVZ in the 1.0 Version? Theorically speaking, LXC is not planned to be a replacement to OpenVZ. When a specific functionality is missing, it is added. Sometimes that needs a kernel development implying an attempt to mainline inclusion. When the users of LXC want a new functionality, they send a patchset or ask if it possible to implement it. Often, the modifications need a kernel modification at that takes sometime to reach the upstream kernel (eg. sysfs per namespace). Practically speaking, LXC evolves following the needs (eg. entering a container) of the users and that may lead to a replacement of OpenVZ. The version 1.0 is planned to be a stable version, with documentation and frozen API. As of the current version 2) is there IPTable support, any sort of control like the OpenVZ IPTable config. The iptables support in the container is depending on the kernel version you are using. AFAICS, iptables per namespace is implemented now. 3) Is there support for tun/tap device The drivers are ready to be used in the container but not sysfs and that unfortunately prevent to create a tun/tap in a container. sysfs per namespace is on the way to be merged upstream. 4) is there support for correct memory info and disk space info (are df and top are showing the container ressources or the resources of the hardwarenode) No and that will not be supported by the kernel but it is possible to do that with fuse. I did a prototype here: http://lxc.sourceforge.net/download/procfs/procfs.tar.gz But I gave up with it because I have too much things to do with lxc and not enough free time. Anyone is welcome to improve it ;) 5) is there something compared to the fine grained controll about memory resources like vmguarpages/privmpages/oomguarpages in LXC? I don't know these controls you are talking about but LXC is plugged with the cgroups. One of the subsystem of the cgroup is the memory controller allowing to assign an amount of physical memory and swap space to the container. There are some mechanism for notification as well. There are some other resource controller like io (new), freezer, cpuset, net_cls and device whitelist (googling one of these name + lwn may help). 6) is LXC production ready? yes and no :) If you plan to run several webserver (not a full system) or non-root applications, then yes IMHO it is ready for production. If you plan to run a full system and you have very aggressive users inside with root privilege then it may not be ready yet. If you setup a full system and you plan to have only the administrator of the host to be the administrator of the containers, and the users inside the container are never root, then IMHO it ready if you accept for example to have the iptables logs to go to the host system. Really, it depends of what you want to do ... I don't know OpenVZ very well, but AFAIK it is focused on system container while LXC can setup different level of isolation allowing to run an application sharing a filesystem or a network for example, as well as running a full system. But this flexibility is a drawback too because the administrator of the container needs a bit of knowledge on the system administration and the container technology. Thanks in Advance, and we are looking forward to switch to Linux Containers when all Questions are answered with yes :-) Hope that helped. Thanks -- Daniel -- ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
