Re: [Users] Docker inside an OpenVZ container
Hello Vasily, thank you for the answer. I have expected all of the points you mention and let me assure you, that I fully understand all of them. These days there isn't a single project other than OpenVZ, for which I'm looking forward to release its latestgreatest - namely the RHEL7 based kernel. It shouldn't be too hard to backport these things I'm mentioning to RHEL7 kernel, RHEL6 might be a bit hard in this regard. I have two additional questions, I'm not sure whether you're bound by NDA so that you can/can't answer them, but it would mean a great deal to me if you could: - do you guys at Parallels have access to separated-out patches for RHEL kernels / to their git? Because if you don't, then you guys are real heros for doing your work on top of RHEL kernel. And if you do, that would mean that backporting the shrinker patches from 3.12 shouldn't be too hard - and I would like to help, if possible. - do you have an ETA for releasing any RHEL7 vz kernel preview? I would be interested in testing as soon as you have anything, even if it's the most unstable thing in the world. Thank you for your time! /snajpa ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users
Re: [Users] Docker inside an OpenVZ container
On 03/24/2015 02:37 PM, Pavel Snajdr wrote: Hello Vasily, thank you for the answer. I have expected all of the points you mention and let me assure you, that I fully understand all of them. These days there isn't a single project other than OpenVZ, for which I'm looking forward to release its latestgreatest - namely the RHEL7 based kernel. It shouldn't be too hard to backport these things I'm mentioning to RHEL7 kernel, RHEL6 might be a bit hard in this regard. I have two additional questions, I'm not sure whether you're bound by NDA so that you can/can't answer them, but it would mean a great deal to me if you could: - do you guys at Parallels have access to separated-out patches for RHEL kernels / to their git? Because if you don't, then you guys are real heros for doing your work on top of RHEL kernel. And if you do, that would mean that backporting the shrinker patches from 3.12 shouldn't be too hard - and I would like to help, if possible. - do you have an ETA for releasing any RHEL7 vz kernel preview? I would be interested in testing as soon as you have anything, even if it's the most unstable thing in the world. Oh and I see you have answered this one. However, I would like to ask in addition to your answer - is there any channel, through which I could take a look at that work and how it's evolving even if it's not usable? This is what I mean by OpenVZ project 'not being open', this is not how we do open-source in linux community :( /snajpa Thank you for your time! /snajpa ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users
Re: [Users] Docker inside an OpenVZ container
Yep, nice suggestion! An every switch between Red Hat kernel 2.6.32 milestones produce big amount of bugs in OpenVZ. But do not provide enough benefits for customers. And I'm sure number of bugs in case of switch between major upstream kernel versions (for example 3.10~3.11) will be same or even much times less because every new version bring new code from OpenVZ patch to upstream. From my point of view, an ideal approach about kernel version used in CoreOS. They follow vanilla kernel and provide new features as soon as possible (overlayfs, vxvlan, dpdk, vswitch, syn proxy, batch routing, tcp/ip stack optimization, live kernel patching). But the work with really huge companies and with important data and know what is reliability. And they use vanilla kernel. Yep, security flaws is and issue but we use containers and 90% of kernel bugs is not affect us. For very important bugs kpatch could be used for live patching. Second nice idea from CoreOS project is consistent upgrade when we could automatically switch to next OpenVZ kernel version on next reboot (or kernel panic, it's more recent for OpenVZ). I'm really sure old approach we use old kernel and we assume it's stable become completely useless today. Because old kernel has bunch of problems (route cache? syn cookies? listen block on socket?) even in subsystems architecture and very important things like tcp routing subsystem. On Tue, Mar 24, 2015 at 2:57 PM, Narcis Garcia informat...@actiu.net wrote: A good strategy could be to make OpenVZ become fully as an LXC enhancement, and apply patches thinking in datacenter scenario for LXC. This focus could make easier to follow Linux kernel versions. OpenVZ for Linux 2.6.32 is excellent, but the time makes grow some matters that didn't seem a problem in 2010. El 24/03/15 a les 10:05, Pavel Odintsov ha escrit: Hello, folks! CentOS 6 with 2.6.32 kernel is real nightmare (even in case of networking) from my point of view. Simple syn flood could KILL my HWN's (I can share details off list). You (Parallels) and RH do big amount of backporting but upstream is far away in future. Difference even between 3.17 and 3.18 kernels is EXTREMELY HUGE. You could look at diffs here https://github.com/torvalds/linux/commits/master. And performance could be improved for 50-70% with this upgrade (I assume even more speed up with update from 2.6.32RH to 3.18). Red Hat interested only in few things about 2.6.32 kernel for running Java and KVM (with Oracle?). But storages (what about stable filesystem like ZFS instead bunch-of-crap-ext4 and it-will-be-non-stable-for-ever-btrfs?), networking (routing! routing! routing!) and many other extremely important things is out of focus. And I have innate desire to test new RHEL 7 kernel and switch my hundreds of servers to it :) But this kernel based on enough old 3.10 kernel will be obsolete since release (look at my comparison about 3.17 3.18 kernel). IMHO, best approach for OpenVZ will be follow the upstream because your patch have significantly reduced since 2.6.32. You are trying to keep up stability with following to Red Hat with old kernels but in reality OpenVZ kernel is not really stable (you could grep my bug reports at bugzilla.openvz.org and found hundreds of issues with stability) even with RH kernel. My instances with upstream kernel could work many months without any issues. Yep, it's enough stable and completely suitable for OpenVZ HWN's. ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users -- Sincerely yours, Pavel Odintsov ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users
Re: [Users] Docker inside an OpenVZ container
A good strategy could be to make OpenVZ become fully as an LXC enhancement, and apply patches thinking in datacenter scenario for LXC. This focus could make easier to follow Linux kernel versions. OpenVZ for Linux 2.6.32 is excellent, but the time makes grow some matters that didn't seem a problem in 2010. El 24/03/15 a les 10:05, Pavel Odintsov ha escrit: Hello, folks! CentOS 6 with 2.6.32 kernel is real nightmare (even in case of networking) from my point of view. Simple syn flood could KILL my HWN's (I can share details off list). You (Parallels) and RH do big amount of backporting but upstream is far away in future. Difference even between 3.17 and 3.18 kernels is EXTREMELY HUGE. You could look at diffs here https://github.com/torvalds/linux/commits/master. And performance could be improved for 50-70% with this upgrade (I assume even more speed up with update from 2.6.32RH to 3.18). Red Hat interested only in few things about 2.6.32 kernel for running Java and KVM (with Oracle?). But storages (what about stable filesystem like ZFS instead bunch-of-crap-ext4 and it-will-be-non-stable-for-ever-btrfs?), networking (routing! routing! routing!) and many other extremely important things is out of focus. And I have innate desire to test new RHEL 7 kernel and switch my hundreds of servers to it :) But this kernel based on enough old 3.10 kernel will be obsolete since release (look at my comparison about 3.17 3.18 kernel). IMHO, best approach for OpenVZ will be follow the upstream because your patch have significantly reduced since 2.6.32. You are trying to keep up stability with following to Red Hat with old kernels but in reality OpenVZ kernel is not really stable (you could grep my bug reports at bugzilla.openvz.org and found hundreds of issues with stability) even with RH kernel. My instances with upstream kernel could work many months without any issues. Yep, it's enough stable and completely suitable for OpenVZ HWN's. ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users
Re: [Users] Docker inside an OpenVZ container
Hello, folks! CentOS 6 with 2.6.32 kernel is real nightmare (even in case of networking) from my point of view. Simple syn flood could KILL my HWN's (I can share details off list). You (Parallels) and RH do big amount of backporting but upstream is far away in future. Difference even between 3.17 and 3.18 kernels is EXTREMELY HUGE. You could look at diffs here https://github.com/torvalds/linux/commits/master. And performance could be improved for 50-70% with this upgrade (I assume even more speed up with update from 2.6.32RH to 3.18). Red Hat interested only in few things about 2.6.32 kernel for running Java and KVM (with Oracle?). But storages (what about stable filesystem like ZFS instead bunch-of-crap-ext4 and it-will-be-non-stable-for-ever-btrfs?), networking (routing! routing! routing!) and many other extremely important things is out of focus. And I have innate desire to test new RHEL 7 kernel and switch my hundreds of servers to it :) But this kernel based on enough old 3.10 kernel will be obsolete since release (look at my comparison about 3.17 3.18 kernel). IMHO, best approach for OpenVZ will be follow the upstream because your patch have significantly reduced since 2.6.32. You are trying to keep up stability with following to Red Hat with old kernels but in reality OpenVZ kernel is not really stable (you could grep my bug reports at bugzilla.openvz.org and found hundreds of issues with stability) even with RH kernel. My instances with upstream kernel could work many months without any issues. Yep, it's enough stable and completely suitable for OpenVZ HWN's. On Tue, Mar 24, 2015 at 11:42 AM, Vasily Averin v...@parallels.com wrote: On 24.03.2015 02:59, Pavel Snajdr wrote: On 03/24/2015 12:48 AM, Pavel Snajdr wrote: On 03/23/2015 11:01 PM, Kir Kolyshkin wrote: On 03/23/2015 03:12 AM, Benjamin Henrion wrote: On Mon, Mar 23, 2015 at 10:55 AM, Narcis Garcia informat...@actiu.net wrote: As I read from Ubuntu/Debian package (version 0.9.1): Docker complements kernel namespacing with a high-level API which operates at the process level. It runs unix processes with strong guarantees of isolation and repeatability across servers. Docker is a great building block for automating distributed systems: large-scale web deployments, database clusters, continuous deployment systems, private PaaS, service-oriented architectures, etc. This package contains the daemon and client. *Using docker.io on non-amd64 hosts is not supported at this time*. Please be careful when using it on anything besides amd64. Also, note that *kernel version 3.8 or above is required* for proper operation of the daemon process, and that any lower versions may have subtle and/or glaring issues. Redhat backported a lot of LXC features to 2.6.32, so that's one of the reasons you can run docker/lxc on top of the openvz kernel. In addition to that, we did a significant amount of kernel work to allow running Docker inside our containers. In general, OpenVZ kernel version (which is 2.6.32) has very little to do with vanilla 2.6.32, so this number doesn't really mean anything except that Red Hat kernel team branched their kernel off this version when they started working on RHEL6. Currently this is 2.6.32 plus tons of patches from Red Hat plus a pretty big patchset from OpenVZ. In particular, we make sure all the recent distros work inside containers, so sometimes we have to backport some new syscall or other feature from recent kernels. From time to time I see people saying OpenVZ kernel is very old and obsoleted. It happens because the judge by the label, and the label starts with 2.6.32. Indeed, 2.6.32 is a very old kernel, but as I explained above our kernel has very little to do with 2.6.32. Hi Kir, I've been telling those people just about the same thing, but recently I don't think I can agree. I've become more involved in ZFS on Linux development and we run on RHEL6 OpenVZ kernel. The core of RHEL6 is getting old and there are lots and lots of improvement, that have gone upstream, which improve mainly performance and multi-core scalability. For instance and probably the most importantly for me now, I would like to have VFS scalability patches, that have gone into upstream, in OpenVZ RHEL6 kernel as well, as I'd like to see doing more granular eviction on dentries. (http://lwn.net/Articles/636133/) Sorry, linked the wrong thing, I mean two separate things. Primary objective is to have the shrinker work, which has gone to 3.12, secondary and nice to have would be these VFS scalability patches. Dear Pavel, such backports are quite hard for us. I'm not decision maker, but it's unlikely that we'll do it in RHEL6-based kernels. We inherit all infrastructure changes from RHEL6 kernels, and do not changes it without urgent need. - we risks to broke stable RHEL6 kernel - it makes hard maintenance, we'll need to re-apply these patches after rebase to new RHEL6
Re: [Users] Docker inside an OpenVZ container
On 24.03.2015 02:59, Pavel Snajdr wrote: On 03/24/2015 12:48 AM, Pavel Snajdr wrote: On 03/23/2015 11:01 PM, Kir Kolyshkin wrote: On 03/23/2015 03:12 AM, Benjamin Henrion wrote: On Mon, Mar 23, 2015 at 10:55 AM, Narcis Garcia informat...@actiu.net wrote: As I read from Ubuntu/Debian package (version 0.9.1): Docker complements kernel namespacing with a high-level API which operates at the process level. It runs unix processes with strong guarantees of isolation and repeatability across servers. Docker is a great building block for automating distributed systems: large-scale web deployments, database clusters, continuous deployment systems, private PaaS, service-oriented architectures, etc. This package contains the daemon and client. *Using docker.io on non-amd64 hosts is not supported at this time*. Please be careful when using it on anything besides amd64. Also, note that *kernel version 3.8 or above is required* for proper operation of the daemon process, and that any lower versions may have subtle and/or glaring issues. Redhat backported a lot of LXC features to 2.6.32, so that's one of the reasons you can run docker/lxc on top of the openvz kernel. In addition to that, we did a significant amount of kernel work to allow running Docker inside our containers. In general, OpenVZ kernel version (which is 2.6.32) has very little to do with vanilla 2.6.32, so this number doesn't really mean anything except that Red Hat kernel team branched their kernel off this version when they started working on RHEL6. Currently this is 2.6.32 plus tons of patches from Red Hat plus a pretty big patchset from OpenVZ. In particular, we make sure all the recent distros work inside containers, so sometimes we have to backport some new syscall or other feature from recent kernels. From time to time I see people saying OpenVZ kernel is very old and obsoleted. It happens because the judge by the label, and the label starts with 2.6.32. Indeed, 2.6.32 is a very old kernel, but as I explained above our kernel has very little to do with 2.6.32. Hi Kir, I've been telling those people just about the same thing, but recently I don't think I can agree. I've become more involved in ZFS on Linux development and we run on RHEL6 OpenVZ kernel. The core of RHEL6 is getting old and there are lots and lots of improvement, that have gone upstream, which improve mainly performance and multi-core scalability. For instance and probably the most importantly for me now, I would like to have VFS scalability patches, that have gone into upstream, in OpenVZ RHEL6 kernel as well, as I'd like to see doing more granular eviction on dentries. (http://lwn.net/Articles/636133/) Sorry, linked the wrong thing, I mean two separate things. Primary objective is to have the shrinker work, which has gone to 3.12, secondary and nice to have would be these VFS scalability patches. Dear Pavel, such backports are quite hard for us. I'm not decision maker, but it's unlikely that we'll do it in RHEL6-based kernels. We inherit all infrastructure changes from RHEL6 kernels, and do not changes it without urgent need. - we risks to broke stable RHEL6 kernel - it makes hard maintenance, we'll need to re-apply these patches after rebase to new RHEL6 kernels. - it requires careful testing, so it's an additional load for our QA, - of course it's an additional task for our developers, but let's forget about this. You could push RHEL6 and convince them that they need to do this job. If you'll have hard arguments, they will add the patches into next major update, update 7 is expected in few months, update 8 most likely will be released in next year. However nobody guarantees that they will do it too. Good news is that we're rebasing to RHEL7-kenrels. So I would recommend you to look at RHEL7 kernels right now, and wait for our rhel7-based kernel. Docker support in RHEL6 kernels delayed us a little, but anyway we expect to publish first beta in few months. http://openvz.livejournal.com/49158.html Thank you, Vasily Averin But this seems rather impossible, due to git/separated-out-patches not being available for RHEL6 kernel and OpenVZ project following the suit. I would have to invest a lot of time every time a rebase onto a newer RHEL6 kernel release is made. I would like to help out with OpenVZ development from time to time, especially with things related to storage, but the project doesn't seem all that open, you guys only publish your final results, but nothing from the process of getting to them. I don't mean to criticize or I don't mean it in any other bad way, here's me just sighing at how things are. Do you see any room for change in this regard? Or should we just leverage Parallels paid support for OpenVZ to have you guys pull in the patches by yourselves? I love open-source and doing things openly, I know that you guys don't have a whole lot of breathing room thanks to Red
Re: [Users] Docker inside an OpenVZ container
Hi, Benjamin Henrion zoo...@gmail.com hat am 23. März 2015 um 09:54 geschrieben: Hi, Anyone has tried this docker inside an OpenVZ container: https://openvz.org/Docker_inside_CT I discovered the page yesterday. I tried to upgrade a Proxmox server with the OpenVZ kernel, did not survived a reboot. Just to note, Proxmox VE already have a kernel based on 042stab105.14 but I did not tried with docker. http://download.proxmox.com/debian/dists/wheezy/pve-no-subscription/binary-amd64/pve-kernel-2.6.32-37-pve_2.6.32-150_amd64.deb http://download.proxmox.com/debian/dists/wheezy/pve-no-subscription/binary-amd64/pve-kernel-2.6.32-37-pve_2.6.32-150.changelog Martin ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users
Re: [Users] Docker inside an OpenVZ container
As I read from Ubuntu/Debian package (version 0.9.1): Docker complements kernel namespacing with a high-level API which operates at the process level. It runs unix processes with strong guarantees of isolation and repeatability across servers. Docker is a great building block for automating distributed systems: large-scale web deployments, database clusters, continuous deployment systems, private PaaS, service-oriented architectures, etc. This package contains the daemon and client. *Using docker.io on non-amd64 hosts is not supported at this time*. Please be careful when using it on anything besides amd64. Also, note that *kernel version 3.8 or above is required* for proper operation of the daemon process, and that any lower versions may have subtle and/or glaring issues. El 23/03/15 a les 10:49, Martin Maurer ha escrit: Hi, Benjamin Henrion zoo...@gmail.com hat am 23. März 2015 um 09:54 geschrieben: Hi, Anyone has tried this docker inside an OpenVZ container: https://openvz.org/Docker_inside_CT I discovered the page yesterday. I tried to upgrade a Proxmox server with the OpenVZ kernel, did not survived a reboot. Just to note, Proxmox VE already have a kernel based on 042stab105.14 but I did not tried with docker. http://download.proxmox.com/debian/dists/wheezy/pve-no-subscription/binary-amd64/pve-kernel-2.6.32-37-pve_2.6.32-150_amd64.deb http://download.proxmox.com/debian/dists/wheezy/pve-no-subscription/binary-amd64/pve-kernel-2.6.32-37-pve_2.6.32-150.changelog Martin ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users
[Users] Docker inside an OpenVZ container
Hi, Anyone has tried this docker inside an OpenVZ container: https://openvz.org/Docker_inside_CT I discovered the page yesterday. I tried to upgrade a Proxmox server with the OpenVZ kernel, did not survived a reboot. Will try to find some time today to give it a shot. Best, -- Benjamin Henrion bhenrion at ffii.org FFII Brussels - +32-484-566109 - +32-2-4148403 In July 2005, after several failed attempts to legalise software patents in Europe, the patent establishment changed its strategy. Instead of explicitly seeking to sanction the patentability of software, they are now seeking to create a central European patent court, which would establish and enforce patentability rules in their favor, without any possibility of correction by competing courts or democratically elected legislators. ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users
Re: [Users] Docker inside an OpenVZ container
On 03/24/2015 12:48 AM, Pavel Snajdr wrote: On 03/23/2015 11:01 PM, Kir Kolyshkin wrote: On 03/23/2015 03:12 AM, Benjamin Henrion wrote: On Mon, Mar 23, 2015 at 10:55 AM, Narcis Garcia informat...@actiu.net wrote: As I read from Ubuntu/Debian package (version 0.9.1): Docker complements kernel namespacing with a high-level API which operates at the process level. It runs unix processes with strong guarantees of isolation and repeatability across servers. Docker is a great building block for automating distributed systems: large-scale web deployments, database clusters, continuous deployment systems, private PaaS, service-oriented architectures, etc. This package contains the daemon and client. *Using docker.io on non-amd64 hosts is not supported at this time*. Please be careful when using it on anything besides amd64. Also, note that *kernel version 3.8 or above is required* for proper operation of the daemon process, and that any lower versions may have subtle and/or glaring issues. Redhat backported a lot of LXC features to 2.6.32, so that's one of the reasons you can run docker/lxc on top of the openvz kernel. In addition to that, we did a significant amount of kernel work to allow running Docker inside our containers. In general, OpenVZ kernel version (which is 2.6.32) has very little to do with vanilla 2.6.32, so this number doesn't really mean anything except that Red Hat kernel team branched their kernel off this version when they started working on RHEL6. Currently this is 2.6.32 plus tons of patches from Red Hat plus a pretty big patchset from OpenVZ. In particular, we make sure all the recent distros work inside containers, so sometimes we have to backport some new syscall or other feature from recent kernels. From time to time I see people saying OpenVZ kernel is very old and obsoleted. It happens because the judge by the label, and the label starts with 2.6.32. Indeed, 2.6.32 is a very old kernel, but as I explained above our kernel has very little to do with 2.6.32. Hi Kir, I've been telling those people just about the same thing, but recently I don't think I can agree. I've become more involved in ZFS on Linux development and we run on RHEL6 OpenVZ kernel. The core of RHEL6 is getting old and there are lots and lots of improvement, that have gone upstream, which improve mainly performance and multi-core scalability. For instance and probably the most importantly for me now, I would like to have VFS scalability patches, that have gone into upstream, in OpenVZ RHEL6 kernel as well, as I'd like to see doing more granular eviction on dentries. (http://lwn.net/Articles/636133/) Sorry, linked the wrong thing, I mean two separate things. Primary objective is to have the shrinker work, which has gone to 3.12, secondary and nice to have would be these VFS scalability patches. But this seems rather impossible, due to git/separated-out-patches not being available for RHEL6 kernel and OpenVZ project following the suit. I would have to invest a lot of time every time a rebase onto a newer RHEL6 kernel release is made. I would like to help out with OpenVZ development from time to time, especially with things related to storage, but the project doesn't seem all that open, you guys only publish your final results, but nothing from the process of getting to them. I don't mean to criticize or I don't mean it in any other bad way, here's me just sighing at how things are. Do you see any room for change in this regard? Or should we just leverage Parallels paid support for OpenVZ to have you guys pull in the patches by yourselves? I love open-source and doing things openly, I know that you guys don't have a whole lot of breathing room thanks to Red Hat here, but is there any possibility of opening the project up more? Finally, I would like to thank all of you at the OpenVZ project, there is no other usable container technology for Linux without you guys. I highly respect that fact despite the relative closedness of the project. /snajpa (vpsFree.cz) ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users
