Re: [lxc-devel] [PATCH] ubuntu: Fix regression in post-process
On Tue, 3 Dec 2013 23:34:43 -0500 S.Çağlar Onur cag...@10ur.org wrote: Hi, On Tue, Dec 3, 2013 at 6:53 PM, Stéphane Graber stgra...@ubuntu.com wrote: On Tue, Dec 03, 2013 at 05:43:47PM -0600, Serge Hallyn wrote: Quoting S.Çağlar Onur (cag...@10ur.org): Hey Stéphane, On Wed, Nov 27, 2013 at 7:49 PM, Stéphane Graber stgra...@ubuntu.com wrote: THe recent reorg of lxc-ubuntu introduced some package installation in post-process but without first disabling service startup. As a result, if the cache is a bit out of date and a ssh update is available, post-process will apply that update (as it does apt-get install ssh vim) which in turn will attemp to start sshd. This will either lead to ssh on the host being restarted or if there's no sshd on the host, will fail the container creation as the postinst will get an error from upstart. The fix is very simply to add the same policy-rc.d trick when running post-process. I'm not sure whether this is the desired outcome (I haven't taken a look at it yet) but it looks like after this change lxc-create -n t -t ubuntu started to take more time (order of couple of minutes) to Can you reproduce this at will - revert the change and it's faster, reapply and it's slower? -serge Yes, lxc-ubuntu is now slower when creating a container as an extra apt-get update run and installation of vim was moved from being done when creating the cache to being done at container creation time. That change was caused by the addition of --packages which allows users to specify extra packages that should be available in the container. I believe at least one problem with that is that ssh is apt-get install'ed both at cache creation time and at container creation time. In most cases this would be a no-op, but if the cache is outdated, then ssh and openssh-server will get upgraded at container creation time, creating an extra delay. I think we should just be dropping ssh from the list of packages installed at creation time (since it's already in the original install) and I'd be happy to change the behaviour so that vim is always installed (as it used to be) and that this extra apt-get update + apt-get install run would only happen when extra packages are actually passed on the command line. To be honest that would be my preference otherwise creating containers in different times will produce different results and it might not be a good thing depending on how you look :) I believe we should be accepting the cache as the authoritative state over what distro provides on its repositories. Users can upgrade their containers or install/uninstall packages after the initial creation as they wish (or using this new --packages parameter) or could invalidate the cache by flushing it. One of the reasons I didn't do the cache thing in the Oracle template (for lxc-create it always fetches pkgs from upstream) was so that if you for example say lxc-create -n ol -t oracle -- -R 6.latest you actually get a container based on latest. My thought was that if you want faster creates, then you can do the create once and then just use lxc-clone after that. This also avoids any cache coherency problems that could crop up. -- Stéphane Graber Ubuntu developer http://www.ubuntu.com -- Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631iu=/4140/ostg.clktrk ___ lxc-devel mailing list lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH] ubuntu: Fix regression in post-process
Hi Dwight, On Wed, Dec 4, 2013 at 3:47 PM, Dwight Engen dwight.en...@oracle.com wrote: On Tue, 3 Dec 2013 23:34:43 -0500 S.Çağlar Onur cag...@10ur.org wrote: Hi, On Tue, Dec 3, 2013 at 6:53 PM, Stéphane Graber stgra...@ubuntu.com wrote: On Tue, Dec 03, 2013 at 05:43:47PM -0600, Serge Hallyn wrote: Quoting S.Çağlar Onur (cag...@10ur.org): Hey Stéphane, On Wed, Nov 27, 2013 at 7:49 PM, Stéphane Graber stgra...@ubuntu.com wrote: THe recent reorg of lxc-ubuntu introduced some package installation in post-process but without first disabling service startup. As a result, if the cache is a bit out of date and a ssh update is available, post-process will apply that update (as it does apt-get install ssh vim) which in turn will attemp to start sshd. This will either lead to ssh on the host being restarted or if there's no sshd on the host, will fail the container creation as the postinst will get an error from upstart. The fix is very simply to add the same policy-rc.d trick when running post-process. I'm not sure whether this is the desired outcome (I haven't taken a look at it yet) but it looks like after this change lxc-create -n t -t ubuntu started to take more time (order of couple of minutes) to Can you reproduce this at will - revert the change and it's faster, reapply and it's slower? -serge Yes, lxc-ubuntu is now slower when creating a container as an extra apt-get update run and installation of vim was moved from being done when creating the cache to being done at container creation time. That change was caused by the addition of --packages which allows users to specify extra packages that should be available in the container. I believe at least one problem with that is that ssh is apt-get install'ed both at cache creation time and at container creation time. In most cases this would be a no-op, but if the cache is outdated, then ssh and openssh-server will get upgraded at container creation time, creating an extra delay. I think we should just be dropping ssh from the list of packages installed at creation time (since it's already in the original install) and I'd be happy to change the behaviour so that vim is always installed (as it used to be) and that this extra apt-get update + apt-get install run would only happen when extra packages are actually passed on the command line. To be honest that would be my preference otherwise creating containers in different times will produce different results and it might not be a good thing depending on how you look :) I believe we should be accepting the cache as the authoritative state over what distro provides on its repositories. Users can upgrade their containers or install/uninstall packages after the initial creation as they wish (or using this new --packages parameter) or could invalidate the cache by flushing it. One of the reasons I didn't do the cache thing in the Oracle template (for lxc-create it always fetches pkgs from upstream) was so that if you for example say lxc-create -n ol -t oracle -- -R 6.latest you actually get a container based on latest. My thought was that if you want faster creates, then you can do the create once and then just use lxc-clone after that. This also avoids any cache coherency problems that could crop up. I understand this desire but then question becomes why do we bother to keep a cache if we are going to download stuff from internet and loose that time? IMHO only a freshly created cache can help avoid downloading stuff from internet in this case. If your cache is let's say Oracle 6.3 and latest points to 6.5 then I bet you will end up downloading same if not more stuff from the internet while trying to create the container. And if we think cloning is the way to go then there is no need to do caching at all. Of course this is just a gut feeling and I have no empirical data to prove that. And just to be clear, I'm not opposing this behavior but just saying that I was expecting create to be an disconnected operation as long as there is a cache on FS backing that distro/release/arch. -- Stéphane Graber Ubuntu developer http://www.ubuntu.com Cheers, -- S.Çağlar Onur cag...@10ur.org -- Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631iu=/4140/ostg.clktrk ___ lxc-devel mailing list lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH] ubuntu: Fix regression in post-process
Quoting S.Çağlar Onur (cag...@10ur.org): Hey Stéphane, On Wed, Nov 27, 2013 at 7:49 PM, Stéphane Graber stgra...@ubuntu.com wrote: THe recent reorg of lxc-ubuntu introduced some package installation in post-process but without first disabling service startup. As a result, if the cache is a bit out of date and a ssh update is available, post-process will apply that update (as it does apt-get install ssh vim) which in turn will attemp to start sshd. This will either lead to ssh on the host being restarted or if there's no sshd on the host, will fail the container creation as the postinst will get an error from upstart. The fix is very simply to add the same policy-rc.d trick when running post-process. I'm not sure whether this is the desired outcome (I haven't taken a look at it yet) but it looks like after this change lxc-create -n t -t ubuntu started to take more time (order of couple of minutes) to Can you reproduce this at will - revert the change and it's faster, reapply and it's slower? -serge -- Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631iu=/4140/ostg.clktrk ___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH] ubuntu: Fix regression in post-process
On Tue, Dec 03, 2013 at 05:43:47PM -0600, Serge Hallyn wrote: Quoting S.Çağlar Onur (cag...@10ur.org): Hey Stéphane, On Wed, Nov 27, 2013 at 7:49 PM, Stéphane Graber stgra...@ubuntu.com wrote: THe recent reorg of lxc-ubuntu introduced some package installation in post-process but without first disabling service startup. As a result, if the cache is a bit out of date and a ssh update is available, post-process will apply that update (as it does apt-get install ssh vim) which in turn will attemp to start sshd. This will either lead to ssh on the host being restarted or if there's no sshd on the host, will fail the container creation as the postinst will get an error from upstart. The fix is very simply to add the same policy-rc.d trick when running post-process. I'm not sure whether this is the desired outcome (I haven't taken a look at it yet) but it looks like after this change lxc-create -n t -t ubuntu started to take more time (order of couple of minutes) to Can you reproduce this at will - revert the change and it's faster, reapply and it's slower? -serge Yes, lxc-ubuntu is now slower when creating a container as an extra apt-get update run and installation of vim was moved from being done when creating the cache to being done at container creation time. That change was caused by the addition of --packages which allows users to specify extra packages that should be available in the container. I believe at least one problem with that is that ssh is apt-get install'ed both at cache creation time and at container creation time. In most cases this would be a no-op, but if the cache is outdated, then ssh and openssh-server will get upgraded at container creation time, creating an extra delay. I think we should just be dropping ssh from the list of packages installed at creation time (since it's already in the original install) and I'd be happy to change the behaviour so that vim is always installed (as it used to be) and that this extra apt-get update + apt-get install run would only happen when extra packages are actually passed on the command line. -- Stéphane Graber Ubuntu developer http://www.ubuntu.com signature.asc Description: Digital signature -- Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631iu=/4140/ostg.clktrk___ Lxc-devel mailing list Lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH] ubuntu: Fix regression in post-process
Hi, On Tue, Dec 3, 2013 at 6:53 PM, Stéphane Graber stgra...@ubuntu.com wrote: On Tue, Dec 03, 2013 at 05:43:47PM -0600, Serge Hallyn wrote: Quoting S.Çağlar Onur (cag...@10ur.org): Hey Stéphane, On Wed, Nov 27, 2013 at 7:49 PM, Stéphane Graber stgra...@ubuntu.com wrote: THe recent reorg of lxc-ubuntu introduced some package installation in post-process but without first disabling service startup. As a result, if the cache is a bit out of date and a ssh update is available, post-process will apply that update (as it does apt-get install ssh vim) which in turn will attemp to start sshd. This will either lead to ssh on the host being restarted or if there's no sshd on the host, will fail the container creation as the postinst will get an error from upstart. The fix is very simply to add the same policy-rc.d trick when running post-process. I'm not sure whether this is the desired outcome (I haven't taken a look at it yet) but it looks like after this change lxc-create -n t -t ubuntu started to take more time (order of couple of minutes) to Can you reproduce this at will - revert the change and it's faster, reapply and it's slower? -serge Yes, lxc-ubuntu is now slower when creating a container as an extra apt-get update run and installation of vim was moved from being done when creating the cache to being done at container creation time. That change was caused by the addition of --packages which allows users to specify extra packages that should be available in the container. I believe at least one problem with that is that ssh is apt-get install'ed both at cache creation time and at container creation time. In most cases this would be a no-op, but if the cache is outdated, then ssh and openssh-server will get upgraded at container creation time, creating an extra delay. I think we should just be dropping ssh from the list of packages installed at creation time (since it's already in the original install) and I'd be happy to change the behaviour so that vim is always installed (as it used to be) and that this extra apt-get update + apt-get install run would only happen when extra packages are actually passed on the command line. To be honest that would be my preference otherwise creating containers in different times will produce different results and it might not be a good thing depending on how you look :) I believe we should be accepting the cache as the authoritative state over what distro provides on its repositories. Users can upgrade their containers or install/uninstall packages after the initial creation as they wish (or using this new --packages parameter) or could invalidate the cache by flushing it. -- Stéphane Graber Ubuntu developer http://www.ubuntu.com -- S.Çağlar Onur cag...@10ur.org -- Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631iu=/4140/ostg.clktrk ___ lxc-devel mailing list lxc-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH] ubuntu: Fix regression in post-process
Hey Stéphane,
On Wed, Nov 27, 2013 at 7:49 PM, Stéphane Graber stgra...@ubuntu.com wrote:
THe recent reorg of lxc-ubuntu introduced some package installation in
post-process but without first disabling service startup.
As a result, if the cache is a bit out of date and a ssh update is
available, post-process will apply that update (as it does apt-get
install ssh vim) which in turn will attemp to start sshd. This will
either lead to ssh on the host being restarted or if there's no sshd on
the host, will fail the container creation as the postinst will get an
error from upstart.
The fix is very simply to add the same policy-rc.d trick when running
post-process.
I'm not sure whether this is the desired outcome (I haven't taken a
look at it yet) but it looks like after this change lxc-create -n t
-t ubuntu started to take more time (order of couple of minutes) to
complete. This wasn't the case couple of weeks ago, when I last tried
creating an ubuntu container. I flushed the cache thinking that it's
age is the problem but it didn't change anything, container still
tries to install (or maybe upgrade) additional packages (I think vim
and ssh) via apt. Is there a reason not to include those additional
packages in the initial bootstrap phase?
Signed-off-by: Stéphane Graber stgra...@ubuntu.com
---
templates/lxc-ubuntu.in | 10 ++
1 file changed, 10 insertions(+)
diff --git a/templates/lxc-ubuntu.in b/templates/lxc-ubuntu.in
index ac39ed2..4e6a54f 100644
--- a/templates/lxc-ubuntu.in
+++ b/templates/lxc-ubuntu.in
@@ -467,6 +467,13 @@ post_process()
release=$2
packages=$3
+# Disable service startup
+cat $rootfs/usr/sbin/policy-rc.d EOF
+#!/bin/sh
+exit 101
+EOF
+chmod +x $rootfs/usr/sbin/policy-rc.d
+
if [ ! -f $rootfs/etc/init/container-detect.conf ]; then
# Make sure we have a working resolv.conf
cresolvonf=${rootfs}/etc/resolv.conf
@@ -538,6 +545,9 @@ post_process()
mv $rootfs/dev/shm $rootfs/dev/shm.bak
ln -s /run/shm $rootfs/dev/shm
fi
+
+# Re-enable service startup
+rm $rootfs/usr/sbin/policy-rc.d
}
do_bindhome()
--
1.8.4.4
--
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk
___
Lxc-devel mailing list
Lxc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-devel
--
S.Çağlar Onur cag...@10ur.org
--
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk
___
Lxc-devel mailing list
Lxc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH] ubuntu: Fix regression in post-process
Quoting Stéphane Graber (stgra...@ubuntu.com):
THe recent reorg of lxc-ubuntu introduced some package installation in
post-process but without first disabling service startup.
As a result, if the cache is a bit out of date and a ssh update is
available, post-process will apply that update (as it does apt-get
install ssh vim) which in turn will attemp to start sshd. This will
either lead to ssh on the host being restarted or if there's no sshd on
the host, will fail the container creation as the postinst will get an
error from upstart.
The fix is very simply to add the same policy-rc.d trick when running
post-process.
If we're on a new enough kernel we might want to do all apt-gets
in the template by doing lxc-start -d and then lxc-attach. Or
we could just do it through lxc-execute I suppose, and always
put lxc-init into the container.
Signed-off-by: Stéphane Graber stgra...@ubuntu.com
Acked-by: Serge E. Hallyn serge.hal...@ubuntu.com
---
templates/lxc-ubuntu.in | 10 ++
1 file changed, 10 insertions(+)
diff --git a/templates/lxc-ubuntu.in b/templates/lxc-ubuntu.in
index ac39ed2..4e6a54f 100644
--- a/templates/lxc-ubuntu.in
+++ b/templates/lxc-ubuntu.in
@@ -467,6 +467,13 @@ post_process()
release=$2
packages=$3
+# Disable service startup
+cat $rootfs/usr/sbin/policy-rc.d EOF
+#!/bin/sh
+exit 101
+EOF
+chmod +x $rootfs/usr/sbin/policy-rc.d
+
if [ ! -f $rootfs/etc/init/container-detect.conf ]; then
# Make sure we have a working resolv.conf
cresolvonf=${rootfs}/etc/resolv.conf
@@ -538,6 +545,9 @@ post_process()
mv $rootfs/dev/shm $rootfs/dev/shm.bak
ln -s /run/shm $rootfs/dev/shm
fi
+
+# Re-enable service startup
+rm $rootfs/usr/sbin/policy-rc.d
}
do_bindhome()
--
1.8.4.4
--
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk
___
Lxc-devel mailing list
Lxc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-devel
--
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk
___
Lxc-devel mailing list
Lxc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-devel
Re: [lxc-devel] [PATCH] ubuntu: Fix regression in post-process
On Wed, Nov 27, 2013 at 09:14:24PM -0600, Serge Hallyn wrote:
Quoting Stéphane Graber (stgra...@ubuntu.com):
THe recent reorg of lxc-ubuntu introduced some package installation in
post-process but without first disabling service startup.
As a result, if the cache is a bit out of date and a ssh update is
available, post-process will apply that update (as it does apt-get
install ssh vim) which in turn will attemp to start sshd. This will
either lead to ssh on the host being restarted or if there's no sshd on
the host, will fail the container creation as the postinst will get an
error from upstart.
The fix is very simply to add the same policy-rc.d trick when running
post-process.
If we're on a new enough kernel we might want to do all apt-gets
in the template by doing lxc-start -d and then lxc-attach. Or
we could just do it through lxc-execute I suppose, and always
put lxc-init into the container.
Unfortunately upstart talks using an abstract unix socket, so the only
way to avoid those from messing with the host's upstart is to setup the
whole network stack for the container and put it in its network
namespace.
I very regularly lxc-create containers that won't start immediately
post-creation so I don't think relying on lxc-start is really a good
idea there. Instead we should just make sure to consistently block
service startup just like we would with a standard chroot (and so far
we've done that relatively well, I just missed that one case in my
review last week ...).
Signed-off-by: Stéphane Graber stgra...@ubuntu.com
Acked-by: Serge E. Hallyn serge.hal...@ubuntu.com
---
templates/lxc-ubuntu.in | 10 ++
1 file changed, 10 insertions(+)
diff --git a/templates/lxc-ubuntu.in b/templates/lxc-ubuntu.in
index ac39ed2..4e6a54f 100644
--- a/templates/lxc-ubuntu.in
+++ b/templates/lxc-ubuntu.in
@@ -467,6 +467,13 @@ post_process()
release=$2
packages=$3
+# Disable service startup
+cat $rootfs/usr/sbin/policy-rc.d EOF
+#!/bin/sh
+exit 101
+EOF
+chmod +x $rootfs/usr/sbin/policy-rc.d
+
if [ ! -f $rootfs/etc/init/container-detect.conf ]; then
# Make sure we have a working resolv.conf
cresolvonf=${rootfs}/etc/resolv.conf
@@ -538,6 +545,9 @@ post_process()
mv $rootfs/dev/shm $rootfs/dev/shm.bak
ln -s /run/shm $rootfs/dev/shm
fi
+
+# Re-enable service startup
+rm $rootfs/usr/sbin/policy-rc.d
}
do_bindhome()
--
1.8.4.4
--
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics
Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk
___
Lxc-devel mailing list
Lxc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-devel
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
signature.asc
Description: Digital signature
--
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk___
Lxc-devel mailing list
Lxc-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-devel
