[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Robie Basak]

> Doing so would require the CPC team to update /etc/default/lxc-net,
setting USE_LXC_BRIDGE to false.

Note that this would cause a conffile prompt for all users using cloud
images who dist-upgrade to pick up the latest updates after another lxc
SRU, which breaks people doing automatic deployments (even though they
could override, many don't). I think you're not planning to do this now
anyway? Anyway, I think it's important for everyone to understand that
altering conffiles as part of cloud image builds causes future problems
and so need to be avoided - especially for default cases.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Scott Moser]

I've opened bug 1510108 to address 'Stage 2' of this fix.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Scott Moser]

I'm marking this verification-done based on comments:
   25 : https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/comments/25
   23 : https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/comments/23
   21 : https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/comments/21
   20 : https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/comments/20


** Tags removed: verification-needed
** Tags added: verification-done

** Description changed:

  [Problem]
  The released wily image preinstalls lxc, which breaks the assumption that 
lxc's preinst packaging script makes:
  
  It inspects the network to try to pick a 10.0.N.0 network that isn't
  being used, with N starting at 3, so this appears to have picked
  10.0.3.0 when it was installed on whatever system was generating the
  image.
  
  When a container is started, it will dhcp on eth0 and get 10.0.3.X as
  expected.  The problem comes when the lxc-net service that is already
  installed in that container starts and configures *its* lxcbr0 with
  10.0.3.X.  The networking inside the container is broken at that point.
  
  This affects LXC containers, and should affect LXD containers but
  doesn't currently, as the metadata used for lxd images is still pointing
  to a beta2 release (bug 1509390).
  
  The easiest way to reproduce this is to use the ubuntu-cloud lxc
  template on a wily host.
  
  [Test Case]
  
  1.) Verify expectation for each image
     - -disk1.img cloud image, check for file
     - -root.tar.xz image (used by lxd) and check for file
     - -root.tar.gz image (used by lxc)
  
     For each of those images, verify:
     a.) A cloud image should not have /etc/default/lxc-net
     b.) lxd should be installed (dpkg-query --show | grep lxd)
  
  2.) Start instance from updated image and start instance in lxc inside
     launch instance on openstack or kvm or other
     verify lxcbr0 bridge exists
     lxc-create -t ubuntu-cloud -n bugcheck -- --release=wily --stream=daily
     # wait until lxc-ls --fancy shows 'running'
     lxc-attach -n bugcheck wget http://ubuntu.com
  
  3.) Start instance from updated image and start instance in lxd inside
     launch instance on openstack or kvm or other
     verify lxcbr0 bridge exists
     lxd import-images ubuntu wily
     lxc launch ubuntu
     # wait some amount
     lxc attach bugcheck wget http://ubuntu.com
  
  [Regression Potentional]
  The highest chance for fallout is a change in the /24 network that is chosen 
conflicting with some existing service.
  
  [Other Info]
  Default apt install of lxc has always picked some 10.0.X.0/24 network to use 
for its lxcbr0 bridge.  That network (often 10.0.3.0/24) would then be 
unreachable from the host.  The same behavior occurs with libvirt-bin and many 
other such services.
  
  We are moving that logic to happen the first time that 'lxc-net' service
  starts.
  
  This means first boot for a cloud instance rather than cloud-image build
  time.
  
  [Work around]
  To patch / fix an existing cloud image to make lxc and lxd guests start 
simply change the config of /etc/default/lxc-net to have something other than 
10.0.3.0.
  
  sudo sed -i '/^LXC.*10[.]0[.][0-9][.]/s/10.0.[0-9]./10.0.4./g' 
/etc/default/lxc-net &&
  sudo service lxc-net stop &&
  sudo service lxc-net start
+ 
+ === End SRU Report ===
+ 
+ Related bugs:
+  * bug 1510108: pre-installed lxc in cloud-image means loss of access to some 
10.0.X.0/24

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Launchpad Bug Tracker]

This bug was fixed in the package lxc - 1.1.4-0ubuntu1.1

---
lxc (1.1.4-0ubuntu1.1) wily-proposed; urgency=medium

  * lxc-net init script: update to select the default lxc bridge network
at first service start time rather than install time.  (LP: #1509414)
  * lxc-net init script: also move cleanup() definition as it was undefined
when called after failed dnsmasq.
  * lxc.preinst:
- remove code for writing /etc/default/lxc-net (moved to lxc-net service)
- add code removing just the known-potentially-bad /etc/default/lxc-net
- if user had deleted /etc/default/lxc-net (intending to disable lxcbr0),
   honor that by creating one which says not to use lxcbr0.

 -- Serge Hallyn   Fri, 23 Oct 2015 19:29:23
-0500

** Changed in: lxc (Ubuntu)
   Status: Confirmed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Scott Moser]

** Description changed:

  [Problem]
  The released wily image preinstalls lxc, which breaks the assumption that 
lxc's preinst packaging script makes:
  
  It inspects the network to try to pick a 10.0.N.0 network that isn't
  being used, with N starting at 3, so this appears to have picked
  10.0.3.0 when it was installed on whatever system was generating the
  image.
  
  When a container is started, it will dhcp on eth0 and get 10.0.3.X as
  expected.  The problem comes when the lxc-net service that is already
  installed in that container starts and configures *its* lxcbr0 with
  10.0.3.X.  The networking inside the container is broken at that point.
  
  This affects LXC containers, and should affect LXD containers but
  doesn't currently, as the metadata used for lxd images is still pointing
  to a beta2 release (bug 1509390).
  
  The easiest way to reproduce this is to use the ubuntu-cloud lxc
  template on a wily host.
  
  [Test Case]
  
  1.) Verify expectation for each image
     - -disk1.img cloud image, check for file
     - -root.tar.xz image (used by lxd) and check for file
     - -root.tar.gz image (used by lxc)
  
     For each of those images, verify:
     a.) A cloud image should not have /etc/default/lxc-net
     b.) lxd should be installed (dpkg-query --show | grep lxd)
  
  2.) Start instance from updated image and start instance in lxc inside
     launch instance on openstack or kvm or other
     verify lxcbr0 bridge exists
     lxc-create -t ubuntu-cloud -n bugcheck -- --release=wily --stream=daily
     # wait until lxc-ls --fancy shows 'running'
     lxc-attach -n bugcheck wget http://ubuntu.com
  
  3.) Start instance from updated image and start instance in lxd inside
     launch instance on openstack or kvm or other
     verify lxcbr0 bridge exists
     lxd import-images ubuntu wily
     lxc launch ubuntu
     # wait some amount
     lxc attach bugcheck wget http://ubuntu.com
  
  [Regression Potentional]
- The highest chance for fallout is a change in the /16 network that is chosen 
conflicting with some existing service.
+ The highest chance for fallout is a change in the /24 network that is chosen 
conflicting with some existing service.
  
  [Other Info]
- Default apt install of lxc has always picked some 10.0.X.0/16 network to use 
for its lxcbr0 bridge.  That network (often 10.0.3.0/16) would then be 
unreachable from the host.  The same behavior occurs with libvirt-bin and many 
other such services.
+ Default apt install of lxc has always picked some 10.0.X.0/24 network to use 
for its lxcbr0 bridge.  That network (often 10.0.3.0/24) would then be 
unreachable from the host.  The same behavior occurs with libvirt-bin and many 
other such services.
  
  We are moving that logic to happen the first time that 'lxc-net' service
  starts.
  
  This means first boot for a cloud instance rather than cloud-image build
  time.
  
  [Work around]
  To patch / fix an existing cloud image to make lxc and lxd guests start 
simply change the config of /etc/default/lxc-net to have something other than 
10.0.3.0.
  
  sudo sed -i '/^LXC.*10[.]0[.][0-9][.]/s/10.0.[0-9]./10.0.4./g' 
/etc/default/lxc-net &&
  sudo service lxc-net stop &&
  sudo service lxc-net start

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Stéphane Graber]

A pre-start lxc hook with sufficient privileges to start lxc-net would
cover all use cases as far as I can tell and would only require the
addition of two files to the lxc package.

Such a hook would also cover LXD as LXD does exec all LXC hooks, so we
wouldn't even have to mess with those init scripts at all. Just ship
such a hook and have the cloud-images ship with the lxc-net job
disabled. At container startup time, the hook fires and if the job isn't
running, it gets started.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Stéphane Graber]

Such shuffling around as an SRU seems pretty risky to me. Having the
main lxc package be essentially empty except for the systemd postinst
also feels weird.

This would also further complicate things when I then break lxc into lxc
and lxc-common this cycle (lxc-common will include the apparmor profiles
and the binaries that are used by liblxc1).

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Robert C Jennings]

I agree that this shuffling around is not pretty, but we need a solution
that makes 10.0.0.0/16 routable in cloud images where lxc/lxd are not in
use as had prior to http://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-
seeds/ubuntu.wily/revision/2360

The current situation conflicts with how clouds are instructing users to
set up private networks. [1] [2]

[1] http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Subnets.html
[2] 
https://azure.microsoft.com/en-us/documentation/articles/virtual-networks-reserved-private-ip/

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

Re: [Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Serge Hallyn]

Quoting Stéphane Graber (stgra...@stgraber.org):
> I agree, the stage 2 fix for this issue concerns me with regard to
> regressing current use cases.
> 
> As much as I'd like to get rid of the rest of this issue (any user of
> 10.0.4.0/24 behind a router looses connectivity to that subnet), we must
> make sure we do not regress anyone who's been relying on "apt-get
> install lxc" providing something that can immediately be used both by
> root and for unprivileged users.
> 
> Serge: We may be able to provide a hook, added to
> /usr/share/lxc/config/common.conf.d which will bring the bridge up
> automatically at first lxc container start. Such a hook would
> unfortunately need to be setuid so that it also works for unprivileged
> users. We'd also need to make sure that the current lxc hooks are
> sufficient from a timing point of view to do so (run before lxc checks
> whether the requested bridge exists).

smoser and I had considered creating a new lxc-base (I'm making that
name up) package which is the current lxc without the multi-user.target
wants symlink for lxc, and making lxd depend on that package.  Regular
lxc then would add the multiuser.target wants symlink for lxc.

Juju would not regress, regular cloud users would not have lxcbr0 until
they used lxd.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Robert C Jennings]

Séphane,

When this was added to the cloud seed we changed this from "users that
install LXC will loose connectivity to a 10.0.x.0/24 network" to "all
cloud users do not have connectivity to a 10.0.x.0/24 network at boot"
and the cause/effect will not be as clear to an end user.  This had come
up in conversation but had not been documented in the bug.   So let us
continue to search for a solution like what you have outlined in the
last paragraph of comment #29.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Cheryl Jennings]

Kicked off instances  for testing with Juju.  Will update with results
once my testing is complete.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Robert C Jennings]

Cloud images build from proposed are available.

Next action:

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Robert C Jennings]

Cloud image downloads for amd64, i386, and ppc64el are available @ http
://cloud-images.ubuntu.com/proposed/wily/20151024/

The amd64 image is also available in canonistack lcy02 region as
lp1509414/wily-proposed

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Robert C Jennings]

Be aware in your testing that the lxd-net's service can come up slowly
depending on the speed of your cloud instance. Without the bridge
(lxcbr0) the container's networking will function prior to that service
starting; watch out for this false positive in your testing.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Robert C Jennings]

Cloud images build from proposed are available.

Next action:
 - Verification of proposed package.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Serge Hallyn]

New image works for me in lxc:

lxcbr0Link encap:Ethernet  HWaddr 76:79:3e:90:1c:88
inet addr:10.0.4.1  Bcast:0.0.0.0  Mask:255.255.255.0

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Jon Grimm]

Tested wily-proposed cloud-image running in local kvm.   Started wily-
proposed container via lxc (using ubuntu-cloud template), verified the
container's lxcbr0 looks fine (10.0.4.1), verified networking works (via
www.google.com).

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Serge Hallyn]

I was able to For stage two, at least with systemd, I changed
/lib/systemd/system/lxd-startup.service to:

[Unit]
Description=Container hypervisor based on LXC - boot time check
After=cgmanager.service lxd-unix.socket
Requires=cgmanager.service lxd-unix.socket

[Service]
Type=oneshot
ExecStart=/usr/bin/lxd activateifneeded
TimeoutSec=30s

[Install]
WantedBy=multi-user.target


while removing the files

/etc/systemd/system/multi-user.target.wants/lxc-net.service
/etc/systemd/system/multi-user.target.wants/lxc.service

(i.e. in packaging we would remove

[Install]
WantedBy=multi-user.target

from /lib/systemd/system/lxc{,-net}.service)

Now when the system comes up, lxcbr0 is not there.  When I do 'lxc
list', it comes up.

A user who wants to use non-lxd lxc can do

sudo systemctl add-wants multi-user.target lxc.service

to make lxc and lxc-net start back up at boot.

The lxd-startup.service also still works, since it works by activating
lxd if the database shows containers need to be started.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Adam Stokes]

I've also tested manually and via our OpenStack installer and lxcbr0 is
assigned (10.0.4.1) and the network is able to reach out to the internet
as before.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Tycho Andersen]

The lxc in wily-proposed also works for me: inet addr:10.0.4.1

Thanks all.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Cheryl Jennings]

Verified that Juju can start lxc containers with the proposed changes.
The containers came up and were able to communicate with the state
server, even when on a separate machine.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Robert C Jennings]

My testing with the cloud image containing the proposed package has been
successful.

Just a note again that the test case detailed in the description is fine
with the understanding that network testing needs to ensure the lxc-net
service has started lxcbr0 or a false positive is possible (per comment
#18).

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Cheryl Jennings]

Serge - your changes from comment #22 will break juju with lxc.  juju
will need to be modified to call systemctl add-wants multi-user.target
lxc.service

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Stéphane Graber]

I agree, the stage 2 fix for this issue concerns me with regard to
regressing current use cases.

As much as I'd like to get rid of the rest of this issue (any user of
10.0.4.0/24 behind a router looses connectivity to that subnet), we must
make sure we do not regress anyone who's been relying on "apt-get
install lxc" providing something that can immediately be used both by
root and for unprivileged users.

Serge: We may be able to provide a hook, added to
/usr/share/lxc/config/common.conf.d which will bring the bridge up
automatically at first lxc container start. Such a hook would
unfortunately need to be setuid so that it also works for unprivileged
users. We'd also need to make sure that the current lxc hooks are
sufficient from a timing point of view to do so (run before lxc checks
whether the requested bridge exists).

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Serge Hallyn]

This lxc debdiff (not appropriate upstream lxc) and a pull request
against lxd-pkg-ubuntu (https://github.com/lxc/lxd-pkg-ubuntu/pull/7)
combined should implement stage 2 of the fix.

Note I've tested these when separately implemented by hand, but have not
built packages with this debdiff+pull-request.


** Patch added: "lxcnet-sysd.debdiff"
   
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+attachment/4504366/+files/lxcnet-sysd.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

Re: [Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Scott Kitterman]

On Sunday, October 25, 2015 03:12:44 AM you wrote:
> Serge - your changes from comment #22 will break juju with lxc.  juju
> will need to be modified to call systemctl add-wants multi-user.target
> lxc.service

If that's the case, this approach for a fix probably isn't appropriate for an 
SRU.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Robert C Jennings]

The cloud-image builder picked up the change and is building images.
They are with the LP buildds now.  I will update this bug once
publication completes.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Stéphane Graber]

Hello Mike, or anyone else affected,

Accepted lxc into wily-proposed. The package will build now and be
available at https://launchpad.net/ubuntu/+source/lxc/1.1.4-0ubuntu1.1
in a few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to
enable and use -proposed.  Your feedback will aid us getting this update
out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed.  In either case, details of your testing will help
us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Tags added: verification-needed

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Serge Hallyn]

** Patch added: "And one more to fix in vms"
   
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+attachment/4503681/+files/lxcneta.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Serge Hallyn]

Handle one more corner case

** Patch added: "lxcnet9.debdiff"
   
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+attachment/4503630/+files/lxcnet9.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Dustin Kirkland ]

** Changed in: lxc (Ubuntu)
 Assignee: (unassigned) => Stéphane Graber (stgraber)

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Scott Moser]

** Summary changed:

- lxc postinst script checks available interfaces, can choose 
+ pre-installed lxc in cloud image produces broken lxc (and later lxd) 
containers

** Description changed:

  [Problem]
  The released wily image preinstalls lxc, which breaks the assumption that 
lxc's preinst packaging script makes:
  
  It inspects the network to try to pick a 10.0.N.0 network that isn't
  being used, with N starting at 3, so this appears to have picked
  10.0.3.0 when it was installed on whatever system was generating the
- image. This conflicts with the network that eth0 gets attached to when
- the image is brought up in a container, because it gets attached to the
- host's lxcbr0, which is using 10.0.3.x.
+ image.
  
- This affects LXC, and should affect LXD but doesn't currently, as the 
metadata used for lxd images is still pointing to a beta2 release.
- The easiest way to reproduce this is to use the ubuntu-cloud lxc template on 
a wily host:
+ When a container is started, it will dhcp on eth0 and get 10.0.3.X as
+ expected.  The problem comes when the lxc-net service that is already
+ installed in that container starts and configures *its* lxcbr0 with
+ 10.0.3.X.  The networking inside the container is broken at that point.
+ 
+ This affects LXC containers, and should affect LXD containers but
+ doesn't currently, as the metadata used for lxd images is still pointing
+ to a beta2 release (bug 1509390).
+ 
+ The easiest way to reproduce this is to use the ubuntu-cloud lxc
+ template on a wily host.
  
  [Test Case]
  
  1.) Verify expectation for each image
     - -disk1.img cloud image, check for file
     - -root.tar.xz image (used by lxd) and check for file
     - -root.tar.gz image (used by lxc)
  
     For each of those images, verify:
     a.) A cloud image should not have /etc/default/lxc-net
     b.) lxd should be installed (dpkg-query --show | grep lxd)
  
  2.) Start instance from updated image and start instance in lxc inside
     launch instance on openstack or kvm or other
     verify lxcbr0 bridge exists
     lxc-create -t ubuntu-cloud -n bugcheck -- --release=wily --stream=daily
     # wait until lxc-ls --fancy shows 'running'
     lxc-attach -n bugcheck wget http://ubuntu.com
  
  3.) Start instance from updated image and start instance in lxd inside
     launch instance on openstack or kvm or other
     verify lxcbr0 bridge exists
     lxd import-images ubuntu wily
     lxc launch ubuntu
     # wait some amount
     lxc attach bugcheck wget http://ubuntu.com
  
  [Regression Potentional]
  The highest chance for fallout is a change in the /16 network that is chosen 
conflicting with some existing service.
  
  [Other Info]
  Default apt install of lxc has always picked some 10.0.X.0/16 network to use 
for its lxcbr0 bridge.  That network (often 10.0.3.0/16) would then be 
unreachable from the host.  The same behavior occurs with libvirt-bin and many 
other such services.
  
  We are moving that logic to happen the first time that 'lxc-net' service
  starts.
  
  This means first boot for a cloud instance rather than cloud-image build
  time.
  
  [Work around]
  To patch / fix an existing cloud image to make lxc and lxd guests start 
simply change the config of /etc/default/lxc-net to have something other than 
10.0.3.0.
  
- sudo sed -i '/^LXC.*10[.]0[.][0-9][.]/s/10.0.[0-9]./10.0.4./g' 
/etc/default/lxc-net && 
- sudo service lxc-net stop && 
- sudo service lxc-net start
+ sudo sed -i '/^LXC.*10[.]0[.][0-9][.]/s/10.0.[0-9]./10.0.4./g' 
/etc/default/lxc-net &&
+ sudo service lxc-net stop &&
+ sudo service lxc-net start

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Robert C Jennings]

Action plan:
Stage 1 - Configure lxc-net at boot rather than at install.
 * This addresses the network failure for 15.10 containers started on 15.10 
hosts (patch  above in comment #6)
Stage 2 - Start lxc-net through systemd on the first launch of an LXC container.
 * This mitigates the unroutable 10.0.x.0/16 network issue for general cloud 
image users.  With this step we’re back to Trusty function.
 * This will work for privileged users, like Juju, without any interaction.  
Unprivileged users will be directed to start the service on the first container 
launch.

Next action:
 - serge-hallyn working on patch (last update in comment #7) and code in 
ppa:serge-hallyn/lxc-natty.  Patch is not yet ready for upload and serge will 
update as he progresses.
 - Once ready, uploaded, and published in -proposed, email ring rcj/utlemming 
and we'll ensure cloud image builder picks this up ASAP to build -proposed 
images

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Serge Hallyn]

Final proposed patch for now.  Uploaded to ppa:serge-hallyn/lxc-natty
for wily.

Installing this on a fresh ubuntu-cloud wily container (i.e. a broken
one) results in working lxcbr0 on new subnet.

** Patch added: "lxcnet8.debdiff"
   
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+attachment/4503564/+files/lxcnet8.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs

[Bug 1509414] Re: pre-installed lxc in cloud image produces broken lxc (and later lxd) containers

[Serge Hallyn]

new patch.

It upgrades a broken container fine, but lxc-net is not properly started
until I manually call

/usr/lib/x86_64-linux-gnu/lxc/lxc-net stop
/usr/lib/x86_64-linux-gnu/lxc/lxc-net start

or reboot


** Patch added: "lxcnet6.debdiff"
   
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+attachment/4503549/+files/lxcnet6.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1509414

Title:
  pre-installed lxc in cloud image produces broken lxc (and later lxd)
  containers

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1509414/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs