I just used the focal cloud images on systems with no internet
connectivity and snapd is adding at least 20 seconds of delay:
Jun 18 14:04:59 ubuntu systemd[1]: Finished Service for snap application
lxd.activate.
Jun 18 14:05:21 ubuntu snapd[1013]: stateengine.go:150: state ensure error:
@stgraber,
The bug subject says "Huge and slow". Shame on me to make 1 bug for 2 issues.
But wrt "is this fixed", groovy images are still dramatically bigger *and*
slower to boot than bionic images.
a.) Huge. Still looks much bigger, 'Huge' can obviously be argued.
346M for current groovy
I believe work in snapd was done to improve seeding so that this
wouldn't be such an issue anymore.
Has this been done and are the cloud images making use of it now?
If so, this can probably be closed.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is
I'm assign Pawel here as he has been working on some first-boot speed
improvements for cloud images specifically. Also, since this was
reported we now have `snap debug timings` command, which would be really
useful to look at for slow first boot. For a first-boot, the first snap
change is always
Subscribed the LXD team directly as we are obviously interested in this
issue.
Then moved the bug from lxd over to snapd since as far as LXD's
concerned, we've fixed the bug on our side (activate hanging for 10s),
with this fixed, LXD spends less than a second on boot to check whether
it's
Some work has been done to make connections after in
https://github.com/snapcore/snapd/pull/5469 - this will be part of the
upcoming 2.36 release which will most likely not be ready before cosmic
though :/
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is
Thanks Dan! Thats really helpful. So from the log you provided it looks
like the follow happend:
Boot time went from up by 8s (from 15s to 23s):
- "graphical.target reached after 15.614s in userspace" (before)
- "graphical.target reached after 23.527s in userspace" (after)
Seeding time went up
>From a pre-LXD-seeding instance, the plot is at
http://people.canonical.com/~dwatkins/1796137-before.svg and the `snap
change` output is:
$ snap change --abs-time 1
Status Spawn Ready Summary
Done2018-10-09T20:41:32Z 2018-10-09T20:41:33Z Ensure
@Dan could we get the same data (plot + output of snap changes/tasks)
for the image *before* lxd was seeded as well? This way we can more
easily compare.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
Some more details about what's happening inside snapd:
$ snap changes --abs-time
ID Status Spawn Ready Summary
1Done2018-10-09T20:35:36Z 2018-10-09T20:35:47Z Initialize system state
2Done2018-10-09T20:35:36Z 2018-10-09T20:35:38Z Initialize
As requested on IRC:
$ systemctl cat snap.lxd.daemon.unix.socket
# /etc/systemd/system/snap.lxd.daemon.unix.socket
[Unit]
# Auto-generated, DO NOT EDIT
Description=Socket unix for snap application lxd.daemon
Requires=snap-lxd-9010.mount
After=snap-lxd-9010.mount
X-Snappy=yes
[Socket]
http://people.canonical.com/~dwatkins/1796137.svg is a `systemd-analyze
plot` run on the instance with lxd.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1796137
Title:
huge and slow image 20181002
This is from a cosmic GCE instance _after_ the lxd snap has been seeded:
$ systemd-analyze critical-chain
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.
graphical.target @23.527s
This is from a cosmic GCE instance _before_ the lxd snap was seeded
(i.e. with just the google-cloud-sdk snap seeded):
$ systemd-analyze critical-chain
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+"
I've heard that even cloud images that were shipping snaps before the
LXD inclusion are seeing a ~10s slowdown. If that's indeed the case, can
we get a before/after for those with:
- systemd-analyze critical-chain
- journalctl --boot 0
- ps fauxww
The LXD snap at boot time should be spending
Just as a point of reference, a single container booted on the same system I
showed the 38 second boot time before, seems to be down to 25 or so now.
Much better, but still not 10 seconds as it was.
Note that I don't call this a scientific test. I've made no real effort to
control the
snapd has now migrated to the release pocket
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1796137
Title:
huge and slow image 20181002 due to seeded lxd snap
To manage notifications about this bug
The updated LXD snap should be available now in the stable channel.
The updated snapd is in cosmic-proposed and will hopefully migrate over the
weekend.
Assuming that does happen, it'd be good to re-check cloud image on
Monday.
--
You received this bug notification because you are a member of
Bah. my comment 8 above is wrong.
I meant to reference bug 1796165 (seeded snaps never cleaned up.)
That could affect boot time of subsequent boots, if snapd is re-reading those
preseed snaps.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to
I've updated the LXD snap, note that there's also a snapd issue which
may be in play here as it's been known to needlessly delay boot with
snaps using socket activation.
I've accepted the new snapd into cosmic now, so I think we'll want for
both the lxd snap to be in stable (should be in the next
Oh, I see what's going on, this piece of code used to be running AFTER
we setup our custom /etc for the snap (which includes the real host
nsswitch.conf) but now that it's moved over when we setup socket
activation, this isn't true anymore.
I'll push a fix to the snap tomorrow, that should fix
As Scott showed in comment #9, /etc/nsswitch.conf in the images do not
contain extrausers, but the core snap does.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1796137
Title:
huge and slow image
@stgraber:
# Grab the latest cosmic daily (20181002 is what I get)
#lxc launch ubuntu-daily:c recreate
$ lxc exec recreate -- grep "^passwd.*extrausers"
/snap/core/current/etc/nsswitch.conf
passwd: compat extrausers
$ lxc exec recreate -- snap list
Name VersionRev Tracking
To get the extrausers in the log above, you need this "grep -q
"^passwd.*extrausers" /etc/nsswitch.conf" to be true, so I'm wondering
what else may be different in this image's nsswitch and also how we
ended up with extrausers in nsswitch but no extrausers file on disk.
I suspect the solution
Because I suspect the GCE image may have a different looking one.
The one in the base image that you posted wouldn't trip the code that
uses extrausers.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
@Stgraber,
I'm not sure why you wouldnt' have just done this yourself, so maybe I'm
missing your request.
$ lxc init ubuntu-daily:cosmic my-test
Creating my-test
$ lxc file pull my-test/etc/nsswitch.conf - | pastebinit
http://paste.ubuntu.com/p/7rpZ2N4grn/
$ lxc start my-test
$ lxc exec my-test
repeat boots could I guess be slow as a result of bug 1786165.
The snaps that are seeded are never removed. perhaps snapd doesn't realize (or
has to re-check) to verify that they are not new.
I'd suggest debugging and re-create in lxd itself.
Doing so allows you to jump into the container before
Can you paste the content of /etc/nsswitch.conf from an affected
instanced?
It looks like you may have nss-extrausers installed, causing a mis-
trigger of part of our activate logic. Hopefully that's the source of
the issue and I can solved it easy enough in our snap.
--
You received this bug
Attaching the 'journalctl --boot' output after first boot of an AWS AMI
from today (ubuntu/images-testing/hvm-ssd/ubuntu-cosmic-daily-
amd64-server-20181002 - ami-03a19e7ba42cf1f55 in us-west-2) on an
m5.large instance
** Attachment added: "journalctl.txt"
No output was returned for:
$ snap logs lxd.activate
$ snap logs lxd.daemon
Here's the rest of the info on the instance/image used to capture the
journalctl output:
ubuntu@ip-172-31-35-137:~$ cat /etc/cloud/build.info
build_name: server
serial: 20181002
ubuntu@ip-172-31-35-137:~$ snap list
I was able to SSH in before the service completed, so I don't think it's
just a lack of network connectivity.
http://paste.ubuntu.com/p/pSrJJVD8zt/ is the full journalctl output from
the (GCE) instance.
(I'm not 100% sure if this is relevant to the second part of your
comment, Dimitri, but this
31 matches
Mail list logo
