I was getting a bit desperate to recover my containers as I had invested a lot of time configuring them. I had backups of the containers (on ZFS with snapshots and replicas) but that did me no good as long as lxc could not "see" them.
It occurred to me maybe I could swap out 'dummy' containers with my real ones. So I tried the following, which seems to have worked - no issues with the 'recovered' containers as yet. I created a dummy template [template] using the same image as my valuable containers. I used the template to create new containers [newName]. LXC could see and start the new container. I stopped the container and deleted it, then copied one of my existing containers in its place and renamed it to the new name. I applied the correct profile. Then, whew!, lxc could see and start the container and I had my container back, albeit with a new name. After tidying up the hostname of the container, it was good to go. I repeated the exercise for all the containers I needed to recover. I still have the ghost names of the containers in lxd.db but I am not touching that. $ lxc launch images:501a7fb84afe [template] $ lxc stop [template] $ lxc copy [template] [newName] $ sudo rm -r /var/lib/lxd/containers/[newName] $ sudo cp -R --preserve=all /backup/location/containers/[backup] /var/lib/lxd/containers $ sudo mv /var/lib/lxd/containers/[backup] /var/lib/lxd/containers/[newName] $ lxc profile apply [newName] [profileName] $ lxc start [newName] $ lxc exec [newName] -- /bin/bash # nano /etc/hosts # nano /etc/hostname Apologies to Stéphane and the LXD/LXC team for what must appear an appalling solution. On Thu, Nov 19, 2015 at 11:30 PM, Matthew McKennirey <[email protected]> wrote: > Thanks for the rapid response. I rebooted the server, then tried > again. Host is Ubuntu 15.04 fully patched, kernel 3.19.0-25 (with later > kernels lxd will not start because of a permission issue with proc) > > $ sudo uname -a > Linux zeus 3.19.0-25-generic #26-Ubuntu SMP Fri Jul 24 21:17:31 UTC 2015 > x86_64 x86_64 x86_64 GNU/Linux > > lxc is 1.1.5-0ubuntu3~ubuntu > lxd is 0.22-0ubuntu2~ubuntu1 > > I am running all client commands on the host machine > $ lxc --version > 0.22 > $ lxd --version > 0.22 > > Containers are present on the file system > $ ls -l /var/lib/lxd/containers/ > -rwxrwxrwx 1 root root 27854 Nov 14 23:20 lxc-monitord.log > drwxrwxrwx 3 100000 100000 5 Feb 14 2015 ub1404_sg > drwx--x--x 3 root root 3 Aug 16 14:29 ubuntu14-java-utils > drwx--x--x 3 100000 100000 3 Aug 18 23:12 ubuntu-build > drwxr-xr-x 3 root root 3 Oct 18 17:24 ubuntu-build-clone > drwx--x--x 4 100000 100000 5 Aug 16 14:09 ubuntu-sg > drwx--x--x 3 root root 3 Aug 16 15:38 ubuntu-sg-base > drwxr-xr-x 3 100000 100000 3 Oct 18 18:25 ubuntu-sg-tst > drwxr-xr-x 3 100000 100000 3 Oct 18 17:30 ubuntu-tst > drwxr-xr-x 3 root root 3 Oct 18 17:18 ubuntu-web > > Containers are in lxd.db (see prior message) > > After the reboot I had trouble starting lxd > > $ sudo cat /var/log/syslog > kernel: [ 6575.572166] init: lxd main process (5391) terminated with > status 1 > kernel: [ 6575.572179] init: lxd main process ended, respawning > kernel: [ 6575.588459] init: lxd main process (5396) terminated with > status 1 > kernel: [ 6575.588474] init: lxd main process ended, respawning > kernel: [ 6575.604184] init: lxd main process (5401) terminated with > status 1 > kernel: [ 6575.604198] init: lxd respawning too fast, stopped > > $ sudo cat /var/log/upstart/lxd.log > error: mkdir /var/lib/lxd/images: file exists > error: mkdir /var/lib/lxd/images: file exists > error: mkdir /var/lib/lxd/images: file exists > error: mkdir /var/lib/lxd/images: file exists > > I'm really not sure why lxd was trying to create the images folder at > start-up. > Anyway I renamed the folder and lxd would start > > $ sudo mv /var/lib/lxd/images /var/lib/lxd/images-back > > $ sudo service lxd start --debug -v > $ lxc list --debug > DBUG[11-19|22:52:29] Fingering the daemon > DBUG[11-19|22:52:29] Raw response: > {"type":"sync","status":"Success","status_code":200,"metadata":{"api_compat":1,"auth":"trusted","config":{"images.remote_cache_expiry":"10"},"environment":{"addresses":[],"architectures":[2,1],"driver":"lxc","driver_version":"1.1.5","kernel":"Linux","kernel_architecture":"x86_64","kernel_version":"3.19.0-25-generic","server":"lxd","server_pid":5613,"server_version":"0.22","storage":"dir","storage_version":""}}} > > DBUG[11-19|22:52:29] Pong received > DBUG[11-19|22:52:29] Raw response: > {"type":"sync","status":"Success","status_code":200,"metadata":[]} > > +------+-------+------+------+-----------+-----------+ > | NAME | STATE | IPV4 | IPV6 | EPHEMERAL | SNAPSHOTS | > +------+-------+------+------+-----------+-----------+ > +------+-------+------+------+-----------+-----------+ > > Still no joy. > > $ sudo cat /var/log/lxd/lxd.log > t=2015-11-19T22:45:51-0500 lvl=info msg="LXD is starting" path=/var/lib/lxd > t=2015-11-19T22:45:51-0500 lvl=info msg="Default uid/gid map:" > t=2015-11-19T22:45:51-0500 lvl=info msg=" - u 0 100000 65536" > t=2015-11-19T22:45:51-0500 lvl=info msg=" - g 0 100000 65536" > t=2015-11-19T22:45:51-0500 lvl=info msg=Init driver=storage/dir > t=2015-11-19T22:45:51-0500 lvl=info msg="Looking for existing > certificates" cert=/var/lib/lxd/server.crt key=/var/lib/lxd/server.key > t=2015-11-19T22:45:51-0500 lvl=info msg="LXD isn't socket activated" > t=2015-11-19T22:45:51-0500 lvl=info msg="REST API daemon:" > t=2015-11-19T22:45:51-0500 lvl=info msg=" - binding socket" > socket=/var/lib/lxd/unix.socket > t=2015-11-19T22:46:09-0500 lvl=info msg=handling method=GET url=/1.0 ip=@ > t=2015-11-19T22:46:09-0500 lvl=info msg=handling method=GET > url="/1.0/containers?recursion=1" ip=@ > t=2015-11-19T22:48:20-0500 lvl=info msg=handling method=GET url=/1.0 ip=@ > t=2015-11-19T22:48:20-0500 lvl=info msg=handling method=GET > url="/1.0/containers?recursion=1" ip=@ > t=2015-11-19T22:49:06-0500 lvl=info msg=handling method=GET url=/1.0 ip=@ > t=2015-11-19T22:49:06-0500 lvl=info msg=handling method=GET > url="/1.0/containers?recursion=1" ip=@ > t=2015-11-19T22:52:29-0500 lvl=info msg=handling ip=@ method=GET url=/1.0 > t=2015-11-19T22:52:29-0500 lvl=info msg=handling method=GET > url="/1.0/containers?recursion=1" ip=@ > > thanks very much > > > > From: "Stéphane Graber" <[email protected]> > To: LXC users mailing-list <[email protected]> > Cc: > Date: Tue, 17 Nov 2015 22:09:17 -0500 > Subject: Re: [lxc-users] lxc list fails to show containers after upgrade > to lxd 0.22 / lxc 1.1.5 > Can you confirm your client and server are running the exact same > version of LXD? > > The fact that the containers appear to be properly in the database > typically indicates an API mismatch between the client and the server. > > Our API isn't stable yet (it will be when we tag LXD 2.0 early next > year), so mismatches can give you random results. > > > If "lxc version" and "lxd --version" match, then I'd suggest running > both of them with --debug, that should give us more of an idea of what's > going on. > > On Tue, Nov 17, 2015 at 09:57:50PM -0500, Matthew McKennirey wrote: > > I am running LXD/LXC on two hosts, each with multiple containers. > > I upgraded one host to the latest versions of LXD /LXC (had been on lxd > > 0.21 / lxc 1.1.4) > > > > After the upgrade. lxc list shows no containers >
_______________________________________________ lxc-users mailing list [email protected] http://lists.linuxcontainers.org/listinfo/lxc-users
