On Tue, Mar 29, 2016 at 09:30:19AM -0700, jjs - mainphrame wrote: > On Tue, Mar 29, 2016 at 7:18 AM, Tycho Andersen < > [email protected]> wrote: > > > On Mon, Mar 28, 2016 at 08:47:24PM -0700, jjs - mainphrame wrote: > > > I've looked at ct migration between 2 ubuntu 16.04 hosts today, and had > > > some interesting problems; I find that migration of stopped containers > > > works fairly reliably; but live migration, well, it transfers a lot of > > > data, then exits with a failure message. I can then move the same > > > container, stopped, with no problem. > > > > > > The error is the same every time, a failure of "mkdtemp" - > > > > It looks like your host /tmp isn't writable by the uid map that the > > container is being restored as? > > > > Which is odd, since /tmp has 1777 perms on both hosts, so I don't see how > it could be a permissions problem. Surely the default apparmor profile is > not the cause? You did give me a new idea though, and I'll set up a test > with privileged containers for comparison. Is there a switch to enable > verbose logging?
It already is enabled, you can find the full logs in /var/log/lxd/$container/migration_* Perhaps the pwd of the CRIU task is what's broken instead, since CRIU isn't supplying a full mkdtemp template. I'll have a deeper look in a bit. Tycho > > > > > > > root@ronnie:~# lxc move third lxd: > > > error: Error transferring container data: restore failed: > > > (00.033172) 1: Error (cr-restore.c:1489): mkdtemp failed > > > crtools-proc.x9p5OH: Permission denied > > > (00.060072) Error (cr-restore.c:1352): 9188 killed by signal 9 > > > (00.117126) Error (cr-restore.c:2182): Restoring FAILED. > > > > > > root@raskolnikov:~# lxc move third lxd2: > > > error: Error transferring container data: restore failed: > > > (00.039099) 1: Error (cr-restore.c:1489): mkdtemp failed > > > crtools-proc.a3U2t5: Permission denied > > > (00.063015) Error (cr-restore.c:1352): 1771 killed by signal 9 > > > (00.115261) Error (cr-restore.c:2182): Restoring FAILED. > > > > > > root@ronnie:~# lxc move third lxd: > > > error: Error transferring container data: restore failed: > > > (00.034542) 1: Error (cr-restore.c:1489): mkdtemp failed > > > crtools-proc.gee5YS: Permission denied > > > (00.059955) Error (cr-restore.c:1352): 9274 killed by signal 9 > > > (00.109272) Error (cr-restore.c:2182): Restoring FAILED. > > > > > > > > > Thanks for any pointers you can provide - > > > > > > PS - on a side note, I'd originally set up the new box with a separate > > > /var partition on btrfs. As a result, "criu check" would die instantly > > with > > > a segmentation error. After putting /var back on / (ext4) criu check > > > produces the message "Looks good." > > > > Hmm. Can you get the backtrace? > > > > Tycho > > > > Sorry to say, I moved on quickly and the btrfs filesystem is no more, but I > can recreate that scenario again later. > > > Regards, > > Jake > > > > > > > > Jake > > > > > > > > > > > > On Mon, Mar 28, 2016 at 7:47 AM, Tycho Andersen < > > > [email protected]> wrote: > > > > > > > On Sun, Mar 27, 2016 at 09:22:44PM -0700, jjs - mainphrame wrote: > > > > > You've had some success with live migrations? At any rate, I'm > > upgrading > > > > my > > > > > ubuntu 15.10 test box to 16.04 so that I'll have 2 of them. I'll > > give it > > > > a > > > > > whirl. > > > > > > > > Yes, there are still various unsupported kernel features, but it > > > > should work out of the box for a modern stock xenial image on a xenial > > > > host. There is one more xenial kernel patch which will eventually be > > > > released that will break things, but I'm in the process of upstreaming > > > > some CRIU patches to handle that case, and we'll distro patch those > > > > when they're ready. > > > > > > > > Tycho > > > > > > > > > On Sun, Mar 27, 2016 at 9:20 PM, Fajar A. Nugraha <[email protected]> > > > > wrote: > > > > > > > > > > > On Sun, Mar 27, 2016 at 11:31 PM, jjs - mainphrame < > > [email protected] > > > > > > > > > > > wrote: > > > > > > > The 2nd link you sent seems to indicate that > > > > > > > live migration wants to work, but I haven't been able to find any > > > > reports > > > > > > > from normal users in the field who've actually succeeded with > > live > > > > > > > migration. if I've missed something, please let me know. > > > > > > > > > > > > My best advice is to try yourself with latest 16.04 daily. > > > > > > > > > > > > I've had both success and failure with it. > > > > > > > > > > > > -- > > > > > > Fajar > > > > > > _______________________________________________ > > > > > > lxc-users mailing list > > > > > > [email protected] > > > > > > http://lists.linuxcontainers.org/listinfo/lxc-users > > > > > > > > > > > > > > > _______________________________________________ > > > > > lxc-users mailing list > > > > > [email protected] > > > > > http://lists.linuxcontainers.org/listinfo/lxc-users > > > > > > > > _______________________________________________ > > > > lxc-users mailing list > > > > [email protected] > > > > http://lists.linuxcontainers.org/listinfo/lxc-users > > > > > > > > > _______________________________________________ > > > lxc-users mailing list > > > [email protected] > > > http://lists.linuxcontainers.org/listinfo/lxc-users > > > > _______________________________________________ > > lxc-users mailing list > > [email protected] > > http://lists.linuxcontainers.org/listinfo/lxc-users > > > _______________________________________________ > lxc-users mailing list > [email protected] > http://lists.linuxcontainers.org/listinfo/lxc-users _______________________________________________ lxc-users mailing list [email protected] http://lists.linuxcontainers.org/listinfo/lxc-users
