On Mon, Jan 30, 2017 at 5:13 PM, Reed O'Brien <[email protected]> wrote:
> KVM as a juju deploy target now works on AMD64 and ARM64. There is an > issue on ARM64 using trusty, so it only works with xenial on the hardware I > could test on atm. There are a number of issues that prevent running juju > on trusty on ARM64. Following is a support matrix, an outline of known > issues, and scenarios where we might run into the issue of trusty not > working as expected. Finally there are a couple thoughts about handling the > issue. Any input on options/solutions for this issue are appreciated. > > ARM64 QEMU/KVM UEFI Support Matrix > I've put the support matrix here: http://pastebin.ubuntu.com/23907437/ since the html table was stripped from the list archive. > > - EFI guests require qemu-efi, which is only available in xenial. > > > - Trusty’s 3.13 kernel can’t boot on GICv3 systems, nor as a guest on > GICv3 systems. > > > - Trusty’s virt stack does not support GICv3 guests, but that support > is available in the cloud archive. > > > The first issue is that there is a very limited set of hardware that was > certified for trusty on ARM64. So that severely limits the number of > systems where we would expect juju to be able to deploy to trusty. As noted > in the next issue is possible using the hwe-x kernel, but that is not > something juju can control. > > The second issue is that those systems include GICv2 (General Interrupt > Controller v2), whereas newer systems include GICv3. This would require > trusty to boot with hwe-x kernel. AFAIU there are no official cloud images > with that kernel pre-installed. Additionally, it currently seems it isn't > possible to set hwe-x as the minimum kernel version in MAAS: > https://bugs.launchpad.net/maas/+bug/1659694 > > Newer versions of qemu/libvirt support setting the GIC version to "host" > which allows the VM to use whatever the host supports as long as the guest > supports. Trusty guests will not boot on a GICv3 host. Also "host" as a > GIC element’s version attribute isn't supported by the virt stack that > ships with trusty. > > One final issue is that there is no qemu-efi package built for trusty. It > is feasible that someone could build their own ppa for use. Other options > include: > > - adding qemu-efi to trusty-backports which is not easy to use, > - push a stable release update back to trusty, which is a regression risk > for x86 by updating the source that also builds ovmf, > - introduce qemu-efi in trusty which would require an exception from the > security team. > > > There are a few scenarios which involve deploying trusty. > > - juju bootstrap a-cloud arm64-controller --default-series trusty > This would a require a trusty cloud image with hwe-x kernel > pre-installed. > - juju add-machine --series trusty > This would require the underlying provider to boot with hwe-x kernel > minimum. This also seems like it is going to fail to deploy with the > wily kernel. > - juju deploy wordpress --to kvm:0 > This would require the machine-0 to be able to install qemu-efi and > it > would also require a trusty cloud image with hwe-x kernel > pre-installed. > > > > So given all the possible knobs one would need to turn to make it work, > should we: > > - block it from working in juju and return an error noting that trusty on > ARM64 isn’t supported? > > This in conjunction with the deploy scenario in the last section seems to > be a real issue. One could decide to deploy only xenial hosts and guests on > ARM64, but it would hobble the usefulness of quite a few charms that > require trusty or bundles that require such charms. This seems draconian > when selecting a particular minimum kernel in MAAS could make it work -- > that is if qemu-efi was available and there weren’t hardware version issues. > > - warn that it won't work without a custom setup and document it as a > known issue pointing in to a wiki page in the warning/error that contains > the information above? > - something else/other ideas? > > > Cheers, > Reed > > -- Reed O'Brien ✉ [email protected] ✆ 415-562-6797 💻 redir
-- Juju-dev mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
