I think this is possibly a bug in CS 4.15.2 with KVM hypervisor on Ubuntu 20.04
I have evicted one of our hosts from the cloudstack cluster and added the
/etc/cloudstack/agent/uefi.properties file.
Cleared out the keystore and set the libvirtd.conf file back to listen_tls=0,
listen_tcp=1 and re-added the host back in to the cluster in Cloudstack
In the agent logs I can see that it detects the uefi.properties file and
enumerates the paths.
The host is added back into Cloudstack – but in the database in the
“host_details” table I see the “host.uefi.enable” value is set to “false” for
this host ?
We then manually set “host.uefi.enable” to true in the database
I then provision a new instance and use a Windows Server2016 ISO to provision
the machine on this uefi enabled host. I set the adv settings to BIOS: UEFI
BOOT MODE: Secure
The VM starts but when I console on to it there is an error message on the
console window saying “Guest has not initialized the display (yet)”
So at this point it appears we are unable to create any VM’s with uefi – secure
boot enabled
Has anyone suucessfully managed to get Windows VM’s with uefi secure boot
enabled working in Cloudstack 4.15.2 with KVM hypervisor on Ubuntu 20.04 hosts ?
A virsh dumpxml shows this:
<description>Windows Server 2016 (64-bit)</description>
<memory unit='KiB'>8388608</memory>
<currentMemory unit='KiB'>8388608</currentMemory>
<vcpu placement='static'>4</vcpu>
<cputune>
<shares>3240</shares>
</cputune>
<resource>
<partition>/machine</partition>
</resource>
<sysinfo type='smbios'>
<system>
<entry name='manufacturer'>Apache Software Foundation</entry>
<entry name='product'>CloudStack KVM Hypervisor</entry>
<entry name='uuid'>39c9fa33-0ef2-463a-aff6-45b6e77d1c4d</entry>
</system>
</sysinfo>
<os>
<type arch='x86_64' machine='pc-q35-4.2'>hvm</type>
<loader readonly='yes' secure='yes'
type='pflash'>/usr/share/OVMF/OVMF_CODE.secboot.fd</loader>
<nvram
template='/usr/share/OVMF/OVMF_VARS.fd'>/var/lib/libvirt/qemu/nvram/39c9fa33-0ef2-463a-aff6-45b6e77d1c4d.fd</nvram>
<boot dev='cdrom'/>
<boot dev='hd'/>
<smbios mode='sysinfo'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
<smm state='on'/>
</features>
<cpu mode='host-passthrough' check='none'>
<topology sockets='1' cores='4' threads='1'/>
</cpu>
<clock offset='localtime'>
<timer name='hypervclock' present='yes'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source dev='/dev/storpool-byid/n91t.b.brrdr' index='2'/>
<backingStore/>
<target dev='sda' bus='sata'/>
<serial>69bcfffc3c8a41ab876b</serial>
<alias name='sata0-0-0'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
<disk type='file' device='cdrom'>
<driver name='qemu' type='raw'/>
<source
file='/mnt/45d6d957-afa2-371a-b0dc-b6e70ef17d97/035fa65a-4556-47b0-95c1-ac2db8ee054e.iso'
index='1'/>
<backingStore/>
<target dev='sdd' bus='sata'/>
<readonly/>
<alias name='sata0-0-3'/>
<address type='drive' controller='0' bus='0' target='0' unit='3'/>
</disk>
<controller type='usb' index='0' model='qemu-xhci'>
<alias name='usb'/>
<address type='pci' domain='0x0000' bus='0x03' slot='0x00'
function='0x0'/>
</controller>
<controller type='sata' index='0'>
<alias name='ide'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1f'
function='0x2'/>
</controller>
<controller type='pci' index='0' model='pcie-root'>
<alias name='pcie.0'/>
</controller>
<controller type='virtio-serial' index='0'>
<alias name='virtio-serial0'/>
<address type='pci' domain='0x0000' bus='0x04' slot='0x00'
function='0x0'/>
</controller>
<controller type='pci' index='1' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='1' port='0x10'/>
<alias name='pci.1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'
multifunction='on'/>
</controller>
<controller type='pci' index='2' model='pcie-to-pci-bridge'>
<model name='pcie-pci-bridge'/>
<alias name='pci.2'/>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00'
function='0x0'/>
</controller>
<controller type='pci' index='3' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='3' port='0x11'/>
<alias name='pci.3'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x1'/>
</controller>
<controller type='pci' index='4' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='4' port='0x12'/>
<alias name='pci.4'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x2'/>
</controller>
<controller type='pci' index='5' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='5' port='0x13'/>
<alias name='pci.5'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x3'/>
</controller>
<interface type='bridge'>
<mac address='02:00:0d:ea:00:0f'/>
<source bridge='brvx-17906'/>
<bandwidth>
<inbound average='25600' peak='25600'/>
<outbound average='25600' peak='25600'/>
</bandwidth>
<target dev='vnet0'/>
<model type='e1000'/>
<link state='up'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x02' slot='0x01'
function='0x0'/>
</interface>
<serial type='pty'>
<source path='/dev/pts/2'/>
<target type='isa-serial' port='0'>
<model name='isa-serial'/>
</target>
<alias name='serial0'/>
</serial>
<console type='pty' tty='/dev/pts/2'>
<source path='/dev/pts/2'/>
<target type='serial' port='0'/>
<alias name='serial0'/>
</console>
<channel type='unix'>
<source mode='bind'
path='/var/lib/libvirt/qemu/i-2-1811-VM.org.qemu.guest_agent.0'/>
<target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
<alias name='channel0'/>
<address type='virtio-serial' controller='0' bus='0' port='1'/>
</channel>
<input type='tablet' bus='usb'>
<alias name='input0'/>
<address type='usb' bus='0' port='1'/>
</input>
<input type='mouse' bus='ps2'>
<alias name='input1'/>
</input>
<input type='keyboard' bus='ps2'>
<alias name='input2'/>
</input>
<graphics type='vnc' port='5900' autoport='yes' listen='10.255.4.14'>
<listen type='address' address='10.255.4.14'/>
</graphics>
<video>
<model type='cirrus' vram='16384' heads='1' primary='yes'/>
<alias name='video0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x0'/>
</video>
<watchdog model='i6300esb' action='none'>
<alias name='watchdog0'/>
<address type='pci' domain='0x0000' bus='0x02' slot='0x02'
function='0x0'/>
</watchdog>
<memballoon model='none'/>
</devices>
<seclabel type='dynamic' model='dac' relabel='yes'>
<label>+0:+0</label>
<imagelabel>+0:+0</imagelabel>
</seclabel>
</domain>
Gary Dixon
Senior Technical Consultant
T: +44 161 537 4990
E: v...@quadris-support.com
W: www.quadris.co.uk
The information contained in this e-mail from Quadris may be confidential and
privileged for the private use of the named recipient. The contents of this
e-mail may not necessarily represent the official views of Quadris. If you
have received this information in error you must not copy, distribute or take
any action or reliance on its contents. Please destroy any hard copies and
delete this message.
From: Gary Dixon <gary.di...@quadris.co.uk.INVALID>
Sent: 19 January 2023 14:35
To: users@cloudstack.apache.org
Subject: RE: KVM host UEFI allow guest UEFI Secure boot
I think I just solved this myself – in the qemu.conf file I see :
#nvram = [
# "/usr/share/OVMF/OVMF_CODE.fd:/usr/share/OVMF/OVMF_VARS.fd",
# "/usr/share/OVMF/OVMF_CODE.secboot.fd:/usr/share/OVMF/OVMF_VARS.fd",
# "/usr/share/AAVMF/AAVMF_CODE.fd:/usr/share/AAVMF/AAVMF_VARS.fd",
# "/usr/share/AAVMF/AAVMF32_CODE.fd:/usr/share/AAVMF/AAVMF32_VARS.fd",
# "/usr/share/OVMF/OVMF_CODE.ms.fd:/usr/share/OVMF/OVMF_VARS.ms.fd"
#]
So in Ubuntu 20.04 there is no reference to OVMF_VARS.secure.fd for the nvram
template
Gary Dixon
Senior Technical Consultant
T: +44 161 537 4990
E: v<tel:+44%207989717661>ms@quadris‑support.com
W: www.quadris.co.uk<http://www.quadris.co.uk>
[cid:image828463.png@1B150A60.0CBE8265]
The information contained in this e-mail from Quadris may be confidential and
privileged for the private use of the named recipient. The contents of this
e-mail may not necessarily represent the official views of Quadris. If you
have received this information in error you must not copy, distribute or take
any action or reliance on its contents. Please destroy any hard copies and
delete this message.
From: Gary Dixon
<gary.di...@quadris.co.uk.INVALID<mailto:gary.di...@quadris.co.uk.INVALID>>
Sent: 19 January 2023 13:55
To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>
Subject: RE: KVM host UEFI allow guest UEFI Secure boot
Thanks for all your quick responses
On our Ubuntu 20.04 hosts it appears that the OVMF files are located in
"/usr/share/OVMF/" directory - however the OVMF_VARS.secboot.fd file is not
there ? :
root@qcloud-s2-p1-c1-kvm4:~# ls -al /usr/share/OVMF/
total 4232
drwxr-xr-x 2 root root 4096 Mar 9 2022 .
drwxr-xr-x 151 root root 4096 Apr 2 2022 ..
-rw-r--r-- 1 root root 1966080 Sep 20 2021 OVMF_CODE.fd
lrwxrwxrwx 1 root root 20 Sep 20 2021 OVMF_CODE.ms.fd -> OVMF_CODE.secboot.fd
-rw-r--r-- 1 root root 1966080 Sep 20 2021 OVMF_CODE.secboot.fd
-rw-r--r-- 1 root root 131072 Sep 20 2021 OVMF_VARS.fd
-rw-r--r-- 1 root root 131072 Sep 20 2021 OVMF_VARS.ms.fd
-rw-r--r-- 1 root root 131072 Sep 20 2021 OVMF_VARS.snakeoil.fd
Is this needed in the uefi.properties config file ?
BR
Gary
Gary Dixon
Senior Technical Consultant
T: +44 161 537 4990
E: v<tel:+44%207989717661>ms@quadris‑support.com
W:
www.quadris.co.uk<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.quadris.co.uk%2F&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7Cbdeec2af0e4a4a6774a908dafa2a65c4%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C1%7C638097357263190623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=TpoRJ6htE7TVpNmxq8hpoKarJKQOXM2Z1qY%2FWqA%2BuA8%3D&reserved=0>
[cid:image385073.png@E0A53755.B8760DA1]
The information contained in this e-mail from Quadris may be confidential and
privileged for the private use of the named recipient. The contents of this
e-mail may not necessarily represent the official views of Quadris. If you
have received this information in error you must not copy, distribute or take
any action or reliance on its contents. Please destroy any hard copies and
delete this message.
-----Original Message-----
From: vas...@gmx.de<mailto:vas...@gmx.de> <vas...@gmx.de<mailto:vas...@gmx.de>>
Sent: 19 January 2023 13:42
To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>
Subject: Re: KVM host UEFI allow guest UEFI Secure boot
Not the direct solution but maybe some bits of information for your further
efforts:
Overall description of the feature
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FCLOUDSTACK%2FEnable%2BUEFI%2Bbooting%2Bfor%2BInstance&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7C8057c1b2e3bd4f13beae08dafa231af3%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C0%7C638097325927612509%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=8Q5jWBGmCYA82hk6NmrVESq%2F%2BwkdzSKKn9MbJsPjA%2BM%3D&reserved=0<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FCLOUDSTACK%2FEnable%2BUEFI%2Bbooting%2Bfor%2BInstance&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7Cbdeec2af0e4a4a6774a908dafa2a65c4%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C1%7C638097357263190623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=oge%2BrXNhgInAvEh6kBBXqT4Rbo8hpIb73LgiXT8JlMI%3D&reserved=0>
User guide + example to enable secure boot
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flab.piszki.pl%2Fcloudstack-vm-with-vtpm-and-secure-boot-uefi%2F&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7C8057c1b2e3bd4f13beae08dafa231af3%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C0%7C638097325927612509%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=o312PhI9IqAYJEgn8dY4EQliP4p4W4Ry9iJ4XuKsSVA%3D&reserved=0<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flab.piszki.pl%2Fcloudstack-vm-with-vtpm-and-secure-boot-uefi%2F&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7Cbdeec2af0e4a4a6774a908dafa2a65c4%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C1%7C638097357263190623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=m49P0WhvyeipIRcp%2FmgnijIBR5ohIUPoWFCKm5RrBho%3D&reserved=0>
Gitlab - Issue with further informations on deploying that capability
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcloudstack%2Fissues%2F4238&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7C8057c1b2e3bd4f13beae08dafa231af3%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C0%7C638097325927612509%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HDMzobnzON4SpjRT9VZFXNtvd7RMpVluNwjcF1TQDvo%3D&reserved=0<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcloudstack%2Fissues%2F4238&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7Cbdeec2af0e4a4a6774a908dafa2a65c4%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C1%7C638097357263190623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=D86CHfpq3hTR366f%2FmxjdJO4J03it5LKR6HFxDrGSsg%3D&reserved=0>
regards,
Chris
Am Do., 19. Jan. 2023 um 14:09 Uhr schrieb Gary Dixon
<gary.di...@quadris.co.uk.invalid<mailto:gary.di...@quadris.co.uk.invalid>>:
> Hi everyone
>
>
>
> CS : 4.15.2
>
> Hypervisor: KVM
>
> OS: Ubuntu 20.04
>
>
>
> Apologies if this has been discussed before.
>
> We have a requirement to create Windows server templates with UEFI
> Secure boot enabled and in testing find that our instances are being
> created with Legacy BIOS enabled.
>
> I checked our KVM hosts and they have the ovmf package installed –
> however there is no uefi.properties file in the /etc/cloudstack/agent
> directory
>
> How do I enable the KVM hosts to support Cloudstack guests with UEFI
> Secure boot bios ?
>
> Also will this ‘break’ all current running VM’s that have the Legacy
> BIOS enabled or will they still be able to run ?
>
>
>
> BR
>
>
>
> Gary
> Gary Dixon
> Senior Technical Consultant
> T: +44 161 537 4990
> E: *v*
> <+44%207989717661>ms@quadris‑support.com<mailto:ms@quadris%1esupport.com>
> W:
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.q<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.q%2F&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7Cbdeec2af0e4a4a6774a908dafa2a65c4%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C1%7C638097357263190623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=SMOS6Ij7daZPgT0%2FwodZB9TqTVZMeyy4%2BaTtrucvbrs%3D&reserved=0>
> uadris.co.uk%2F&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7C8057c1b2e3
> bd4f13beae08dafa231af3%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C0%7C63
> 8097325927612509%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV
> 2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=M9uXGY9aAT
> 4z8oYezjiqrFQ6%2FH9nDV4ZmDOXn6RxUB4%3D&reserved=0
> The information contained in this e-mail from Quadris may be
> confidential and privileged for the private use of the named
> recipient. The contents of this e-mail may not necessarily represent the
> official views of Quadris.
> If you have received this information in error you must not copy,
> distribute or take any action or reliance on its contents. Please
> destroy any hard copies and delete this message.
>
