I see wrong vars configured for secure VAR. *<nvram template='/usr/share/OVMF/OVMF_VARS.fd'> *
It should be something like "/usr/share/OVMF/OVMF_CODE.secboot.fd:/usr/share/OVMF/OVMF_VARS.fd", or the file should be like path to the OVMF_VARS.secboot.fd inside uefi.properties on the Ubuntu Host. I hope this helps. Thanks & Regards, Pavan Aravapalli. Architect. https://www.linkedin.com/in/pavan-a-70995a27/ On Fri, 20 Jan 2023 at 16:01, Gary Dixon <gary.di...@quadris.co.uk.invalid> wrote: > I think this is possibly a bug in CS 4.15.2 with KVM hypervisor on Ubuntu > 20.04 > > > > I have evicted one of our hosts from the cloudstack cluster and added the > /etc/cloudstack/agent/uefi.properties file. > > > > Cleared out the keystore and set the libvirtd.conf file back to > listen_tls=0, listen_tcp=1 and re-added the host back in to the cluster in > Cloudstack > > > > In the agent logs I can see that it detects the uefi.properties file and > enumerates the paths. > > > > The host is added back into Cloudstack – but in the database in the > “host_details” table I see the “host.uefi.enable” value is set to “false” > for this host ? > > > > We then manually set “host.uefi.enable” to true in the database > > > > I then provision a new instance and use a Windows Server2016 ISO to > provision the machine on this uefi enabled host. I set the adv settings to > BIOS: UEFI BOOT MODE: Secure > > The VM starts but when I console on to it there is an error message on the > console window saying “*Guest has not initialized the display (yet)”* > > So at this point it appears we are unable to create any VM’s with uefi – > secure boot enabled > > > > Has anyone suucessfully managed to get Windows VM’s with uefi secure boot > enabled working in Cloudstack 4.15.2 with KVM hypervisor on Ubuntu 20.04 > hosts ? > > > > > > A virsh dumpxml shows this: > > > > <description>Windows Server 2016 (64-bit)</description> > > <memory unit='KiB'>8388608</memory> > > <currentMemory unit='KiB'>8388608</currentMemory> > > <vcpu placement='static'>4</vcpu> > > <cputune> > > <shares>3240</shares> > > </cputune> > > <resource> > > <partition>/machine</partition> > > </resource> > > <sysinfo type='smbios'> > > <system> > > <entry name='manufacturer'>Apache Software Foundation</entry> > > <entry name='product'>CloudStack KVM Hypervisor</entry> > > <entry name='uuid'>39c9fa33-0ef2-463a-aff6-45b6e77d1c4d</entry> > > </system> > > </sysinfo> > > <os> > > <type arch='x86_64' machine='pc-q35-4.2'>hvm</type> > > <loader readonly='yes' secure='yes' > type='pflash'>/usr/share/OVMF/OVMF_CODE.secboot.fd</loader> > > <nvram > template='/usr/share/OVMF/OVMF_VARS.fd'>/var/lib/libvirt/qemu/nvram/39c9fa33-0ef2-463a-aff6-45b6e77d1c4d.fd</nvram> > > <boot dev='cdrom'/> > > <boot dev='hd'/> > > <smbios mode='sysinfo'/> > > </os> > > <features> > > <acpi/> > > <apic/> > > <pae/> > > <smm state='on'/> > > </features> > > <cpu mode='host-passthrough' check='none'> > > <topology sockets='1' cores='4' threads='1'/> > > </cpu> > > <clock offset='localtime'> > > <timer name='hypervclock' present='yes'/> > > </clock> > > <on_poweroff>destroy</on_poweroff> > > <on_reboot>restart</on_reboot> > > <on_crash>destroy</on_crash> > > <devices> > > <emulator>/usr/bin/qemu-system-x86_64</emulator> > > <disk type='block' device='disk'> > > <driver name='qemu' type='raw' cache='none'/> > > <source dev='/dev/storpool-byid/n91t.b.brrdr' index='2'/> > > <backingStore/> > > <target dev='sda' bus='sata'/> > > <serial>69bcfffc3c8a41ab876b</serial> > > <alias name='sata0-0-0'/> > > <address type='drive' controller='0' bus='0' target='0' unit='0'/> > > </disk> > > <disk type='file' device='cdrom'> > > <driver name='qemu' type='raw'/> > > <source > file='/mnt/45d6d957-afa2-371a-b0dc-b6e70ef17d97/035fa65a-4556-47b0-95c1-ac2db8ee054e.iso' > index='1'/> > > <backingStore/> > > <target dev='sdd' bus='sata'/> > > <readonly/> > > <alias name='sata0-0-3'/> > > <address type='drive' controller='0' bus='0' target='0' unit='3'/> > > </disk> > > <controller type='usb' index='0' model='qemu-xhci'> > > <alias name='usb'/> > > <address type='pci' domain='0x0000' bus='0x03' slot='0x00' > function='0x0'/> > > </controller> > > <controller type='sata' index='0'> > > <alias name='ide'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' > function='0x2'/> > > </controller> > > <controller type='pci' index='0' model='pcie-root'> > > <alias name='pcie.0'/> > > </controller> > > <controller type='virtio-serial' index='0'> > > <alias name='virtio-serial0'/> > > <address type='pci' domain='0x0000' bus='0x04' slot='0x00' > function='0x0'/> > > </controller> > > <controller type='pci' index='1' model='pcie-root-port'> > > <model name='pcie-root-port'/> > > <target chassis='1' port='0x10'/> > > <alias name='pci.1'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x0' multifunction='on'/> > > </controller> > > <controller type='pci' index='2' model='pcie-to-pci-bridge'> > > <model name='pcie-pci-bridge'/> > > <alias name='pci.2'/> > > <address type='pci' domain='0x0000' bus='0x01' slot='0x00' > function='0x0'/> > > </controller> > > <controller type='pci' index='3' model='pcie-root-port'> > > <model name='pcie-root-port'/> > > <target chassis='3' port='0x11'/> > > <alias name='pci.3'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x1'/> > > </controller> > > <controller type='pci' index='4' model='pcie-root-port'> > > <model name='pcie-root-port'/> > > <target chassis='4' port='0x12'/> > > <alias name='pci.4'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x2'/> > > </controller> > > <controller type='pci' index='5' model='pcie-root-port'> > > <model name='pcie-root-port'/> > > <target chassis='5' port='0x13'/> > > <alias name='pci.5'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x3'/> > > </controller> > > <interface type='bridge'> > > <mac address='02:00:0d:ea:00:0f'/> > > <source bridge='brvx-17906'/> > > <bandwidth> > > <inbound average='25600' peak='25600'/> > > <outbound average='25600' peak='25600'/> > > </bandwidth> > > <target dev='vnet0'/> > > <model type='e1000'/> > > <link state='up'/> > > <alias name='net0'/> > > <address type='pci' domain='0x0000' bus='0x02' slot='0x01' > function='0x0'/> > > </interface> > > <serial type='pty'> > > <source path='/dev/pts/2'/> > > <target type='isa-serial' port='0'> > > <model name='isa-serial'/> > > </target> > > <alias name='serial0'/> > > </serial> > > <console type='pty' tty='/dev/pts/2'> > > <source path='/dev/pts/2'/> > > <target type='serial' port='0'/> > > <alias name='serial0'/> > > </console> > > <channel type='unix'> > > <source mode='bind' > path='/var/lib/libvirt/qemu/i-2-1811-VM.org.qemu.guest_agent.0'/> > > <target type='virtio' name='org.qemu.guest_agent.0' > state='disconnected'/> > > <alias name='channel0'/> > > <address type='virtio-serial' controller='0' bus='0' port='1'/> > > </channel> > > <input type='tablet' bus='usb'> > > <alias name='input0'/> > > <address type='usb' bus='0' port='1'/> > > </input> > > <input type='mouse' bus='ps2'> > > <alias name='input1'/> > > </input> > > <input type='keyboard' bus='ps2'> > > <alias name='input2'/> > > </input> > > <graphics type='vnc' port='5900' autoport='yes' listen='10.255.4.14'> > > <listen type='address' address='10.255.4.14'/> > > </graphics> > > <video> > > <model type='cirrus' vram='16384' heads='1' primary='yes'/> > > <alias name='video0'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x01' > function='0x0'/> > > </video> > > <watchdog model='i6300esb' action='none'> > > <alias name='watchdog0'/> > > <address type='pci' domain='0x0000' bus='0x02' slot='0x02' > function='0x0'/> > > </watchdog> > > <memballoon model='none'/> > > </devices> > > <seclabel type='dynamic' model='dac' relabel='yes'> > > <label>+0:+0</label> > > <imagelabel>+0:+0</imagelabel> > > </seclabel> > > </domain> > > > > > > > > > Gary Dixon > Senior Technical Consultant > T: +44 161 537 4990 > E: *v* <+44%207989717661>ms@quadris‑support.com > W: www.quadris.co.uk > The information contained in this e-mail from Quadris may be confidential > and privileged for the private use of the named recipient. The contents of > this e-mail may not necessarily represent the official views of Quadris. > If you have received this information in error you must not copy, > distribute or take any action or reliance on its contents. Please destroy > any hard copies and delete this message. > > *From:* Gary Dixon <gary.di...@quadris.co.uk.INVALID> > *Sent:* 19 January 2023 14:35 > *To:* users@cloudstack.apache.org > *Subject:* RE: KVM host UEFI allow guest UEFI Secure boot > > > > I think I just solved this myself – in the qemu.conf file I see : > > > > #nvram = [ > > # "/usr/share/OVMF/OVMF_CODE.fd:/usr/share/OVMF/OVMF_VARS.fd", > > # "/usr/share/OVMF/OVMF_CODE.secboot.fd:/usr/share/OVMF/OVMF_VARS.fd", > > # "/usr/share/AAVMF/AAVMF_CODE.fd:/usr/share/AAVMF/AAVMF_VARS.fd", > > # "/usr/share/AAVMF/AAVMF32_CODE.fd:/usr/share/AAVMF/AAVMF32_VARS.fd", > > # "/usr/share/OVMF/OVMF_CODE.ms.fd:/usr/share/OVMF/OVMF_VARS.ms.fd" > > #] > > > > So in Ubuntu 20.04 there is no reference to OVMF_VARS.secure.fd for the > nvram template > > > > > > *Gary Dixon*** > > Senior Technical Consultant > > T: +44 161 537 4990 > > E: *v* <+44%207989717661>ms@quadris‑support.com > > W: www.quadris.co.uk > > *The information contained in this e-mail from Quadris may be confidential > and privileged for the private use of the named recipient. The contents of > this e-mail may not necessarily represent the official views of Quadris. > If you have received this information in error you must not copy, > distribute or take any action or reliance on its contents. Please destroy > any hard copies and delete this message.* > > *From:* Gary Dixon <gary.di...@quadris.co.uk.INVALID> > *Sent:* 19 January 2023 13:55 > *To:* users@cloudstack.apache.org > *Subject:* RE: KVM host UEFI allow guest UEFI Secure boot > > > > Thanks for all your quick responses > > On our Ubuntu 20.04 hosts it appears that the OVMF files are located in > "/usr/share/OVMF/" directory - however the OVMF_VARS.secboot.fd file is not > there ? : > > root@qcloud-s2-p1-c1-kvm4:~# ls -al /usr/share/OVMF/ > total 4232 > drwxr-xr-x 2 root root 4096 Mar 9 2022 . > drwxr-xr-x 151 root root 4096 Apr 2 2022 .. > -rw-r--r-- 1 root root 1966080 Sep 20 2021 OVMF_CODE.fd > lrwxrwxrwx 1 root root 20 Sep 20 2021 OVMF_CODE.ms.fd -> > OVMF_CODE.secboot.fd > -rw-r--r-- 1 root root 1966080 Sep 20 2021 OVMF_CODE.secboot.fd > -rw-r--r-- 1 root root 131072 Sep 20 2021 OVMF_VARS.fd > -rw-r--r-- 1 root root 131072 Sep 20 2021 OVMF_VARS.ms.fd > -rw-r--r-- 1 root root 131072 Sep 20 2021 OVMF_VARS.snakeoil.fd > > Is this needed in the uefi.properties config file ? > > > BR > > Gary > > *Gary Dixon*** > > Senior Technical Consultant > > T: +44 161 537 4990 > > E: *v* <+44%207989717661>ms@quadris‑support.com > > W: www.quadris.co.uk > <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.quadris.co.uk%2F&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7Cbdeec2af0e4a4a6774a908dafa2a65c4%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C1%7C638097357263190623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=TpoRJ6htE7TVpNmxq8hpoKarJKQOXM2Z1qY%2FWqA%2BuA8%3D&reserved=0> > > *The information contained in this e-mail from Quadris may be confidential > and privileged for the private use of the named recipient. The contents of > this e-mail may not necessarily represent the official views of Quadris. > If you have received this information in error you must not copy, > distribute or take any action or reliance on its contents. Please destroy > any hard copies and delete this message.* > > -----Original Message----- > From: vas...@gmx.de <vas...@gmx.de> > Sent: 19 January 2023 13:42 > To: users@cloudstack.apache.org > Subject: Re: KVM host UEFI allow guest UEFI Secure boot > > Not the direct solution but maybe some bits of information for your further > efforts: > > Overall description of the feature > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FCLOUDSTACK%2FEnable%2BUEFI%2Bbooting%2Bfor%2BInstance&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7C8057c1b2e3bd4f13beae08dafa231af3%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C0%7C638097325927612509%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=8Q5jWBGmCYA82hk6NmrVESq%2F%2BwkdzSKKn9MbJsPjA%2BM%3D&reserved=0 > <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FCLOUDSTACK%2FEnable%2BUEFI%2Bbooting%2Bfor%2BInstance&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7Cbdeec2af0e4a4a6774a908dafa2a65c4%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C1%7C638097357263190623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=oge%2BrXNhgInAvEh6kBBXqT4Rbo8hpIb73LgiXT8JlMI%3D&reserved=0> > > User guide + example to enable secure boot > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flab.piszki.pl%2Fcloudstack-vm-with-vtpm-and-secure-boot-uefi%2F&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7C8057c1b2e3bd4f13beae08dafa231af3%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C0%7C638097325927612509%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=o312PhI9IqAYJEgn8dY4EQliP4p4W4Ry9iJ4XuKsSVA%3D&reserved=0 > <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flab.piszki.pl%2Fcloudstack-vm-with-vtpm-and-secure-boot-uefi%2F&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7Cbdeec2af0e4a4a6774a908dafa2a65c4%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C1%7C638097357263190623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=m49P0WhvyeipIRcp%2FmgnijIBR5ohIUPoWFCKm5RrBho%3D&reserved=0> > > Gitlab - Issue with further informations on deploying that capability > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcloudstack%2Fissues%2F4238&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7C8057c1b2e3bd4f13beae08dafa231af3%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C0%7C638097325927612509%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HDMzobnzON4SpjRT9VZFXNtvd7RMpVluNwjcF1TQDvo%3D&reserved=0 > <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcloudstack%2Fissues%2F4238&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7Cbdeec2af0e4a4a6774a908dafa2a65c4%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C1%7C638097357263190623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=D86CHfpq3hTR366f%2FmxjdJO4J03it5LKR6HFxDrGSsg%3D&reserved=0> > > regards, > Chris > > Am Do., 19. Jan. 2023 um 14:09 Uhr schrieb Gary Dixon > <gary.di...@quadris.co.uk.invalid>: > > > Hi everyone > > > > > > > > CS : 4.15.2 > > > > Hypervisor: KVM > > > > OS: Ubuntu 20.04 > > > > > > > > Apologies if this has been discussed before. > > > > We have a requirement to create Windows server templates with UEFI > > Secure boot enabled and in testing find that our instances are being > > created with Legacy BIOS enabled. > > > > I checked our KVM hosts and they have the ovmf package installed – > > however there is no uefi.properties file in the /etc/cloudstack/agent > > directory > > > > How do I enable the KVM hosts to support Cloudstack guests with UEFI > > Secure boot bios ? > > > > Also will this ‘break’ all current running VM’s that have the Legacy > > BIOS enabled or will they still be able to run ? > > > > > > > > BR > > > > > > > > Gary > > Gary Dixon > > Senior Technical Consultant > > T: +44 161 537 4990 > > E: *v* <+44%207989717661>ms@quadris‑support.com > <ms@quadris%1esupport.com> > > W: > > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.q > <https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.q%2F&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7Cbdeec2af0e4a4a6774a908dafa2a65c4%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C1%7C638097357263190623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=SMOS6Ij7daZPgT0%2FwodZB9TqTVZMeyy4%2BaTtrucvbrs%3D&reserved=0> > > uadris.co.uk%2F&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7C8057c1b2e3 > > bd4f13beae08dafa231af3%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C0%7C63 > > 8097325927612509%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV > > 2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=M9uXGY9aAT > > 4z8oYezjiqrFQ6%2FH9nDV4ZmDOXn6RxUB4%3D&reserved=0 > > The information contained in this e-mail from Quadris may be > > confidential and privileged for the private use of the named > > recipient. The contents of this e-mail may not necessarily represent the > official views of Quadris. > > If you have received this information in error you must not copy, > > distribute or take any action or reliance on its contents. Please > > destroy any hard copies and delete this message. > > >
