The SEV memory encryption works without problems. Parameters for 'cbitpos' and 'reduced-phys-bits' were successfully and automatically applied, when not supplied in the config. I could run up to 15 SEV-enabled VMs concurrently, which is in line with the expected maximum capabilities of a 1st gen Epyc CPU. Memory settings in the range of 512MiB to 56GiB all worked for me without problems.
Tested on an AMD Epyc 7351P with a SuperMicro H11SSL-i Motherboard (Bios Revision 1.3) I tried to get SEV-ES running but even though my CPU signals support for it, starting VMs with the feature enabled always yields following error: > kvm: sev_kvm_init: failed to initialize ret=-25 fw_error=0 '' > kvm: failed to initialize kvm: Operation not permitted > kvm: falling back to tcg > kvm: ../softmmu/vl.c:2738: qemu_machine_creation_done: Assertion `machine->cgs->ready' failed. I suspect that this feature is simply not supported by my motherboard/BIOS yet, but did not find the actual cause for this problem. I did not test SEV-SNP as it seems to be a 3rd gen Epyc CPU feature. Tested-by: Daniel Tschlatscher <d.tschlatsc...@proxmox.com> Some more comments inline in the code, all of which are non-breaking though: On 6/9/22 13:14, Markus Frank wrote: > This Patch is for enabling AMD SEV (Secure Encrypted Virtualization) support > in > QEMU and for supporting other memory encryption technologies like INTEL MKTME > (Multi-key Total Memory Encryption) and AMD-SNP in the future. > > Config-Example: > memory_encryption: type=sev,cbitpos=47,policy=0x0005,reduced-phys-bits=1 > > "reduced-phys-bios" and "cbitpos" are system specific and can be read out with > QMP. If not set by the user, a dummy-vm gets started to read QMP for these > variables out and save them to config. Afterwards, the dummy-vm gets stopped. > > For a more detailed Explanation plus Requirements & Limitations > see my coherent pve-docs patch and the qemu documentation. > > Signed-off-by: Markus Frank <m.fr...@proxmox.com> > --- > I could not test SEV-ES because I get a similar error to > https://www.mail-archive.com/devel@edk2.groups.io/msg38521.html > But I still get the same error on master and mentioned patched versions. > On some older versions I just get "kvm: SEV-ES reset address is zero > kvm: failed to locate and/or save reset vector" > > Maybe I will report my error to the edk2 project or find a way to fix > it, when I know more about it. > > I also could not test SEV-SNP, because I do not have a EPYC 7003 to test > on and there is also no support in the Linux Kernel yet. SEV-SNP support > should be in Linux 5.19: > https://www.phoronix.com/scan.php?page=news_item&px=AMD-SEV-SNP-Arrives-Linux-5.19 > patched kernel-fork: https://github.com/AMDESE/linux/tree/sev-snp-5.18-rc3 > > PVE/QemuServer.pm | 133 ++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 133 insertions(+) > > diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm > index e9aa248..abc21d4 100644 > --- a/PVE/QemuServer.pm > +++ b/PVE/QemuServer.pm > @@ -174,6 +174,55 @@ my $agent_fmt = { > }, > }; > > +my $memory_encryption_fmt = { > + type => { > + type => 'string', > + default_key => 1, > + description => "Memory Encryption Type:" > + ."for AMD SEV -> 'memory_encryption: type=sev'" > + ."for AMD SEV-SNP -> 'memory_encryption: type=sev-snp'" > + ."for AMD SEV-ES -> use 'sev' and change policy to between 0x4 and > 0x7" > + ."(Bit-2 has to be set 1 (LSB 0 bit numbering))" > + ."(sev requires edk2-ovmf & on guest: up-to-date kernel + sev > support &" > + ."on host: add kernel-parameters 'mem_encrypt=on kvm_amd.sev=1')" > + ."see https://github.com/AMDESE/AMDSEV";, > + format_description => "qemu-memory-encryption-type", > + #pattern => '(sev|sev-snp|mktme)', > + pattern => '(sev|sev-snp)', > + default => "", > + maxLength => 10, > + }, > + 'reduced-phys-bits' => { > + description => "Number of bits the physical address space is reduced > by. System dependent", > + type => 'integer', > + default => 1, > + optional => 1, > + minium => 0, Says "minium" instead of "minimum" > + maximum => 100, > + }, > + cbitpos => { > + description => "C-bit: marks if a memory page is protected. System > dependent", > + type => 'integer', > + default => 47, > + optional => 1, > + minium => 0, Same as above > + maximum => 100, > + }, > + policy => { > + description => "SEV Guest Policy" > + ."see Capter 3:" > + > ."https://www.amd.com/system/files/TechDocs/55766_SEV-KM_API_Specification.pdf"; > + ."& > https://www.qemu.org/docs/master/system/i386/amd-memory-encryption.html";, > + format_description => "qemu-memory-encryption-policy", > + type => 'string', > + default => '0x0000', > + pattern => '0[xX][0-9a-fA-F]{1,4}', > + optional => 1, > + maxLength => 6, > + }, > +}; > +PVE::JSONSchema::register_format('pve-qemu-memory-encryption-fmt', > $memory_encryption_fmt); > + > my $vga_fmt = { > type => { > description => "Select the VGA type.", > @@ -348,6 +397,12 @@ my $confdesc = { > minimum => 16, > default => 512, > }, > + memory_encryption => { > + description => "Memory Encryption", > + optional => 1, > + format => 'pve-qemu-memory-encryption-fmt', > + type => 'string', > + }, > balloon => { > optional => 1, > type => 'integer', > @@ -2107,6 +2162,16 @@ sub parse_guest_agent { > return $res; > } > > +sub parse_memory_encryption { > + my ($value) = @_; > + > + return if !$value; > + > + my $res = eval { parse_property_string($memory_encryption_fmt, $value) }; > + warn $@ if $@; > + return $res; > +} > + > sub get_qga_key { > my ($conf, $key) = @_; > return undef if !defined($conf->{agent}); > @@ -4079,6 +4144,74 @@ sub config_to_command { > } > push @$machineFlags, "type=${machine_type_min}"; > > + # Memory Encryption configuration > + my $memory_encryption = > parse_memory_encryption($conf->{'memory_encryption'}); > + > + # Die if bios is not ovmf > + if ( > + $conf->{'memory_encryption'} > + && $memory_encryption->{type} eq 'sev' > + && !$conf->{bios} eq 'ovmf' !$conf->{bios} eq 'ovmf' will never yield true (did you forget the parenthesis?) and die with the error This creates a problem where the VM status changes to running but noVNC will forever show "Guest has not initialized display yet" when run with anything other than OVMF > + ) { > + die "SEV needs ovmf"; Nit: \n at the end here to suppress the "at .../QemuServer.pm line xxxx" message > + } > + > + # AMD SEV > + if ($conf->{'memory_encryption'} && $memory_encryption->{type} =~ > /(sev|sev-snp)/) { > + # Get reduced-phys-bits & cbitpos from QMP, if not set > + if ( > + !$memory_encryption->{'reduced-phys-bits'} > + || !$memory_encryption->{cbitpos} > + ) { > + my $fakevmid = -1; > + my $qemu_cmd = get_command_for_arch($arch); > + my $pidfile = PVE::QemuServer::Helpers::pidfile_name($fakevmid); > + my $default_machine = $default_machines->{$arch}; > + my $cmd = [ > + $qemu_cmd, > + '-machine', $default_machine, > + '-display', 'none', > + '-chardev', > "socket,id=qmp,path=/var/run/qemu-server/$fakevmid.qmp,server=on,wait=off", > + '-mon', 'chardev=qmp,mode=control', > + '-pidfile', $pidfile, > + '-S', '-daemonize' > + ]; > + if (!$kvm) { > + push @$cmd, '-accel', 'tcg'; > + } > + my $rc = run_command($cmd, noerr => 1, quiet => 0); > + die "QEMU flag querying VM exited with code " . $rc if $rc; > + my $res = mon_cmd($fakevmid, 'query-sev-capabilities'); > + $memory_encryption->{'reduced-phys-bits'} = > $res->{'reduced-phys-bits'}; > + $memory_encryption->{cbitpos} = $res->{cbitpos}; > + $conf->{'memory_encryption'} = > PVE::JSONSchema::print_property_string( > + $memory_encryption, > + $memory_encryption_fmt > + ); > + PVE::QemuConfig->write_config($vmid, $conf); > + vm_stop(undef, $fakevmid, 1, 1, 10, 0, 1); > + } > + > + # qemu-Example: -object > 'sev-guest,id=sev0,cbitpos=47,reduced-phys-bits=1'; > + # see > https://www.qemu.org/docs/master/system/i386/amd-memory-encryption.html > + my $memobjcmd = ""; > + if ($memory_encryption->{type} eq 'sev-snp') { > + # https://github.com/AMDESE/AMDSEV/tree/sev-snp-devel > + $memobjcmd = $memobjcmd.'sev-snp-guest,'; > + } else { > + $memobjcmd = $memobjcmd.'sev-guest,'; > + } > + $memobjcmd = $memobjcmd . > 'id=sev0,cbitpos='.$memory_encryption->{cbitpos} > + .',reduced-phys-bits='.$memory_encryption->{'reduced-phys-bits'}; > + if ($memory_encryption->{type} eq 'sev' && > $memory_encryption->{policy}) { > + $memobjcmd = $memobjcmd.',policy='.$memory_encryption->{policy} > + } > + push @$devices, '-object' , $memobjcmd; > + # old qemu-Example: -machine 'type=q35+pve0,memory-encryption=sev0' > + # > https://fossies.org/linux/qemu/docs/system/confidential-guest-support.rst > + push @$machineFlags, 'confidential-guest-support=sev0'; > + } > + > push @$cmd, @$devices; > push @$cmd, '-rtc', join(',', @$rtcFlags) if scalar(@$rtcFlags); > push @$cmd, '-machine', join(',', @$machineFlags) if > scalar(@$machineFlags); _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
