Re: [PATCH 0/3] KVM: x86: guest interface for SEV live migration

[Ashish Kalra]

On Tue, Apr 20, 2021 at 05:31:07PM +, Sean Christopherson wrote:
> On Tue, Apr 20, 2021, Paolo Bonzini wrote:
> > From ef78673f78e3f2eedc498c1fbf9271146caa83cb Mon Sep 17 00:00:00 2001
> > From: Ashish Kalra 
> > Date: Thu, 15 Apr 2021 15:57:02 +
> > Subject: [PATCH 2/3] KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall
> > 
> > This hypercall is used by the SEV guest to notify a change in the page
> > encryption status to the hypervisor. The hypercall should be invoked
> > only when the encryption attribute is changed from encrypted -> decrypted
> > and vice versa. By default all guest pages are considered encrypted.
> > 
> > The hypercall exits to userspace to manage the guest shared regions and
> > integrate with the userspace VMM's migration code.
> 
> ...
> 
> > diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
> > index fd4a84911355..2bc353d1f356 100644
> > --- a/Documentation/virt/kvm/api.rst
> > +++ b/Documentation/virt/kvm/api.rst
> > @@ -6766,3 +6766,14 @@ they will get passed on to user space. So user space 
> > still has to have
> >  an implementation for these despite the in kernel acceleration.
> >  
> >  This capability is always enabled.
> > +
> > +8.32 KVM_CAP_EXIT_HYPERCALL
> > +---
> > +
> > +:Capability: KVM_CAP_EXIT_HYPERCALL
> > +:Architectures: x86
> > +:Type: vm
> > +
> > +This capability, if enabled, will cause KVM to exit to userspace
> > +with KVM_EXIT_HYPERCALL exit reason to process some hypercalls.
> > +Right now, the only such hypercall is KVM_HC_PAGE_ENC_STATUS.
> > diff --git a/Documentation/virt/kvm/cpuid.rst 
> > b/Documentation/virt/kvm/cpuid.rst
> > index cf62162d4be2..c9378d163b5a 100644
> > --- a/Documentation/virt/kvm/cpuid.rst
> > +++ b/Documentation/virt/kvm/cpuid.rst
> > @@ -96,6 +96,11 @@ KVM_FEATURE_MSI_EXT_DEST_ID15  guest 
> > checks this feature bit
> > before using extended 
> > destination
> > ID bits in MSI address bits 
> > 11-5.
> >  
> > +KVM_FEATURE_HC_PAGE_ENC_STATUS 16  guest checks this feature 
> > bit before
> > +   using the page encryption 
> > state
> > +   hypercall to notify the 
> > page state
> > +   change
> 
> ...
> 
> >  int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
> >  {
> > unsigned long nr, a0, a1, a2, a3, ret;
> > @@ -8334,6 +8346,28 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
> > kvm_sched_yield(vcpu, a0);
> > ret = 0;
> > break;
> > +   case KVM_HC_PAGE_ENC_STATUS: {
> > +   u64 gpa = a0, npages = a1, enc = a2;
> > +
> > +   ret = -KVM_ENOSYS;
> > +   if (!vcpu->kvm->arch.hypercall_exit_enabled)
> 
> I don't follow, why does the hypercall need to be gated by a capability?  What
> would break if this were changed to?
> 
>   if (!guest_pv_has(vcpu, KVM_FEATURE_HC_PAGE_ENC_STATUS))
> 

But, the above indicates host support for page_enc_status_hc, so we want
to ensure that host supports and has enabled support for the hypercall
exit, i.e., hypercall has been enabled.

Thanks,
Ashish

> > +   break;
> > +
> > +   if (!PAGE_ALIGNED(gpa) || !npages ||
> > +   gpa_to_gfn(gpa) + npages <= gpa_to_gfn(gpa)) {
> > +   ret = -EINVAL;
> > +   break;
> > +   }
> > +
> > +   vcpu->run->exit_reason= KVM_EXIT_HYPERCALL;
> > +   vcpu->run->hypercall.nr   = KVM_HC_PAGE_ENC_STATUS;
> > +   vcpu->run->hypercall.args[0]  = gpa;
> > +   vcpu->run->hypercall.args[1]  = npages;
> > +   vcpu->run->hypercall.args[2]  = enc;
> > +   vcpu->run->hypercall.longmode = op_64_bit;
> > +   vcpu->arch.complete_userspace_io = complete_hypercall_exit;
> > +   return 0;
> > +   }
> > default:
> > ret = -KVM_ENOSYS;
> > break;
> 
> ...
> 
> > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> > index 590cc811c99a..d696a9f13e33 100644
> > --- a/arch/x86/kvm/x86.c
> > +++ b/arch/x86/kvm/x86.c
> > @@ -3258,6 +3258,14 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, str

[PATCH] staging: wlan-ng: silence incorrect type in argument 1 (different address spaces) warning

[Ashish Kalra]

Upon running sparse, "warning: incorrect type in argument 1 (different address 
spaces)
is brought to notice for this file.let's add correct typecast to make it 
cleaner and
silence the Sparse warning.

Signed-off-by: Ashish Kalra 
---
 drivers/staging/wlan-ng/p80211netdev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/wlan-ng/p80211netdev.c 
b/drivers/staging/wlan-ng/p80211netdev.c
index 6f9666dc0277..70570e8a5ad2 100644
--- a/drivers/staging/wlan-ng/p80211netdev.c
+++ b/drivers/staging/wlan-ng/p80211netdev.c
@@ -569,7 +569,7 @@ static int p80211knetdev_do_ioctl(struct net_device *dev,
goto bail;
}
 
-   msgbuf = memdup_user(req->data, req->len);
+   msgbuf = memdup_user((void __user *)req->data, req->len);
if (IS_ERR(msgbuf)) {
result = PTR_ERR(msgbuf);
goto bail;
-- 
2.25.1

Re: [PATCH] staging: wfx: silence symbol 'wfx_get_ps_timeout' was not declared warning

[Ashish Kalra]

On Mon, Apr 19, 2021 at 05:55:34PM +0200, Jérôme Pouiller wrote:
> On Monday 19 April 2021 17:33:48 CEST Ashish Kalra wrote:
> > 
> > Upon running sparse, "warning: symbol 'wfx_get_ps_timeout' was not declared.
> > Should it be static?" and "warning: symbol 'wfx_update_pm' was not declared.
> > Should it be static?" is brought to notice for this file.  static keyword
> > should be added to prevent this warning. let's add it to make it cleaner and
> > silence the Sparse warning.
> 
> Hi Ashish,
> 
> Thank you for your contribution.
> 
> It seems that this issue is already fixed by commit ce59858bbc10 "staging:
> wfx: make methods 'wfx_get_ps_timeout' and 'wfx_update_pm' static" (merged
> in master in version 5.8). Can you check you are working on the last tree?
> 
> 
> -- 
> Jérôme Pouiller
> 
> 
Thanks Jerome
It seems some sync issue with my source code.I will update to 
latest and correct

[PATCH] staging: wfx: silence symbol 'wfx_get_ps_timeout' was not declared warning

[Ashish Kalra]

Upon running sparse, "warning: symbol 'wfx_get_ps_timeout' was not declared.
Should it be static?" and "warning: symbol 'wfx_update_pm' was not declared.
Should it be static?" is brought to notice for this file.  static keyword
should be added to prevent this warning. let's add it to make it cleaner and
silence the Sparse warning.

Signed-off-by: Ashish Kalra 
---
 drivers/staging/wfx/sta.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/wfx/sta.c b/drivers/staging/wfx/sta.c
index 12e8a5b638f1..d855d87c2102 100644
--- a/drivers/staging/wfx/sta.c
+++ b/drivers/staging/wfx/sta.c
@@ -200,7 +200,7 @@ void wfx_configure_filter(struct ieee80211_hw *hw, unsigned 
int changed_flags,
mutex_unlock(>conf_mutex);
 }
 
-int wfx_get_ps_timeout(struct wfx_vif *wvif, bool *enable_ps)
+static int wfx_get_ps_timeout(struct wfx_vif *wvif, bool *enable_ps)
 {
struct ieee80211_channel *chan0 = NULL, *chan1 = NULL;
struct ieee80211_conf *conf = >wdev->hw->conf;
@@ -230,7 +230,7 @@ int wfx_get_ps_timeout(struct wfx_vif *wvif, bool 
*enable_ps)
return -1;
 }
 
-int wfx_update_pm(struct wfx_vif *wvif)
+static int wfx_update_pm(struct wfx_vif *wvif)
 {
int ps_timeout;
bool ps;
-- 
2.25.1

Re: [PATCH v13 00/12] Add AMD SEV guest live migration support

[Ashish Kalra]

On Fri, Apr 16, 2021 at 02:43:48PM -0700, Steve Rutherford wrote:
> On Thu, Apr 15, 2021 at 8:52 AM Ashish Kalra  wrote:
> >
> > From: Ashish Kalra 
> >
> > The series add support for AMD SEV guest live migration commands. To 
> > protect the
> > confidentiality of an SEV protected guest memory while in transit we need to
> > use the SEV commands defined in SEV API spec [1].
> >
> > SEV guest VMs have the concept of private and shared memory. Private memory
> > is encrypted with the guest-specific key, while shared memory may be 
> > encrypted
> > with hypervisor key. The commands provided by the SEV FW are meant to be 
> > used
> > for the private memory only. The patch series introduces a new hypercall.
> > The guest OS can use this hypercall to notify the page encryption status.
> > If the page is encrypted with guest specific-key then we use SEV command 
> > during
> > the migration. If page is not encrypted then fallback to default.
> >
> > The patch uses the KVM_EXIT_HYPERCALL exitcode and hypercall to
> > userspace exit functionality as a common interface from the guest back to 
> > the
> > VMM and passing on the guest shared/unencrypted page information to the
> > userspace VMM/Qemu. Qemu can consult this information during migration to 
> > know
> > whether the page is encrypted.
> >
> > This section descibes how the SEV live migration feature is negotiated
> > between the host and guest, the host indicates this feature support via
> > KVM_FEATURE_CPUID. The guest firmware (OVMF) detects this feature and
> > sets a UEFI enviroment variable indicating OVMF support for live
> > migration, the guest kernel also detects the host support for this
> > feature via cpuid and in case of an EFI boot verifies if OVMF also
> > supports this feature by getting the UEFI enviroment variable and if it
> > set then enables live migration feature on host by writing to a custom
> > MSR, if not booted under EFI, then it simply enables the feature by
> > again writing to the custom MSR. The MSR is also handled by the
> > userspace VMM/Qemu.
> >
> > A branch containing these patches is available here:
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAMDESE%2Flinux%2Ftree%2Fsev-migration-v13data=04%7C01%7CAshish.Kalra%40amd.com%7C7bee6d5c907b46d0998508d90120ce2d%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637542063133830260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=FkKrciL41GDNyNDqrPMVblRa%2FaReogW4OzhbYaSYs04%3Dreserved=0
> >
> > [1] 
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.amd.com%2Fwp-content%2Fresources%2F55766.PDFdata=04%7C01%7CAshish.Kalra%40amd.com%7C7bee6d5c907b46d0998508d90120ce2d%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637542063133830260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=%2FLFBR9ean0acMmR8WTLUHZsAynYPRAa7%2FeZEVVdpCo8%3Dreserved=0
> >
> > Changes since v12:
> > - Reset page encryption status during early boot instead of just
> >   before the kexec to avoid SMP races during kvm_pv_guest_cpu_reboot().
> 
> Does this series need to disable the MSR during kvm_pv_guest_cpu_reboot()?
> 

Yes, i think that will make sense, it will be similar to the first time
VM boot where the MSR will be disabled till it is enabled at early
kernel boot. I will add this to the current patch series.

Thanks,
Ashish

> I _think_ going into blackout during the window after restart, but
> before the MSR is explicitly reenabled, would cause corruption. The
> historical shared pages could be re-allocated as non-shared pages
> during restart.
> 
> Steve

Re: [PATCH] media: atomisp: silence "dubious: !x | !y" warning

[Ashish Kalra]

On Sat, Apr 17, 2021 at 08:56:13PM +0200, Mauro Carvalho Chehab wrote:
> Em Sat, 17 Apr 2021 21:06:27 +0530
> Ashish Kalra  escreveu:
> 
> > Upon running sparse, "warning: dubious: !x | !y" is brought to notice
> > for this file.  Logical and bitwise OR are basically the same in this
> > context so it doesn't cause a runtime bug.  But let's change it to
> > logical OR to make it cleaner and silence the Sparse warning.
> > 
> > Signed-off-by: Ashish Kalra 
> > ---
> >  .../media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c| 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git 
> > a/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c 
> > b/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
> > index 358cb7d2cd4c..3b850bb2d39d 100644
> > --- 
> > a/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
> > +++ 
> > b/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
> > @@ -58,7 +58,7 @@ sh_css_vf_downscale_log2(
> > unsigned int ds_log2 = 0;
> > unsigned int out_width;
> >  
> > -   if ((!out_info) | (!vf_info))
> > +   if ((!out_info) || (!vf_info))
> 
> 
> While here, please get rid of the unneeded parenthesis:
> 
>   if (!out_info || !vf_info)
> 
> 
> > return -EINVAL;
> >  
> >     out_width = out_info->res.width;
> 
> 
> 
> Thanks,
> Mauro
Updated Patch as per your feedback

Thanks
Ashish
>From 770317157c3a7abf2fda1d71b0bd651c33bf0bfa Mon Sep 17 00:00:00 2001
From: Ashish Kalra 
Date: Sun, 18 Apr 2021 06:52:03 +0530
Subject: [PATCH] media: atomisp: silence "dubious: !x | !y" warning

Upon running sparse, "warning: dubious: !x | !y" is brought to notice
for this file.  Logical and bitwise OR are basically the same in this
context so it doesn't cause a runtime bug.  But let's change it to
logical OR to make it cleaner and silence the Sparse warning.

Signed-off-by: Ashish Kalra 
---
 .../media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c| 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c b/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
index 358cb7d2cd4c..71c3e7dac052 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
@@ -58,7 +58,7 @@ sh_css_vf_downscale_log2(
 	unsigned int ds_log2 = 0;
 	unsigned int out_width;
 
-	if ((!out_info) | (!vf_info))
+	if (!out_info || !vf_info)
 		return -EINVAL;
 
 	out_width = out_info->res.width;
-- 
2.25.1

Re: [PATCH] media: atomisp: silence "dubious: !x | !y" warning

[Ashish Kalra]

On Sat, Apr 17, 2021 at 09:31:32PM +, David Laight wrote:
> From: Mauro Carvalho Chehab
> > Sent: 17 April 2021 19:56
> > 
> > Em Sat, 17 Apr 2021 21:06:27 +0530
> > Ashish Kalra  escreveu:
> > 
> > > Upon running sparse, "warning: dubious: !x | !y" is brought to notice
> > > for this file.  Logical and bitwise OR are basically the same in this
> > > context so it doesn't cause a runtime bug.  But let's change it to
> > > logical OR to make it cleaner and silence the Sparse warning.
> 
> The old code is very likely to by slightly more efficient.
> 
> It may not matter here, but it might in a really hot path.
> 
> Since !x | !y and !x || !y always have the same value
> why is sparse complaining at all.
> 
>   David
This warning is coming from sparse as per below explanation

As the name suggests, a "bitwise" expression is one that is restricted to
only a certain "bitwise" operations that make sense within that class. In
particular, you can't mix a "bitwise" class with a normal integer
expression
Warning:
int __bitwise i;
int __bitwise j;
the two variables "i" and "j" are _not_ compatible, simply because they
were declared separately, while in the case of
int __bitwise i, j;
they _are_ compatible.

https://yarchive.net/comp/linux/sparse.html
> 
> > >
> > > Signed-off-by: Ashish Kalra 
> > > ---
> > >  .../media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c| 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git 
> > > a/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
> > b/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
> > > index 358cb7d2cd4c..3b850bb2d39d 100644
> > > --- 
> > > a/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
> > > +++ 
> > > b/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
> > > @@ -58,7 +58,7 @@ sh_css_vf_downscale_log2(
> > >   unsigned int ds_log2 = 0;
> > >   unsigned int out_width;
> > >
> > > - if ((!out_info) | (!vf_info))
> > > + if ((!out_info) || (!vf_info))
> > 
> > 
> > While here, please get rid of the unneeded parenthesis:
> > 
> > if (!out_info || !vf_info)
> > 
> > 
> > >   return -EINVAL;
> > >
> > >   out_width = out_info->res.width;
> > 
> > 
> > 
> > Thanks,
> > Mauro
> 
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 
> 1PT, UK
> Registration No: 1397386 (Wales)
> 

[PATCH] media: atomisp: silence "dubious: !x | !y" warning

[Ashish Kalra]

Upon running sparse, "warning: dubious: !x | !y" is brought to notice
for this file.  Logical and bitwise OR are basically the same in this
context so it doesn't cause a runtime bug.  But let's change it to
logical OR to make it cleaner and silence the Sparse warning.

Signed-off-by: Ashish Kalra 
---
 .../media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c| 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 
a/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c 
b/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
index 358cb7d2cd4c..3b850bb2d39d 100644
--- a/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
+++ b/drivers/staging/media/atomisp/pci/isp/kernels/vf/vf_1.0/ia_css_vf.host.c
@@ -58,7 +58,7 @@ sh_css_vf_downscale_log2(
unsigned int ds_log2 = 0;
unsigned int out_width;
 
-   if ((!out_info) | (!vf_info))
+   if ((!out_info) || (!vf_info))
return -EINVAL;
 
out_width = out_info->res.width;
-- 
2.25.1

[PATCH v13 12/12] x86/kvm: Add guest support for detecting and enabling SEV Live Migration feature.

[Ashish Kalra]

From: Ashish Kalra 

The guest support for detecting and enabling SEV Live migration
feature uses the following logic :

 - kvm_init_plaform() invokes check_kvm_sev_migration() which
   checks if its booted under the EFI

   - If not EFI,

 i) check for the KVM_FEATURE_CPUID

 ii) if CPUID reports that migration is supported, issue a wrmsrl()
 to enable the SEV live migration support

   - If EFI,

 i) check for the KVM_FEATURE_CPUID

 ii) If CPUID reports that migration is supported, read the UEFI variable 
which
 indicates OVMF support for live migration

 iii) the variable indicates live migration is supported, issue a wrmsrl() 
to
  enable the SEV live migration support

The EFI live migration check is done using a late_initcall() callback.

Also, ensure that _bss_decrypted section is marked as decrypted in the
shared pages list.

Also adds kexec support for SEV Live Migration.

Reset the host's shared pages list related to kernel
specific page encryption status settings before we load a
new kernel by kexec. We cannot reset the complete
shared pages list here as we need to retain the
UEFI/OVMF firmware specific settings.

The host's shared pages list is maintained for the
guest to keep track of all unencrypted guest memory regions,
therefore we need to explicitly mark all shared pages as
encrypted again before rebooting into the new guest kernel.

Signed-off-by: Ashish Kalra 
---
 arch/x86/include/asm/mem_encrypt.h |  8 
 arch/x86/kernel/kvm.c  | 55 +
 arch/x86/mm/mem_encrypt.c  | 64 ++
 3 files changed, 127 insertions(+)

diff --git a/arch/x86/include/asm/mem_encrypt.h 
b/arch/x86/include/asm/mem_encrypt.h
index 31c4df123aa0..19b77f3a62dc 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -21,6 +21,7 @@
 extern u64 sme_me_mask;
 extern u64 sev_status;
 extern bool sev_enabled;
+extern bool sev_live_migration_enabled;
 
 void sme_encrypt_execute(unsigned long encrypted_kernel_vaddr,
 unsigned long decrypted_kernel_vaddr,
@@ -44,8 +45,11 @@ void __init sme_enable(struct boot_params *bp);
 
 int __init early_set_memory_decrypted(unsigned long vaddr, unsigned long size);
 int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size);
+void __init early_set_mem_enc_dec_hypercall(unsigned long vaddr, int npages,
+   bool enc);
 
 void __init mem_encrypt_free_decrypted_mem(void);
+void __init check_kvm_sev_migration(void);
 
 /* Architecture __weak replacement functions */
 void __init mem_encrypt_init(void);
@@ -60,6 +64,7 @@ bool sev_es_active(void);
 #else  /* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define sme_me_mask0ULL
+#define sev_live_migration_enabled false
 
 static inline void __init sme_early_encrypt(resource_size_t paddr,
unsigned long size) { }
@@ -84,8 +89,11 @@ static inline int __init
 early_set_memory_decrypted(unsigned long vaddr, unsigned long size) { return 
0; }
 static inline int __init
 early_set_memory_encrypted(unsigned long vaddr, unsigned long size) { return 
0; }
+static inline void __init
+early_set_mem_enc_dec_hypercall(unsigned long vaddr, int npages, bool enc) {}
 
 static inline void mem_encrypt_free_decrypted_mem(void) { }
+static inline void check_kvm_sev_migration(void) { }
 
 #define __bss_decrypted
 
diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index 78bb0fae3982..94ef16d263a7 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -26,6 +26,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -429,6 +430,59 @@ static inline void __set_percpu_decrypted(void *ptr, 
unsigned long size)
early_set_memory_decrypted((unsigned long) ptr, size);
 }
 
+static int __init setup_kvm_sev_migration(void)
+{
+   efi_char16_t efi_sev_live_migration_enabled[] = 
L"SevLiveMigrationEnabled";
+   efi_guid_t efi_variable_guid = MEM_ENCRYPT_GUID;
+   efi_status_t status;
+   unsigned long size;
+   bool enabled;
+
+   /*
+* check_kvm_sev_migration() invoked via kvm_init_platform() before
+* this callback would have setup the indicator that live migration
+* feature is supported/enabled.
+*/
+   if (!sev_live_migration_enabled)
+   return 0;
+
+   if (!efi_enabled(EFI_BOOT))
+   return 0;
+
+   if (!efi_enabled(EFI_RUNTIME_SERVICES)) {
+   pr_info("%s : EFI runtime services are not enabled\n", 
__func__);
+   return 0;
+   }
+
+   size = sizeof(enabled);
+
+   /* Get variable contents into buffer */
+   status = efi.get_variable(efi_sev_live_migration_enabled,
+ _variable_guid, NULL, , );
+
+   if (status == EFI_NOT_FOUND) {
+   pr_info("%s : EF

[PATCH v13 11/12] EFI: Introduce the new AMD Memory Encryption GUID.

[Ashish Kalra]

From: Ashish Kalra 

Introduce a new AMD Memory Encryption GUID which is currently
used for defining a new UEFI environment variable which indicates
UEFI/OVMF support for the SEV live migration feature. This variable
is setup when UEFI/OVMF detects host/hypervisor support for SEV
live migration and later this variable is read by the kernel using
EFI runtime services to verify if OVMF supports the live migration
feature.

Signed-off-by: Ashish Kalra 
Reviewed-by: Steve Rutherford 
---
 include/linux/efi.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/linux/efi.h b/include/linux/efi.h
index 6b5d36babfcc..6f364ace82cb 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -362,6 +362,7 @@ void efi_native_runtime_setup(void);
 
 /* OEM GUIDs */
 #define DELLEMC_EFI_RCI2_TABLE_GUIDEFI_GUID(0x2d9f28a2, 0xa886, 
0x456a,  0x97, 0xa8, 0xf1, 0x1e, 0xf2, 0x4f, 0xf4, 0x55)
+#define MEM_ENCRYPT_GUID   EFI_GUID(0x0cf29b71, 0x9e51, 
0x433a,  0xa3, 0xb7, 0x81, 0xf3, 0xab, 0x16, 0xb8, 0x75)
 
 typedef struct {
efi_guid_t guid;
-- 
2.17.1

[PATCH v13 10/12] KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature & Custom MSR.

[Ashish Kalra]

From: Ashish Kalra 

Add new KVM_FEATURE_SEV_LIVE_MIGRATION feature for guest to check
for host-side support for SEV live migration. Also add a new custom
MSR_KVM_SEV_LIVE_MIGRATION for guest to enable the SEV live migration
feature.

MSR is handled by userspace using MSR filters.

Signed-off-by: Ashish Kalra 
Reviewed-by: Steve Rutherford 
---
 Documentation/virt/kvm/cpuid.rst |  5 +
 Documentation/virt/kvm/msr.rst   | 12 
 arch/x86/include/uapi/asm/kvm_para.h |  4 
 arch/x86/kvm/cpuid.c |  3 ++-
 4 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/Documentation/virt/kvm/cpuid.rst b/Documentation/virt/kvm/cpuid.rst
index cf62162d4be2..0bdb6cdb12d3 100644
--- a/Documentation/virt/kvm/cpuid.rst
+++ b/Documentation/virt/kvm/cpuid.rst
@@ -96,6 +96,11 @@ KVM_FEATURE_MSI_EXT_DEST_ID15  guest checks 
this feature bit
before using extended 
destination
ID bits in MSI address bits 
11-5.
 
+KVM_FEATURE_SEV_LIVE_MIGRATION 16  guest checks this feature bit 
before
+   using the page encryption state
+   hypercall to notify the page 
state
+   change
+
 KVM_FEATURE_CLOCKSOURCE_STABLE_BIT 24  host will warn if no guest-side
per-cpu warps are expected in
kvmclock
diff --git a/Documentation/virt/kvm/msr.rst b/Documentation/virt/kvm/msr.rst
index e37a14c323d2..020245d16087 100644
--- a/Documentation/virt/kvm/msr.rst
+++ b/Documentation/virt/kvm/msr.rst
@@ -376,3 +376,15 @@ data:
write '1' to bit 0 of the MSR, this causes the host to re-scan its queue
and check if there are more notifications pending. The MSR is available
if KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
+
+MSR_KVM_SEV_LIVE_MIGRATION:
+0x4b564d08
+
+   Control SEV Live Migration features.
+
+data:
+Bit 0 enables (1) or disables (0) host-side SEV Live Migration feature,
+in other words, this is guest->host communication that it's properly
+handling the shared pages list.
+
+All other bits are reserved.
diff --git a/arch/x86/include/uapi/asm/kvm_para.h 
b/arch/x86/include/uapi/asm/kvm_para.h
index 950afebfba88..f6bfa138874f 100644
--- a/arch/x86/include/uapi/asm/kvm_para.h
+++ b/arch/x86/include/uapi/asm/kvm_para.h
@@ -33,6 +33,7 @@
 #define KVM_FEATURE_PV_SCHED_YIELD 13
 #define KVM_FEATURE_ASYNC_PF_INT   14
 #define KVM_FEATURE_MSI_EXT_DEST_ID15
+#define KVM_FEATURE_SEV_LIVE_MIGRATION 16
 
 #define KVM_HINTS_REALTIME  0
 
@@ -54,6 +55,7 @@
 #define MSR_KVM_POLL_CONTROL   0x4b564d05
 #define MSR_KVM_ASYNC_PF_INT   0x4b564d06
 #define MSR_KVM_ASYNC_PF_ACK   0x4b564d07
+#define MSR_KVM_SEV_LIVE_MIGRATION 0x4b564d08
 
 struct kvm_steal_time {
__u64 steal;
@@ -136,4 +138,6 @@ struct kvm_vcpu_pv_apf_data {
 #define KVM_PV_EOI_ENABLED KVM_PV_EOI_MASK
 #define KVM_PV_EOI_DISABLED 0x0
 
+#define KVM_SEV_LIVE_MIGRATION_ENABLED BIT_ULL(0)
+
 #endif /* _UAPI_ASM_X86_KVM_PARA_H */
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 6bd2f8b830e4..4e2e69a692aa 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -812,7 +812,8 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array 
*array, u32 function)
 (1 << KVM_FEATURE_PV_SEND_IPI) |
 (1 << KVM_FEATURE_POLL_CONTROL) |
 (1 << KVM_FEATURE_PV_SCHED_YIELD) |
-(1 << KVM_FEATURE_ASYNC_PF_INT);
+(1 << KVM_FEATURE_ASYNC_PF_INT) |
+(1 << KVM_FEATURE_SEV_LIVE_MIGRATION);
 
if (sched_info_on())
entry->eax |= (1 << KVM_FEATURE_STEAL_TIME);
-- 
2.17.1

[PATCH v13 09/12] mm: x86: Invoke hypercall when page encryption status is changed

[Ashish Kalra]

From: Brijesh Singh 

Invoke a hypercall when a memory region is changed from encrypted ->
decrypted and vice versa. Hypervisor needs to know the page encryption
status during the guest migration.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Reviewed-by: Venu Busireddy 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 arch/x86/include/asm/paravirt.h   | 10 +
 arch/x86/include/asm/paravirt_types.h |  2 +
 arch/x86/kernel/paravirt.c|  1 +
 arch/x86/mm/mem_encrypt.c | 57 ++-
 arch/x86/mm/pat/set_memory.c  |  7 
 5 files changed, 76 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index 4abf110e2243..efaa3e628967 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -84,6 +84,12 @@ static inline void paravirt_arch_exit_mmap(struct mm_struct 
*mm)
PVOP_VCALL1(mmu.exit_mmap, mm);
 }
 
+static inline void page_encryption_changed(unsigned long vaddr, int npages,
+   bool enc)
+{
+   PVOP_VCALL3(mmu.page_encryption_changed, vaddr, npages, enc);
+}
+
 #ifdef CONFIG_PARAVIRT_XXL
 static inline void load_sp0(unsigned long sp0)
 {
@@ -799,6 +805,10 @@ static inline void paravirt_arch_dup_mmap(struct mm_struct 
*oldmm,
 static inline void paravirt_arch_exit_mmap(struct mm_struct *mm)
 {
 }
+
+static inline void page_encryption_changed(unsigned long vaddr, int npages, 
bool enc)
+{
+}
 #endif
 #endif /* __ASSEMBLY__ */
 #endif /* _ASM_X86_PARAVIRT_H */
diff --git a/arch/x86/include/asm/paravirt_types.h 
b/arch/x86/include/asm/paravirt_types.h
index de87087d3bde..69ef9c207b38 100644
--- a/arch/x86/include/asm/paravirt_types.h
+++ b/arch/x86/include/asm/paravirt_types.h
@@ -195,6 +195,8 @@ struct pv_mmu_ops {
 
/* Hook for intercepting the destruction of an mm_struct. */
void (*exit_mmap)(struct mm_struct *mm);
+   void (*page_encryption_changed)(unsigned long vaddr, int npages,
+   bool enc);
 
 #ifdef CONFIG_PARAVIRT_XXL
struct paravirt_callee_save read_cr2;
diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index c60222ab8ab9..9f206e192f6b 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -335,6 +335,7 @@ struct paravirt_patch_template pv_ops = {
(void (*)(struct mmu_gather *, void *))tlb_remove_page,
 
.mmu.exit_mmap  = paravirt_nop,
+   .mmu.page_encryption_changed= paravirt_nop,
 
 #ifdef CONFIG_PARAVIRT_XXL
.mmu.read_cr2   = __PV_IS_CALLEE_SAVE(native_read_cr2),
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index ae78cef79980..fae9ccbd0da7 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -19,6 +19,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -29,6 +30,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "mm_internal.h"
 
@@ -229,6 +231,47 @@ void __init sev_setup_arch(void)
swiotlb_adjust_size(size);
 }
 
+static void set_memory_enc_dec_hypercall(unsigned long vaddr, int npages,
+   bool enc)
+{
+   unsigned long sz = npages << PAGE_SHIFT;
+   unsigned long vaddr_end, vaddr_next;
+
+   vaddr_end = vaddr + sz;
+
+   for (; vaddr < vaddr_end; vaddr = vaddr_next) {
+   int psize, pmask, level;
+   unsigned long pfn;
+   pte_t *kpte;
+
+   kpte = lookup_address(vaddr, );
+   if (!kpte || pte_none(*kpte))
+   return;
+
+   switch (level) {
+   case PG_LEVEL_4K:
+   pfn = pte_pfn(*kpte);
+   break;
+   case PG_LEVEL_2M:
+   pfn = pmd_pfn(*(pmd_t *)kpte);
+   break;
+   case PG_LEVEL_1G:
+   pfn = pud_pfn(*(pud_t *)kpte);
+   break;
+   default:
+   return;
+   }
+
+   psize = page_level_size(level);
+   pmask = page_level_mask(level);
+
+   kvm_sev_hypercall3(KVM_HC_PAGE_ENC_STATUS,
+  pfn << PAGE_SHIFT, psize >> PAGE_SHIFT, enc);
+
+   vaddr_next = (vaddr & pmask) + psize;
+   }
+}
+
 static void __init __set_clr_pte_enc(pte_t *kpte, int level, bool enc)
 {
pgprot_t old_prot, new_prot;
@@ -286,12 +329,13 @@ static void __init __set_clr_pte_enc(pte_t *kpte, int 
level, bool enc)
 static int __init early_set_memory_enc_dec(unsigned long vaddr,
   

[PATCH v13 08/12] KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall

[Ashish Kalra]

From: Ashish Kalra 

This hypercall is used by the SEV guest to notify a change in the page
encryption status to the hypervisor. The hypercall should be invoked
only when the encryption attribute is changed from encrypted -> decrypted
and vice versa. By default all guest pages are considered encrypted.

The hypercall exits to userspace to manage the guest shared regions and
integrate with the userspace VMM's migration code.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
Co-developed-by: Sean Christopherson 
Signed-off-by: Sean Christopherson 
---
 Documentation/virt/kvm/hypercalls.rst | 15 ++
 arch/x86/include/asm/kvm_host.h   |  2 ++
 arch/x86/kvm/svm/sev.c|  1 +
 arch/x86/kvm/x86.c| 29 +++
 include/uapi/linux/kvm_para.h |  1 +
 5 files changed, 48 insertions(+)

diff --git a/Documentation/virt/kvm/hypercalls.rst 
b/Documentation/virt/kvm/hypercalls.rst
index ed4fddd364ea..7aff0cebab7c 100644
--- a/Documentation/virt/kvm/hypercalls.rst
+++ b/Documentation/virt/kvm/hypercalls.rst
@@ -169,3 +169,18 @@ a0: destination APIC ID
 
 :Usage example: When sending a call-function IPI-many to vCPUs, yield if
any of the IPI target vCPUs was preempted.
+
+
+8. KVM_HC_PAGE_ENC_STATUS
+-
+:Architecture: x86
+:Status: active
+:Purpose: Notify the encryption status changes in guest page table (SEV guest)
+
+a0: the guest physical address of the start page
+a1: the number of pages
+a2: encryption attribute
+
+   Where:
+   * 1: Encryption attribute is set
+   * 0: Encryption attribute is cleared
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 3768819693e5..42eb0fe3df5d 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1050,6 +1050,8 @@ struct kvm_arch {
 
bool bus_lock_detection_enabled;
 
+   bool page_enc_hc_enable;
+
/* Deflect RDMSR and WRMSR to user space when they trigger a #GP */
u32 user_space_msr_mask;
struct kvm_x86_msr_filter __rcu *msr_filter;
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index c9795a22e502..5184a0c0131a 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -197,6 +197,7 @@ static int sev_guest_init(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
sev->active = true;
sev->asid = asid;
INIT_LIST_HEAD(>regions_list);
+   kvm->arch.page_enc_hc_enable = true;
 
return 0;
 
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index f7d12fca397b..e8986478b653 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8208,6 +8208,13 @@ static void kvm_sched_yield(struct kvm *kvm, unsigned 
long dest_id)
kvm_vcpu_yield_to(target);
 }
 
+static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
+{
+   kvm_rax_write(vcpu, vcpu->run->hypercall.ret);
+   ++vcpu->stat.hypercalls;
+   return kvm_skip_emulated_instruction(vcpu);
+}
+
 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
 {
unsigned long nr, a0, a1, a2, a3, ret;
@@ -8273,6 +8280,28 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
kvm_sched_yield(vcpu->kvm, a0);
ret = 0;
break;
+   case KVM_HC_PAGE_ENC_STATUS: {
+   u64 gpa = a0, npages = a1, enc = a2;
+
+   ret = -KVM_ENOSYS;
+   if (!vcpu->kvm->arch.page_enc_hc_enable)
+   break;
+
+   if (!PAGE_ALIGNED(gpa) || !npages ||
+   gpa_to_gfn(gpa) + npages <= gpa_to_gfn(gpa)) {
+   ret = -EINVAL;
+   break;
+   }
+
+   vcpu->run->exit_reason= KVM_EXIT_HYPERCALL;
+   vcpu->run->hypercall.nr   = KVM_HC_PAGE_ENC_STATUS;
+   vcpu->run->hypercall.args[0]  = gpa;
+   vcpu->run->hypercall.args[1]  = npages;
+   vcpu->run->hypercall.args[2]  = enc;
+   vcpu->run->hypercall.longmode = op_64_bit;
+   vcpu->arch.complete_userspace_io = complete_hypercall_exit;
+   return 0;
+   }
default:
ret = -KVM_ENOSYS;
break;
diff --git a/include/uapi/linux/kvm_para.h b/include/uapi/linux/kvm_para.h
index 8b86609849b9..847b83b75dc8 100644
--- a/include/uapi/linux/kvm_para.h
+++ b/include/uapi/linux/kvm_para.h
@@ -29,6 +29,7 @@
 #define KVM_HC_CLOCK_PAIRING   9
 #define KVM_HC_SEND_IPI10
 #define KVM_HC_SCHED_YIELD 11
+#define KVM_HC_PAGE_ENC_STATUS 12
 
 /*
  * hypercalls use architecture specific
-- 
2.17.1

[PATCH v13 07/12] KVM: x86: Add AMD SEV specific Hypercall3

[Ashish Kalra]

From: Brijesh Singh 

KVM hypercall framework relies on alternative framework to patch the
VMCALL -> VMMCALL on AMD platform. If a hypercall is made before
apply_alternative() is called then it defaults to VMCALL. The approach
works fine on non SEV guest. A VMCALL would causes #UD, and hypervisor
will be able to decode the instruction and do the right things. But
when SEV is active, guest memory is encrypted with guest key and
hypervisor will not be able to decode the instruction bytes.

Add SEV specific hypercall3, it unconditionally uses VMMCALL. The hypercall
will be used by the SEV guest to notify encrypted pages to the hypervisor.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Reviewed-by: Venu Busireddy 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 arch/x86/include/asm/kvm_para.h | 12 
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 338119852512..bc1b11d057fc 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -85,6 +85,18 @@ static inline long kvm_hypercall4(unsigned int nr, unsigned 
long p1,
return ret;
 }
 
+static inline long kvm_sev_hypercall3(unsigned int nr, unsigned long p1,
+ unsigned long p2, unsigned long p3)
+{
+   long ret;
+
+   asm volatile("vmmcall"
+: "=a"(ret)
+: "a"(nr), "b"(p1), "c"(p2), "d"(p3)
+: "memory");
+   return ret;
+}
+
 #ifdef CONFIG_KVM_GUEST
 bool kvm_para_available(void);
 unsigned int kvm_arch_para_features(void);
-- 
2.17.1

[PATCH v13 06/12] KVM: SVM: Add KVM_SEV_RECEIVE_FINISH command

[Ashish Kalra]

From: Brijesh Singh 

The command finalize the guest receiving process and make the SEV guest
ready for the execution.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  8 +++
 arch/x86/kvm/svm/sev.c| 23 +++
 2 files changed, 31 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index c6ed5b26d841..0466c0febff9 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -396,6 +396,14 @@ Returns: 0 on success, -negative on error
 __u32 trans_len;
 };
 
+15. KVM_SEV_RECEIVE_FINISH
+
+
+After completion of the migration flow, the KVM_SEV_RECEIVE_FINISH command can 
be
+issued by the hypervisor to make the guest ready for execution.
+
+Returns: 0 on success, -negative on error
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 2c95657cc9bf..c9795a22e502 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1524,6 +1524,26 @@ static int sev_receive_update_data(struct kvm *kvm, 
struct kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_receive_finish(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_receive_finish *data;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL);
+   if (!data)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_RECEIVE_FINISH, data, >error);
+
+   kfree(data);
+   return ret;
+}
+
 int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
 {
struct kvm_sev_cmd sev_cmd;
@@ -1592,6 +1612,9 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
case KVM_SEV_RECEIVE_UPDATE_DATA:
r = sev_receive_update_data(kvm, _cmd);
break;
+   case KVM_SEV_RECEIVE_FINISH:
+   r = sev_receive_finish(kvm, _cmd);
+   break;
default:
r = -EINVAL;
goto out;
-- 
2.17.1

[PATCH v13 05/12] KVM: SVM: Add KVM_SEV_RECEIVE_UPDATE_DATA command

[Ashish Kalra]

From: Brijesh Singh 

The command is used for copying the incoming buffer into the
SEV guest memory space.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst| 24 ++
 arch/x86/kvm/svm/sev.c| 79 +++
 include/uapi/linux/kvm.h  |  9 +++
 3 files changed, 112 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index c86c1ded8dd8..c6ed5b26d841 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -372,6 +372,30 @@ On success, the 'handle' field contains a new handle and 
on error, a negative va
 
 For more details, see SEV spec Section 6.12.
 
+14. KVM_SEV_RECEIVE_UPDATE_DATA
+
+
+The KVM_SEV_RECEIVE_UPDATE_DATA command can be used by the hypervisor to copy
+the incoming buffers into the guest memory region with encryption context
+created during the KVM_SEV_RECEIVE_START.
+
+Parameters (in): struct kvm_sev_receive_update_data
+
+Returns: 0 on success, -negative on error
+
+::
+
+struct kvm_sev_launch_receive_update_data {
+__u64 hdr_uaddr;/* userspace address containing the 
packet header */
+__u32 hdr_len;
+
+__u64 guest_uaddr;  /* the destination guest memory region 
*/
+__u32 guest_len;
+
+__u64 trans_uaddr;  /* the incoming buffer memory region  
*/
+__u32 trans_len;
+};
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index e530c2b34b5e..2c95657cc9bf 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1448,6 +1448,82 @@ static int sev_receive_start(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_receive_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct kvm_sev_receive_update_data params;
+   struct sev_data_receive_update_data *data;
+   void *hdr = NULL, *trans = NULL;
+   struct page **guest_page;
+   unsigned long n;
+   int ret, offset;
+
+   if (!sev_guest(kvm))
+   return -EINVAL;
+
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_receive_update_data)))
+   return -EFAULT;
+
+   if (!params.hdr_uaddr || !params.hdr_len ||
+   !params.guest_uaddr || !params.guest_len ||
+   !params.trans_uaddr || !params.trans_len)
+   return -EINVAL;
+
+   /* Check if we are crossing the page boundary */
+   offset = params.guest_uaddr & (PAGE_SIZE - 1);
+   if ((params.guest_len + offset > PAGE_SIZE))
+   return -EINVAL;
+
+   hdr = psp_copy_user_blob(params.hdr_uaddr, params.hdr_len);
+   if (IS_ERR(hdr))
+   return PTR_ERR(hdr);
+
+   trans = psp_copy_user_blob(params.trans_uaddr, params.trans_len);
+   if (IS_ERR(trans)) {
+   ret = PTR_ERR(trans);
+   goto e_free_hdr;
+   }
+
+   ret = -ENOMEM;
+   data = kzalloc(sizeof(*data), GFP_KERNEL);
+   if (!data)
+   goto e_free_trans;
+
+   data->hdr_address = __psp_pa(hdr);
+   data->hdr_len = params.hdr_len;
+   data->trans_address = __psp_pa(trans);
+   data->trans_len = params.trans_len;
+
+   /* Pin guest memory */
+   ret = -EFAULT;
+   guest_page = sev_pin_memory(kvm, params.guest_uaddr & PAGE_MASK,
+   PAGE_SIZE, , 0);
+   if (!guest_page)
+   goto e_free;
+
+   /* The RECEIVE_UPDATE_DATA command requires C-bit to be always set. */
+   data->guest_address = (page_to_pfn(guest_page[0]) << PAGE_SHIFT) +
+   offset;
+   data->guest_address |= sev_me_mask;
+   data->guest_len = params.guest_len;
+   data->handle = sev->handle;
+
+   ret = sev_issue_cmd(kvm, SEV_CMD_RECEIVE_UPDATE_DATA, data,
+   >error);
+
+   sev_unpin_memory(kvm, guest_page, n);
+
+e_free:
+   kfree(data);
+e_free_trans:
+   kfree(trans);
+e_free_hdr:
+   kfree(hdr);
+
+   return ret;
+}
+
 int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
 {
struct kvm_sev_cmd sev_cmd;
@@ -1513,6 +1589,9 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
case KVM_SEV_RECEIVE_START:
r = sev_receive_start(kvm, _cmd);
break;
+   case KVM_SEV_R

[PATCH v13 04/12] KVM: SVM: Add support for KVM_SEV_RECEIVE_START command

[Ashish Kalra]

From: Brijesh Singh 

The command is used to create the encryption context for an incoming
SEV guest. The encryption context can be later used by the hypervisor
to import the incoming data into the SEV guest memory space.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst| 29 +++
 arch/x86/kvm/svm/sev.c| 81 +++
 include/uapi/linux/kvm.h  |  9 +++
 3 files changed, 119 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index 26c4e6c83f62..c86c1ded8dd8 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -343,6 +343,35 @@ issued by the hypervisor to delete the encryption context.
 
 Returns: 0 on success, -negative on error
 
+13. KVM_SEV_RECEIVE_START
+
+
+The KVM_SEV_RECEIVE_START command is used for creating the memory encryption
+context for an incoming SEV guest. To create the encryption context, the user 
must
+provide a guest policy, the platform public Diffie-Hellman (PDH) key and 
session
+information.
+
+Parameters: struct  kvm_sev_receive_start (in/out)
+
+Returns: 0 on success, -negative on error
+
+::
+
+struct kvm_sev_receive_start {
+__u32 handle;   /* if zero then firmware creates a new 
handle */
+__u32 policy;   /* guest's policy */
+
+__u64 pdh_uaddr;/* userspace address pointing to the 
PDH key */
+__u32 pdh_len;
+
+__u64 session_uaddr;/* userspace address which points to 
the guest session information */
+__u32 session_len;
+};
+
+On success, the 'handle' field contains a new handle and on error, a negative 
value.
+
+For more details, see SEV spec Section 6.12.
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 92325d9527ce..e530c2b34b5e 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1370,6 +1370,84 @@ static int sev_send_finish(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_receive_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_receive_start *start;
+   struct kvm_sev_receive_start params;
+   int *error = >error;
+   void *session_data;
+   void *pdh_data;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   /* Get parameter from the userspace */
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_receive_start)))
+   return -EFAULT;
+
+   /* some sanity checks */
+   if (!params.pdh_uaddr || !params.pdh_len ||
+   !params.session_uaddr || !params.session_len)
+   return -EINVAL;
+
+   pdh_data = psp_copy_user_blob(params.pdh_uaddr, params.pdh_len);
+   if (IS_ERR(pdh_data))
+   return PTR_ERR(pdh_data);
+
+   session_data = psp_copy_user_blob(params.session_uaddr,
+   params.session_len);
+   if (IS_ERR(session_data)) {
+   ret = PTR_ERR(session_data);
+   goto e_free_pdh;
+   }
+
+   ret = -ENOMEM;
+   start = kzalloc(sizeof(*start), GFP_KERNEL);
+   if (!start)
+   goto e_free_session;
+
+   start->handle = params.handle;
+   start->policy = params.policy;
+   start->pdh_cert_address = __psp_pa(pdh_data);
+   start->pdh_cert_len = params.pdh_len;
+   start->session_address = __psp_pa(session_data);
+   start->session_len = params.session_len;
+
+   /* create memory encryption context */
+   ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_RECEIVE_START, start,
+   error);
+   if (ret)
+   goto e_free;
+
+   /* Bind ASID to this guest */
+   ret = sev_bind_asid(kvm, start->handle, error);
+   if (ret)
+   goto e_free;
+
+   params.handle = start->handle;
+   if (copy_to_user((void __user *)(uintptr_t)argp->data,
+, sizeof(struct kvm_sev_receive_start))) {
+   ret = -EFAULT;
+   sev_unbind_asid(kvm, start->handle);
+   goto e_free;
+   }
+
+   sev->handle = start->handle;
+   sev->fd = argp->sev_fd;
+
+e_free:
+   kfree(start);
+e_free_session:
+   kfree(session_data);
+e_free_pdh:
+   kfree(pdh_data);
+
+   return 

[PATCH v13 03/12] KVM: SVM: Add KVM_SEV_SEND_FINISH command

[Ashish Kalra]

From: Brijesh Singh 

The command is used to finailize the encryption context created with
KVM_SEV_SEND_START command.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  8 +++
 arch/x86/kvm/svm/sev.c| 23 +++
 2 files changed, 31 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index 3c5456e0268a..26c4e6c83f62 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -335,6 +335,14 @@ Returns: 0 on success, -negative on error
 __u32 trans_len;
 };
 
+12. KVM_SEV_SEND_FINISH
+
+
+After completion of the migration flow, the KVM_SEV_SEND_FINISH command can be
+issued by the hypervisor to delete the encryption context.
+
+Returns: 0 on success, -negative on error
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 30527285a39a..92325d9527ce 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1350,6 +1350,26 @@ static int sev_send_update_data(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_send_finish(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_finish *data;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL);
+   if (!data)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_SEND_FINISH, data, >error);
+
+   kfree(data);
+   return ret;
+}
+
 int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
 {
struct kvm_sev_cmd sev_cmd;
@@ -1409,6 +1429,9 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
case KVM_SEV_SEND_UPDATE_DATA:
r = sev_send_update_data(kvm, _cmd);
break;
+   case KVM_SEV_SEND_FINISH:
+   r = sev_send_finish(kvm, _cmd);
+   break;
default:
r = -EINVAL;
goto out;
-- 
2.17.1

[PATCH v13 02/12] KVM: SVM: Add KVM_SEND_UPDATE_DATA command

[Ashish Kalra]

From: Brijesh Singh 

The command is used for encrypting the guest memory region using the encryption
context created with KVM_SEV_SEND_START.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by : Steve Rutherford 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  24 
 arch/x86/kvm/svm/sev.c| 122 ++
 include/uapi/linux/kvm.h  |   9 ++
 3 files changed, 155 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index ac799dd7a618..3c5456e0268a 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -311,6 +311,30 @@ Returns: 0 on success, -negative on error
 __u32 session_len;
 };
 
+11. KVM_SEV_SEND_UPDATE_DATA
+
+
+The KVM_SEV_SEND_UPDATE_DATA command can be used by the hypervisor to encrypt 
the
+outgoing guest memory region with the encryption context creating using
+KVM_SEV_SEND_START.
+
+Parameters (in): struct kvm_sev_send_update_data
+
+Returns: 0 on success, -negative on error
+
+::
+
+struct kvm_sev_launch_send_update_data {
+__u64 hdr_uaddr;/* userspace address containing the 
packet header */
+__u32 hdr_len;
+
+__u64 guest_uaddr;  /* the source memory region to be 
encrypted */
+__u32 guest_len;
+
+__u64 trans_uaddr;  /* the destition memory region  */
+__u32 trans_len;
+};
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 2b65900c05d6..30527285a39a 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -34,6 +34,7 @@ static DECLARE_RWSEM(sev_deactivate_lock);
 static DEFINE_MUTEX(sev_bitmap_lock);
 unsigned int max_sev_asid;
 static unsigned int min_sev_asid;
+static unsigned long sev_me_mask;
 static unsigned long *sev_asid_bitmap;
 static unsigned long *sev_reclaim_asid_bitmap;
 
@@ -1232,6 +1233,123 @@ static int sev_send_start(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+/* Userspace wants to query either header or trans length. */
+static int
+__sev_send_update_data_query_lengths(struct kvm *kvm, struct kvm_sev_cmd *argp,
+struct kvm_sev_send_update_data *params)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_update_data *data;
+   int ret;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL_ACCOUNT);
+   if (!data)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_SEND_UPDATE_DATA, data, >error);
+
+   params->hdr_len = data->hdr_len;
+   params->trans_len = data->trans_len;
+
+   if (copy_to_user((void __user *)(uintptr_t)argp->data, params,
+sizeof(struct kvm_sev_send_update_data)))
+   ret = -EFAULT;
+
+   kfree(data);
+   return ret;
+}
+
+static int sev_send_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_update_data *data;
+   struct kvm_sev_send_update_data params;
+   void *hdr, *trans_data;
+   struct page **guest_page;
+   unsigned long n;
+   int ret, offset;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_send_update_data)))
+   return -EFAULT;
+
+   /* userspace wants to query either header or trans length */
+   if (!params.trans_len || !params.hdr_len)
+   return __sev_send_update_data_query_lengths(kvm, argp, );
+
+   if (!params.trans_uaddr || !params.guest_uaddr ||
+   !params.guest_len || !params.hdr_uaddr)
+   return -EINVAL;
+
+   /* Check if we are crossing the page boundary */
+   offset = params.guest_uaddr & (PAGE_SIZE - 1);
+   if ((params.guest_len + offset > PAGE_SIZE))
+   return -EINVAL;
+
+   /* Pin guest memory */
+   guest_page = sev_pin_memory(kvm, params.guest_uaddr & PAGE_MASK,
+   PAGE_SIZE, , 0);
+   if (!guest_page)
+   return -EFAULT;
+
+   /* allocate memory for header and transport buffer */
+   ret = -ENOMEM;
+   hdr = kmalloc(params.hdr_len, GFP_KERNEL_ACCOUNT);
+   if (!hdr)
+   goto e_unpin;
+
+   trans_data = kmalloc(params.trans_len, GFP_KERNEL_ACCOUNT);
+   if (!trans_data)
+  

[PATCH v13 01/12] KVM: SVM: Add KVM_SEV SEND_START command

[Ashish Kalra]

From: Brijesh Singh 

The command is used to create an outgoing SEV guest encryption context.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Reviewed-by: Venu Busireddy 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  27 
 arch/x86/kvm/svm/sev.c| 125 ++
 include/linux/psp-sev.h   |   8 +-
 include/uapi/linux/kvm.h  |  12 ++
 4 files changed, 168 insertions(+), 4 deletions(-)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index 469a6308765b..ac799dd7a618 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -284,6 +284,33 @@ Returns: 0 on success, -negative on error
 __u32 len;
 };
 
+10. KVM_SEV_SEND_START
+--
+
+The KVM_SEV_SEND_START command can be used by the hypervisor to create an
+outgoing guest encryption context.
+
+Parameters (in): struct kvm_sev_send_start
+
+Returns: 0 on success, -negative on error
+
+::
+struct kvm_sev_send_start {
+__u32 policy; /* guest policy */
+
+__u64 pdh_cert_uaddr; /* platform Diffie-Hellman 
certificate */
+__u32 pdh_cert_len;
+
+__u64 plat_certs_uaddr;/* platform certificate chain */
+__u32 plat_certs_len;
+
+__u64 amd_certs_uaddr;/* AMD certificate */
+__u32 amd_certs_len;
+
+__u64 session_uaddr;  /* Guest session information */
+__u32 session_len;
+};
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 874ea309279f..2b65900c05d6 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1110,6 +1110,128 @@ static int sev_get_attestation_report(struct kvm *kvm, 
struct kvm_sev_cmd *argp)
return ret;
 }
 
+/* Userspace wants to query session length. */
+static int
+__sev_send_start_query_session_length(struct kvm *kvm, struct kvm_sev_cmd 
*argp,
+ struct kvm_sev_send_start *params)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_start *data;
+   int ret;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL_ACCOUNT);
+   if (data == NULL)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_SEND_START, data, >error);
+
+   params->session_len = data->session_len;
+   if (copy_to_user((void __user *)(uintptr_t)argp->data, params,
+   sizeof(struct kvm_sev_send_start)))
+   ret = -EFAULT;
+
+   kfree(data);
+   return ret;
+}
+
+static int sev_send_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_start *data;
+   struct kvm_sev_send_start params;
+   void *amd_certs, *session_data;
+   void *pdh_cert, *plat_certs;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_send_start)))
+   return -EFAULT;
+
+   /* if session_len is zero, userspace wants to query the session length 
*/
+   if (!params.session_len)
+   return __sev_send_start_query_session_length(kvm, argp,
+   );
+
+   /* some sanity checks */
+   if (!params.pdh_cert_uaddr || !params.pdh_cert_len ||
+   !params.session_uaddr || params.session_len > SEV_FW_BLOB_MAX_SIZE)
+   return -EINVAL;
+
+   /* allocate the memory to hold the session data blob */
+   session_data = kmalloc(params.session_len, GFP_KERNEL_ACCOUNT);
+   if (!session_data)
+   return -ENOMEM;
+
+   /* copy the certificate blobs from userspace */
+   pdh_cert = psp_copy_user_blob(params.pdh_cert_uaddr,
+   params.pdh_cert_len);
+   if (IS_ERR(pdh_cert)) {
+   ret = PTR_ERR(pdh_cert);
+   goto e_free_session;
+   }
+
+   plat_certs = psp_copy_user_blob(params.plat_certs_uaddr,
+   params.plat_certs_len);
+   if (IS_ERR(plat_certs)) {
+   ret = PTR_ERR(plat_certs);
+   goto e_free_pdh;
+   }
+
+   amd_certs = psp_copy_user_blob(params.amd_certs_uaddr,
+   params.amd_certs_len);
+   if (IS_ERR(amd_certs)) {

[PATCH v13 00/12] Add AMD SEV guest live migration support

[Ashish Kalra]

From: Ashish Kalra 

The series add support for AMD SEV guest live migration commands. To protect the
confidentiality of an SEV protected guest memory while in transit we need to
use the SEV commands defined in SEV API spec [1].

SEV guest VMs have the concept of private and shared memory. Private memory
is encrypted with the guest-specific key, while shared memory may be encrypted
with hypervisor key. The commands provided by the SEV FW are meant to be used
for the private memory only. The patch series introduces a new hypercall.
The guest OS can use this hypercall to notify the page encryption status.
If the page is encrypted with guest specific-key then we use SEV command during
the migration. If page is not encrypted then fallback to default.

The patch uses the KVM_EXIT_HYPERCALL exitcode and hypercall to
userspace exit functionality as a common interface from the guest back to the
VMM and passing on the guest shared/unencrypted page information to the
userspace VMM/Qemu. Qemu can consult this information during migration to know 
whether the page is encrypted.

This section descibes how the SEV live migration feature is negotiated
between the host and guest, the host indicates this feature support via 
KVM_FEATURE_CPUID. The guest firmware (OVMF) detects this feature and
sets a UEFI enviroment variable indicating OVMF support for live
migration, the guest kernel also detects the host support for this
feature via cpuid and in case of an EFI boot verifies if OVMF also
supports this feature by getting the UEFI enviroment variable and if it
set then enables live migration feature on host by writing to a custom
MSR, if not booted under EFI, then it simply enables the feature by
again writing to the custom MSR. The MSR is also handled by the
userspace VMM/Qemu.

A branch containing these patches is available here:
https://github.com/AMDESE/linux/tree/sev-migration-v13

[1] https://developer.amd.com/wp-content/resources/55766.PDF

Changes since v12:
- Reset page encryption status during early boot instead of just 
  before the kexec to avoid SMP races during kvm_pv_guest_cpu_reboot().
- Remove incorrect log message in case of non-EFI boot and implicit
  enabling of SEV live migration feature.

Changes since v11:
- Clean up and remove kvm_x86_ops callback for page_enc_status_hc and
  instead add a new per-VM flag to support/enable the page encryption
  status hypercall.
- Remove KVM_EXIT_DMA_SHARE/KVM_EXIT_DMA_UNSHARE exitcodes and instead
  use the KVM_EXIT_HYPERCALL exitcode for page encryption status
  hypercall to userspace functionality. 

Changes since v10:
- Adds new KVM_EXIT_DMA_SHARE/KVM_EXIT_DMA_UNSHARE hypercall to
  userspace exit functionality as a common interface from the guest back to the
  KVM and passing on the guest shared/unencrypted region information to the
  userspace VMM/Qemu. KVM/host kernel does not maintain the guest shared
  memory regions information anymore. 
- Remove implicit enabling of SEV live migration feature for an SEV
  guest, now this is explicitly in control of the userspace VMM/Qemu.
- Custom MSR handling is also now moved into userspace VMM/Qemu.
- As KVM does not maintain the guest shared memory region information
  anymore, sev_dbg_crypt() cannot bypass unencrypted guest memory
  regions without support from userspace VMM/Qemu.

Changes since v9:
- Transitioning from page encryption bitmap to the shared pages list
  to keep track of guest's shared/unencrypted memory regions.
- Move back to marking the complete _bss_decrypted section as 
  decrypted in the shared pages list.
- Invoke a new function check_kvm_sev_migration() via kvm_init_platform()
  for guest to query for host-side support for SEV live migration 
  and to enable the SEV live migration feature, to avoid
  #ifdefs in code 
- Rename MSR_KVM_SEV_LIVE_MIG_EN to MSR_KVM_SEV_LIVE_MIGRATION.
- Invoke a new function handle_unencrypted_region() from 
  sev_dbg_crypt() to bypass unencrypted guest memory regions.

Changes since v8:
- Rebasing to kvm next branch.
- Fixed and added comments as per review feedback on v8 patches.
- Removed implicitly enabling live migration for incoming VMs in
  in KVM_SET_PAGE_ENC_BITMAP, it is now done via KVM_SET_MSR ioctl.
- Adds support for bypassing unencrypted guest memory regions for
  DBG_DECRYPT API calls, guest memory region encryption status in
  sev_dbg_decrypt() is referenced using the page encryption bitmap.

Changes since v7:
- Removed the hypervisor specific hypercall/paravirt callback for
  SEV live migration and moved back to calling kvm_sev_hypercall3 
  directly.
- Fix build errors as
  Reported-by: kbuild test robot , specifically fixed
  build error when CONFIG_HYPERVISOR_GUEST=y and
  CONFIG_AMD_MEM_ENCRYPT=n.
- Implicitly enabled live migration for incoming VM(s) to handle 
  A->B->C->... VM migrations.
- Fixed Documentation as per comments on v6 patches.
- Fixed error return path in sev_send_update_data() as per comments 
  on v6 patches. 

Chan

Re: [PATCH v12 13/13] x86/kvm: Add kexec support for SEV Live Migration.

[Ashish Kalra]

On Mon, Apr 12, 2021 at 07:25:03PM -0700, Steve Rutherford wrote:
> On Mon, Apr 12, 2021 at 6:48 PM Ashish Kalra  wrote:
> >
> > On Mon, Apr 12, 2021 at 06:23:32PM -0700, Steve Rutherford wrote:
> > > On Mon, Apr 12, 2021 at 5:22 PM Steve Rutherford  
> > > wrote:
> > > >
> > > > On Mon, Apr 12, 2021 at 12:48 PM Ashish Kalra  
> > > > wrote:
> > > > >
> > > > > From: Ashish Kalra 
> > > > >
> > > > > Reset the host's shared pages list related to kernel
> > > > > specific page encryption status settings before we load a
> > > > > new kernel by kexec. We cannot reset the complete
> > > > > shared pages list here as we need to retain the
> > > > > UEFI/OVMF firmware specific settings.
> > > > >
> > > > > The host's shared pages list is maintained for the
> > > > > guest to keep track of all unencrypted guest memory regions,
> > > > > therefore we need to explicitly mark all shared pages as
> > > > > encrypted again before rebooting into the new guest kernel.
> > > > >
> > > > > Signed-off-by: Ashish Kalra 
> > > > > ---
> > > > >  arch/x86/kernel/kvm.c | 24 
> > > > >  1 file changed, 24 insertions(+)
> > > > >
> > > > > diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
> > > > > index bcc82e0c9779..4ad3ed547ff1 100644
> > > > > --- a/arch/x86/kernel/kvm.c
> > > > > +++ b/arch/x86/kernel/kvm.c
> > > > > @@ -39,6 +39,7 @@
> > > > >  #include 
> > > > >  #include 
> > > > >  #include 
> > > > > +#include 
> > > > >
> > > > >  DEFINE_STATIC_KEY_FALSE(kvm_async_pf_enabled);
> > > > >
> > > > > @@ -384,6 +385,29 @@ static void kvm_pv_guest_cpu_reboot(void *unused)
> > > > >  */
> > > > > if (kvm_para_has_feature(KVM_FEATURE_PV_EOI))
> > > > > wrmsrl(MSR_KVM_PV_EOI_EN, 0);
> > > > > +   /*
> > > > > +* Reset the host's shared pages list related to kernel
> > > > > +* specific page encryption status settings before we load a
> > > > > +* new kernel by kexec. NOTE: We cannot reset the complete
> > > > > +* shared pages list here as we need to retain the
> > > > > +* UEFI/OVMF firmware specific settings.
> > > > > +*/
> > > > > +   if (sev_live_migration_enabled & (smp_processor_id() == 0)) {
> > > > What happens if the reboot of CPU0 races with another CPU servicing a
> > > > device request (while the reboot is pending for that CPU)?
> > > > Seems like you could run into a scenario where you have hypercalls 
> > > > racing.
> > > >
> > > > Calling this on every core isn't free, but it is an easy way to avoid 
> > > > this race.
> > > > You could also count cores, and have only last core do the job, but
> > > > that seems more complicated.
> > > On second thought, I think this may be insufficient as a fix, since my
> > > read of kernel/reboot.c seems to imply that devices aren't shutdown
> > > until after these notifiers occur. As such, a single thread might be
> > > able to race with itself. I could be wrong here though.
> > >
> > > The heavy hammer would be to disable migration through the MSR (which
> > > the subsequent boot will re-enable).
> > >
> > > I'm curious if there is a less "blocking" way of handling kexecs (that
> > > strategy would block LM while the guest booted).
> > >
> > > One option that comes to mind would be for the guest to "mute" the
> > > encryption status hypercall after the call to reset the encryption
> > > status. The problem would be that the encryption status for pages
> > > would be very temporarily inaccurate in the window between that call
> > > and the start of the next boot. That isn't ideal, but, on the other
> > > hand, the VM was about to reboot anyway, so a corrupted shared page
> > > for device communication probably isn't super important. Still, I'm
> > > not really a fan of that. This would avoid corrupting the next boot,
> > > which is clearly an improvement.
> > >
> > > Each time the kernel boots it could also choose something 

Re: [PATCH v12 13/13] x86/kvm: Add kexec support for SEV Live Migration.

[Ashish Kalra]

On Mon, Apr 12, 2021 at 06:23:32PM -0700, Steve Rutherford wrote:
> On Mon, Apr 12, 2021 at 5:22 PM Steve Rutherford  
> wrote:
> >
> > On Mon, Apr 12, 2021 at 12:48 PM Ashish Kalra  wrote:
> > >
> > > From: Ashish Kalra 
> > >
> > > Reset the host's shared pages list related to kernel
> > > specific page encryption status settings before we load a
> > > new kernel by kexec. We cannot reset the complete
> > > shared pages list here as we need to retain the
> > > UEFI/OVMF firmware specific settings.
> > >
> > > The host's shared pages list is maintained for the
> > > guest to keep track of all unencrypted guest memory regions,
> > > therefore we need to explicitly mark all shared pages as
> > > encrypted again before rebooting into the new guest kernel.
> > >
> > > Signed-off-by: Ashish Kalra 
> > > ---
> > >  arch/x86/kernel/kvm.c | 24 
> > >  1 file changed, 24 insertions(+)
> > >
> > > diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
> > > index bcc82e0c9779..4ad3ed547ff1 100644
> > > --- a/arch/x86/kernel/kvm.c
> > > +++ b/arch/x86/kernel/kvm.c
> > > @@ -39,6 +39,7 @@
> > >  #include 
> > >  #include 
> > >  #include 
> > > +#include 
> > >
> > >  DEFINE_STATIC_KEY_FALSE(kvm_async_pf_enabled);
> > >
> > > @@ -384,6 +385,29 @@ static void kvm_pv_guest_cpu_reboot(void *unused)
> > >  */
> > > if (kvm_para_has_feature(KVM_FEATURE_PV_EOI))
> > > wrmsrl(MSR_KVM_PV_EOI_EN, 0);
> > > +   /*
> > > +* Reset the host's shared pages list related to kernel
> > > +* specific page encryption status settings before we load a
> > > +* new kernel by kexec. NOTE: We cannot reset the complete
> > > +* shared pages list here as we need to retain the
> > > +* UEFI/OVMF firmware specific settings.
> > > +*/
> > > +   if (sev_live_migration_enabled & (smp_processor_id() == 0)) {
> > What happens if the reboot of CPU0 races with another CPU servicing a
> > device request (while the reboot is pending for that CPU)?
> > Seems like you could run into a scenario where you have hypercalls racing.
> >
> > Calling this on every core isn't free, but it is an easy way to avoid this 
> > race.
> > You could also count cores, and have only last core do the job, but
> > that seems more complicated.
> On second thought, I think this may be insufficient as a fix, since my
> read of kernel/reboot.c seems to imply that devices aren't shutdown
> until after these notifiers occur. As such, a single thread might be
> able to race with itself. I could be wrong here though.
> 
> The heavy hammer would be to disable migration through the MSR (which
> the subsequent boot will re-enable).
> 
> I'm curious if there is a less "blocking" way of handling kexecs (that
> strategy would block LM while the guest booted).
> 
> One option that comes to mind would be for the guest to "mute" the
> encryption status hypercall after the call to reset the encryption
> status. The problem would be that the encryption status for pages
> would be very temporarily inaccurate in the window between that call
> and the start of the next boot. That isn't ideal, but, on the other
> hand, the VM was about to reboot anyway, so a corrupted shared page
> for device communication probably isn't super important. Still, I'm
> not really a fan of that. This would avoid corrupting the next boot,
> which is clearly an improvement.
> 
> Each time the kernel boots it could also choose something like a
> generation ID, and pass that down each time it calls the hypercall.
> This would then let userspace identify which requests were coming from
> the subsequent boot.
> 
> Everything here (except, perhaps, disabling migration through the MSR)
> seems kind of complicated. I somewhat hope my interpretation of
> kernel/reboot.c is wrong and this race just is not possible in the
> first place.
> 

Disabling migration through the MSR after resetting the page encryption
status is a reasonable approach. There is a similar window existing for
normal VM boot during which LM is disabled, from the point where OVMF
checks and adds support for SEV LM and the kernel boot checks for the
same and enables LM using the MSR. 

Thanks,
Ashish

> > > +   int i;
> > > +   unsigned long nr_pages;
> > > +
> > > +   for (i = 0; i < e820_table->nr_entries; i++) {
> > > +   struct e820_entry *entry = 
> > > _table->entries[i];
> > > +
> > > +   if (entry->type != E820_TYPE_RAM)
> > > +   continue;
> > > +
> > > +   nr_pages = DIV_ROUND_UP(entry->size, PAGE_SIZE);
> > > +
> > > +   kvm_sev_hypercall3(KVM_HC_PAGE_ENC_STATUS,
> > > +  entry->addr, nr_pages, 1);
> > > +   }
> > > +   }
> > > kvm_pv_disable_apf();
> > > kvm_disable_steal_time();
> > >  }
> > > --
> > > 2.17.1
> > >

Re: [PATCH v12 12/13] x86/kvm: Add guest support for detecting and enabling SEV Live Migration feature.

[Ashish Kalra]

On Mon, Apr 12, 2021 at 05:25:15PM -0700, Steve Rutherford wrote:
> On Mon, Apr 12, 2021 at 12:46 PM Ashish Kalra  wrote:
> >
> > From: Ashish Kalra 
> >
> > The guest support for detecting and enabling SEV Live migration
> > feature uses the following logic :
> >
> >  - kvm_init_plaform() invokes check_kvm_sev_migration() which
> >checks if its booted under the EFI
> >
> >- If not EFI,
> >
> >  i) check for the KVM_FEATURE_CPUID
> >
> >  ii) if CPUID reports that migration is supported, issue a wrmsrl()
> >  to enable the SEV live migration support
> >
> >- If EFI,
> >
> >  i) check for the KVM_FEATURE_CPUID
> >
> >  ii) If CPUID reports that migration is supported, read the UEFI 
> > variable which
> >  indicates OVMF support for live migration
> >
> >  iii) the variable indicates live migration is supported, issue a 
> > wrmsrl() to
> >   enable the SEV live migration support
> >
> > The EFI live migration check is done using a late_initcall() callback.
> >
> > Also, ensure that _bss_decrypted section is marked as decrypted in the
> > shared pages list.
> >
> > Signed-off-by: Ashish Kalra 
> > ---
> >  arch/x86/include/asm/mem_encrypt.h |  8 +
> >  arch/x86/kernel/kvm.c  | 52 ++
> >  arch/x86/mm/mem_encrypt.c  | 41 +++
> >  3 files changed, 101 insertions(+)
> >
> > diff --git a/arch/x86/include/asm/mem_encrypt.h 
> > b/arch/x86/include/asm/mem_encrypt.h
> > index 31c4df123aa0..19b77f3a62dc 100644
> > --- a/arch/x86/include/asm/mem_encrypt.h
> > +++ b/arch/x86/include/asm/mem_encrypt.h
> > @@ -21,6 +21,7 @@
> >  extern u64 sme_me_mask;
> >  extern u64 sev_status;
> >  extern bool sev_enabled;
> > +extern bool sev_live_migration_enabled;
> >
> >  void sme_encrypt_execute(unsigned long encrypted_kernel_vaddr,
> >  unsigned long decrypted_kernel_vaddr,
> > @@ -44,8 +45,11 @@ void __init sme_enable(struct boot_params *bp);
> >
> >  int __init early_set_memory_decrypted(unsigned long vaddr, unsigned long 
> > size);
> >  int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long 
> > size);
> > +void __init early_set_mem_enc_dec_hypercall(unsigned long vaddr, int 
> > npages,
> > +   bool enc);
> >
> >  void __init mem_encrypt_free_decrypted_mem(void);
> > +void __init check_kvm_sev_migration(void);
> >
> >  /* Architecture __weak replacement functions */
> >  void __init mem_encrypt_init(void);
> > @@ -60,6 +64,7 @@ bool sev_es_active(void);
> >  #else  /* !CONFIG_AMD_MEM_ENCRYPT */
> >
> >  #define sme_me_mask0ULL
> > +#define sev_live_migration_enabled false
> >
> >  static inline void __init sme_early_encrypt(resource_size_t paddr,
> > unsigned long size) { }
> > @@ -84,8 +89,11 @@ static inline int __init
> >  early_set_memory_decrypted(unsigned long vaddr, unsigned long size) { 
> > return 0; }
> >  static inline int __init
> >  early_set_memory_encrypted(unsigned long vaddr, unsigned long size) { 
> > return 0; }
> > +static inline void __init
> > +early_set_mem_enc_dec_hypercall(unsigned long vaddr, int npages, bool enc) 
> > {}
> >
> >  static inline void mem_encrypt_free_decrypted_mem(void) { }
> > +static inline void check_kvm_sev_migration(void) { }
> >
> >  #define __bss_decrypted
> >
> > diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
> > index 78bb0fae3982..bcc82e0c9779 100644
> > --- a/arch/x86/kernel/kvm.c
> > +++ b/arch/x86/kernel/kvm.c
> > @@ -26,6 +26,7 @@
> >  #include 
> >  #include 
> >  #include 
> > +#include 
> >  #include 
> >  #include 
> >  #include 
> > @@ -429,6 +430,56 @@ static inline void __set_percpu_decrypted(void *ptr, 
> > unsigned long size)
> > early_set_memory_decrypted((unsigned long) ptr, size);
> >  }
> >
> > +static int __init setup_kvm_sev_migration(void)
> > +{
> > +   efi_char16_t efi_sev_live_migration_enabled[] = 
> > L"SevLiveMigrationEnabled";
> > +   efi_guid_t efi_variable_guid = MEM_ENCRYPT_GUID;
> > +   efi_status_t status;
> > +   unsigned long size;
> > +   bool enabled;
> > +
> > +   /*
> > +* check_kvm_sev_migration() invoked 

[PATCH v12 13/13] x86/kvm: Add kexec support for SEV Live Migration.

[Ashish Kalra]

From: Ashish Kalra 

Reset the host's shared pages list related to kernel
specific page encryption status settings before we load a
new kernel by kexec. We cannot reset the complete
shared pages list here as we need to retain the
UEFI/OVMF firmware specific settings.

The host's shared pages list is maintained for the
guest to keep track of all unencrypted guest memory regions,
therefore we need to explicitly mark all shared pages as
encrypted again before rebooting into the new guest kernel.

Signed-off-by: Ashish Kalra 
---
 arch/x86/kernel/kvm.c | 24 
 1 file changed, 24 insertions(+)

diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index bcc82e0c9779..4ad3ed547ff1 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -39,6 +39,7 @@
 #include 
 #include 
 #include 
+#include 
 
 DEFINE_STATIC_KEY_FALSE(kvm_async_pf_enabled);
 
@@ -384,6 +385,29 @@ static void kvm_pv_guest_cpu_reboot(void *unused)
 */
if (kvm_para_has_feature(KVM_FEATURE_PV_EOI))
wrmsrl(MSR_KVM_PV_EOI_EN, 0);
+   /*
+* Reset the host's shared pages list related to kernel
+* specific page encryption status settings before we load a
+* new kernel by kexec. NOTE: We cannot reset the complete
+* shared pages list here as we need to retain the
+* UEFI/OVMF firmware specific settings.
+*/
+   if (sev_live_migration_enabled & (smp_processor_id() == 0)) {
+   int i;
+   unsigned long nr_pages;
+
+   for (i = 0; i < e820_table->nr_entries; i++) {
+   struct e820_entry *entry = _table->entries[i];
+
+   if (entry->type != E820_TYPE_RAM)
+   continue;
+
+   nr_pages = DIV_ROUND_UP(entry->size, PAGE_SIZE);
+
+   kvm_sev_hypercall3(KVM_HC_PAGE_ENC_STATUS,
+  entry->addr, nr_pages, 1);
+   }
+   }
kvm_pv_disable_apf();
kvm_disable_steal_time();
 }
-- 
2.17.1

[PATCH v12 11/13] EFI: Introduce the new AMD Memory Encryption GUID.

[Ashish Kalra]

From: Ashish Kalra 

Introduce a new AMD Memory Encryption GUID which is currently
used for defining a new UEFI environment variable which indicates
UEFI/OVMF support for the SEV live migration feature. This variable
is setup when UEFI/OVMF detects host/hypervisor support for SEV
live migration and later this variable is read by the kernel using
EFI runtime services to verify if OVMF supports the live migration
feature.

Signed-off-by: Ashish Kalra 
---
 include/linux/efi.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/linux/efi.h b/include/linux/efi.h
index 6b5d36babfcc..6f364ace82cb 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -362,6 +362,7 @@ void efi_native_runtime_setup(void);
 
 /* OEM GUIDs */
 #define DELLEMC_EFI_RCI2_TABLE_GUIDEFI_GUID(0x2d9f28a2, 0xa886, 
0x456a,  0x97, 0xa8, 0xf1, 0x1e, 0xf2, 0x4f, 0xf4, 0x55)
+#define MEM_ENCRYPT_GUID   EFI_GUID(0x0cf29b71, 0x9e51, 
0x433a,  0xa3, 0xb7, 0x81, 0xf3, 0xab, 0x16, 0xb8, 0x75)
 
 typedef struct {
efi_guid_t guid;
-- 
2.17.1

[PATCH v12 09/13] mm: x86: Invoke hypercall when page encryption status is changed

[Ashish Kalra]

From: Brijesh Singh 

Invoke a hypercall when a memory region is changed from encrypted ->
decrypted and vice versa. Hypervisor needs to know the page encryption
status during the guest migration.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Venu Busireddy 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 arch/x86/include/asm/paravirt.h   | 10 +
 arch/x86/include/asm/paravirt_types.h |  2 +
 arch/x86/kernel/paravirt.c|  1 +
 arch/x86/mm/mem_encrypt.c | 57 ++-
 arch/x86/mm/pat/set_memory.c  |  7 
 5 files changed, 76 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index 4abf110e2243..efaa3e628967 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -84,6 +84,12 @@ static inline void paravirt_arch_exit_mmap(struct mm_struct 
*mm)
PVOP_VCALL1(mmu.exit_mmap, mm);
 }
 
+static inline void page_encryption_changed(unsigned long vaddr, int npages,
+   bool enc)
+{
+   PVOP_VCALL3(mmu.page_encryption_changed, vaddr, npages, enc);
+}
+
 #ifdef CONFIG_PARAVIRT_XXL
 static inline void load_sp0(unsigned long sp0)
 {
@@ -799,6 +805,10 @@ static inline void paravirt_arch_dup_mmap(struct mm_struct 
*oldmm,
 static inline void paravirt_arch_exit_mmap(struct mm_struct *mm)
 {
 }
+
+static inline void page_encryption_changed(unsigned long vaddr, int npages, 
bool enc)
+{
+}
 #endif
 #endif /* __ASSEMBLY__ */
 #endif /* _ASM_X86_PARAVIRT_H */
diff --git a/arch/x86/include/asm/paravirt_types.h 
b/arch/x86/include/asm/paravirt_types.h
index de87087d3bde..69ef9c207b38 100644
--- a/arch/x86/include/asm/paravirt_types.h
+++ b/arch/x86/include/asm/paravirt_types.h
@@ -195,6 +195,8 @@ struct pv_mmu_ops {
 
/* Hook for intercepting the destruction of an mm_struct. */
void (*exit_mmap)(struct mm_struct *mm);
+   void (*page_encryption_changed)(unsigned long vaddr, int npages,
+   bool enc);
 
 #ifdef CONFIG_PARAVIRT_XXL
struct paravirt_callee_save read_cr2;
diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index c60222ab8ab9..9f206e192f6b 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -335,6 +335,7 @@ struct paravirt_patch_template pv_ops = {
(void (*)(struct mmu_gather *, void *))tlb_remove_page,
 
.mmu.exit_mmap  = paravirt_nop,
+   .mmu.page_encryption_changed= paravirt_nop,
 
 #ifdef CONFIG_PARAVIRT_XXL
.mmu.read_cr2   = __PV_IS_CALLEE_SAVE(native_read_cr2),
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index ae78cef79980..fae9ccbd0da7 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -19,6 +19,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -29,6 +30,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "mm_internal.h"
 
@@ -229,6 +231,47 @@ void __init sev_setup_arch(void)
swiotlb_adjust_size(size);
 }
 
+static void set_memory_enc_dec_hypercall(unsigned long vaddr, int npages,
+   bool enc)
+{
+   unsigned long sz = npages << PAGE_SHIFT;
+   unsigned long vaddr_end, vaddr_next;
+
+   vaddr_end = vaddr + sz;
+
+   for (; vaddr < vaddr_end; vaddr = vaddr_next) {
+   int psize, pmask, level;
+   unsigned long pfn;
+   pte_t *kpte;
+
+   kpte = lookup_address(vaddr, );
+   if (!kpte || pte_none(*kpte))
+   return;
+
+   switch (level) {
+   case PG_LEVEL_4K:
+   pfn = pte_pfn(*kpte);
+   break;
+   case PG_LEVEL_2M:
+   pfn = pmd_pfn(*(pmd_t *)kpte);
+   break;
+   case PG_LEVEL_1G:
+   pfn = pud_pfn(*(pud_t *)kpte);
+   break;
+   default:
+   return;
+   }
+
+   psize = page_level_size(level);
+   pmask = page_level_mask(level);
+
+   kvm_sev_hypercall3(KVM_HC_PAGE_ENC_STATUS,
+  pfn << PAGE_SHIFT, psize >> PAGE_SHIFT, enc);
+
+   vaddr_next = (vaddr & pmask) + psize;
+   }
+}
+
 static void __init __set_clr_pte_enc(pte_t *kpte, int level, bool enc)
 {
pgprot_t old_prot, new_prot;
@@ -286,12 +329,13 @@ static void __init __set_clr_pte_enc(pte_t *kpte, int 
level, bool enc)
 static int __init early_set_memory_enc_dec(unsigned long vaddr,
   unsign

[PATCH v12 10/13] KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature & Custom MSR.

[Ashish Kalra]

From: Ashish Kalra 

Add new KVM_FEATURE_SEV_LIVE_MIGRATION feature for guest to check
for host-side support for SEV live migration. Also add a new custom
MSR_KVM_SEV_LIVE_MIGRATION for guest to enable the SEV live migration
feature.

MSR is handled by userspace using MSR filters.

Signed-off-by: Ashish Kalra 
---
 Documentation/virt/kvm/cpuid.rst |  5 +
 Documentation/virt/kvm/msr.rst   | 12 
 arch/x86/include/uapi/asm/kvm_para.h |  4 
 arch/x86/kvm/cpuid.c |  3 ++-
 4 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/Documentation/virt/kvm/cpuid.rst b/Documentation/virt/kvm/cpuid.rst
index cf62162d4be2..0bdb6cdb12d3 100644
--- a/Documentation/virt/kvm/cpuid.rst
+++ b/Documentation/virt/kvm/cpuid.rst
@@ -96,6 +96,11 @@ KVM_FEATURE_MSI_EXT_DEST_ID15  guest checks 
this feature bit
before using extended 
destination
ID bits in MSI address bits 
11-5.
 
+KVM_FEATURE_SEV_LIVE_MIGRATION 16  guest checks this feature bit 
before
+   using the page encryption state
+   hypercall to notify the page 
state
+   change
+
 KVM_FEATURE_CLOCKSOURCE_STABLE_BIT 24  host will warn if no guest-side
per-cpu warps are expected in
kvmclock
diff --git a/Documentation/virt/kvm/msr.rst b/Documentation/virt/kvm/msr.rst
index e37a14c323d2..020245d16087 100644
--- a/Documentation/virt/kvm/msr.rst
+++ b/Documentation/virt/kvm/msr.rst
@@ -376,3 +376,15 @@ data:
write '1' to bit 0 of the MSR, this causes the host to re-scan its queue
and check if there are more notifications pending. The MSR is available
if KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
+
+MSR_KVM_SEV_LIVE_MIGRATION:
+0x4b564d08
+
+   Control SEV Live Migration features.
+
+data:
+Bit 0 enables (1) or disables (0) host-side SEV Live Migration feature,
+in other words, this is guest->host communication that it's properly
+handling the shared pages list.
+
+All other bits are reserved.
diff --git a/arch/x86/include/uapi/asm/kvm_para.h 
b/arch/x86/include/uapi/asm/kvm_para.h
index 950afebfba88..f6bfa138874f 100644
--- a/arch/x86/include/uapi/asm/kvm_para.h
+++ b/arch/x86/include/uapi/asm/kvm_para.h
@@ -33,6 +33,7 @@
 #define KVM_FEATURE_PV_SCHED_YIELD 13
 #define KVM_FEATURE_ASYNC_PF_INT   14
 #define KVM_FEATURE_MSI_EXT_DEST_ID15
+#define KVM_FEATURE_SEV_LIVE_MIGRATION 16
 
 #define KVM_HINTS_REALTIME  0
 
@@ -54,6 +55,7 @@
 #define MSR_KVM_POLL_CONTROL   0x4b564d05
 #define MSR_KVM_ASYNC_PF_INT   0x4b564d06
 #define MSR_KVM_ASYNC_PF_ACK   0x4b564d07
+#define MSR_KVM_SEV_LIVE_MIGRATION 0x4b564d08
 
 struct kvm_steal_time {
__u64 steal;
@@ -136,4 +138,6 @@ struct kvm_vcpu_pv_apf_data {
 #define KVM_PV_EOI_ENABLED KVM_PV_EOI_MASK
 #define KVM_PV_EOI_DISABLED 0x0
 
+#define KVM_SEV_LIVE_MIGRATION_ENABLED BIT_ULL(0)
+
 #endif /* _UAPI_ASM_X86_KVM_PARA_H */
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 6bd2f8b830e4..4e2e69a692aa 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -812,7 +812,8 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array 
*array, u32 function)
 (1 << KVM_FEATURE_PV_SEND_IPI) |
 (1 << KVM_FEATURE_POLL_CONTROL) |
 (1 << KVM_FEATURE_PV_SCHED_YIELD) |
-(1 << KVM_FEATURE_ASYNC_PF_INT);
+(1 << KVM_FEATURE_ASYNC_PF_INT) |
+(1 << KVM_FEATURE_SEV_LIVE_MIGRATION);
 
if (sched_info_on())
entry->eax |= (1 << KVM_FEATURE_STEAL_TIME);
-- 
2.17.1

[PATCH v12 06/13] KVM: SVM: Add KVM_SEV_RECEIVE_FINISH command

[Ashish Kalra]

From: Brijesh Singh 

The command finalize the guest receiving process and make the SEV guest
ready for the execution.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  8 +++
 arch/x86/kvm/svm/sev.c| 23 +++
 2 files changed, 31 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index c6ed5b26d841..0466c0febff9 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -396,6 +396,14 @@ Returns: 0 on success, -negative on error
 __u32 trans_len;
 };
 
+15. KVM_SEV_RECEIVE_FINISH
+
+
+After completion of the migration flow, the KVM_SEV_RECEIVE_FINISH command can 
be
+issued by the hypervisor to make the guest ready for execution.
+
+Returns: 0 on success, -negative on error
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 2c95657cc9bf..c9795a22e502 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1524,6 +1524,26 @@ static int sev_receive_update_data(struct kvm *kvm, 
struct kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_receive_finish(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_receive_finish *data;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL);
+   if (!data)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_RECEIVE_FINISH, data, >error);
+
+   kfree(data);
+   return ret;
+}
+
 int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
 {
struct kvm_sev_cmd sev_cmd;
@@ -1592,6 +1612,9 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
case KVM_SEV_RECEIVE_UPDATE_DATA:
r = sev_receive_update_data(kvm, _cmd);
break;
+   case KVM_SEV_RECEIVE_FINISH:
+   r = sev_receive_finish(kvm, _cmd);
+   break;
default:
r = -EINVAL;
goto out;
-- 
2.17.1

[PATCH v12 04/13] KVM: SVM: Add support for KVM_SEV_RECEIVE_START command

[Ashish Kalra]

From: Brijesh Singh 

The command is used to create the encryption context for an incoming
SEV guest. The encryption context can be later used by the hypervisor
to import the incoming data into the SEV guest memory space.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst| 29 +++
 arch/x86/kvm/svm/sev.c| 81 +++
 include/uapi/linux/kvm.h  |  9 +++
 3 files changed, 119 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index 26c4e6c83f62..c86c1ded8dd8 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -343,6 +343,35 @@ issued by the hypervisor to delete the encryption context.
 
 Returns: 0 on success, -negative on error
 
+13. KVM_SEV_RECEIVE_START
+
+
+The KVM_SEV_RECEIVE_START command is used for creating the memory encryption
+context for an incoming SEV guest. To create the encryption context, the user 
must
+provide a guest policy, the platform public Diffie-Hellman (PDH) key and 
session
+information.
+
+Parameters: struct  kvm_sev_receive_start (in/out)
+
+Returns: 0 on success, -negative on error
+
+::
+
+struct kvm_sev_receive_start {
+__u32 handle;   /* if zero then firmware creates a new 
handle */
+__u32 policy;   /* guest's policy */
+
+__u64 pdh_uaddr;/* userspace address pointing to the 
PDH key */
+__u32 pdh_len;
+
+__u64 session_uaddr;/* userspace address which points to 
the guest session information */
+__u32 session_len;
+};
+
+On success, the 'handle' field contains a new handle and on error, a negative 
value.
+
+For more details, see SEV spec Section 6.12.
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 92325d9527ce..e530c2b34b5e 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1370,6 +1370,84 @@ static int sev_send_finish(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_receive_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_receive_start *start;
+   struct kvm_sev_receive_start params;
+   int *error = >error;
+   void *session_data;
+   void *pdh_data;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   /* Get parameter from the userspace */
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_receive_start)))
+   return -EFAULT;
+
+   /* some sanity checks */
+   if (!params.pdh_uaddr || !params.pdh_len ||
+   !params.session_uaddr || !params.session_len)
+   return -EINVAL;
+
+   pdh_data = psp_copy_user_blob(params.pdh_uaddr, params.pdh_len);
+   if (IS_ERR(pdh_data))
+   return PTR_ERR(pdh_data);
+
+   session_data = psp_copy_user_blob(params.session_uaddr,
+   params.session_len);
+   if (IS_ERR(session_data)) {
+   ret = PTR_ERR(session_data);
+   goto e_free_pdh;
+   }
+
+   ret = -ENOMEM;
+   start = kzalloc(sizeof(*start), GFP_KERNEL);
+   if (!start)
+   goto e_free_session;
+
+   start->handle = params.handle;
+   start->policy = params.policy;
+   start->pdh_cert_address = __psp_pa(pdh_data);
+   start->pdh_cert_len = params.pdh_len;
+   start->session_address = __psp_pa(session_data);
+   start->session_len = params.session_len;
+
+   /* create memory encryption context */
+   ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_RECEIVE_START, start,
+   error);
+   if (ret)
+   goto e_free;
+
+   /* Bind ASID to this guest */
+   ret = sev_bind_asid(kvm, start->handle, error);
+   if (ret)
+   goto e_free;
+
+   params.handle = start->handle;
+   if (copy_to_user((void __user *)(uintptr_t)argp->data,
+, sizeof(struct kvm_sev_receive_start))) {
+   ret = -EFAULT;
+   sev_unbind_asid(kvm, start->handle);
+   goto e_free;
+   }
+
+   sev->handle = start->handle;
+   sev->fd = argp->sev_fd;
+
+e_free:
+   kfree(start);
+e_free_session:
+   kfree(session_data);
+e_free_pdh:
+   kfree(pdh_data);
+
+   return ret;
+}
+
 int svm_mem_enc_op(struct kv

[PATCH v12 12/13] x86/kvm: Add guest support for detecting and enabling SEV Live Migration feature.

[Ashish Kalra]

From: Ashish Kalra 

The guest support for detecting and enabling SEV Live migration
feature uses the following logic :

 - kvm_init_plaform() invokes check_kvm_sev_migration() which
   checks if its booted under the EFI

   - If not EFI,

 i) check for the KVM_FEATURE_CPUID

 ii) if CPUID reports that migration is supported, issue a wrmsrl()
 to enable the SEV live migration support

   - If EFI,

 i) check for the KVM_FEATURE_CPUID

 ii) If CPUID reports that migration is supported, read the UEFI variable 
which
 indicates OVMF support for live migration

 iii) the variable indicates live migration is supported, issue a wrmsrl() 
to
  enable the SEV live migration support

The EFI live migration check is done using a late_initcall() callback.

Also, ensure that _bss_decrypted section is marked as decrypted in the
shared pages list.

Signed-off-by: Ashish Kalra 
---
 arch/x86/include/asm/mem_encrypt.h |  8 +
 arch/x86/kernel/kvm.c  | 52 ++
 arch/x86/mm/mem_encrypt.c  | 41 +++
 3 files changed, 101 insertions(+)

diff --git a/arch/x86/include/asm/mem_encrypt.h 
b/arch/x86/include/asm/mem_encrypt.h
index 31c4df123aa0..19b77f3a62dc 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -21,6 +21,7 @@
 extern u64 sme_me_mask;
 extern u64 sev_status;
 extern bool sev_enabled;
+extern bool sev_live_migration_enabled;
 
 void sme_encrypt_execute(unsigned long encrypted_kernel_vaddr,
 unsigned long decrypted_kernel_vaddr,
@@ -44,8 +45,11 @@ void __init sme_enable(struct boot_params *bp);
 
 int __init early_set_memory_decrypted(unsigned long vaddr, unsigned long size);
 int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size);
+void __init early_set_mem_enc_dec_hypercall(unsigned long vaddr, int npages,
+   bool enc);
 
 void __init mem_encrypt_free_decrypted_mem(void);
+void __init check_kvm_sev_migration(void);
 
 /* Architecture __weak replacement functions */
 void __init mem_encrypt_init(void);
@@ -60,6 +64,7 @@ bool sev_es_active(void);
 #else  /* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define sme_me_mask0ULL
+#define sev_live_migration_enabled false
 
 static inline void __init sme_early_encrypt(resource_size_t paddr,
unsigned long size) { }
@@ -84,8 +89,11 @@ static inline int __init
 early_set_memory_decrypted(unsigned long vaddr, unsigned long size) { return 
0; }
 static inline int __init
 early_set_memory_encrypted(unsigned long vaddr, unsigned long size) { return 
0; }
+static inline void __init
+early_set_mem_enc_dec_hypercall(unsigned long vaddr, int npages, bool enc) {}
 
 static inline void mem_encrypt_free_decrypted_mem(void) { }
+static inline void check_kvm_sev_migration(void) { }
 
 #define __bss_decrypted
 
diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index 78bb0fae3982..bcc82e0c9779 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -26,6 +26,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -429,6 +430,56 @@ static inline void __set_percpu_decrypted(void *ptr, 
unsigned long size)
early_set_memory_decrypted((unsigned long) ptr, size);
 }
 
+static int __init setup_kvm_sev_migration(void)
+{
+   efi_char16_t efi_sev_live_migration_enabled[] = 
L"SevLiveMigrationEnabled";
+   efi_guid_t efi_variable_guid = MEM_ENCRYPT_GUID;
+   efi_status_t status;
+   unsigned long size;
+   bool enabled;
+
+   /*
+* check_kvm_sev_migration() invoked via kvm_init_platform() before
+* this callback would have setup the indicator that live migration
+* feature is supported/enabled.
+*/
+   if (!sev_live_migration_enabled)
+   return 0;
+
+   if (!efi_enabled(EFI_RUNTIME_SERVICES)) {
+   pr_info("%s : EFI runtime services are not enabled\n", 
__func__);
+   return 0;
+   }
+
+   size = sizeof(enabled);
+
+   /* Get variable contents into buffer */
+   status = efi.get_variable(efi_sev_live_migration_enabled,
+ _variable_guid, NULL, , );
+
+   if (status == EFI_NOT_FOUND) {
+   pr_info("%s : EFI live migration variable not found\n", 
__func__);
+   return 0;
+   }
+
+   if (status != EFI_SUCCESS) {
+   pr_info("%s : EFI variable retrieval failed\n", __func__);
+   return 0;
+   }
+
+   if (enabled == 0) {
+   pr_info("%s: live migration disabled in EFI\n", __func__);
+   return 0;
+   }
+
+   pr_info("%s : live migration enabled in EFI\n", __func__);
+   wrmsrl(MSR_KVM_SEV_LIVE_MIGRATION, KVM_SEV_LIVE_MIGRATION_ENABLED);
+
+   return true;
+}
+
+late_

[PATCH v12 02/13] KVM: SVM: Add KVM_SEND_UPDATE_DATA command

[Ashish Kalra]

From: Brijesh Singh 

The command is used for encrypting the guest memory region using the encryption
context created with KVM_SEV_SEND_START.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by : Steve Rutherford 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  24 
 arch/x86/kvm/svm/sev.c| 122 ++
 include/uapi/linux/kvm.h  |   9 ++
 3 files changed, 155 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index ac799dd7a618..3c5456e0268a 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -311,6 +311,30 @@ Returns: 0 on success, -negative on error
 __u32 session_len;
 };
 
+11. KVM_SEV_SEND_UPDATE_DATA
+
+
+The KVM_SEV_SEND_UPDATE_DATA command can be used by the hypervisor to encrypt 
the
+outgoing guest memory region with the encryption context creating using
+KVM_SEV_SEND_START.
+
+Parameters (in): struct kvm_sev_send_update_data
+
+Returns: 0 on success, -negative on error
+
+::
+
+struct kvm_sev_launch_send_update_data {
+__u64 hdr_uaddr;/* userspace address containing the 
packet header */
+__u32 hdr_len;
+
+__u64 guest_uaddr;  /* the source memory region to be 
encrypted */
+__u32 guest_len;
+
+__u64 trans_uaddr;  /* the destition memory region  */
+__u32 trans_len;
+};
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 2b65900c05d6..30527285a39a 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -34,6 +34,7 @@ static DECLARE_RWSEM(sev_deactivate_lock);
 static DEFINE_MUTEX(sev_bitmap_lock);
 unsigned int max_sev_asid;
 static unsigned int min_sev_asid;
+static unsigned long sev_me_mask;
 static unsigned long *sev_asid_bitmap;
 static unsigned long *sev_reclaim_asid_bitmap;
 
@@ -1232,6 +1233,123 @@ static int sev_send_start(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+/* Userspace wants to query either header or trans length. */
+static int
+__sev_send_update_data_query_lengths(struct kvm *kvm, struct kvm_sev_cmd *argp,
+struct kvm_sev_send_update_data *params)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_update_data *data;
+   int ret;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL_ACCOUNT);
+   if (!data)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_SEND_UPDATE_DATA, data, >error);
+
+   params->hdr_len = data->hdr_len;
+   params->trans_len = data->trans_len;
+
+   if (copy_to_user((void __user *)(uintptr_t)argp->data, params,
+sizeof(struct kvm_sev_send_update_data)))
+   ret = -EFAULT;
+
+   kfree(data);
+   return ret;
+}
+
+static int sev_send_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_update_data *data;
+   struct kvm_sev_send_update_data params;
+   void *hdr, *trans_data;
+   struct page **guest_page;
+   unsigned long n;
+   int ret, offset;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_send_update_data)))
+   return -EFAULT;
+
+   /* userspace wants to query either header or trans length */
+   if (!params.trans_len || !params.hdr_len)
+   return __sev_send_update_data_query_lengths(kvm, argp, );
+
+   if (!params.trans_uaddr || !params.guest_uaddr ||
+   !params.guest_len || !params.hdr_uaddr)
+   return -EINVAL;
+
+   /* Check if we are crossing the page boundary */
+   offset = params.guest_uaddr & (PAGE_SIZE - 1);
+   if ((params.guest_len + offset > PAGE_SIZE))
+   return -EINVAL;
+
+   /* Pin guest memory */
+   guest_page = sev_pin_memory(kvm, params.guest_uaddr & PAGE_MASK,
+   PAGE_SIZE, , 0);
+   if (!guest_page)
+   return -EFAULT;
+
+   /* allocate memory for header and transport buffer */
+   ret = -ENOMEM;
+   hdr = kmalloc(params.hdr_len, GFP_KERNEL_ACCOUNT);
+   if (!hdr)
+   goto e_unpin;
+
+   trans_data = kmalloc(params.trans_len, GFP_KERNEL_ACCOUNT);
+   if (!trans_data)
+  

[PATCH v12 07/13] KVM: x86: Add AMD SEV specific Hypercall3

[Ashish Kalra]

From: Brijesh Singh 

KVM hypercall framework relies on alternative framework to patch the
VMCALL -> VMMCALL on AMD platform. If a hypercall is made before
apply_alternative() is called then it defaults to VMCALL. The approach
works fine on non SEV guest. A VMCALL would causes #UD, and hypervisor
will be able to decode the instruction and do the right things. But
when SEV is active, guest memory is encrypted with guest key and
hypervisor will not be able to decode the instruction bytes.

Add SEV specific hypercall3, it unconditionally uses VMMCALL. The hypercall
will be used by the SEV guest to notify encrypted pages to the hypervisor.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Reviewed-by: Venu Busireddy 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 arch/x86/include/asm/kvm_para.h | 12 
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 338119852512..bc1b11d057fc 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -85,6 +85,18 @@ static inline long kvm_hypercall4(unsigned int nr, unsigned 
long p1,
return ret;
 }
 
+static inline long kvm_sev_hypercall3(unsigned int nr, unsigned long p1,
+ unsigned long p2, unsigned long p3)
+{
+   long ret;
+
+   asm volatile("vmmcall"
+: "=a"(ret)
+: "a"(nr), "b"(p1), "c"(p2), "d"(p3)
+: "memory");
+   return ret;
+}
+
 #ifdef CONFIG_KVM_GUEST
 bool kvm_para_available(void);
 unsigned int kvm_arch_para_features(void);
-- 
2.17.1

[PATCH v12 08/13] KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall

[Ashish Kalra]

From: Ashish Kalra 

This hypercall is used by the SEV guest to notify a change in the page
encryption status to the hypervisor. The hypercall should be invoked
only when the encryption attribute is changed from encrypted -> decrypted
and vice versa. By default all guest pages are considered encrypted.

The hypercall exits to userspace to manage the guest shared regions and
integrate with the userspace VMM's migration code.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
Co-developed-by: Sean Christopherson 
Signed-off-by: Sean Christopherson 
---
 Documentation/virt/kvm/hypercalls.rst | 15 ++
 arch/x86/include/asm/kvm_host.h   |  2 ++
 arch/x86/kvm/svm/sev.c|  1 +
 arch/x86/kvm/x86.c| 29 +++
 include/uapi/linux/kvm_para.h |  1 +
 5 files changed, 48 insertions(+)

diff --git a/Documentation/virt/kvm/hypercalls.rst 
b/Documentation/virt/kvm/hypercalls.rst
index ed4fddd364ea..7aff0cebab7c 100644
--- a/Documentation/virt/kvm/hypercalls.rst
+++ b/Documentation/virt/kvm/hypercalls.rst
@@ -169,3 +169,18 @@ a0: destination APIC ID
 
 :Usage example: When sending a call-function IPI-many to vCPUs, yield if
any of the IPI target vCPUs was preempted.
+
+
+8. KVM_HC_PAGE_ENC_STATUS
+-
+:Architecture: x86
+:Status: active
+:Purpose: Notify the encryption status changes in guest page table (SEV guest)
+
+a0: the guest physical address of the start page
+a1: the number of pages
+a2: encryption attribute
+
+   Where:
+   * 1: Encryption attribute is set
+   * 0: Encryption attribute is cleared
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 3768819693e5..42eb0fe3df5d 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1050,6 +1050,8 @@ struct kvm_arch {
 
bool bus_lock_detection_enabled;
 
+   bool page_enc_hc_enable;
+
/* Deflect RDMSR and WRMSR to user space when they trigger a #GP */
u32 user_space_msr_mask;
struct kvm_x86_msr_filter __rcu *msr_filter;
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index c9795a22e502..5184a0c0131a 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -197,6 +197,7 @@ static int sev_guest_init(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
sev->active = true;
sev->asid = asid;
INIT_LIST_HEAD(>regions_list);
+   kvm->arch.page_enc_hc_enable = true;
 
return 0;
 
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index f7d12fca397b..e8986478b653 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8208,6 +8208,13 @@ static void kvm_sched_yield(struct kvm *kvm, unsigned 
long dest_id)
kvm_vcpu_yield_to(target);
 }
 
+static int complete_hypercall_exit(struct kvm_vcpu *vcpu)
+{
+   kvm_rax_write(vcpu, vcpu->run->hypercall.ret);
+   ++vcpu->stat.hypercalls;
+   return kvm_skip_emulated_instruction(vcpu);
+}
+
 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
 {
unsigned long nr, a0, a1, a2, a3, ret;
@@ -8273,6 +8280,28 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
kvm_sched_yield(vcpu->kvm, a0);
ret = 0;
break;
+   case KVM_HC_PAGE_ENC_STATUS: {
+   u64 gpa = a0, npages = a1, enc = a2;
+
+   ret = -KVM_ENOSYS;
+   if (!vcpu->kvm->arch.page_enc_hc_enable)
+   break;
+
+   if (!PAGE_ALIGNED(gpa) || !npages ||
+   gpa_to_gfn(gpa) + npages <= gpa_to_gfn(gpa)) {
+   ret = -EINVAL;
+   break;
+   }
+
+   vcpu->run->exit_reason= KVM_EXIT_HYPERCALL;
+   vcpu->run->hypercall.nr   = KVM_HC_PAGE_ENC_STATUS;
+   vcpu->run->hypercall.args[0]  = gpa;
+   vcpu->run->hypercall.args[1]  = npages;
+   vcpu->run->hypercall.args[2]  = enc;
+   vcpu->run->hypercall.longmode = op_64_bit;
+   vcpu->arch.complete_userspace_io = complete_hypercall_exit;
+   return 0;
+   }
default:
ret = -KVM_ENOSYS;
break;
diff --git a/include/uapi/linux/kvm_para.h b/include/uapi/linux/kvm_para.h
index 8b86609849b9..847b83b75dc8 100644
--- a/include/uapi/linux/kvm_para.h
+++ b/include/uapi/linux/kvm_para.h
@@ -29,6 +29,7 @@
 #define KVM_HC_CLOCK_PAIRING   9
 #define KVM_HC_SEND_IPI10
 #define KVM_HC_SCHED_YIELD 11
+#define KVM_HC_PAGE_ENC_STATUS 12
 
 /*
  * hypercalls use architecture specific
-- 
2.17.1

[PATCH v12 05/13] KVM: SVM: Add KVM_SEV_RECEIVE_UPDATE_DATA command

[Ashish Kalra]

From: Brijesh Singh 

The command is used for copying the incoming buffer into the
SEV guest memory space.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst| 24 ++
 arch/x86/kvm/svm/sev.c| 79 +++
 include/uapi/linux/kvm.h  |  9 +++
 3 files changed, 112 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index c86c1ded8dd8..c6ed5b26d841 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -372,6 +372,30 @@ On success, the 'handle' field contains a new handle and 
on error, a negative va
 
 For more details, see SEV spec Section 6.12.
 
+14. KVM_SEV_RECEIVE_UPDATE_DATA
+
+
+The KVM_SEV_RECEIVE_UPDATE_DATA command can be used by the hypervisor to copy
+the incoming buffers into the guest memory region with encryption context
+created during the KVM_SEV_RECEIVE_START.
+
+Parameters (in): struct kvm_sev_receive_update_data
+
+Returns: 0 on success, -negative on error
+
+::
+
+struct kvm_sev_launch_receive_update_data {
+__u64 hdr_uaddr;/* userspace address containing the 
packet header */
+__u32 hdr_len;
+
+__u64 guest_uaddr;  /* the destination guest memory region 
*/
+__u32 guest_len;
+
+__u64 trans_uaddr;  /* the incoming buffer memory region  
*/
+__u32 trans_len;
+};
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index e530c2b34b5e..2c95657cc9bf 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1448,6 +1448,82 @@ static int sev_receive_start(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_receive_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct kvm_sev_receive_update_data params;
+   struct sev_data_receive_update_data *data;
+   void *hdr = NULL, *trans = NULL;
+   struct page **guest_page;
+   unsigned long n;
+   int ret, offset;
+
+   if (!sev_guest(kvm))
+   return -EINVAL;
+
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_receive_update_data)))
+   return -EFAULT;
+
+   if (!params.hdr_uaddr || !params.hdr_len ||
+   !params.guest_uaddr || !params.guest_len ||
+   !params.trans_uaddr || !params.trans_len)
+   return -EINVAL;
+
+   /* Check if we are crossing the page boundary */
+   offset = params.guest_uaddr & (PAGE_SIZE - 1);
+   if ((params.guest_len + offset > PAGE_SIZE))
+   return -EINVAL;
+
+   hdr = psp_copy_user_blob(params.hdr_uaddr, params.hdr_len);
+   if (IS_ERR(hdr))
+   return PTR_ERR(hdr);
+
+   trans = psp_copy_user_blob(params.trans_uaddr, params.trans_len);
+   if (IS_ERR(trans)) {
+   ret = PTR_ERR(trans);
+   goto e_free_hdr;
+   }
+
+   ret = -ENOMEM;
+   data = kzalloc(sizeof(*data), GFP_KERNEL);
+   if (!data)
+   goto e_free_trans;
+
+   data->hdr_address = __psp_pa(hdr);
+   data->hdr_len = params.hdr_len;
+   data->trans_address = __psp_pa(trans);
+   data->trans_len = params.trans_len;
+
+   /* Pin guest memory */
+   ret = -EFAULT;
+   guest_page = sev_pin_memory(kvm, params.guest_uaddr & PAGE_MASK,
+   PAGE_SIZE, , 0);
+   if (!guest_page)
+   goto e_free;
+
+   /* The RECEIVE_UPDATE_DATA command requires C-bit to be always set. */
+   data->guest_address = (page_to_pfn(guest_page[0]) << PAGE_SHIFT) +
+   offset;
+   data->guest_address |= sev_me_mask;
+   data->guest_len = params.guest_len;
+   data->handle = sev->handle;
+
+   ret = sev_issue_cmd(kvm, SEV_CMD_RECEIVE_UPDATE_DATA, data,
+   >error);
+
+   sev_unpin_memory(kvm, guest_page, n);
+
+e_free:
+   kfree(data);
+e_free_trans:
+   kfree(trans);
+e_free_hdr:
+   kfree(hdr);
+
+   return ret;
+}
+
 int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
 {
struct kvm_sev_cmd sev_cmd;
@@ -1513,6 +1589,9 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
case KVM_SEV_RECEIVE_START:
r = sev_receive_start(kvm, _cmd);
break;
+   case KVM_SEV_RECEIVE_UPD

[PATCH v12 03/13] KVM: SVM: Add KVM_SEV_SEND_FINISH command

[Ashish Kalra]

From: Brijesh Singh 

The command is used to finailize the encryption context created with
KVM_SEV_SEND_START command.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  8 +++
 arch/x86/kvm/svm/sev.c| 23 +++
 2 files changed, 31 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index 3c5456e0268a..26c4e6c83f62 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -335,6 +335,14 @@ Returns: 0 on success, -negative on error
 __u32 trans_len;
 };
 
+12. KVM_SEV_SEND_FINISH
+
+
+After completion of the migration flow, the KVM_SEV_SEND_FINISH command can be
+issued by the hypervisor to delete the encryption context.
+
+Returns: 0 on success, -negative on error
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 30527285a39a..92325d9527ce 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1350,6 +1350,26 @@ static int sev_send_update_data(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_send_finish(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_finish *data;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL);
+   if (!data)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_SEND_FINISH, data, >error);
+
+   kfree(data);
+   return ret;
+}
+
 int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
 {
struct kvm_sev_cmd sev_cmd;
@@ -1409,6 +1429,9 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
case KVM_SEV_SEND_UPDATE_DATA:
r = sev_send_update_data(kvm, _cmd);
break;
+   case KVM_SEV_SEND_FINISH:
+   r = sev_send_finish(kvm, _cmd);
+   break;
default:
r = -EINVAL;
goto out;
-- 
2.17.1

[PATCH v12 01/13] KVM: SVM: Add KVM_SEV SEND_START command

[Ashish Kalra]

From: Brijesh Singh 

The command is used to create an outgoing SEV guest encryption context.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Reviewed-by: Venu Busireddy 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  27 
 arch/x86/kvm/svm/sev.c| 125 ++
 include/linux/psp-sev.h   |   8 +-
 include/uapi/linux/kvm.h  |  12 ++
 4 files changed, 168 insertions(+), 4 deletions(-)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index 469a6308765b..ac799dd7a618 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -284,6 +284,33 @@ Returns: 0 on success, -negative on error
 __u32 len;
 };
 
+10. KVM_SEV_SEND_START
+--
+
+The KVM_SEV_SEND_START command can be used by the hypervisor to create an
+outgoing guest encryption context.
+
+Parameters (in): struct kvm_sev_send_start
+
+Returns: 0 on success, -negative on error
+
+::
+struct kvm_sev_send_start {
+__u32 policy; /* guest policy */
+
+__u64 pdh_cert_uaddr; /* platform Diffie-Hellman 
certificate */
+__u32 pdh_cert_len;
+
+__u64 plat_certs_uaddr;/* platform certificate chain */
+__u32 plat_certs_len;
+
+__u64 amd_certs_uaddr;/* AMD certificate */
+__u32 amd_certs_len;
+
+__u64 session_uaddr;  /* Guest session information */
+__u32 session_len;
+};
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 874ea309279f..2b65900c05d6 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1110,6 +1110,128 @@ static int sev_get_attestation_report(struct kvm *kvm, 
struct kvm_sev_cmd *argp)
return ret;
 }
 
+/* Userspace wants to query session length. */
+static int
+__sev_send_start_query_session_length(struct kvm *kvm, struct kvm_sev_cmd 
*argp,
+ struct kvm_sev_send_start *params)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_start *data;
+   int ret;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL_ACCOUNT);
+   if (data == NULL)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_SEND_START, data, >error);
+
+   params->session_len = data->session_len;
+   if (copy_to_user((void __user *)(uintptr_t)argp->data, params,
+   sizeof(struct kvm_sev_send_start)))
+   ret = -EFAULT;
+
+   kfree(data);
+   return ret;
+}
+
+static int sev_send_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_start *data;
+   struct kvm_sev_send_start params;
+   void *amd_certs, *session_data;
+   void *pdh_cert, *plat_certs;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_send_start)))
+   return -EFAULT;
+
+   /* if session_len is zero, userspace wants to query the session length 
*/
+   if (!params.session_len)
+   return __sev_send_start_query_session_length(kvm, argp,
+   );
+
+   /* some sanity checks */
+   if (!params.pdh_cert_uaddr || !params.pdh_cert_len ||
+   !params.session_uaddr || params.session_len > SEV_FW_BLOB_MAX_SIZE)
+   return -EINVAL;
+
+   /* allocate the memory to hold the session data blob */
+   session_data = kmalloc(params.session_len, GFP_KERNEL_ACCOUNT);
+   if (!session_data)
+   return -ENOMEM;
+
+   /* copy the certificate blobs from userspace */
+   pdh_cert = psp_copy_user_blob(params.pdh_cert_uaddr,
+   params.pdh_cert_len);
+   if (IS_ERR(pdh_cert)) {
+   ret = PTR_ERR(pdh_cert);
+   goto e_free_session;
+   }
+
+   plat_certs = psp_copy_user_blob(params.plat_certs_uaddr,
+   params.plat_certs_len);
+   if (IS_ERR(plat_certs)) {
+   ret = PTR_ERR(plat_certs);
+   goto e_free_pdh;
+   }
+
+   amd_certs = psp_copy_user_blob(params.amd_certs_uaddr,
+   params.amd_certs_len);
+   if (IS_ERR(amd_certs)) {

[PATCH v12 00/13] Add AMD SEV guest live migration support

[Ashish Kalra]

From: Ashish Kalra 

The series add support for AMD SEV guest live migration commands. To protect the
confidentiality of an SEV protected guest memory while in transit we need to
use the SEV commands defined in SEV API spec [1].

SEV guest VMs have the concept of private and shared memory. Private memory
is encrypted with the guest-specific key, while shared memory may be encrypted
with hypervisor key. The commands provided by the SEV FW are meant to be used
for the private memory only. The patch series introduces a new hypercall.
The guest OS can use this hypercall to notify the page encryption status.
If the page is encrypted with guest specific-key then we use SEV command during
the migration. If page is not encrypted then fallback to default.

The patch uses the KVM_EXIT_HYPERCALL exitcode and hypercall to
userspace exit functionality as a common interface from the guest back to the
VMM and passing on the guest shared/unencrypted page information to the
userspace VMM/Qemu. Qemu can consult this information during migration to know 
whether the page is encrypted.

This section descibes how the SEV live migration feature is negotiated
between the host and guest, the host indicates this feature support via 
KVM_FEATURE_CPUID. The guest firmware (OVMF) detects this feature and
sets a UEFI enviroment variable indicating OVMF support for live
migration, the guest kernel also detects the host support for this
feature via cpuid and in case of an EFI boot verifies if OVMF also
supports this feature by getting the UEFI enviroment variable and if it
set then enables live migration feature on host by writing to a custom
MSR, if not booted under EFI, then it simply enables the feature by
again writing to the custom MSR. The MSR is also handled by the
userspace VMM/Qemu.

A branch containing these patches is available here:
https://github.com/AMDESE/linux/tree/sev-migration-v12

[1] https://developer.amd.com/wp-content/resources/55766.PDF

Changes since v11:
- Clean up and remove kvm_x86_ops callback for page_enc_status_hc and
  instead add a new per-VM flag to support/enable the page encryption
  status hypercall.
- Remove KVM_EXIT_DMA_SHARE/KVM_EXIT_DMA_UNSHARE exitcodes and instead
  use the KVM_EXIT_HYPERCALL exitcode for page encryption status
  hypercall to userspace functionality. 

Changes since v10:
- Adds new KVM_EXIT_DMA_SHARE/KVM_EXIT_DMA_UNSHARE hypercall to
  userspace exit functionality as a common interface from the guest back to the
  KVM and passing on the guest shared/unencrypted region information to the
  userspace VMM/Qemu. KVM/host kernel does not maintain the guest shared
  memory regions information anymore. 
- Remove implicit enabling of SEV live migration feature for an SEV
  guest, now this is explicitly in control of the userspace VMM/Qemu.
- Custom MSR handling is also now moved into userspace VMM/Qemu.
- As KVM does not maintain the guest shared memory region information
  anymore, sev_dbg_crypt() cannot bypass unencrypted guest memory
  regions without support from userspace VMM/Qemu.

Changes since v9:
- Transitioning from page encryption bitmap to the shared pages list
  to keep track of guest's shared/unencrypted memory regions.
- Move back to marking the complete _bss_decrypted section as 
  decrypted in the shared pages list.
- Invoke a new function check_kvm_sev_migration() via kvm_init_platform()
  for guest to query for host-side support for SEV live migration 
  and to enable the SEV live migration feature, to avoid
  #ifdefs in code 
- Rename MSR_KVM_SEV_LIVE_MIG_EN to MSR_KVM_SEV_LIVE_MIGRATION.
- Invoke a new function handle_unencrypted_region() from 
  sev_dbg_crypt() to bypass unencrypted guest memory regions.

Changes since v8:
- Rebasing to kvm next branch.
- Fixed and added comments as per review feedback on v8 patches.
- Removed implicitly enabling live migration for incoming VMs in
  in KVM_SET_PAGE_ENC_BITMAP, it is now done via KVM_SET_MSR ioctl.
- Adds support for bypassing unencrypted guest memory regions for
  DBG_DECRYPT API calls, guest memory region encryption status in
  sev_dbg_decrypt() is referenced using the page encryption bitmap.

Changes since v7:
- Removed the hypervisor specific hypercall/paravirt callback for
  SEV live migration and moved back to calling kvm_sev_hypercall3 
  directly.
- Fix build errors as
  Reported-by: kbuild test robot , specifically fixed
  build error when CONFIG_HYPERVISOR_GUEST=y and
  CONFIG_AMD_MEM_ENCRYPT=n.
- Implicitly enabled live migration for incoming VM(s) to handle 
  A->B->C->... VM migrations.
- Fixed Documentation as per comments on v6 patches.
- Fixed error return path in sev_send_update_data() as per comments 
  on v6 patches. 

Changes since v6:
- Rebasing to mainline and refactoring to the new split SVM
  infrastructre.
- Move to static allocation of the unified Page Encryption bitmap
  instead of the dynamic resizing of the bitmap, the static allocation
  is done implicitly by 

Re: [PATCH v11 08/13] KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall

[Ashish Kalra]

On Tue, Apr 06, 2021 at 03:48:20PM +, Sean Christopherson wrote:
> On Mon, Apr 05, 2021, Ashish Kalra wrote:
> > From: Ashish Kalra 
> 
> ...
> 
> > diff --git a/arch/x86/include/asm/kvm_host.h 
> > b/arch/x86/include/asm/kvm_host.h
> > index 3768819693e5..78284ebbbee7 100644
> > --- a/arch/x86/include/asm/kvm_host.h
> > +++ b/arch/x86/include/asm/kvm_host.h
> > @@ -1352,6 +1352,8 @@ struct kvm_x86_ops {
> > int (*complete_emulated_msr)(struct kvm_vcpu *vcpu, int err);
> >  
> > void (*vcpu_deliver_sipi_vector)(struct kvm_vcpu *vcpu, u8 vector);
> > +   int (*page_enc_status_hc)(struct kvm_vcpu *vcpu, unsigned long gpa,
> > + unsigned long sz, unsigned long mode);
> >  };
> >  
> >  struct kvm_x86_nested_ops {
> > diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
> > index c9795a22e502..fb3a315e5827 100644
> > --- a/arch/x86/kvm/svm/sev.c
> > +++ b/arch/x86/kvm/svm/sev.c
> > @@ -1544,6 +1544,67 @@ static int sev_receive_finish(struct kvm *kvm, 
> > struct kvm_sev_cmd *argp)
> > return ret;
> >  }
> >  
> > +static int sev_complete_userspace_page_enc_status_hc(struct kvm_vcpu *vcpu)
> > +{
> > +   vcpu->run->exit_reason = 0;
> > +   kvm_rax_write(vcpu, vcpu->run->dma_sharing.ret);
> > +   ++vcpu->stat.hypercalls;
> > +   return kvm_skip_emulated_instruction(vcpu);
> > +}
> > +
> > +int svm_page_enc_status_hc(struct kvm_vcpu *vcpu, unsigned long gpa,
> > +  unsigned long npages, unsigned long enc)
> > +{
> > +   kvm_pfn_t pfn_start, pfn_end;
> > +   struct kvm *kvm = vcpu->kvm;
> > +   gfn_t gfn_start, gfn_end;
> > +
> > +   if (!sev_guest(kvm))
> > +   return -EINVAL;
> > +
> > +   if (!npages)
> > +   return 0;
> 
> Parth of me thinks passing a zero size should be an error not a nop.  Either 
> way
> works, just feels a bit weird to allow this to be a nop.
> 
> > +
> > +   gfn_start = gpa_to_gfn(gpa);
> 
> This should check that @gpa is aligned.
> 
> > +   gfn_end = gfn_start + npages;
> > +
> > +   /* out of bound access error check */
> > +   if (gfn_end <= gfn_start)
> > +   return -EINVAL;
> > +
> > +   /* lets make sure that gpa exist in our memslot */
> > +   pfn_start = gfn_to_pfn(kvm, gfn_start);
> > +   pfn_end = gfn_to_pfn(kvm, gfn_end);
> > +
> > +   if (is_error_noslot_pfn(pfn_start) && !is_noslot_pfn(pfn_start)) {
> > +   /*
> > +* Allow guest MMIO range(s) to be added
> > +* to the shared pages list.
> > +*/
> > +   return -EINVAL;
> > +   }
> > +
> > +   if (is_error_noslot_pfn(pfn_end) && !is_noslot_pfn(pfn_end)) {
> > +   /*
> > +* Allow guest MMIO range(s) to be added
> > +* to the shared pages list.
> > +*/
> > +   return -EINVAL;
> > +   }
> 
> I don't think KVM should do any checks beyond gfn_end <= gfn_start.  Just punt
> to userspace and give userspace full say over what is/isn't legal.
> 
> > +
> > +   if (enc)
> > +   vcpu->run->exit_reason = KVM_EXIT_DMA_UNSHARE;
> > +   else
> > +   vcpu->run->exit_reason = KVM_EXIT_DMA_SHARE;
> 
> Use a single exit and pass "enc" via kvm_run.  I also strongly dislike "DMA",
> there's no guarantee the guest is sharing memory for DMA.
> 
> I think we can usurp KVM_EXIT_HYPERCALL for this?  E.g.
> 
>   vcpu->run->exit_reason= KVM_EXIT_HYPERCALL;
>   vcpu->run->hypercall.nr   = KVM_HC_PAGE_ENC_STATUS;
>   vcpu->run->hypercall.args[0]  = gfn_start << PAGE_SHIFT;
>   vcpu->run->hypercall.args[1]  = npages * PAGE_SIZE;
>   vcpu->run->hypercall.args[2]  = enc;
>   vcpu->run->hypercall.longmode = is_64_bit_mode(vcpu);
> 
> > +
> > +   vcpu->run->dma_sharing.addr = gfn_start;
> 
> Addresses and pfns are not the same thing.  If you're passing the size in 
> bytes,
> then it's probably best to pass the gpa, not the gfn.  Same for the params 
> from
> the guest, they should be in the same "domain".
> 
> > +   vcpu->run->dma_sharing.len = npages * PAGE_SIZE;
> > +   vcpu->arch.complete_userspace_io =
> > +   sev_complete_userspace_page_enc_status_hc;
> 
> I vote to drop the "userspace" part, it's already quite verbose.
> 
&g

Re: [PATCH v11 08/13] KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall

[Ashish Kalra]

On Tue, Apr 06, 2021 at 06:22:48AM +, Ashish Kalra wrote:
> On Mon, Apr 05, 2021 at 01:42:42PM -0700, Steve Rutherford wrote:
> > On Mon, Apr 5, 2021 at 7:28 AM Ashish Kalra  wrote:
> > >
> > > From: Ashish Kalra 
> > >
> > > This hypercall is used by the SEV guest to notify a change in the page
> > > encryption status to the hypervisor. The hypercall should be invoked
> > > only when the encryption attribute is changed from encrypted -> decrypted
> > > and vice versa. By default all guest pages are considered encrypted.
> > >
> > > The hypercall exits to userspace to manage the guest shared regions and
> > > integrate with the userspace VMM's migration code.
> > >
> > > The patch integrates and extends DMA_SHARE/UNSHARE hypercall to
> > > userspace exit functionality (arm64-specific) patch from Marc Zyngier,
> > > to avoid arch-specific stuff and have a common interface
> > > from the guest back to the VMM and sharing of the host handling of the
> > > hypercall to support use case for a guest to share memory with a host.
> > >
> > > Cc: Thomas Gleixner 
> > > Cc: Ingo Molnar 
> > > Cc: "H. Peter Anvin" 
> > > Cc: Paolo Bonzini 
> > > Cc: Joerg Roedel 
> > > Cc: Borislav Petkov 
> > > Cc: Tom Lendacky 
> > > Cc: x...@kernel.org
> > > Cc: k...@vger.kernel.org
> > > Cc: linux-kernel@vger.kernel.org
> > > Signed-off-by: Brijesh Singh 
> > > Signed-off-by: Ashish Kalra 
> > > ---
> > >  Documentation/virt/kvm/api.rst| 18 
> > >  Documentation/virt/kvm/hypercalls.rst | 15 +++
> > >  arch/x86/include/asm/kvm_host.h   |  2 +
> > >  arch/x86/kvm/svm/sev.c| 61 +++
> > >  arch/x86/kvm/svm/svm.c|  2 +
> > >  arch/x86/kvm/svm/svm.h|  2 +
> > >  arch/x86/kvm/vmx/vmx.c|  1 +
> > >  arch/x86/kvm/x86.c| 12 ++
> > >  include/uapi/linux/kvm.h  |  8 
> > >  include/uapi/linux/kvm_para.h |  1 +
> > >  10 files changed, 122 insertions(+)
> > >
> > > diff --git a/Documentation/virt/kvm/api.rst 
> > > b/Documentation/virt/kvm/api.rst
> > > index 307f2fcf1b02..52bd7e475fd6 100644
> > > --- a/Documentation/virt/kvm/api.rst
> > > +++ b/Documentation/virt/kvm/api.rst
> > > @@ -5475,6 +5475,24 @@ Valid values for 'type' are:
> > >  Userspace is expected to place the hypercall result into the 
> > > appropriate
> > >  field before invoking KVM_RUN again.
> > >
> > > +::
> > > +
> > > +   /* KVM_EXIT_DMA_SHARE / KVM_EXIT_DMA_UNSHARE */
> > > +   struct {
> > > +   __u64 addr;
> > > +   __u64 len;
> > > +   __u64 ret;
> > > +   } dma_sharing;
> > > +
> > > +This defines a common interface from the guest back to the KVM to support
> > > +use case for a guest to share memory with a host.
> > > +
> > > +The addr and len fields define the starting address and length of the
> > > +shared memory region.
> > > +
> > > +Userspace is expected to place the hypercall result into the "ret" field
> > > +before invoking KVM_RUN again.
> > > +
> > >  ::
> > >
> > > /* Fix the size of the union. */
> > > diff --git a/Documentation/virt/kvm/hypercalls.rst 
> > > b/Documentation/virt/kvm/hypercalls.rst
> > > index ed4fddd364ea..7aff0cebab7c 100644
> > > --- a/Documentation/virt/kvm/hypercalls.rst
> > > +++ b/Documentation/virt/kvm/hypercalls.rst
> > > @@ -169,3 +169,18 @@ a0: destination APIC ID
> > >
> > >  :Usage example: When sending a call-function IPI-many to vCPUs, yield if
> > > any of the IPI target vCPUs was preempted.
> > > +
> > > +
> > > +8. KVM_HC_PAGE_ENC_STATUS
> > > +-
> > > +:Architecture: x86
> > > +:Status: active
> > > +:Purpose: Notify the encryption status changes in guest page table (SEV 
> > > guest)
> > > +
> > > +a0: the guest physical address of the start page
> > > +a1: the number of pages
> > > +a2: encryption attribute
> > > +
> > > +   Where:
> > > +   * 1: Encryption attribute is set
> > > +   * 0: E

Re: [PATCH v11 08/13] KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall

[Ashish Kalra]

On Tue, Apr 06, 2021 at 03:48:20PM +, Sean Christopherson wrote:
> On Mon, Apr 05, 2021, Ashish Kalra wrote:
> > From: Ashish Kalra 
> 
> ...
> 
> > diff --git a/arch/x86/include/asm/kvm_host.h 
> > b/arch/x86/include/asm/kvm_host.h
> > index 3768819693e5..78284ebbbee7 100644
> > --- a/arch/x86/include/asm/kvm_host.h
> > +++ b/arch/x86/include/asm/kvm_host.h
> > @@ -1352,6 +1352,8 @@ struct kvm_x86_ops {
> > int (*complete_emulated_msr)(struct kvm_vcpu *vcpu, int err);
> >  
> > void (*vcpu_deliver_sipi_vector)(struct kvm_vcpu *vcpu, u8 vector);
> > +   int (*page_enc_status_hc)(struct kvm_vcpu *vcpu, unsigned long gpa,
> > + unsigned long sz, unsigned long mode);
> >  };
> >  
> >  struct kvm_x86_nested_ops {
> > diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
> > index c9795a22e502..fb3a315e5827 100644
> > --- a/arch/x86/kvm/svm/sev.c
> > +++ b/arch/x86/kvm/svm/sev.c
> > @@ -1544,6 +1544,67 @@ static int sev_receive_finish(struct kvm *kvm, 
> > struct kvm_sev_cmd *argp)
> > return ret;
> >  }
> >  
> > +static int sev_complete_userspace_page_enc_status_hc(struct kvm_vcpu *vcpu)
> > +{
> > +   vcpu->run->exit_reason = 0;
> > +   kvm_rax_write(vcpu, vcpu->run->dma_sharing.ret);
> > +   ++vcpu->stat.hypercalls;
> > +   return kvm_skip_emulated_instruction(vcpu);
> > +}
> > +
> > +int svm_page_enc_status_hc(struct kvm_vcpu *vcpu, unsigned long gpa,
> > +  unsigned long npages, unsigned long enc)
> > +{
> > +   kvm_pfn_t pfn_start, pfn_end;
> > +   struct kvm *kvm = vcpu->kvm;
> > +   gfn_t gfn_start, gfn_end;
> > +
> > +   if (!sev_guest(kvm))
> > +   return -EINVAL;
> > +
> > +   if (!npages)
> > +   return 0;
> 
> Parth of me thinks passing a zero size should be an error not a nop.  Either 
> way
> works, just feels a bit weird to allow this to be a nop.
> 
> > +
> > +   gfn_start = gpa_to_gfn(gpa);
> 
> This should check that @gpa is aligned.
> 
> > +   gfn_end = gfn_start + npages;
> > +
> > +   /* out of bound access error check */
> > +   if (gfn_end <= gfn_start)
> > +   return -EINVAL;
> > +
> > +   /* lets make sure that gpa exist in our memslot */
> > +   pfn_start = gfn_to_pfn(kvm, gfn_start);
> > +   pfn_end = gfn_to_pfn(kvm, gfn_end);
> > +
> > +   if (is_error_noslot_pfn(pfn_start) && !is_noslot_pfn(pfn_start)) {
> > +   /*
> > +* Allow guest MMIO range(s) to be added
> > +* to the shared pages list.
> > +*/
> > +   return -EINVAL;
> > +   }
> > +
> > +   if (is_error_noslot_pfn(pfn_end) && !is_noslot_pfn(pfn_end)) {
> > +   /*
> > +* Allow guest MMIO range(s) to be added
> > +* to the shared pages list.
> > +*/
> > +   return -EINVAL;
> > +   }
> 
> I don't think KVM should do any checks beyond gfn_end <= gfn_start.  Just punt
> to userspace and give userspace full say over what is/isn't legal.
> 
> > +
> > +   if (enc)
> > +   vcpu->run->exit_reason = KVM_EXIT_DMA_UNSHARE;
> > +   else
> > +   vcpu->run->exit_reason = KVM_EXIT_DMA_SHARE;
> 
> Use a single exit and pass "enc" via kvm_run.  I also strongly dislike "DMA",
> there's no guarantee the guest is sharing memory for DMA.
> 
> I think we can usurp KVM_EXIT_HYPERCALL for this?  E.g.
> 

I see the following in Documentation/virt/kvm/api.rst :
..
..
/* KVM_EXIT_HYPERCALL */
struct {
__u64 nr;
__u64 args[6];
__u64 ret;
__u32 longmode;
__u32 pad;
} hypercall;

Unused.  This was once used for 'hypercall to userspace'.  To implement
such functionality, use KVM_EXIT_IO (x86) or KVM_EXIT_MMIO (all except s390).

This mentions this exitcode to be unused and implementing this
functionality using KVM_EXIT_IO for x86?

Thanks,
Ashish

>   vcpu->run->exit_reason= KVM_EXIT_HYPERCALL;
>   vcpu->run->hypercall.nr   = KVM_HC_PAGE_ENC_STATUS;
>   vcpu->run->hypercall.args[0]  = gfn_start << PAGE_SHIFT;
>   vcpu->run->hypercall.args[1]  = npages * PAGE_SIZE;
>   vcpu->run->hypercall.args[2]  = enc;
>   vcpu->run->hypercall.longmode = is_64_bit_mode(vcpu);
> 
> > +
> > +   vcpu->run->dma_sha

Re: [PATCH v11 10/13] KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature & Custom MSR.

[Ashish Kalra]

Hello Paolo,

On Tue, Apr 06, 2021 at 03:47:59PM +0200, Paolo Bonzini wrote:
> On 06/04/21 15:26, Ashish Kalra wrote:
> > > It's a little unintuitive to see KVM_MSR_RET_FILTERED here, since
> > > userspace can make this happen on its own without having an entry in
> > > this switch statement (by setting it in the msr filter bitmaps). When
> > > using MSR filters, I would only expect to get MSR filter exits for
> > > MSRs I specifically asked for.
> > > 
> > > Not a huge deal, just a little unintuitive. I'm not sure other options
> > > are much better (you could put KVM_MSR_RET_INVALID, or you could just
> > > not have these entries in svm_{get,set}_msr).
> > > 
> > Actually KVM_MSR_RET_FILTERED seems more logical to use, especially in
> > comparison with KVM_MSR_RET_INVALID.
> > 
> > Also, hooking this msr in svm_{get,set}_msr allows some in-kernel error
> > pre-processsing before doing the pass-through to userspace.
> 
> I agree that it should be up to userspace to set up the filter since we now
> have that functionality.
> 

The userspace is still setting up the filter and handling this MSR, it
is only some basic error pre-processing being done in-kernel here.

Thanks,
Ashish

> Let me read the whole threads for the past versions to see what the
> objections were...
> 
> Paolo
> 

Re: [PATCH v11 10/13] KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature & Custom MSR.

[Ashish Kalra]

Hello Steve,

On Mon, Apr 05, 2021 at 06:39:03PM -0700, Steve Rutherford wrote:
> On Mon, Apr 5, 2021 at 7:30 AM Ashish Kalra  wrote:
> >
> > From: Ashish Kalra 
> >
> > Add new KVM_FEATURE_SEV_LIVE_MIGRATION feature for guest to check
> > for host-side support for SEV live migration. Also add a new custom
> > MSR_KVM_SEV_LIVE_MIGRATION for guest to enable the SEV live migration
> > feature.
> >
> > MSR is handled by userspace using MSR filters.
> >
> > Signed-off-by: Ashish Kalra 
> > ---
> >  Documentation/virt/kvm/cpuid.rst |  5 +
> >  Documentation/virt/kvm/msr.rst   | 12 
> >  arch/x86/include/uapi/asm/kvm_para.h |  4 
> >  arch/x86/kvm/cpuid.c |  3 ++-
> >  arch/x86/kvm/svm/svm.c   | 22 ++
> >  5 files changed, 45 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/virt/kvm/cpuid.rst 
> > b/Documentation/virt/kvm/cpuid.rst
> > index cf62162d4be2..0bdb6cdb12d3 100644
> > --- a/Documentation/virt/kvm/cpuid.rst
> > +++ b/Documentation/virt/kvm/cpuid.rst
> > @@ -96,6 +96,11 @@ KVM_FEATURE_MSI_EXT_DEST_ID15  guest 
> > checks this feature bit
> > before using extended 
> > destination
> > ID bits in MSI address bits 
> > 11-5.
> >
> > +KVM_FEATURE_SEV_LIVE_MIGRATION 16  guest checks this feature 
> > bit before
> > +   using the page encryption 
> > state
> > +   hypercall to notify the 
> > page state
> > +   change
> > +
> >  KVM_FEATURE_CLOCKSOURCE_STABLE_BIT 24  host will warn if no 
> > guest-side
> > per-cpu warps are expected 
> > in
> > kvmclock
> > diff --git a/Documentation/virt/kvm/msr.rst b/Documentation/virt/kvm/msr.rst
> > index e37a14c323d2..020245d16087 100644
> > --- a/Documentation/virt/kvm/msr.rst
> > +++ b/Documentation/virt/kvm/msr.rst
> > @@ -376,3 +376,15 @@ data:
> > write '1' to bit 0 of the MSR, this causes the host to re-scan its 
> > queue
> > and check if there are more notifications pending. The MSR is 
> > available
> > if KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
> > +
> > +MSR_KVM_SEV_LIVE_MIGRATION:
> > +0x4b564d08
> > +
> > +   Control SEV Live Migration features.
> > +
> > +data:
> > +Bit 0 enables (1) or disables (0) host-side SEV Live Migration 
> > feature,
> > +in other words, this is guest->host communication that it's 
> > properly
> > +handling the shared pages list.
> > +
> > +All other bits are reserved.
> > diff --git a/arch/x86/include/uapi/asm/kvm_para.h 
> > b/arch/x86/include/uapi/asm/kvm_para.h
> > index 950afebfba88..f6bfa138874f 100644
> > --- a/arch/x86/include/uapi/asm/kvm_para.h
> > +++ b/arch/x86/include/uapi/asm/kvm_para.h
> > @@ -33,6 +33,7 @@
> >  #define KVM_FEATURE_PV_SCHED_YIELD 13
> >  #define KVM_FEATURE_ASYNC_PF_INT   14
> >  #define KVM_FEATURE_MSI_EXT_DEST_ID15
> > +#define KVM_FEATURE_SEV_LIVE_MIGRATION 16
> >
> >  #define KVM_HINTS_REALTIME  0
> >
> > @@ -54,6 +55,7 @@
> >  #define MSR_KVM_POLL_CONTROL   0x4b564d05
> >  #define MSR_KVM_ASYNC_PF_INT   0x4b564d06
> >  #define MSR_KVM_ASYNC_PF_ACK   0x4b564d07
> > +#define MSR_KVM_SEV_LIVE_MIGRATION 0x4b564d08
> >
> >  struct kvm_steal_time {
> > __u64 steal;
> > @@ -136,4 +138,6 @@ struct kvm_vcpu_pv_apf_data {
> >  #define KVM_PV_EOI_ENABLED KVM_PV_EOI_MASK
> >  #define KVM_PV_EOI_DISABLED 0x0
> >
> > +#define KVM_SEV_LIVE_MIGRATION_ENABLED BIT_ULL(0)
> > +
> >  #endif /* _UAPI_ASM_X86_KVM_PARA_H */
> > diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
> > index 6bd2f8b830e4..4e2e69a692aa 100644
> > --- a/arch/x86/kvm/cpuid.c
> > +++ b/arch/x86/kvm/cpuid.c
> > @@ -812,7 +812,8 @@ static inline int __do_cpuid_func(struct 
> > kvm_cpuid_array *array, u32 function)
> >  (1 << KVM_FEATURE_PV_SEND_IPI) |
> >  (1 << KVM_FEATURE_POLL_CONTROL) |
> >  (1 << KVM_FEATURE_PV_SCHED_YIELD) |
> > -  

Re: [PATCH v11 08/13] KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall

[Ashish Kalra]

On Mon, Apr 05, 2021 at 01:42:42PM -0700, Steve Rutherford wrote:
> On Mon, Apr 5, 2021 at 7:28 AM Ashish Kalra  wrote:
> >
> > From: Ashish Kalra 
> >
> > This hypercall is used by the SEV guest to notify a change in the page
> > encryption status to the hypervisor. The hypercall should be invoked
> > only when the encryption attribute is changed from encrypted -> decrypted
> > and vice versa. By default all guest pages are considered encrypted.
> >
> > The hypercall exits to userspace to manage the guest shared regions and
> > integrate with the userspace VMM's migration code.
> >
> > The patch integrates and extends DMA_SHARE/UNSHARE hypercall to
> > userspace exit functionality (arm64-specific) patch from Marc Zyngier,
> > to avoid arch-specific stuff and have a common interface
> > from the guest back to the VMM and sharing of the host handling of the
> > hypercall to support use case for a guest to share memory with a host.
> >
> > Cc: Thomas Gleixner 
> > Cc: Ingo Molnar 
> > Cc: "H. Peter Anvin" 
> > Cc: Paolo Bonzini 
> > Cc: Joerg Roedel 
> > Cc: Borislav Petkov 
> > Cc: Tom Lendacky 
> > Cc: x...@kernel.org
> > Cc: k...@vger.kernel.org
> > Cc: linux-kernel@vger.kernel.org
> > Signed-off-by: Brijesh Singh 
> > Signed-off-by: Ashish Kalra 
> > ---
> >  Documentation/virt/kvm/api.rst| 18 
> >  Documentation/virt/kvm/hypercalls.rst | 15 +++
> >  arch/x86/include/asm/kvm_host.h   |  2 +
> >  arch/x86/kvm/svm/sev.c| 61 +++
> >  arch/x86/kvm/svm/svm.c|  2 +
> >  arch/x86/kvm/svm/svm.h|  2 +
> >  arch/x86/kvm/vmx/vmx.c|  1 +
> >  arch/x86/kvm/x86.c| 12 ++
> >  include/uapi/linux/kvm.h  |  8 
> >  include/uapi/linux/kvm_para.h |  1 +
> >  10 files changed, 122 insertions(+)
> >
> > diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
> > index 307f2fcf1b02..52bd7e475fd6 100644
> > --- a/Documentation/virt/kvm/api.rst
> > +++ b/Documentation/virt/kvm/api.rst
> > @@ -5475,6 +5475,24 @@ Valid values for 'type' are:
> >  Userspace is expected to place the hypercall result into the 
> > appropriate
> >  field before invoking KVM_RUN again.
> >
> > +::
> > +
> > +   /* KVM_EXIT_DMA_SHARE / KVM_EXIT_DMA_UNSHARE */
> > +   struct {
> > +   __u64 addr;
> > +   __u64 len;
> > +   __u64 ret;
> > +   } dma_sharing;
> > +
> > +This defines a common interface from the guest back to the KVM to support
> > +use case for a guest to share memory with a host.
> > +
> > +The addr and len fields define the starting address and length of the
> > +shared memory region.
> > +
> > +Userspace is expected to place the hypercall result into the "ret" field
> > +before invoking KVM_RUN again.
> > +
> >  ::
> >
> > /* Fix the size of the union. */
> > diff --git a/Documentation/virt/kvm/hypercalls.rst 
> > b/Documentation/virt/kvm/hypercalls.rst
> > index ed4fddd364ea..7aff0cebab7c 100644
> > --- a/Documentation/virt/kvm/hypercalls.rst
> > +++ b/Documentation/virt/kvm/hypercalls.rst
> > @@ -169,3 +169,18 @@ a0: destination APIC ID
> >
> >  :Usage example: When sending a call-function IPI-many to vCPUs, yield if
> > any of the IPI target vCPUs was preempted.
> > +
> > +
> > +8. KVM_HC_PAGE_ENC_STATUS
> > +-
> > +:Architecture: x86
> > +:Status: active
> > +:Purpose: Notify the encryption status changes in guest page table (SEV 
> > guest)
> > +
> > +a0: the guest physical address of the start page
> > +a1: the number of pages
> > +a2: encryption attribute
> > +
> > +   Where:
> > +   * 1: Encryption attribute is set
> > +   * 0: Encryption attribute is cleared
> > diff --git a/arch/x86/include/asm/kvm_host.h 
> > b/arch/x86/include/asm/kvm_host.h
> > index 3768819693e5..78284ebbbee7 100644
> > --- a/arch/x86/include/asm/kvm_host.h
> > +++ b/arch/x86/include/asm/kvm_host.h
> > @@ -1352,6 +1352,8 @@ struct kvm_x86_ops {
> > int (*complete_emulated_msr)(struct kvm_vcpu *vcpu, int err);
> >
> > void (*vcpu_deliver_sipi_vector)(struct kvm_vcpu *vcpu, u8 vector);
> > +   int (*page_enc_status_hc)(struct kvm_vcpu 

[PATCH v11 13/13] x86/kvm: Add kexec support for SEV Live Migration.

[Ashish Kalra]

From: Ashish Kalra 

Reset the host's shared pages list related to kernel
specific page encryption status settings before we load a
new kernel by kexec. We cannot reset the complete
shared pages list here as we need to retain the
UEFI/OVMF firmware specific settings.

The host's shared pages list is maintained for the
guest to keep track of all unencrypted guest memory regions,
therefore we need to explicitly mark all shared pages as
encrypted again before rebooting into the new guest kernel.

Signed-off-by: Ashish Kalra 
---
 arch/x86/kernel/kvm.c | 24 
 1 file changed, 24 insertions(+)

diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index bcc82e0c9779..4ad3ed547ff1 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -39,6 +39,7 @@
 #include 
 #include 
 #include 
+#include 
 
 DEFINE_STATIC_KEY_FALSE(kvm_async_pf_enabled);
 
@@ -384,6 +385,29 @@ static void kvm_pv_guest_cpu_reboot(void *unused)
 */
if (kvm_para_has_feature(KVM_FEATURE_PV_EOI))
wrmsrl(MSR_KVM_PV_EOI_EN, 0);
+   /*
+* Reset the host's shared pages list related to kernel
+* specific page encryption status settings before we load a
+* new kernel by kexec. NOTE: We cannot reset the complete
+* shared pages list here as we need to retain the
+* UEFI/OVMF firmware specific settings.
+*/
+   if (sev_live_migration_enabled & (smp_processor_id() == 0)) {
+   int i;
+   unsigned long nr_pages;
+
+   for (i = 0; i < e820_table->nr_entries; i++) {
+   struct e820_entry *entry = _table->entries[i];
+
+   if (entry->type != E820_TYPE_RAM)
+   continue;
+
+   nr_pages = DIV_ROUND_UP(entry->size, PAGE_SIZE);
+
+   kvm_sev_hypercall3(KVM_HC_PAGE_ENC_STATUS,
+  entry->addr, nr_pages, 1);
+   }
+   }
kvm_pv_disable_apf();
kvm_disable_steal_time();
 }
-- 
2.17.1

[PATCH v11 11/13] EFI: Introduce the new AMD Memory Encryption GUID.

[Ashish Kalra]

From: Ashish Kalra 

Introduce a new AMD Memory Encryption GUID which is currently
used for defining a new UEFI environment variable which indicates
UEFI/OVMF support for the SEV live migration feature. This variable
is setup when UEFI/OVMF detects host/hypervisor support for SEV
live migration and later this variable is read by the kernel using
EFI runtime services to verify if OVMF supports the live migration
feature.

Signed-off-by: Ashish Kalra 
---
 include/linux/efi.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/linux/efi.h b/include/linux/efi.h
index 6b5d36babfcc..6f364ace82cb 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -362,6 +362,7 @@ void efi_native_runtime_setup(void);
 
 /* OEM GUIDs */
 #define DELLEMC_EFI_RCI2_TABLE_GUIDEFI_GUID(0x2d9f28a2, 0xa886, 
0x456a,  0x97, 0xa8, 0xf1, 0x1e, 0xf2, 0x4f, 0xf4, 0x55)
+#define MEM_ENCRYPT_GUID   EFI_GUID(0x0cf29b71, 0x9e51, 
0x433a,  0xa3, 0xb7, 0x81, 0xf3, 0xab, 0x16, 0xb8, 0x75)
 
 typedef struct {
efi_guid_t guid;
-- 
2.17.1

[PATCH v11 12/13] x86/kvm: Add guest support for detecting and enabling SEV Live Migration feature.

[Ashish Kalra]

From: Ashish Kalra 

The guest support for detecting and enabling SEV Live migration
feature uses the following logic :

 - kvm_init_plaform() invokes check_kvm_sev_migration() which
   checks if its booted under the EFI

   - If not EFI,

 i) check for the KVM_FEATURE_CPUID

 ii) if CPUID reports that migration is supported, issue a wrmsrl()
 to enable the SEV live migration support

   - If EFI,

 i) check for the KVM_FEATURE_CPUID

 ii) If CPUID reports that migration is supported, read the UEFI variable 
which
 indicates OVMF support for live migration

 iii) the variable indicates live migration is supported, issue a wrmsrl() 
to
  enable the SEV live migration support

The EFI live migration check is done using a late_initcall() callback.

Also, ensure that _bss_decrypted section is marked as decrypted in the
shared pages list.

Signed-off-by: Ashish Kalra 
---
 arch/x86/include/asm/mem_encrypt.h |  8 +
 arch/x86/kernel/kvm.c  | 52 ++
 arch/x86/mm/mem_encrypt.c  | 41 +++
 3 files changed, 101 insertions(+)

diff --git a/arch/x86/include/asm/mem_encrypt.h 
b/arch/x86/include/asm/mem_encrypt.h
index 31c4df123aa0..19b77f3a62dc 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -21,6 +21,7 @@
 extern u64 sme_me_mask;
 extern u64 sev_status;
 extern bool sev_enabled;
+extern bool sev_live_migration_enabled;
 
 void sme_encrypt_execute(unsigned long encrypted_kernel_vaddr,
 unsigned long decrypted_kernel_vaddr,
@@ -44,8 +45,11 @@ void __init sme_enable(struct boot_params *bp);
 
 int __init early_set_memory_decrypted(unsigned long vaddr, unsigned long size);
 int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size);
+void __init early_set_mem_enc_dec_hypercall(unsigned long vaddr, int npages,
+   bool enc);
 
 void __init mem_encrypt_free_decrypted_mem(void);
+void __init check_kvm_sev_migration(void);
 
 /* Architecture __weak replacement functions */
 void __init mem_encrypt_init(void);
@@ -60,6 +64,7 @@ bool sev_es_active(void);
 #else  /* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define sme_me_mask0ULL
+#define sev_live_migration_enabled false
 
 static inline void __init sme_early_encrypt(resource_size_t paddr,
unsigned long size) { }
@@ -84,8 +89,11 @@ static inline int __init
 early_set_memory_decrypted(unsigned long vaddr, unsigned long size) { return 
0; }
 static inline int __init
 early_set_memory_encrypted(unsigned long vaddr, unsigned long size) { return 
0; }
+static inline void __init
+early_set_mem_enc_dec_hypercall(unsigned long vaddr, int npages, bool enc) {}
 
 static inline void mem_encrypt_free_decrypted_mem(void) { }
+static inline void check_kvm_sev_migration(void) { }
 
 #define __bss_decrypted
 
diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index 78bb0fae3982..bcc82e0c9779 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -26,6 +26,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -429,6 +430,56 @@ static inline void __set_percpu_decrypted(void *ptr, 
unsigned long size)
early_set_memory_decrypted((unsigned long) ptr, size);
 }
 
+static int __init setup_kvm_sev_migration(void)
+{
+   efi_char16_t efi_sev_live_migration_enabled[] = 
L"SevLiveMigrationEnabled";
+   efi_guid_t efi_variable_guid = MEM_ENCRYPT_GUID;
+   efi_status_t status;
+   unsigned long size;
+   bool enabled;
+
+   /*
+* check_kvm_sev_migration() invoked via kvm_init_platform() before
+* this callback would have setup the indicator that live migration
+* feature is supported/enabled.
+*/
+   if (!sev_live_migration_enabled)
+   return 0;
+
+   if (!efi_enabled(EFI_RUNTIME_SERVICES)) {
+   pr_info("%s : EFI runtime services are not enabled\n", 
__func__);
+   return 0;
+   }
+
+   size = sizeof(enabled);
+
+   /* Get variable contents into buffer */
+   status = efi.get_variable(efi_sev_live_migration_enabled,
+ _variable_guid, NULL, , );
+
+   if (status == EFI_NOT_FOUND) {
+   pr_info("%s : EFI live migration variable not found\n", 
__func__);
+   return 0;
+   }
+
+   if (status != EFI_SUCCESS) {
+   pr_info("%s : EFI variable retrieval failed\n", __func__);
+   return 0;
+   }
+
+   if (enabled == 0) {
+   pr_info("%s: live migration disabled in EFI\n", __func__);
+   return 0;
+   }
+
+   pr_info("%s : live migration enabled in EFI\n", __func__);
+   wrmsrl(MSR_KVM_SEV_LIVE_MIGRATION, KVM_SEV_LIVE_MIGRATION_ENABLED);
+
+   return true;
+}
+
+late_

[PATCH v11 10/13] KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature & Custom MSR.

[Ashish Kalra]

From: Ashish Kalra 

Add new KVM_FEATURE_SEV_LIVE_MIGRATION feature for guest to check
for host-side support for SEV live migration. Also add a new custom
MSR_KVM_SEV_LIVE_MIGRATION for guest to enable the SEV live migration
feature.

MSR is handled by userspace using MSR filters.

Signed-off-by: Ashish Kalra 
---
 Documentation/virt/kvm/cpuid.rst |  5 +
 Documentation/virt/kvm/msr.rst   | 12 
 arch/x86/include/uapi/asm/kvm_para.h |  4 
 arch/x86/kvm/cpuid.c |  3 ++-
 arch/x86/kvm/svm/svm.c   | 22 ++
 5 files changed, 45 insertions(+), 1 deletion(-)

diff --git a/Documentation/virt/kvm/cpuid.rst b/Documentation/virt/kvm/cpuid.rst
index cf62162d4be2..0bdb6cdb12d3 100644
--- a/Documentation/virt/kvm/cpuid.rst
+++ b/Documentation/virt/kvm/cpuid.rst
@@ -96,6 +96,11 @@ KVM_FEATURE_MSI_EXT_DEST_ID15  guest checks 
this feature bit
before using extended 
destination
ID bits in MSI address bits 
11-5.
 
+KVM_FEATURE_SEV_LIVE_MIGRATION 16  guest checks this feature bit 
before
+   using the page encryption state
+   hypercall to notify the page 
state
+   change
+
 KVM_FEATURE_CLOCKSOURCE_STABLE_BIT 24  host will warn if no guest-side
per-cpu warps are expected in
kvmclock
diff --git a/Documentation/virt/kvm/msr.rst b/Documentation/virt/kvm/msr.rst
index e37a14c323d2..020245d16087 100644
--- a/Documentation/virt/kvm/msr.rst
+++ b/Documentation/virt/kvm/msr.rst
@@ -376,3 +376,15 @@ data:
write '1' to bit 0 of the MSR, this causes the host to re-scan its queue
and check if there are more notifications pending. The MSR is available
if KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
+
+MSR_KVM_SEV_LIVE_MIGRATION:
+0x4b564d08
+
+   Control SEV Live Migration features.
+
+data:
+Bit 0 enables (1) or disables (0) host-side SEV Live Migration feature,
+in other words, this is guest->host communication that it's properly
+handling the shared pages list.
+
+All other bits are reserved.
diff --git a/arch/x86/include/uapi/asm/kvm_para.h 
b/arch/x86/include/uapi/asm/kvm_para.h
index 950afebfba88..f6bfa138874f 100644
--- a/arch/x86/include/uapi/asm/kvm_para.h
+++ b/arch/x86/include/uapi/asm/kvm_para.h
@@ -33,6 +33,7 @@
 #define KVM_FEATURE_PV_SCHED_YIELD 13
 #define KVM_FEATURE_ASYNC_PF_INT   14
 #define KVM_FEATURE_MSI_EXT_DEST_ID15
+#define KVM_FEATURE_SEV_LIVE_MIGRATION 16
 
 #define KVM_HINTS_REALTIME  0
 
@@ -54,6 +55,7 @@
 #define MSR_KVM_POLL_CONTROL   0x4b564d05
 #define MSR_KVM_ASYNC_PF_INT   0x4b564d06
 #define MSR_KVM_ASYNC_PF_ACK   0x4b564d07
+#define MSR_KVM_SEV_LIVE_MIGRATION 0x4b564d08
 
 struct kvm_steal_time {
__u64 steal;
@@ -136,4 +138,6 @@ struct kvm_vcpu_pv_apf_data {
 #define KVM_PV_EOI_ENABLED KVM_PV_EOI_MASK
 #define KVM_PV_EOI_DISABLED 0x0
 
+#define KVM_SEV_LIVE_MIGRATION_ENABLED BIT_ULL(0)
+
 #endif /* _UAPI_ASM_X86_KVM_PARA_H */
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 6bd2f8b830e4..4e2e69a692aa 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -812,7 +812,8 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array 
*array, u32 function)
 (1 << KVM_FEATURE_PV_SEND_IPI) |
 (1 << KVM_FEATURE_POLL_CONTROL) |
 (1 << KVM_FEATURE_PV_SCHED_YIELD) |
-(1 << KVM_FEATURE_ASYNC_PF_INT);
+(1 << KVM_FEATURE_ASYNC_PF_INT) |
+(1 << KVM_FEATURE_SEV_LIVE_MIGRATION);
 
if (sched_info_on())
entry->eax |= (1 << KVM_FEATURE_STEAL_TIME);
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 3cbf000beff1..1ac79e2f2a6c 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -2800,6 +2800,17 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct 
msr_data *msr_info)
case MSR_F10H_DECFG:
msr_info->data = svm->msr_decfg;
break;
+   case MSR_KVM_SEV_LIVE_MIGRATION:
+   if (!sev_guest(vcpu->kvm))
+   return 1;
+
+   if (!guest_cpuid_has(vcpu, KVM_FEATURE_SEV_LIVE_MIGRATION))
+   return 1;
+
+   /*
+* Let userspace handle the MSR using MSR filters.
+*/
+   return KVM_MSR_RET_FILTERED;
default:
return kvm_get_msr_common(vcpu, msr_info);
}
@@ -2996,6 +3007,17 @@ static int svm_set_msr(struct k

[PATCH v11 09/13] mm: x86: Invoke hypercall when page encryption status is changed

[Ashish Kalra]

From: Brijesh Singh 

Invoke a hypercall when a memory region is changed from encrypted ->
decrypted and vice versa. Hypervisor needs to know the page encryption
status during the guest migration.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Venu Busireddy 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 arch/x86/include/asm/paravirt.h   | 10 +
 arch/x86/include/asm/paravirt_types.h |  2 +
 arch/x86/kernel/paravirt.c|  1 +
 arch/x86/mm/mem_encrypt.c | 57 ++-
 arch/x86/mm/pat/set_memory.c  |  7 
 5 files changed, 76 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index 4abf110e2243..efaa3e628967 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -84,6 +84,12 @@ static inline void paravirt_arch_exit_mmap(struct mm_struct 
*mm)
PVOP_VCALL1(mmu.exit_mmap, mm);
 }
 
+static inline void page_encryption_changed(unsigned long vaddr, int npages,
+   bool enc)
+{
+   PVOP_VCALL3(mmu.page_encryption_changed, vaddr, npages, enc);
+}
+
 #ifdef CONFIG_PARAVIRT_XXL
 static inline void load_sp0(unsigned long sp0)
 {
@@ -799,6 +805,10 @@ static inline void paravirt_arch_dup_mmap(struct mm_struct 
*oldmm,
 static inline void paravirt_arch_exit_mmap(struct mm_struct *mm)
 {
 }
+
+static inline void page_encryption_changed(unsigned long vaddr, int npages, 
bool enc)
+{
+}
 #endif
 #endif /* __ASSEMBLY__ */
 #endif /* _ASM_X86_PARAVIRT_H */
diff --git a/arch/x86/include/asm/paravirt_types.h 
b/arch/x86/include/asm/paravirt_types.h
index de87087d3bde..69ef9c207b38 100644
--- a/arch/x86/include/asm/paravirt_types.h
+++ b/arch/x86/include/asm/paravirt_types.h
@@ -195,6 +195,8 @@ struct pv_mmu_ops {
 
/* Hook for intercepting the destruction of an mm_struct. */
void (*exit_mmap)(struct mm_struct *mm);
+   void (*page_encryption_changed)(unsigned long vaddr, int npages,
+   bool enc);
 
 #ifdef CONFIG_PARAVIRT_XXL
struct paravirt_callee_save read_cr2;
diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index c60222ab8ab9..9f206e192f6b 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -335,6 +335,7 @@ struct paravirt_patch_template pv_ops = {
(void (*)(struct mmu_gather *, void *))tlb_remove_page,
 
.mmu.exit_mmap  = paravirt_nop,
+   .mmu.page_encryption_changed= paravirt_nop,
 
 #ifdef CONFIG_PARAVIRT_XXL
.mmu.read_cr2   = __PV_IS_CALLEE_SAVE(native_read_cr2),
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index ae78cef79980..fae9ccbd0da7 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -19,6 +19,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -29,6 +30,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "mm_internal.h"
 
@@ -229,6 +231,47 @@ void __init sev_setup_arch(void)
swiotlb_adjust_size(size);
 }
 
+static void set_memory_enc_dec_hypercall(unsigned long vaddr, int npages,
+   bool enc)
+{
+   unsigned long sz = npages << PAGE_SHIFT;
+   unsigned long vaddr_end, vaddr_next;
+
+   vaddr_end = vaddr + sz;
+
+   for (; vaddr < vaddr_end; vaddr = vaddr_next) {
+   int psize, pmask, level;
+   unsigned long pfn;
+   pte_t *kpte;
+
+   kpte = lookup_address(vaddr, );
+   if (!kpte || pte_none(*kpte))
+   return;
+
+   switch (level) {
+   case PG_LEVEL_4K:
+   pfn = pte_pfn(*kpte);
+   break;
+   case PG_LEVEL_2M:
+   pfn = pmd_pfn(*(pmd_t *)kpte);
+   break;
+   case PG_LEVEL_1G:
+   pfn = pud_pfn(*(pud_t *)kpte);
+   break;
+   default:
+   return;
+   }
+
+   psize = page_level_size(level);
+   pmask = page_level_mask(level);
+
+   kvm_sev_hypercall3(KVM_HC_PAGE_ENC_STATUS,
+  pfn << PAGE_SHIFT, psize >> PAGE_SHIFT, enc);
+
+   vaddr_next = (vaddr & pmask) + psize;
+   }
+}
+
 static void __init __set_clr_pte_enc(pte_t *kpte, int level, bool enc)
 {
pgprot_t old_prot, new_prot;
@@ -286,12 +329,13 @@ static void __init __set_clr_pte_enc(pte_t *kpte, int 
level, bool enc)
 static int __init early_set_memory_enc_dec(unsigned long vaddr,
   unsign

[PATCH v11 08/13] KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall

[Ashish Kalra]

From: Ashish Kalra 

This hypercall is used by the SEV guest to notify a change in the page
encryption status to the hypervisor. The hypercall should be invoked
only when the encryption attribute is changed from encrypted -> decrypted
and vice versa. By default all guest pages are considered encrypted.

The hypercall exits to userspace to manage the guest shared regions and
integrate with the userspace VMM's migration code.

The patch integrates and extends DMA_SHARE/UNSHARE hypercall to
userspace exit functionality (arm64-specific) patch from Marc Zyngier,
to avoid arch-specific stuff and have a common interface
from the guest back to the VMM and sharing of the host handling of the
hypercall to support use case for a guest to share memory with a host.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 Documentation/virt/kvm/api.rst| 18 
 Documentation/virt/kvm/hypercalls.rst | 15 +++
 arch/x86/include/asm/kvm_host.h   |  2 +
 arch/x86/kvm/svm/sev.c| 61 +++
 arch/x86/kvm/svm/svm.c|  2 +
 arch/x86/kvm/svm/svm.h|  2 +
 arch/x86/kvm/vmx/vmx.c|  1 +
 arch/x86/kvm/x86.c| 12 ++
 include/uapi/linux/kvm.h  |  8 
 include/uapi/linux/kvm_para.h |  1 +
 10 files changed, 122 insertions(+)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 307f2fcf1b02..52bd7e475fd6 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -5475,6 +5475,24 @@ Valid values for 'type' are:
 Userspace is expected to place the hypercall result into the appropriate
 field before invoking KVM_RUN again.
 
+::
+
+   /* KVM_EXIT_DMA_SHARE / KVM_EXIT_DMA_UNSHARE */
+   struct {
+   __u64 addr;
+   __u64 len;
+   __u64 ret;
+   } dma_sharing;
+
+This defines a common interface from the guest back to the KVM to support
+use case for a guest to share memory with a host.
+
+The addr and len fields define the starting address and length of the
+shared memory region.
+
+Userspace is expected to place the hypercall result into the "ret" field
+before invoking KVM_RUN again.
+
 ::
 
/* Fix the size of the union. */
diff --git a/Documentation/virt/kvm/hypercalls.rst 
b/Documentation/virt/kvm/hypercalls.rst
index ed4fddd364ea..7aff0cebab7c 100644
--- a/Documentation/virt/kvm/hypercalls.rst
+++ b/Documentation/virt/kvm/hypercalls.rst
@@ -169,3 +169,18 @@ a0: destination APIC ID
 
 :Usage example: When sending a call-function IPI-many to vCPUs, yield if
any of the IPI target vCPUs was preempted.
+
+
+8. KVM_HC_PAGE_ENC_STATUS
+-
+:Architecture: x86
+:Status: active
+:Purpose: Notify the encryption status changes in guest page table (SEV guest)
+
+a0: the guest physical address of the start page
+a1: the number of pages
+a2: encryption attribute
+
+   Where:
+   * 1: Encryption attribute is set
+   * 0: Encryption attribute is cleared
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 3768819693e5..78284ebbbee7 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1352,6 +1352,8 @@ struct kvm_x86_ops {
int (*complete_emulated_msr)(struct kvm_vcpu *vcpu, int err);
 
void (*vcpu_deliver_sipi_vector)(struct kvm_vcpu *vcpu, u8 vector);
+   int (*page_enc_status_hc)(struct kvm_vcpu *vcpu, unsigned long gpa,
+ unsigned long sz, unsigned long mode);
 };
 
 struct kvm_x86_nested_ops {
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index c9795a22e502..fb3a315e5827 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1544,6 +1544,67 @@ static int sev_receive_finish(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_complete_userspace_page_enc_status_hc(struct kvm_vcpu *vcpu)
+{
+   vcpu->run->exit_reason = 0;
+   kvm_rax_write(vcpu, vcpu->run->dma_sharing.ret);
+   ++vcpu->stat.hypercalls;
+   return kvm_skip_emulated_instruction(vcpu);
+}
+
+int svm_page_enc_status_hc(struct kvm_vcpu *vcpu, unsigned long gpa,
+  unsigned long npages, unsigned long enc)
+{
+   kvm_pfn_t pfn_start, pfn_end;
+   struct kvm *kvm = vcpu->kvm;
+   gfn_t gfn_start, gfn_end;
+
+   if (!sev_guest(kvm))
+   return -EINVAL;
+
+   if (!npages)
+   return 0;
+
+   gfn_start = gpa_to_gfn(gpa);
+   gfn_end = gfn_start + npages;
+
+   /* out of bound access error 

[PATCH v11 07/13] KVM: x86: Add AMD SEV specific Hypercall3

[Ashish Kalra]

From: Brijesh Singh 

KVM hypercall framework relies on alternative framework to patch the
VMCALL -> VMMCALL on AMD platform. If a hypercall is made before
apply_alternative() is called then it defaults to VMCALL. The approach
works fine on non SEV guest. A VMCALL would causes #UD, and hypervisor
will be able to decode the instruction and do the right things. But
when SEV is active, guest memory is encrypted with guest key and
hypervisor will not be able to decode the instruction bytes.

Add SEV specific hypercall3, it unconditionally uses VMMCALL. The hypercall
will be used by the SEV guest to notify encrypted pages to the hypervisor.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Reviewed-by: Venu Busireddy 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 arch/x86/include/asm/kvm_para.h | 12 
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 338119852512..bc1b11d057fc 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -85,6 +85,18 @@ static inline long kvm_hypercall4(unsigned int nr, unsigned 
long p1,
return ret;
 }
 
+static inline long kvm_sev_hypercall3(unsigned int nr, unsigned long p1,
+ unsigned long p2, unsigned long p3)
+{
+   long ret;
+
+   asm volatile("vmmcall"
+: "=a"(ret)
+: "a"(nr), "b"(p1), "c"(p2), "d"(p3)
+: "memory");
+   return ret;
+}
+
 #ifdef CONFIG_KVM_GUEST
 bool kvm_para_available(void);
 unsigned int kvm_arch_para_features(void);
-- 
2.17.1

[PATCH v11 06/13] KVM: SVM: Add KVM_SEV_RECEIVE_FINISH command

[Ashish Kalra]

From: Brijesh Singh 

The command finalize the guest receiving process and make the SEV guest
ready for the execution.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  8 +++
 arch/x86/kvm/svm/sev.c| 23 +++
 2 files changed, 31 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index c6ed5b26d841..0466c0febff9 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -396,6 +396,14 @@ Returns: 0 on success, -negative on error
 __u32 trans_len;
 };
 
+15. KVM_SEV_RECEIVE_FINISH
+
+
+After completion of the migration flow, the KVM_SEV_RECEIVE_FINISH command can 
be
+issued by the hypervisor to make the guest ready for execution.
+
+Returns: 0 on success, -negative on error
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 2c95657cc9bf..c9795a22e502 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1524,6 +1524,26 @@ static int sev_receive_update_data(struct kvm *kvm, 
struct kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_receive_finish(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_receive_finish *data;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL);
+   if (!data)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_RECEIVE_FINISH, data, >error);
+
+   kfree(data);
+   return ret;
+}
+
 int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
 {
struct kvm_sev_cmd sev_cmd;
@@ -1592,6 +1612,9 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
case KVM_SEV_RECEIVE_UPDATE_DATA:
r = sev_receive_update_data(kvm, _cmd);
break;
+   case KVM_SEV_RECEIVE_FINISH:
+   r = sev_receive_finish(kvm, _cmd);
+   break;
default:
r = -EINVAL;
goto out;
-- 
2.17.1

[PATCH v11 05/13] KVM: SVM: Add KVM_SEV_RECEIVE_UPDATE_DATA command

[Ashish Kalra]

From: Brijesh Singh 

The command is used for copying the incoming buffer into the
SEV guest memory space.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst| 24 ++
 arch/x86/kvm/svm/sev.c| 79 +++
 include/uapi/linux/kvm.h  |  9 +++
 3 files changed, 112 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index c86c1ded8dd8..c6ed5b26d841 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -372,6 +372,30 @@ On success, the 'handle' field contains a new handle and 
on error, a negative va
 
 For more details, see SEV spec Section 6.12.
 
+14. KVM_SEV_RECEIVE_UPDATE_DATA
+
+
+The KVM_SEV_RECEIVE_UPDATE_DATA command can be used by the hypervisor to copy
+the incoming buffers into the guest memory region with encryption context
+created during the KVM_SEV_RECEIVE_START.
+
+Parameters (in): struct kvm_sev_receive_update_data
+
+Returns: 0 on success, -negative on error
+
+::
+
+struct kvm_sev_launch_receive_update_data {
+__u64 hdr_uaddr;/* userspace address containing the 
packet header */
+__u32 hdr_len;
+
+__u64 guest_uaddr;  /* the destination guest memory region 
*/
+__u32 guest_len;
+
+__u64 trans_uaddr;  /* the incoming buffer memory region  
*/
+__u32 trans_len;
+};
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index e530c2b34b5e..2c95657cc9bf 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1448,6 +1448,82 @@ static int sev_receive_start(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_receive_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct kvm_sev_receive_update_data params;
+   struct sev_data_receive_update_data *data;
+   void *hdr = NULL, *trans = NULL;
+   struct page **guest_page;
+   unsigned long n;
+   int ret, offset;
+
+   if (!sev_guest(kvm))
+   return -EINVAL;
+
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_receive_update_data)))
+   return -EFAULT;
+
+   if (!params.hdr_uaddr || !params.hdr_len ||
+   !params.guest_uaddr || !params.guest_len ||
+   !params.trans_uaddr || !params.trans_len)
+   return -EINVAL;
+
+   /* Check if we are crossing the page boundary */
+   offset = params.guest_uaddr & (PAGE_SIZE - 1);
+   if ((params.guest_len + offset > PAGE_SIZE))
+   return -EINVAL;
+
+   hdr = psp_copy_user_blob(params.hdr_uaddr, params.hdr_len);
+   if (IS_ERR(hdr))
+   return PTR_ERR(hdr);
+
+   trans = psp_copy_user_blob(params.trans_uaddr, params.trans_len);
+   if (IS_ERR(trans)) {
+   ret = PTR_ERR(trans);
+   goto e_free_hdr;
+   }
+
+   ret = -ENOMEM;
+   data = kzalloc(sizeof(*data), GFP_KERNEL);
+   if (!data)
+   goto e_free_trans;
+
+   data->hdr_address = __psp_pa(hdr);
+   data->hdr_len = params.hdr_len;
+   data->trans_address = __psp_pa(trans);
+   data->trans_len = params.trans_len;
+
+   /* Pin guest memory */
+   ret = -EFAULT;
+   guest_page = sev_pin_memory(kvm, params.guest_uaddr & PAGE_MASK,
+   PAGE_SIZE, , 0);
+   if (!guest_page)
+   goto e_free;
+
+   /* The RECEIVE_UPDATE_DATA command requires C-bit to be always set. */
+   data->guest_address = (page_to_pfn(guest_page[0]) << PAGE_SHIFT) +
+   offset;
+   data->guest_address |= sev_me_mask;
+   data->guest_len = params.guest_len;
+   data->handle = sev->handle;
+
+   ret = sev_issue_cmd(kvm, SEV_CMD_RECEIVE_UPDATE_DATA, data,
+   >error);
+
+   sev_unpin_memory(kvm, guest_page, n);
+
+e_free:
+   kfree(data);
+e_free_trans:
+   kfree(trans);
+e_free_hdr:
+   kfree(hdr);
+
+   return ret;
+}
+
 int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
 {
struct kvm_sev_cmd sev_cmd;
@@ -1513,6 +1589,9 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
case KVM_SEV_RECEIVE_START:
r = sev_receive_start(kvm, _cmd);
break;
+   case KVM_SEV_RECEIVE_UPD

[PATCH v11 04/13] KVM: SVM: Add support for KVM_SEV_RECEIVE_START command

[Ashish Kalra]

From: Brijesh Singh 

The command is used to create the encryption context for an incoming
SEV guest. The encryption context can be later used by the hypervisor
to import the incoming data into the SEV guest memory space.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst| 29 +++
 arch/x86/kvm/svm/sev.c| 81 +++
 include/uapi/linux/kvm.h  |  9 +++
 3 files changed, 119 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index 26c4e6c83f62..c86c1ded8dd8 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -343,6 +343,35 @@ issued by the hypervisor to delete the encryption context.
 
 Returns: 0 on success, -negative on error
 
+13. KVM_SEV_RECEIVE_START
+
+
+The KVM_SEV_RECEIVE_START command is used for creating the memory encryption
+context for an incoming SEV guest. To create the encryption context, the user 
must
+provide a guest policy, the platform public Diffie-Hellman (PDH) key and 
session
+information.
+
+Parameters: struct  kvm_sev_receive_start (in/out)
+
+Returns: 0 on success, -negative on error
+
+::
+
+struct kvm_sev_receive_start {
+__u32 handle;   /* if zero then firmware creates a new 
handle */
+__u32 policy;   /* guest's policy */
+
+__u64 pdh_uaddr;/* userspace address pointing to the 
PDH key */
+__u32 pdh_len;
+
+__u64 session_uaddr;/* userspace address which points to 
the guest session information */
+__u32 session_len;
+};
+
+On success, the 'handle' field contains a new handle and on error, a negative 
value.
+
+For more details, see SEV spec Section 6.12.
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 92325d9527ce..e530c2b34b5e 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1370,6 +1370,84 @@ static int sev_send_finish(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_receive_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_receive_start *start;
+   struct kvm_sev_receive_start params;
+   int *error = >error;
+   void *session_data;
+   void *pdh_data;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   /* Get parameter from the userspace */
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_receive_start)))
+   return -EFAULT;
+
+   /* some sanity checks */
+   if (!params.pdh_uaddr || !params.pdh_len ||
+   !params.session_uaddr || !params.session_len)
+   return -EINVAL;
+
+   pdh_data = psp_copy_user_blob(params.pdh_uaddr, params.pdh_len);
+   if (IS_ERR(pdh_data))
+   return PTR_ERR(pdh_data);
+
+   session_data = psp_copy_user_blob(params.session_uaddr,
+   params.session_len);
+   if (IS_ERR(session_data)) {
+   ret = PTR_ERR(session_data);
+   goto e_free_pdh;
+   }
+
+   ret = -ENOMEM;
+   start = kzalloc(sizeof(*start), GFP_KERNEL);
+   if (!start)
+   goto e_free_session;
+
+   start->handle = params.handle;
+   start->policy = params.policy;
+   start->pdh_cert_address = __psp_pa(pdh_data);
+   start->pdh_cert_len = params.pdh_len;
+   start->session_address = __psp_pa(session_data);
+   start->session_len = params.session_len;
+
+   /* create memory encryption context */
+   ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_RECEIVE_START, start,
+   error);
+   if (ret)
+   goto e_free;
+
+   /* Bind ASID to this guest */
+   ret = sev_bind_asid(kvm, start->handle, error);
+   if (ret)
+   goto e_free;
+
+   params.handle = start->handle;
+   if (copy_to_user((void __user *)(uintptr_t)argp->data,
+, sizeof(struct kvm_sev_receive_start))) {
+   ret = -EFAULT;
+   sev_unbind_asid(kvm, start->handle);
+   goto e_free;
+   }
+
+   sev->handle = start->handle;
+   sev->fd = argp->sev_fd;
+
+e_free:
+   kfree(start);
+e_free_session:
+   kfree(session_data);
+e_free_pdh:
+   kfree(pdh_data);
+
+   return ret;
+}
+
 int svm_mem_enc_op(struct kv

[PATCH v11 03/13] KVM: SVM: Add KVM_SEV_SEND_FINISH command

[Ashish Kalra]

From: Brijesh Singh 

The command is used to finailize the encryption context created with
KVM_SEV_SEND_START command.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  8 +++
 arch/x86/kvm/svm/sev.c| 23 +++
 2 files changed, 31 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index 3c5456e0268a..26c4e6c83f62 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -335,6 +335,14 @@ Returns: 0 on success, -negative on error
 __u32 trans_len;
 };
 
+12. KVM_SEV_SEND_FINISH
+
+
+After completion of the migration flow, the KVM_SEV_SEND_FINISH command can be
+issued by the hypervisor to delete the encryption context.
+
+Returns: 0 on success, -negative on error
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 30527285a39a..92325d9527ce 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1350,6 +1350,26 @@ static int sev_send_update_data(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_send_finish(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_finish *data;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL);
+   if (!data)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_SEND_FINISH, data, >error);
+
+   kfree(data);
+   return ret;
+}
+
 int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
 {
struct kvm_sev_cmd sev_cmd;
@@ -1409,6 +1429,9 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
case KVM_SEV_SEND_UPDATE_DATA:
r = sev_send_update_data(kvm, _cmd);
break;
+   case KVM_SEV_SEND_FINISH:
+   r = sev_send_finish(kvm, _cmd);
+   break;
default:
r = -EINVAL;
goto out;
-- 
2.17.1

[PATCH v11 02/13] KVM: SVM: Add KVM_SEND_UPDATE_DATA command

[Ashish Kalra]

From: Brijesh Singh 

The command is used for encrypting the guest memory region using the encryption
context created with KVM_SEV_SEND_START.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by : Steve Rutherford 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  24 
 arch/x86/kvm/svm/sev.c| 122 ++
 include/uapi/linux/kvm.h  |   9 ++
 3 files changed, 155 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index ac799dd7a618..3c5456e0268a 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -311,6 +311,30 @@ Returns: 0 on success, -negative on error
 __u32 session_len;
 };
 
+11. KVM_SEV_SEND_UPDATE_DATA
+
+
+The KVM_SEV_SEND_UPDATE_DATA command can be used by the hypervisor to encrypt 
the
+outgoing guest memory region with the encryption context creating using
+KVM_SEV_SEND_START.
+
+Parameters (in): struct kvm_sev_send_update_data
+
+Returns: 0 on success, -negative on error
+
+::
+
+struct kvm_sev_launch_send_update_data {
+__u64 hdr_uaddr;/* userspace address containing the 
packet header */
+__u32 hdr_len;
+
+__u64 guest_uaddr;  /* the source memory region to be 
encrypted */
+__u32 guest_len;
+
+__u64 trans_uaddr;  /* the destition memory region  */
+__u32 trans_len;
+};
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 2b65900c05d6..30527285a39a 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -34,6 +34,7 @@ static DECLARE_RWSEM(sev_deactivate_lock);
 static DEFINE_MUTEX(sev_bitmap_lock);
 unsigned int max_sev_asid;
 static unsigned int min_sev_asid;
+static unsigned long sev_me_mask;
 static unsigned long *sev_asid_bitmap;
 static unsigned long *sev_reclaim_asid_bitmap;
 
@@ -1232,6 +1233,123 @@ static int sev_send_start(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+/* Userspace wants to query either header or trans length. */
+static int
+__sev_send_update_data_query_lengths(struct kvm *kvm, struct kvm_sev_cmd *argp,
+struct kvm_sev_send_update_data *params)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_update_data *data;
+   int ret;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL_ACCOUNT);
+   if (!data)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_SEND_UPDATE_DATA, data, >error);
+
+   params->hdr_len = data->hdr_len;
+   params->trans_len = data->trans_len;
+
+   if (copy_to_user((void __user *)(uintptr_t)argp->data, params,
+sizeof(struct kvm_sev_send_update_data)))
+   ret = -EFAULT;
+
+   kfree(data);
+   return ret;
+}
+
+static int sev_send_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_update_data *data;
+   struct kvm_sev_send_update_data params;
+   void *hdr, *trans_data;
+   struct page **guest_page;
+   unsigned long n;
+   int ret, offset;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_send_update_data)))
+   return -EFAULT;
+
+   /* userspace wants to query either header or trans length */
+   if (!params.trans_len || !params.hdr_len)
+   return __sev_send_update_data_query_lengths(kvm, argp, );
+
+   if (!params.trans_uaddr || !params.guest_uaddr ||
+   !params.guest_len || !params.hdr_uaddr)
+   return -EINVAL;
+
+   /* Check if we are crossing the page boundary */
+   offset = params.guest_uaddr & (PAGE_SIZE - 1);
+   if ((params.guest_len + offset > PAGE_SIZE))
+   return -EINVAL;
+
+   /* Pin guest memory */
+   guest_page = sev_pin_memory(kvm, params.guest_uaddr & PAGE_MASK,
+   PAGE_SIZE, , 0);
+   if (!guest_page)
+   return -EFAULT;
+
+   /* allocate memory for header and transport buffer */
+   ret = -ENOMEM;
+   hdr = kmalloc(params.hdr_len, GFP_KERNEL_ACCOUNT);
+   if (!hdr)
+   goto e_unpin;
+
+   trans_data = kmalloc(params.trans_len, GFP_KERNEL_ACCOUNT);
+   if (!trans_data)
+  

[PATCH v11 01/13] KVM: SVM: Add KVM_SEV SEND_START command

[Ashish Kalra]

From: Brijesh Singh 

The command is used to create an outgoing SEV guest encryption context.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Reviewed-by: Venu Busireddy 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  27 
 arch/x86/kvm/svm/sev.c| 125 ++
 include/linux/psp-sev.h   |   8 +-
 include/uapi/linux/kvm.h  |  12 ++
 4 files changed, 168 insertions(+), 4 deletions(-)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index 469a6308765b..ac799dd7a618 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -284,6 +284,33 @@ Returns: 0 on success, -negative on error
 __u32 len;
 };
 
+10. KVM_SEV_SEND_START
+--
+
+The KVM_SEV_SEND_START command can be used by the hypervisor to create an
+outgoing guest encryption context.
+
+Parameters (in): struct kvm_sev_send_start
+
+Returns: 0 on success, -negative on error
+
+::
+struct kvm_sev_send_start {
+__u32 policy; /* guest policy */
+
+__u64 pdh_cert_uaddr; /* platform Diffie-Hellman 
certificate */
+__u32 pdh_cert_len;
+
+__u64 plat_certs_uaddr;/* platform certificate chain */
+__u32 plat_certs_len;
+
+__u64 amd_certs_uaddr;/* AMD certificate */
+__u32 amd_certs_len;
+
+__u64 session_uaddr;  /* Guest session information */
+__u32 session_len;
+};
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 874ea309279f..2b65900c05d6 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1110,6 +1110,128 @@ static int sev_get_attestation_report(struct kvm *kvm, 
struct kvm_sev_cmd *argp)
return ret;
 }
 
+/* Userspace wants to query session length. */
+static int
+__sev_send_start_query_session_length(struct kvm *kvm, struct kvm_sev_cmd 
*argp,
+ struct kvm_sev_send_start *params)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_start *data;
+   int ret;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL_ACCOUNT);
+   if (data == NULL)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_SEND_START, data, >error);
+
+   params->session_len = data->session_len;
+   if (copy_to_user((void __user *)(uintptr_t)argp->data, params,
+   sizeof(struct kvm_sev_send_start)))
+   ret = -EFAULT;
+
+   kfree(data);
+   return ret;
+}
+
+static int sev_send_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_start *data;
+   struct kvm_sev_send_start params;
+   void *amd_certs, *session_data;
+   void *pdh_cert, *plat_certs;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_send_start)))
+   return -EFAULT;
+
+   /* if session_len is zero, userspace wants to query the session length 
*/
+   if (!params.session_len)
+   return __sev_send_start_query_session_length(kvm, argp,
+   );
+
+   /* some sanity checks */
+   if (!params.pdh_cert_uaddr || !params.pdh_cert_len ||
+   !params.session_uaddr || params.session_len > SEV_FW_BLOB_MAX_SIZE)
+   return -EINVAL;
+
+   /* allocate the memory to hold the session data blob */
+   session_data = kmalloc(params.session_len, GFP_KERNEL_ACCOUNT);
+   if (!session_data)
+   return -ENOMEM;
+
+   /* copy the certificate blobs from userspace */
+   pdh_cert = psp_copy_user_blob(params.pdh_cert_uaddr,
+   params.pdh_cert_len);
+   if (IS_ERR(pdh_cert)) {
+   ret = PTR_ERR(pdh_cert);
+   goto e_free_session;
+   }
+
+   plat_certs = psp_copy_user_blob(params.plat_certs_uaddr,
+   params.plat_certs_len);
+   if (IS_ERR(plat_certs)) {
+   ret = PTR_ERR(plat_certs);
+   goto e_free_pdh;
+   }
+
+   amd_certs = psp_copy_user_blob(params.amd_certs_uaddr,
+   params.amd_certs_len);
+   if (IS_ERR(amd_certs)) {

[PATCH v11 00/13] Add AMD SEV guest live migration support

[Ashish Kalra]

From: Ashish Kalra 

The series add support for AMD SEV guest live migration commands. To protect the
confidentiality of an SEV protected guest memory while in transit we need to
use the SEV commands defined in SEV API spec [1].

SEV guest VMs have the concept of private and shared memory. Private memory
is encrypted with the guest-specific key, while shared memory may be encrypted
with hypervisor key. The commands provided by the SEV FW are meant to be used
for the private memory only. The patch series introduces a new hypercall.
The guest OS can use this hypercall to notify the page encryption status.
If the page is encrypted with guest specific-key then we use SEV command during
the migration. If page is not encrypted then fallback to default.

The patch adds new KVM_EXIT_DMA_SHARE/KVM_EXIT_DMA_UNSHARE hypercall to
userspace exit functionality as a common interface from the guest back to the
VMM and passing on the guest shared/unencrypted page information to the
userspace VMM/Qemu. Qemu can consult this information during migration to know 
whether the page is encrypted.

This section descibes how the SEV live migration feature is negotiated
between the host and guest, the host indicates this feature support via 
KVM_FEATURE_CPUID. The guest firmware (OVMF) detects this feature and
sets a UEFI enviroment variable indicating OVMF support for live
migration, the guest kernel also detects the host support for this
feature via cpuid and in case of an EFI boot verifies if OVMF also
supports this feature by getting the UEFI enviroment variable and if it
set then enables live migration feature on host by writing to a custom
MSR, if not booted under EFI, then it simply enables the feature by
again writing to the custom MSR. The MSR is also handled by the
userspace VMM/Qemu.

A branch containing these patches is available here:
https://github.com/AMDESE/linux/tree/sev-migration-v11

[1] https://developer.amd.com/wp-content/resources/55766.PDF

Changes since v10:
- Adds new KVM_EXIT_DMA_SHARE/KVM_EXIT_DMA_UNSHARE hypercall to
  userspace exit functionality as a common interface from the guest back to the
  KVM and passing on the guest shared/unencrypted region information to the
  userspace VMM/Qemu. KVM/host kernel does not maintain the guest shared
  memory regions information anymore. 
- Remove implicit enabling of SEV live migration feature for an SEV
  guest, now this is explicitly in control of the userspace VMM/Qemu.
- Custom MSR handling is also now moved into userspace VMM/Qemu.
- As KVM does not maintain the guest shared memory region information
  anymore, sev_dbg_crypt() cannot bypass unencrypted guest memory
  regions without support from userspace VMM/Qemu.

Changes since v9:
- Transitioning from page encryption bitmap to the shared pages list
  to keep track of guest's shared/unencrypted memory regions.
- Move back to marking the complete _bss_decrypted section as 
  decrypted in the shared pages list.
- Invoke a new function check_kvm_sev_migration() via kvm_init_platform()
  for guest to query for host-side support for SEV live migration 
  and to enable the SEV live migration feature, to avoid
  #ifdefs in code 
- Rename MSR_KVM_SEV_LIVE_MIG_EN to MSR_KVM_SEV_LIVE_MIGRATION.
- Invoke a new function handle_unencrypted_region() from 
  sev_dbg_crypt() to bypass unencrypted guest memory regions.

Changes since v8:
- Rebasing to kvm next branch.
- Fixed and added comments as per review feedback on v8 patches.
- Removed implicitly enabling live migration for incoming VMs in
  in KVM_SET_PAGE_ENC_BITMAP, it is now done via KVM_SET_MSR ioctl.
- Adds support for bypassing unencrypted guest memory regions for
  DBG_DECRYPT API calls, guest memory region encryption status in
  sev_dbg_decrypt() is referenced using the page encryption bitmap.

Changes since v7:
- Removed the hypervisor specific hypercall/paravirt callback for
  SEV live migration and moved back to calling kvm_sev_hypercall3 
  directly.
- Fix build errors as
  Reported-by: kbuild test robot , specifically fixed
  build error when CONFIG_HYPERVISOR_GUEST=y and
  CONFIG_AMD_MEM_ENCRYPT=n.
- Implicitly enabled live migration for incoming VM(s) to handle 
  A->B->C->... VM migrations.
- Fixed Documentation as per comments on v6 patches.
- Fixed error return path in sev_send_update_data() as per comments 
  on v6 patches. 

Changes since v6:
- Rebasing to mainline and refactoring to the new split SVM
  infrastructre.
- Move to static allocation of the unified Page Encryption bitmap
  instead of the dynamic resizing of the bitmap, the static allocation
  is done implicitly by extending kvm_arch_commit_memory_region() callack
  to add svm specific x86_ops which can read the userspace provided memory
  region/memslots and calculate the amount of guest RAM managed by the KVM
  and grow the bitmap.
- Fixed KVM_SET_PAGE_ENC_BITMAP ioctl to set the whole bitmap instead
  of simply clearing specific bits.
- Removed KVM_PAGE_ENC_BITMAP_RE

Re: [RFC v2] KVM: x86: Support KVM VMs sharing SEV context

[Ashish Kalra]

Hi Nathan,

Will you be posting a corresponding Qemu patch for this ?

Thanks,
Ashish

On Tue, Mar 16, 2021 at 01:40:27AM +, Nathan Tempelman wrote:
> Add a capability for userspace to mirror SEV encryption context from
> one vm to another. On our side, this is intended to support a
> Migration Helper vCPU, but it can also be used generically to support
> other in-guest workloads scheduled by the host. The intention is for
> the primary guest and the mirror to have nearly identical memslots.
> 
> The primary benefits of this are that:
> 1) The VMs do not share KVM contexts (think APIC/MSRs/etc), so they
> can't accidentally clobber each other.
> 2) The VMs can have different memory-views, which is necessary for post-copy
> migration (the migration vCPUs on the target need to read and write to
> pages, when the primary guest would VMEXIT).
> 
> This does not change the threat model for AMD SEV. Any memory involved
> is still owned by the primary guest and its initial state is still
> attested to through the normal SEV_LAUNCH_* flows. If userspace wanted
> to circumvent SEV, they could achieve the same effect by simply attaching
> a vCPU to the primary VM.
> This patch deliberately leaves userspace in charge of the memslots for the
> mirror, as it already has the power to mess with them in the primary guest.
> 
> This patch does not support SEV-ES (much less SNP), as it does not
> handle handing off attested VMSAs to the mirror.
> 
> For additional context, we need a Migration Helper because SEV PSP migration
> is far too slow for our live migration on its own. Using an in-guest
> migrator lets us speed this up significantly.
> 
> Signed-off-by: Nathan Tempelman 
> ---
>  Documentation/virt/kvm/api.rst  | 17 +++
>  arch/x86/include/asm/kvm_host.h |  1 +
>  arch/x86/kvm/svm/sev.c  | 88 +
>  arch/x86/kvm/svm/svm.c  |  2 +
>  arch/x86/kvm/svm/svm.h  |  2 +
>  arch/x86/kvm/x86.c  |  7 ++-
>  include/linux/kvm_host.h|  1 +
>  include/uapi/linux/kvm.h|  1 +
>  virt/kvm/kvm_main.c |  6 +++
>  9 files changed, 124 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
> index 482508ec7cc4..332ba8b5b6f4 100644
> --- a/Documentation/virt/kvm/api.rst
> +++ b/Documentation/virt/kvm/api.rst
> @@ -6213,6 +6213,23 @@ the bus lock vm exit can be preempted by a higher 
> priority VM exit, the exit
>  notifications to userspace can be KVM_EXIT_BUS_LOCK or other reasons.
>  KVM_RUN_BUS_LOCK flag is used to distinguish between them.
>  
> +7.23 KVM_CAP_VM_COPY_ENC_CONTEXT_FROM
> +-
> +
> +Architectures: x86 SEV enabled
> +Type: vm
> +Parameters: args[0] is the fd of the source vm
> +Returns: 0 on success; ENOTTY on error
> +
> +This capability enables userspace to copy encryption context from the vm
> +indicated by the fd to the vm this is called on.
> +
> +This is intended to support in-guest workloads scheduled by the host. This
> +allows the in-guest workload to maintain its own NPTs and keeps the two vms
> +from accidentally clobbering each other with interrupts and the like 
> (separate
> +APIC/MSRs/etc).
> +
> +
>  8. Other capabilities.
>  ==
>  
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 84499aad01a4..46df415a8e91 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1334,6 +1334,7 @@ struct kvm_x86_ops {
>   int (*mem_enc_op)(struct kvm *kvm, void __user *argp);
>   int (*mem_enc_reg_region)(struct kvm *kvm, struct kvm_enc_region *argp);
>   int (*mem_enc_unreg_region)(struct kvm *kvm, struct kvm_enc_region 
> *argp);
> + int (*vm_copy_enc_context_from)(struct kvm *kvm, unsigned int 
> source_fd);
>  
>   int (*get_msr_feature)(struct kvm_msr_entry *entry);
>  
> diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
> index 874ea309279f..b2c90c67a0d9 100644
> --- a/arch/x86/kvm/svm/sev.c
> +++ b/arch/x86/kvm/svm/sev.c
> @@ -66,6 +66,11 @@ static int sev_flush_asids(void)
>   return ret;
>  }
>  
> +static inline bool is_mirroring_enc_context(struct kvm *kvm)
> +{
> + return to_kvm_svm(kvm)->sev_info.enc_context_owner;
> +}
> +
>  /* Must be called with the sev_bitmap_lock held */
>  static bool __sev_recycle_asids(int min_asid, int max_asid)
>  {
> @@ -1124,6 +1129,10 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
>   if (copy_from_user(_cmd, argp, sizeof(struct kvm_sev_cmd)))
>   return -EFAULT;
>  
> + /* enc_context_owner handles all memory enc operations */
> + if (is_mirroring_enc_context(kvm))
> + return -ENOTTY;
> +
>   mutex_lock(>lock);
>  
>   switch (sev_cmd.id) {
> @@ -1186,6 +1195,10 @@ int svm_register_enc_region(struct kvm *kvm,
>   if (!sev_guest(kvm))
>   return -ENOTTY;
>  
> + /* If kvm is 

Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

Hello Steve,

On Thu, Apr 01, 2021 at 06:40:06PM -0700, Steve Rutherford wrote:
> On Fri, Mar 19, 2021 at 11:00 AM Ashish Kalra  wrote:
> >
> > On Thu, Mar 11, 2021 at 12:48:07PM -0800, Steve Rutherford wrote:
> > > On Thu, Mar 11, 2021 at 10:15 AM Ashish Kalra  
> > > wrote:
> > > >
> > > > On Wed, Mar 03, 2021 at 06:54:41PM +, Will Deacon wrote:
> > > > > [+Marc]
> > > > >
> > > > > On Tue, Mar 02, 2021 at 02:55:43PM +, Ashish Kalra wrote:
> > > > > > On Fri, Feb 26, 2021 at 09:44:41AM -0800, Sean Christopherson wrote:
> > > > > > > On Fri, Feb 26, 2021, Ashish Kalra wrote:
> > > > > > > > On Thu, Feb 25, 2021 at 02:59:27PM -0800, Steve Rutherford 
> > > > > > > > wrote:
> > > > > > > > > On Thu, Feb 25, 2021 at 12:20 PM Ashish Kalra 
> > > > > > > > >  wrote:
> > > > > > > > > Thanks for grabbing the data!
> > > > > > > > >
> > > > > > > > > I am fine with both paths. Sean has stated an explicit desire 
> > > > > > > > > for
> > > > > > > > > hypercall exiting, so I think that would be the current 
> > > > > > > > > consensus.
> > > > > > >
> > > > > > > Yep, though it'd be good to get Paolo's input, too.
> > > > > > >
> > > > > > > > > If we want to do hypercall exiting, this should be in a 
> > > > > > > > > follow-up
> > > > > > > > > series where we implement something more generic, e.g. a 
> > > > > > > > > hypercall
> > > > > > > > > exiting bitmap or hypercall exit list. If we are taking the 
> > > > > > > > > hypercall
> > > > > > > > > exit route, we can drop the kvm side of the hypercall.
> > > > > > >
> > > > > > > I don't think this is a good candidate for arbitrary hypercall 
> > > > > > > interception.  Or
> > > > > > > rather, I think hypercall interception should be an orthogonal 
> > > > > > > implementation.
> > > > > > >
> > > > > > > The guest, including guest firmware, needs to be aware that the 
> > > > > > > hypercall is
> > > > > > > supported, and the ABI needs to be well-defined.  Relying on 
> > > > > > > userspace VMMs to
> > > > > > > implement a common ABI is an unnecessary risk.
> > > > > > >
> > > > > > > We could make KVM's default behavior be a nop, i.e. have KVM 
> > > > > > > enforce the ABI but
> > > > > > > require further VMM intervention.  But, I just don't see the 
> > > > > > > point, it would
> > > > > > > save only a few lines of code.  It would also limit what KVM 
> > > > > > > could do in the
> > > > > > > future, e.g. if KVM wanted to do its own bookkeeping _and_ exit 
> > > > > > > to userspace,
> > > > > > > then mandatory interception would essentially make it impossible 
> > > > > > > for KVM to do
> > > > > > > bookkeeping while still honoring the interception request.
> > > > > > >
> > > > > > > However, I do think it would make sense to have the userspace 
> > > > > > > exit be a generic
> > > > > > > exit type.  But hey, we already have the necessary ABI defined 
> > > > > > > for that!  It's
> > > > > > > just not used anywhere.
> > > > > > >
> > > > > > >   /* KVM_EXIT_HYPERCALL */
> > > > > > >   struct {
> > > > > > >   __u64 nr;
> > > > > > >   __u64 args[6];
> > > > > > >   __u64 ret;
> > > > > > >   __u32 longmode;
> > > > > > >   __u32 pad;
> > > > > > >   } hypercall;
> > > > > > >
> > > > > > >
> > > > > > > > > Userspace could also handle the MSR using MSR filters (would 
> > > > > > > > > need to
> > > > > > > > > confirm that).  The

Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

On Thu, Mar 11, 2021 at 12:48:07PM -0800, Steve Rutherford wrote:
> On Thu, Mar 11, 2021 at 10:15 AM Ashish Kalra  wrote:
> >
> > On Wed, Mar 03, 2021 at 06:54:41PM +, Will Deacon wrote:
> > > [+Marc]
> > >
> > > On Tue, Mar 02, 2021 at 02:55:43PM +, Ashish Kalra wrote:
> > > > On Fri, Feb 26, 2021 at 09:44:41AM -0800, Sean Christopherson wrote:
> > > > > On Fri, Feb 26, 2021, Ashish Kalra wrote:
> > > > > > On Thu, Feb 25, 2021 at 02:59:27PM -0800, Steve Rutherford wrote:
> > > > > > > On Thu, Feb 25, 2021 at 12:20 PM Ashish Kalra 
> > > > > > >  wrote:
> > > > > > > Thanks for grabbing the data!
> > > > > > >
> > > > > > > I am fine with both paths. Sean has stated an explicit desire for
> > > > > > > hypercall exiting, so I think that would be the current consensus.
> > > > >
> > > > > Yep, though it'd be good to get Paolo's input, too.
> > > > >
> > > > > > > If we want to do hypercall exiting, this should be in a follow-up
> > > > > > > series where we implement something more generic, e.g. a hypercall
> > > > > > > exiting bitmap or hypercall exit list. If we are taking the 
> > > > > > > hypercall
> > > > > > > exit route, we can drop the kvm side of the hypercall.
> > > > >
> > > > > I don't think this is a good candidate for arbitrary hypercall 
> > > > > interception.  Or
> > > > > rather, I think hypercall interception should be an orthogonal 
> > > > > implementation.
> > > > >
> > > > > The guest, including guest firmware, needs to be aware that the 
> > > > > hypercall is
> > > > > supported, and the ABI needs to be well-defined.  Relying on 
> > > > > userspace VMMs to
> > > > > implement a common ABI is an unnecessary risk.
> > > > >
> > > > > We could make KVM's default behavior be a nop, i.e. have KVM enforce 
> > > > > the ABI but
> > > > > require further VMM intervention.  But, I just don't see the point, 
> > > > > it would
> > > > > save only a few lines of code.  It would also limit what KVM could do 
> > > > > in the
> > > > > future, e.g. if KVM wanted to do its own bookkeeping _and_ exit to 
> > > > > userspace,
> > > > > then mandatory interception would essentially make it impossible for 
> > > > > KVM to do
> > > > > bookkeeping while still honoring the interception request.
> > > > >
> > > > > However, I do think it would make sense to have the userspace exit be 
> > > > > a generic
> > > > > exit type.  But hey, we already have the necessary ABI defined for 
> > > > > that!  It's
> > > > > just not used anywhere.
> > > > >
> > > > >   /* KVM_EXIT_HYPERCALL */
> > > > >   struct {
> > > > >   __u64 nr;
> > > > >   __u64 args[6];
> > > > >   __u64 ret;
> > > > >   __u32 longmode;
> > > > >   __u32 pad;
> > > > >   } hypercall;
> > > > >
> > > > >
> > > > > > > Userspace could also handle the MSR using MSR filters (would need 
> > > > > > > to
> > > > > > > confirm that).  Then userspace could also be in control of the 
> > > > > > > cpuid bit.
> > > > >
> > > > > An MSR is not a great fit; it's x86 specific and limited to 64 bits 
> > > > > of data.
> > > > > The data limitation could be fudged by shoving data into non-standard 
> > > > > GPRs, but
> > > > > that will result in truly heinous guest code, and extensibility 
> > > > > issues.
> > > > >

We may also need to pass-through the MSR to userspace, as it is a part of this
complete host (userspace/kernel), OVMF and guest kernel negotiation of
the SEV live migration feature. 

Host (userspace/kernel) advertises it's support for SEV live migration
feature via the CPUID bits, which is queried by OVMF and which in turn
adds a new UEFI runtime variable to indicate support for SEV live
migration, which is later queried during guest kernel boot and
accordingly the guest does a wrmrsl() to custom MSR to complete SEV
live mig

Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

On Wed, Mar 03, 2021 at 06:54:41PM +, Will Deacon wrote:
> [+Marc]
> 
> On Tue, Mar 02, 2021 at 02:55:43PM +0000, Ashish Kalra wrote:
> > On Fri, Feb 26, 2021 at 09:44:41AM -0800, Sean Christopherson wrote:
> > > On Fri, Feb 26, 2021, Ashish Kalra wrote:
> > > > On Thu, Feb 25, 2021 at 02:59:27PM -0800, Steve Rutherford wrote:
> > > > > On Thu, Feb 25, 2021 at 12:20 PM Ashish Kalra  
> > > > > wrote:
> > > > > Thanks for grabbing the data!
> > > > > 
> > > > > I am fine with both paths. Sean has stated an explicit desire for
> > > > > hypercall exiting, so I think that would be the current consensus.
> > > 
> > > Yep, though it'd be good to get Paolo's input, too.
> > > 
> > > > > If we want to do hypercall exiting, this should be in a follow-up
> > > > > series where we implement something more generic, e.g. a hypercall
> > > > > exiting bitmap or hypercall exit list. If we are taking the hypercall
> > > > > exit route, we can drop the kvm side of the hypercall.
> > > 
> > > I don't think this is a good candidate for arbitrary hypercall 
> > > interception.  Or
> > > rather, I think hypercall interception should be an orthogonal 
> > > implementation.
> > > 
> > > The guest, including guest firmware, needs to be aware that the hypercall 
> > > is
> > > supported, and the ABI needs to be well-defined.  Relying on userspace 
> > > VMMs to
> > > implement a common ABI is an unnecessary risk.
> > > 
> > > We could make KVM's default behavior be a nop, i.e. have KVM enforce the 
> > > ABI but
> > > require further VMM intervention.  But, I just don't see the point, it 
> > > would
> > > save only a few lines of code.  It would also limit what KVM could do in 
> > > the
> > > future, e.g. if KVM wanted to do its own bookkeeping _and_ exit to 
> > > userspace,
> > > then mandatory interception would essentially make it impossible for KVM 
> > > to do
> > > bookkeeping while still honoring the interception request.
> > > 
> > > However, I do think it would make sense to have the userspace exit be a 
> > > generic
> > > exit type.  But hey, we already have the necessary ABI defined for that!  
> > > It's
> > > just not used anywhere.
> > > 
> > >   /* KVM_EXIT_HYPERCALL */
> > >   struct {
> > >   __u64 nr;
> > >   __u64 args[6];
> > >   __u64 ret;
> > >   __u32 longmode;
> > >   __u32 pad;
> > >   } hypercall;
> > > 
> > > 
> > > > > Userspace could also handle the MSR using MSR filters (would need to
> > > > > confirm that).  Then userspace could also be in control of the cpuid 
> > > > > bit.
> > > 
> > > An MSR is not a great fit; it's x86 specific and limited to 64 bits of 
> > > data.
> > > The data limitation could be fudged by shoving data into non-standard 
> > > GPRs, but
> > > that will result in truly heinous guest code, and extensibility issues.
> > > 
> > > The data limitation is a moot point, because the x86-only thing is a deal
> > > breaker.  arm64's pKVM work has a near-identical use case for a guest to 
> > > share
> > > memory with a host.  I can't think of a clever way to avoid having to 
> > > support
> > > TDX's and SNP's hypervisor-agnostic variants, but we can at least not have
> > > multiple KVM variants.
> > 
> > Looking at arm64's pKVM work, i see that it is a recently introduced RFC
> > patch-set and probably relevant to arm64 nVHE hypervisor
> > mode/implementation, and potentially makes sense as it adds guest
> > memory protection as both host and guest kernels are running on the same
> > privilege level ?
> > 
> > Though i do see that the pKVM stuff adds two hypercalls, specifically :
> > 
> > pkvm_create_mappings() ( I assume this is for setting shared memory
> > regions between host and guest) &
> > pkvm_create_private_mappings().
> > 
> > And the use-cases are quite similar to memory protection architectues
> > use cases, for example, use with virtio devices, guest DMA I/O, etc.
> 
> These hypercalls are both private to the host kernel communicating with
> its hypervisor counterpart, so I don't think they're particularly
> relevant here. As far as I can see, the more useful thing is to allow
> th

Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

On Wed, Mar 03, 2021 at 06:54:41PM +, Will Deacon wrote:
> [+Marc]
> 
> On Tue, Mar 02, 2021 at 02:55:43PM +0000, Ashish Kalra wrote:
> > On Fri, Feb 26, 2021 at 09:44:41AM -0800, Sean Christopherson wrote:
> > > On Fri, Feb 26, 2021, Ashish Kalra wrote:
> > > > On Thu, Feb 25, 2021 at 02:59:27PM -0800, Steve Rutherford wrote:
> > > > > On Thu, Feb 25, 2021 at 12:20 PM Ashish Kalra  
> > > > > wrote:
> > > > > Thanks for grabbing the data!
> > > > > 
> > > > > I am fine with both paths. Sean has stated an explicit desire for
> > > > > hypercall exiting, so I think that would be the current consensus.
> > > 
> > > Yep, though it'd be good to get Paolo's input, too.
> > > 
> > > > > If we want to do hypercall exiting, this should be in a follow-up
> > > > > series where we implement something more generic, e.g. a hypercall
> > > > > exiting bitmap or hypercall exit list. If we are taking the hypercall
> > > > > exit route, we can drop the kvm side of the hypercall.
> > > 
> > > I don't think this is a good candidate for arbitrary hypercall 
> > > interception.  Or
> > > rather, I think hypercall interception should be an orthogonal 
> > > implementation.
> > > 
> > > The guest, including guest firmware, needs to be aware that the hypercall 
> > > is
> > > supported, and the ABI needs to be well-defined.  Relying on userspace 
> > > VMMs to
> > > implement a common ABI is an unnecessary risk.
> > > 
> > > We could make KVM's default behavior be a nop, i.e. have KVM enforce the 
> > > ABI but
> > > require further VMM intervention.  But, I just don't see the point, it 
> > > would
> > > save only a few lines of code.  It would also limit what KVM could do in 
> > > the
> > > future, e.g. if KVM wanted to do its own bookkeeping _and_ exit to 
> > > userspace,
> > > then mandatory interception would essentially make it impossible for KVM 
> > > to do
> > > bookkeeping while still honoring the interception request.
> > > 
> > > However, I do think it would make sense to have the userspace exit be a 
> > > generic
> > > exit type.  But hey, we already have the necessary ABI defined for that!  
> > > It's
> > > just not used anywhere.
> > > 
> > >   /* KVM_EXIT_HYPERCALL */
> > >   struct {
> > >   __u64 nr;
> > >   __u64 args[6];
> > >   __u64 ret;
> > >   __u32 longmode;
> > >   __u32 pad;
> > >   } hypercall;
> > > 

Doc mentions to use KVM_EXIT_IO for x86 to implement "hypercall to userspace"
functionality. 

Any reasons for not using KVM_EXIT_HYPERCALL interface any more, even
though it is still there and not removed ?

> > > 
> > > > > Userspace could also handle the MSR using MSR filters (would need to
> > > > > confirm that).  Then userspace could also be in control of the cpuid 
> > > > > bit.
> > > 
> > > An MSR is not a great fit; it's x86 specific and limited to 64 bits of 
> > > data.
> > > The data limitation could be fudged by shoving data into non-standard 
> > > GPRs, but
> > > that will result in truly heinous guest code, and extensibility issues.
> > > 
> > > The data limitation is a moot point, because the x86-only thing is a deal
> > > breaker.  arm64's pKVM work has a near-identical use case for a guest to 
> > > share
> > > memory with a host.  I can't think of a clever way to avoid having to 
> > > support
> > > TDX's and SNP's hypervisor-agnostic variants, but we can at least not have
> > > multiple KVM variants.
> > 
> > Looking at arm64's pKVM work, i see that it is a recently introduced RFC
> > patch-set and probably relevant to arm64 nVHE hypervisor
> > mode/implementation, and potentially makes sense as it adds guest
> > memory protection as both host and guest kernels are running on the same
> > privilege level ?
> > 
> > Though i do see that the pKVM stuff adds two hypercalls, specifically :
> > 
> > pkvm_create_mappings() ( I assume this is for setting shared memory
> > regions between host and guest) &
> > pkvm_create_private_mappings().
> > 
> > And the use-cases are quite similar to memory protection architectues
> > use cases, for example, use with virtio devices, guest DMA I/O, etc.
> 
> The

Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

On Mon, Mar 08, 2021 at 03:11:41PM -0600, Brijesh Singh wrote:
> 
> On 3/8/21 1:51 PM, Sean Christopherson wrote:
> > On Mon, Mar 08, 2021, Ashish Kalra wrote:
> >> On Fri, Feb 26, 2021 at 09:44:41AM -0800, Sean Christopherson wrote:
> >>> +Will and Quentin (arm64)
> >>>
> >>> Moving the non-KVM x86 folks to bcc, I don't they care about KVM details 
> >>> at this
> >>> point.
> >>>
> >>> On Fri, Feb 26, 2021, Ashish Kalra wrote:
> >>>> On Thu, Feb 25, 2021 at 02:59:27PM -0800, Steve Rutherford wrote:
> >>>>> On Thu, Feb 25, 2021 at 12:20 PM Ashish Kalra  
> >>>>> wrote:
> >>>>> Thanks for grabbing the data!
> >>>>>
> >>>>> I am fine with both paths. Sean has stated an explicit desire for
> >>>>> hypercall exiting, so I think that would be the current consensus.
> >>> Yep, though it'd be good to get Paolo's input, too.
> >>>
> >>>>> If we want to do hypercall exiting, this should be in a follow-up
> >>>>> series where we implement something more generic, e.g. a hypercall
> >>>>> exiting bitmap or hypercall exit list. If we are taking the hypercall
> >>>>> exit route, we can drop the kvm side of the hypercall.
> >>> I don't think this is a good candidate for arbitrary hypercall 
> >>> interception.  Or
> >>> rather, I think hypercall interception should be an orthogonal 
> >>> implementation.
> >>>
> >>> The guest, including guest firmware, needs to be aware that the hypercall 
> >>> is
> >>> supported, and the ABI needs to be well-defined.  Relying on userspace 
> >>> VMMs to
> >>> implement a common ABI is an unnecessary risk.
> >>>
> >>> We could make KVM's default behavior be a nop, i.e. have KVM enforce the 
> >>> ABI but
> >>> require further VMM intervention.  But, I just don't see the point, it 
> >>> would
> >>> save only a few lines of code.  It would also limit what KVM could do in 
> >>> the
> >>> future, e.g. if KVM wanted to do its own bookkeeping _and_ exit to 
> >>> userspace,
> >>> then mandatory interception would essentially make it impossible for KVM 
> >>> to do
> >>> bookkeeping while still honoring the interception request.
> >>>
> >>> However, I do think it would make sense to have the userspace exit be a 
> >>> generic
> >>> exit type.  But hey, we already have the necessary ABI defined for that!  
> >>> It's
> >>> just not used anywhere.
> >>>
> >>>   /* KVM_EXIT_HYPERCALL */
> >>>   struct {
> >>>   __u64 nr;
> >>>   __u64 args[6];
> >>>   __u64 ret;
> >>>   __u32 longmode;
> >>>   __u32 pad;
> >>>   } hypercall;
> >>>
> >>>
> >>>>> Userspace could also handle the MSR using MSR filters (would need to
> >>>>> confirm that).  Then userspace could also be in control of the cpuid 
> >>>>> bit.
> >>> An MSR is not a great fit; it's x86 specific and limited to 64 bits of 
> >>> data.
> >>> The data limitation could be fudged by shoving data into non-standard 
> >>> GPRs, but
> >>> that will result in truly heinous guest code, and extensibility issues.
> >>>
> >>> The data limitation is a moot point, because the x86-only thing is a deal
> >>> breaker.  arm64's pKVM work has a near-identical use case for a guest to 
> >>> share
> >>> memory with a host.  I can't think of a clever way to avoid having to 
> >>> support
> >>> TDX's and SNP's hypervisor-agnostic variants, but we can at least not have
> >>> multiple KVM variants.
> >>>
> >> Potentially, there is another reason for in-kernel hypercall handling
> >> considering SEV-SNP. In case of SEV-SNP the RMP table tracks the state
> >> of each guest page, for instance pages in hypervisor state, i.e., pages
> >> with C=0 and pages in guest valid state with C=1.
> >>
> >> Now, there shouldn't be a need for page encryption status hypercalls on 
> >> SEV-SNP as KVM can track & reference guest page status directly using 
> >> the RMP table.
> > Relying on the RMP table itself would require locking the RMP table for 

Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

On Mon, Mar 08, 2021 at 11:51:57AM -0800, Sean Christopherson wrote:
> On Mon, Mar 08, 2021, Ashish Kalra wrote:
> > On Fri, Feb 26, 2021 at 09:44:41AM -0800, Sean Christopherson wrote:
> > > +Will and Quentin (arm64)
> > > 
> > > Moving the non-KVM x86 folks to bcc, I don't they care about KVM details 
> > > at this
> > > point.
> > > 
> > > On Fri, Feb 26, 2021, Ashish Kalra wrote:
> > > > On Thu, Feb 25, 2021 at 02:59:27PM -0800, Steve Rutherford wrote:
> > > > > On Thu, Feb 25, 2021 at 12:20 PM Ashish Kalra  
> > > > > wrote:
> > > > > Thanks for grabbing the data!
> > > > > 
> > > > > I am fine with both paths. Sean has stated an explicit desire for
> > > > > hypercall exiting, so I think that would be the current consensus.
> > > 
> > > Yep, though it'd be good to get Paolo's input, too.
> > > 
> > > > > If we want to do hypercall exiting, this should be in a follow-up
> > > > > series where we implement something more generic, e.g. a hypercall
> > > > > exiting bitmap or hypercall exit list. If we are taking the hypercall
> > > > > exit route, we can drop the kvm side of the hypercall.
> > > 
> > > I don't think this is a good candidate for arbitrary hypercall 
> > > interception.  Or
> > > rather, I think hypercall interception should be an orthogonal 
> > > implementation.
> > > 
> > > The guest, including guest firmware, needs to be aware that the hypercall 
> > > is
> > > supported, and the ABI needs to be well-defined.  Relying on userspace 
> > > VMMs to
> > > implement a common ABI is an unnecessary risk.
> > > 
> > > We could make KVM's default behavior be a nop, i.e. have KVM enforce the 
> > > ABI but
> > > require further VMM intervention.  But, I just don't see the point, it 
> > > would
> > > save only a few lines of code.  It would also limit what KVM could do in 
> > > the
> > > future, e.g. if KVM wanted to do its own bookkeeping _and_ exit to 
> > > userspace,
> > > then mandatory interception would essentially make it impossible for KVM 
> > > to do
> > > bookkeeping while still honoring the interception request.
> > > 
> > > However, I do think it would make sense to have the userspace exit be a 
> > > generic
> > > exit type.  But hey, we already have the necessary ABI defined for that!  
> > > It's
> > > just not used anywhere.
> > > 
> > >   /* KVM_EXIT_HYPERCALL */
> > >   struct {
> > >   __u64 nr;
> > >   __u64 args[6];
> > >   __u64 ret;
> > >   __u32 longmode;
> > >   __u32 pad;
> > >   } hypercall;
> > > 
> > > 
> > > > > Userspace could also handle the MSR using MSR filters (would need to
> > > > > confirm that).  Then userspace could also be in control of the cpuid 
> > > > > bit.
> > > 
> > > An MSR is not a great fit; it's x86 specific and limited to 64 bits of 
> > > data.
> > > The data limitation could be fudged by shoving data into non-standard 
> > > GPRs, but
> > > that will result in truly heinous guest code, and extensibility issues.
> > > 
> > > The data limitation is a moot point, because the x86-only thing is a deal
> > > breaker.  arm64's pKVM work has a near-identical use case for a guest to 
> > > share
> > > memory with a host.  I can't think of a clever way to avoid having to 
> > > support
> > > TDX's and SNP's hypervisor-agnostic variants, but we can at least not have
> > > multiple KVM variants.
> > > 
> > 
> > Potentially, there is another reason for in-kernel hypercall handling
> > considering SEV-SNP. In case of SEV-SNP the RMP table tracks the state
> > of each guest page, for instance pages in hypervisor state, i.e., pages
> > with C=0 and pages in guest valid state with C=1.
> > 
> > Now, there shouldn't be a need for page encryption status hypercalls on 
> > SEV-SNP as KVM can track & reference guest page status directly using 
> > the RMP table.
> 
> Relying on the RMP table itself would require locking the RMP table for an
> extended duration, and walking the entire RMP to find shared pages would be
> very inefficient.
> 
> > As KVM maintains the RMP table, therefore we will need SET/GET type of
> > interfaces to provide the guest page encryption status to userspace.
> 
> Hrm, somehow I temporarily forgot about SNP and TDX adding their own 
> hypercalls
> for converting between shared and private.  And in the case of TDX, the 
> hypercall
> can't be trusted, i.e. is just a hint, otherwise the guest could induce a #MC 
> in
> the host.

One question here, is this because if hypercall can cause direct
modifications to the shared EPT, it can induce #MC in the host ?

Thanks,
Ashish

Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

On Fri, Feb 26, 2021 at 09:44:41AM -0800, Sean Christopherson wrote:
> +Will and Quentin (arm64)
> 
> Moving the non-KVM x86 folks to bcc, I don't they care about KVM details at 
> this
> point.
> 
> On Fri, Feb 26, 2021, Ashish Kalra wrote:
> > On Thu, Feb 25, 2021 at 02:59:27PM -0800, Steve Rutherford wrote:
> > > On Thu, Feb 25, 2021 at 12:20 PM Ashish Kalra  
> > > wrote:
> > > Thanks for grabbing the data!
> > > 
> > > I am fine with both paths. Sean has stated an explicit desire for
> > > hypercall exiting, so I think that would be the current consensus.
> 
> Yep, though it'd be good to get Paolo's input, too.
> 
> > > If we want to do hypercall exiting, this should be in a follow-up
> > > series where we implement something more generic, e.g. a hypercall
> > > exiting bitmap or hypercall exit list. If we are taking the hypercall
> > > exit route, we can drop the kvm side of the hypercall.
> 
> I don't think this is a good candidate for arbitrary hypercall interception.  
> Or
> rather, I think hypercall interception should be an orthogonal implementation.
> 
> The guest, including guest firmware, needs to be aware that the hypercall is
> supported, and the ABI needs to be well-defined.  Relying on userspace VMMs to
> implement a common ABI is an unnecessary risk.
> 
> We could make KVM's default behavior be a nop, i.e. have KVM enforce the ABI 
> but
> require further VMM intervention.  But, I just don't see the point, it would
> save only a few lines of code.  It would also limit what KVM could do in the
> future, e.g. if KVM wanted to do its own bookkeeping _and_ exit to userspace,
> then mandatory interception would essentially make it impossible for KVM to do
> bookkeeping while still honoring the interception request.
> 
> However, I do think it would make sense to have the userspace exit be a 
> generic
> exit type.  But hey, we already have the necessary ABI defined for that!  It's
> just not used anywhere.
> 
>   /* KVM_EXIT_HYPERCALL */
>   struct {
>   __u64 nr;
>   __u64 args[6];
>   __u64 ret;
>   __u32 longmode;
>   __u32 pad;
>   } hypercall;
> 
> 
> > > Userspace could also handle the MSR using MSR filters (would need to
> > > confirm that).  Then userspace could also be in control of the cpuid bit.
> 
> An MSR is not a great fit; it's x86 specific and limited to 64 bits of data.
> The data limitation could be fudged by shoving data into non-standard GPRs, 
> but
> that will result in truly heinous guest code, and extensibility issues.
> 
> The data limitation is a moot point, because the x86-only thing is a deal
> breaker.  arm64's pKVM work has a near-identical use case for a guest to share
> memory with a host.  I can't think of a clever way to avoid having to support
> TDX's and SNP's hypervisor-agnostic variants, but we can at least not have
> multiple KVM variants.
> 

Potentially, there is another reason for in-kernel hypercall handling
considering SEV-SNP. In case of SEV-SNP the RMP table tracks the state
of each guest page, for instance pages in hypervisor state, i.e., pages
with C=0 and pages in guest valid state with C=1.

Now, there shouldn't be a need for page encryption status hypercalls on 
SEV-SNP as KVM can track & reference guest page status directly using 
the RMP table.

As KVM maintains the RMP table, therefore we will need SET/GET type of
interfaces to provide the guest page encryption status to userspace.

For the above reason if we do in-kernel hypercall handling for page
encryption status (which we probably won't require for SEV-SNP &
correspondingly there will be no hypercall exiting), then we can
implement a standard GET/SET ioctl interface to get/set the guest page
encryption status for userspace, which will work across SEV, SEV-ES and
SEV-SNP.

Thanks,
Ashish

Re: [RFC] KVM: x86: Support KVM VMs sharing SEV context

[Ashish Kalra]

On Thu, Feb 25, 2021 at 10:49:00AM -0800, Steve Rutherford wrote:
> On Thu, Feb 25, 2021 at 6:57 AM Tom Lendacky  wrote:
> > >> +int svm_vm_copy_asid_to(struct kvm *kvm, unsigned int mirror_kvm_fd)
> > >> +{
> > >> +   struct file *mirror_kvm_file;
> > >> +   struct kvm *mirror_kvm;
> > >> +   struct kvm_sev_info *mirror_kvm_sev;
> > >> +   unsigned int asid;
> > >> +   int ret;
> > >> +
> > >> +   if (!sev_guest(kvm))
> > >> +   return -ENOTTY;
> > >
> > > You definitely don't want this: this is the function that turns the vm
> > > into an SEV guest (marks SEV as active).
> >
> > The sev_guest() function does not set sev->active, it only checks it. The
> > sev_guest_init() function is where sev->active is set.
> Sorry, bad use of the english on my part: the "this" was referring to
> svm_vm_copy_asid_to. Right now, you could only pass this sev_guest
> check if you had already called sev_guest_init, which seems incorrect.
> >
> > >
> > > (Not an issue with this patch, but a broader issue) I believe
> > > sev_guest lacks the necessary acquire/release barriers on sev->active,
> >
> > The svm_mem_enc_op() takes the kvm lock and that is the only way into the
> > sev_guest_init() function where sev->active is set.
> There are a few places that check sev->active which don't have the kvm
> lock, which is not problematic if we add in a few compiler barriers
> (ala irqchip_split et al).

Probably, sev->active accesses can be made safe using READ_ONCE() &
WRITE_ONCE().

Thanks,
Ashish

Re: [RFC] KVM: x86: Support KVM VMs sharing SEV context

[Ashish Kalra]

On Wed, Feb 24, 2021 at 08:59:15AM +, Nathan Tempelman wrote:
> Add a capability for userspace to mirror SEV encryption context from
> one vm to another. On our side, this is intended to support a
> Migration Helper vCPU, but it can also be used generically to support
> other in-guest workloads scheduled by the host. The intention is for
> the primary guest and the mirror to have nearly identical memslots.
> 
> The primary benefits of this are that:
> 1) The VMs do not share KVM contexts (think APIC/MSRs/etc), so they
> can't accidentally clobber each other.
> 2) The VMs can have different memory-views, which is necessary for post-copy
> migration (the migration vCPUs on the target need to read and write to
> pages, when the primary guest would VMEXIT).
> 
> This does not change the threat model for AMD SEV. Any memory involved
> is still owned by the primary guest and its initial state is still
> attested to through the normal SEV_LAUNCH_* flows. If userspace wanted
> to circumvent SEV, they could achieve the same effect by simply attaching
> a vCPU to the primary VM.
> This patch deliberately leaves userspace in charge of the memslots for the
> mirror, as it already has the power to mess with them in the primary guest.
> 
> This patch does not support SEV-ES (much less SNP), as it does not
> handle handing off attested VMSAs to the mirror.
> 
> For additional context, we need a Migration Helper because SEV PSP migration
> is far too slow for our live migration on its own. Using an in-guest
> migrator lets us speed this up significantly.
> 
> Signed-off-by: Nathan Tempelman 
> ---
>  Documentation/virt/kvm/api.rst  | 17 +++
>  arch/x86/include/asm/kvm_host.h |  1 +
>  arch/x86/kvm/svm/sev.c  | 82 +
>  arch/x86/kvm/svm/svm.c  |  2 +
>  arch/x86/kvm/svm/svm.h  |  2 +
>  arch/x86/kvm/x86.c  |  7 ++-
>  include/linux/kvm_host.h|  1 +
>  include/uapi/linux/kvm.h|  1 +
>  virt/kvm/kvm_main.c |  8 
>  9 files changed, 120 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
> index 482508ec7cc4..438b647663c9 100644
> --- a/Documentation/virt/kvm/api.rst
> +++ b/Documentation/virt/kvm/api.rst
> @@ -6213,6 +6213,23 @@ the bus lock vm exit can be preempted by a higher 
> priority VM exit, the exit
>  notifications to userspace can be KVM_EXIT_BUS_LOCK or other reasons.
>  KVM_RUN_BUS_LOCK flag is used to distinguish between them.
>  
> +7.23 KVM_CAP_VM_COPY_ENC_CONTEXT_TO
> +---
> +
> +Architectures: x86 SEV enabled
> +Type: system
> +Parameters: args[0] is the fd of the kvm to mirror encryption context to
> +Returns: 0 on success; ENOTTY on error
> +
> +This capability enables userspace to copy encryption context from a primary
> +vm to the vm indicated by the fd.
> +
> +This is intended to support in-guest workloads scheduled by the host. This
> +allows the in-guest workload to maintain its own NPTs and keeps the two vms
> +from accidentally clobbering each other with interrupts and the like 
> (separate
> +APIC/MSRs/etc).
> +
> +
>  8. Other capabilities.
>  ==
>  
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 84499aad01a4..b7636c009647 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1334,6 +1334,7 @@ struct kvm_x86_ops {
>   int (*mem_enc_op)(struct kvm *kvm, void __user *argp);
>   int (*mem_enc_reg_region)(struct kvm *kvm, struct kvm_enc_region *argp);
>   int (*mem_enc_unreg_region)(struct kvm *kvm, struct kvm_enc_region 
> *argp);
> + int (*vm_copy_enc_context_to)(struct kvm *kvm, unsigned int child_fd);
>  
>   int (*get_msr_feature)(struct kvm_msr_entry *entry);
>  
> diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
> index 874ea309279f..2bad6cd2cb4c 100644
> --- a/arch/x86/kvm/svm/sev.c
> +++ b/arch/x86/kvm/svm/sev.c
> @@ -66,6 +66,11 @@ static int sev_flush_asids(void)
>   return ret;
>  }
>  
> +static inline bool is_mirroring_enc_context(struct kvm *kvm)
> +{
> + return _kvm_svm(kvm)->sev_info.enc_context_owner;
> +}
> +
>  /* Must be called with the sev_bitmap_lock held */
>  static bool __sev_recycle_asids(int min_asid, int max_asid)
>  {
> @@ -1124,6 +1129,10 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
>   if (copy_from_user(_cmd, argp, sizeof(struct kvm_sev_cmd)))
>   return -EFAULT;
>  
> + /* enc_context_owner handles all memory enc operations */
> + if (is_mirroring_enc_context(kvm))
> + return -ENOTTY;
> +
>   mutex_lock(>lock);
>  
>   switch (sev_cmd.id) {
> @@ -1186,6 +1195,10 @@ int svm_register_enc_region(struct kvm *kvm,
>   if (!sev_guest(kvm))
>   return -ENOTTY;
>  
> + /* If kvm is mirroring encryption context it isn't responsible for it */
> + if 

Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

On Wed, Mar 03, 2021 at 06:54:41PM +, Will Deacon wrote:
> [+Marc]
> 
> On Tue, Mar 02, 2021 at 02:55:43PM +0000, Ashish Kalra wrote:
> > On Fri, Feb 26, 2021 at 09:44:41AM -0800, Sean Christopherson wrote:
> > > On Fri, Feb 26, 2021, Ashish Kalra wrote:
> > > > On Thu, Feb 25, 2021 at 02:59:27PM -0800, Steve Rutherford wrote:
> > > > > On Thu, Feb 25, 2021 at 12:20 PM Ashish Kalra  
> > > > > wrote:
> > > > > Thanks for grabbing the data!
> > > > > 
> > > > > I am fine with both paths. Sean has stated an explicit desire for
> > > > > hypercall exiting, so I think that would be the current consensus.
> > > 
> > > Yep, though it'd be good to get Paolo's input, too.
> > > 
> > > > > If we want to do hypercall exiting, this should be in a follow-up
> > > > > series where we implement something more generic, e.g. a hypercall
> > > > > exiting bitmap or hypercall exit list. If we are taking the hypercall
> > > > > exit route, we can drop the kvm side of the hypercall.
> > > 
> > > I don't think this is a good candidate for arbitrary hypercall 
> > > interception.  Or
> > > rather, I think hypercall interception should be an orthogonal 
> > > implementation.
> > > 
> > > The guest, including guest firmware, needs to be aware that the hypercall 
> > > is
> > > supported, and the ABI needs to be well-defined.  Relying on userspace 
> > > VMMs to
> > > implement a common ABI is an unnecessary risk.
> > > 
> > > We could make KVM's default behavior be a nop, i.e. have KVM enforce the 
> > > ABI but
> > > require further VMM intervention.  But, I just don't see the point, it 
> > > would
> > > save only a few lines of code.  It would also limit what KVM could do in 
> > > the
> > > future, e.g. if KVM wanted to do its own bookkeeping _and_ exit to 
> > > userspace,
> > > then mandatory interception would essentially make it impossible for KVM 
> > > to do
> > > bookkeeping while still honoring the interception request.
> > > 
> > > However, I do think it would make sense to have the userspace exit be a 
> > > generic
> > > exit type.  But hey, we already have the necessary ABI defined for that!  
> > > It's
> > > just not used anywhere.
> > > 
> > >   /* KVM_EXIT_HYPERCALL */
> > >   struct {
> > >   __u64 nr;
> > >   __u64 args[6];
> > >   __u64 ret;
> > >   __u32 longmode;
> > >   __u32 pad;
> > >   } hypercall;
> > > 
> > > 
> > > > > Userspace could also handle the MSR using MSR filters (would need to
> > > > > confirm that).  Then userspace could also be in control of the cpuid 
> > > > > bit.
> > > 
> > > An MSR is not a great fit; it's x86 specific and limited to 64 bits of 
> > > data.
> > > The data limitation could be fudged by shoving data into non-standard 
> > > GPRs, but
> > > that will result in truly heinous guest code, and extensibility issues.
> > > 
> > > The data limitation is a moot point, because the x86-only thing is a deal
> > > breaker.  arm64's pKVM work has a near-identical use case for a guest to 
> > > share
> > > memory with a host.  I can't think of a clever way to avoid having to 
> > > support
> > > TDX's and SNP's hypervisor-agnostic variants, but we can at least not have
> > > multiple KVM variants.
> > 
> > Looking at arm64's pKVM work, i see that it is a recently introduced RFC
> > patch-set and probably relevant to arm64 nVHE hypervisor
> > mode/implementation, and potentially makes sense as it adds guest
> > memory protection as both host and guest kernels are running on the same
> > privilege level ?
> > 
> > Though i do see that the pKVM stuff adds two hypercalls, specifically :
> > 
> > pkvm_create_mappings() ( I assume this is for setting shared memory
> > regions between host and guest) &
> > pkvm_create_private_mappings().
> > 
> > And the use-cases are quite similar to memory protection architectues
> > use cases, for example, use with virtio devices, guest DMA I/O, etc.
> 
> These hypercalls are both private to the host kernel communicating with
> its hypervisor counterpart, so I don't think they're particularly
> relevant here. 

Yes, i have the same thoughts here as this looked like 

Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

On Fri, Feb 26, 2021 at 09:44:41AM -0800, Sean Christopherson wrote:
> +Will and Quentin (arm64)
> 
> Moving the non-KVM x86 folks to bcc, I don't they care about KVM details at 
> this
> point.
> 
> On Fri, Feb 26, 2021, Ashish Kalra wrote:
> > On Thu, Feb 25, 2021 at 02:59:27PM -0800, Steve Rutherford wrote:
> > > On Thu, Feb 25, 2021 at 12:20 PM Ashish Kalra  
> > > wrote:
> > > Thanks for grabbing the data!
> > > 
> > > I am fine with both paths. Sean has stated an explicit desire for
> > > hypercall exiting, so I think that would be the current consensus.
> 
> Yep, though it'd be good to get Paolo's input, too.
> 
> > > If we want to do hypercall exiting, this should be in a follow-up
> > > series where we implement something more generic, e.g. a hypercall
> > > exiting bitmap or hypercall exit list. If we are taking the hypercall
> > > exit route, we can drop the kvm side of the hypercall.
> 
> I don't think this is a good candidate for arbitrary hypercall interception.  
> Or
> rather, I think hypercall interception should be an orthogonal implementation.
> 
> The guest, including guest firmware, needs to be aware that the hypercall is
> supported, and the ABI needs to be well-defined.  Relying on userspace VMMs to
> implement a common ABI is an unnecessary risk.
> 
> We could make KVM's default behavior be a nop, i.e. have KVM enforce the ABI 
> but
> require further VMM intervention.  But, I just don't see the point, it would
> save only a few lines of code.  It would also limit what KVM could do in the
> future, e.g. if KVM wanted to do its own bookkeeping _and_ exit to userspace,
> then mandatory interception would essentially make it impossible for KVM to do
> bookkeeping while still honoring the interception request.
> 
> However, I do think it would make sense to have the userspace exit be a 
> generic
> exit type.  But hey, we already have the necessary ABI defined for that!  It's
> just not used anywhere.
> 
>   /* KVM_EXIT_HYPERCALL */
>   struct {
>   __u64 nr;
>   __u64 args[6];
>   __u64 ret;
>   __u32 longmode;
>   __u32 pad;
>   } hypercall;
> 
> 
> > > Userspace could also handle the MSR using MSR filters (would need to
> > > confirm that).  Then userspace could also be in control of the cpuid bit.
> 
> An MSR is not a great fit; it's x86 specific and limited to 64 bits of data.
> The data limitation could be fudged by shoving data into non-standard GPRs, 
> but
> that will result in truly heinous guest code, and extensibility issues.
> 
> The data limitation is a moot point, because the x86-only thing is a deal
> breaker.  arm64's pKVM work has a near-identical use case for a guest to share
> memory with a host.  I can't think of a clever way to avoid having to support
> TDX's and SNP's hypervisor-agnostic variants, but we can at least not have
> multiple KVM variants.

Looking at arm64's pKVM work, i see that it is a recently introduced RFC
patch-set and probably relevant to arm64 nVHE hypervisor
mode/implementation, and potentially makes sense as it adds guest
memory protection as both host and guest kernels are running on the same
privilege level ?

Though i do see that the pKVM stuff adds two hypercalls, specifically :

pkvm_create_mappings() ( I assume this is for setting shared memory
regions between host and guest) &
pkvm_create_private_mappings().

And the use-cases are quite similar to memory protection architectues
use cases, for example, use with virtio devices, guest DMA I/O, etc.

But, isn't this patch set still RFC, and though i agree that it adds
an infrastructure for standardised communication between the host and
it's guests for mutually controlled shared memory regions and
surely adds some kind of portability between hypervisor
implementations, but nothing is standardised still, right ?

Thanks,
Ashish

Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

On Tue, Mar 02, 2021 at 02:55:43PM +, Ashish Kalra wrote:
> On Fri, Feb 26, 2021 at 09:44:41AM -0800, Sean Christopherson wrote:
> > +Will and Quentin (arm64)
> > 
> > Moving the non-KVM x86 folks to bcc, I don't they care about KVM details at 
> > this
> > point.
> > 
> > On Fri, Feb 26, 2021, Ashish Kalra wrote:
> > > On Thu, Feb 25, 2021 at 02:59:27PM -0800, Steve Rutherford wrote:
> > > > On Thu, Feb 25, 2021 at 12:20 PM Ashish Kalra  
> > > > wrote:
> > > > Thanks for grabbing the data!
> > > > 
> > > > I am fine with both paths. Sean has stated an explicit desire for
> > > > hypercall exiting, so I think that would be the current consensus.
> > 
> > Yep, though it'd be good to get Paolo's input, too.
> > 
> > > > If we want to do hypercall exiting, this should be in a follow-up
> > > > series where we implement something more generic, e.g. a hypercall
> > > > exiting bitmap or hypercall exit list. If we are taking the hypercall
> > > > exit route, we can drop the kvm side of the hypercall.
> > 
> > I don't think this is a good candidate for arbitrary hypercall 
> > interception.  Or
> > rather, I think hypercall interception should be an orthogonal 
> > implementation.
> > 
> > The guest, including guest firmware, needs to be aware that the hypercall is
> > supported, and the ABI needs to be well-defined.  Relying on userspace VMMs 
> > to
> > implement a common ABI is an unnecessary risk.
> > 
> > We could make KVM's default behavior be a nop, i.e. have KVM enforce the 
> > ABI but
> > require further VMM intervention.  But, I just don't see the point, it would
> > save only a few lines of code.  It would also limit what KVM could do in the
> > future, e.g. if KVM wanted to do its own bookkeeping _and_ exit to 
> > userspace,
> > then mandatory interception would essentially make it impossible for KVM to 
> > do
> > bookkeeping while still honoring the interception request.
> > 
> > However, I do think it would make sense to have the userspace exit be a 
> > generic
> > exit type.  But hey, we already have the necessary ABI defined for that!  
> > It's
> > just not used anywhere.
> > 
> > /* KVM_EXIT_HYPERCALL */
> > struct {
> > __u64 nr;
> > __u64 args[6];
> > __u64 ret;
> > __u32 longmode;
> > __u32 pad;
> > } hypercall;
> > 
> > 
> > > > Userspace could also handle the MSR using MSR filters (would need to
> > > > confirm that).  Then userspace could also be in control of the cpuid 
> > > > bit.
> > 
> > An MSR is not a great fit; it's x86 specific and limited to 64 bits of data.
> > The data limitation could be fudged by shoving data into non-standard GPRs, 
> > but
> > that will result in truly heinous guest code, and extensibility issues.
> > 
> > The data limitation is a moot point, because the x86-only thing is a deal
> > breaker.  arm64's pKVM work has a near-identical use case for a guest to 
> > share
> > memory with a host.  I can't think of a clever way to avoid having to 
> > support
> > TDX's and SNP's hypervisor-agnostic variants, but we can at least not have
> > multiple KVM variants.
> 
> Looking at arm64's pKVM work, i see that it is a recently introduced RFC
> patch-set and probably relevant to arm64 nVHE hypervisor
> mode/implementation, and potentially makes sense as it adds guest
> memory protection as both host and guest kernels are running on the same
> privilege level ?
> 
> Though i do see that the pKVM stuff adds two hypercalls, specifically :
> 
> pkvm_create_mappings() ( I assume this is for setting shared memory
> regions between host and guest) &
> pkvm_create_private_mappings().
> 
> And the use-cases are quite similar to memory protection architectues
> use cases, for example, use with virtio devices, guest DMA I/O, etc.
> 
> But, isn't this patch set still RFC, and though i agree that it adds
> an infrastructure for standardised communication between the host and
> it's guests for mutually controlled shared memory regions and
> surely adds some kind of portability between hypervisor
> implementations, but nothing is standardised still, right ?
> 

And to add here, the hypercall implementation is in-HYP mode,
there is no infrastructure as part of this patch-set to do
hypercall exiting and handling it in user-space. 

Though arguably, we may able to add a hypercall exiting code path on the
amd64 implementation for the same hypercall interfaces ?

Alternatively, we implement this in-kernel and then add SET/GET ioctl
interfaces to export the shared pages/regions list to user-space.

Thanks,
Ashish

Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

Hello Steve,

On Thu, Feb 25, 2021 at 02:59:27PM -0800, Steve Rutherford wrote:
> On Thu, Feb 25, 2021 at 12:20 PM Ashish Kalra  wrote:
> >
> > On Wed, Feb 24, 2021 at 10:22:33AM -0800, Sean Christopherson wrote:
> > > On Wed, Feb 24, 2021, Ashish Kalra wrote:
> > > > # Samples: 19K of event 'kvm:kvm_hypercall'
> > > > # Event count (approx.): 19573
> > > > #
> > > > # Overhead  Command  Shared Object Symbol
> > > > #   ...    .
> > > > #
> > > >100.00%  qemu-system-x86  [kernel.vmlinux]  [k] kvm_emulate_hypercall
> > > >
> > > > Out of these 19573 hypercalls, # of page encryption status hcalls are 
> > > > 19479,
> > > > so almost all hypercalls here are page encryption status hypercalls.
> > >
> > > Oof.
> > >
> > > > The above data indicates that there will be ~2% more Heavyweight VMEXITs
> > > > during SEV guest boot if we do page encryption status hypercalls
> > > > pass-through to host userspace.
> > > >
> > > > But, then Brijesh pointed out to me and highlighted that currently
> > > > OVMF is doing lot of VMEXITs because they don't use the DMA pool to 
> > > > minimize the C-bit toggles,
> > > > in other words, OVMF bounce buffer does page state change on every DMA 
> > > > allocate and free.
> > > >
> > > > So here is the performance analysis after kernel and initrd have been
> > > > loaded into memory using grub and then starting perf just before 
> > > > booting the kernel.
> > > >
> > > > These are the performance #'s after kernel and initrd have been loaded 
> > > > into memory,
> > > > then perf is attached and kernel is booted :
> > > >
> > > > # Samples: 1M of event 'kvm:kvm_userspace_exit'
> > > > # Event count (approx.): 1081235
> > > > #
> > > > # Overhead  Trace output
> > > > #   
> > > > #
> > > > 99.77%  reason KVM_EXIT_IO (2)
> > > >  0.23%  reason KVM_EXIT_MMIO (6)
> > > >
> > > > # Samples: 1K of event 'kvm:kvm_hypercall'
> > > > # Event count (approx.): 1279
> > > > #
> > > >
> > > > So as the above data indicates, Linux is only making ~1K hypercalls,
> > > > compared to ~18K hypercalls made by OVMF in the above use case.
> > > >
> > > > Does the above adds a prerequisite that OVMF needs to be optimized if
> > > > and before hypercall pass-through can be done ?
> > >
> > > Disclaimer: my math could be totally wrong.
> > >
> > > I doubt it's a hard requirement.  Assuming a conversative roundtrip time 
> > > of 50k
> > > cycles, those 18K hypercalls will add well under a 1/2 a second of boot 
> > > time.
> > > If userspace can push the roundtrip time down to 10k cycles, the overhead 
> > > is
> > > more like 50 milliseconds.
> > >
> > > That being said, this does seem like a good OVMF cleanup, irrespective of 
> > > this
> > > new hypercall.  I assume it's not cheap to convert a page between 
> > > encrypted and
> > > decrypted.
> > >
> > > Thanks much for getting the numbers!
> >
> > Considering the above data and guest boot time latencies
> > (and potential issues with OVMF and optimizations required there),
> > do we have any consensus on whether we want to do page encryption
> > status hypercall passthrough or not ?
> >
> > Thanks,
> > Ashish
> 
> Thanks for grabbing the data!
> 
> I am fine with both paths. Sean has stated an explicit desire for
> hypercall exiting, so I think that would be the current consensus.
> 
> If we want to do hypercall exiting, this should be in a follow-up
> series where we implement something more generic, e.g. a hypercall
> exiting bitmap or hypercall exit list. If we are taking the hypercall
> exit route, we can drop the kvm side of the hypercall. Userspace could
> also handle the MSR using MSR filters (would need to confirm that).
> Then userspace could also be in control of the cpuid bit.
> 
> Essentially, I think you could drop most of the host kernel work if
> there were generic support for hypercall exiting. Then userspace would
> be responsible for all of that. Thoughts on this?
> 
So if i understand it correctly, i will submitting v11 of this patch-set
with in-kernel support for page encryption status hypercalls and shared
pages list and the userspace control of SEV live migration feature
support and fixes for MSR handling.

In subsequent follow-up patches we will add generic support for hypercall 
exiting and then drop kvm side of hypercall and also add userspace
support for MSR handling.

Thanks,
Ashish

Re: [RFC] KVM: x86: Support KVM VMs sharing SEV context

[Ashish Kalra]

On Thu, Feb 25, 2021 at 09:33:09PM +0100, Paolo Bonzini wrote:
> On 25/02/21 19:18, Ashish Kalra wrote:
> > I do believe that some of these alternative SEV live migration support
> > or Migration helper (MH) solutions will still use SEV PSP migration for
> > migrating the MH itself, therefore the SEV live migration patches
> > (currently v10 posted upstream) still make sense and will be used.
> 
> I think that since the migration helper (at least for SEV, not -ES) is part
> of the attested firmware, it can be started on the destination before the
> rest of the VM.
> 

We may also need to transport VMSA(s) securely using the PSP migration
support for SEV-ES live migration with MH.

Thanks,
Ashish

Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

On Wed, Feb 24, 2021 at 10:22:33AM -0800, Sean Christopherson wrote:
> On Wed, Feb 24, 2021, Ashish Kalra wrote:
> > # Samples: 19K of event 'kvm:kvm_hypercall'
> > # Event count (approx.): 19573
> > #
> > # Overhead  Command  Shared Object Symbol
> > #   ...    .
> > #
> >100.00%  qemu-system-x86  [kernel.vmlinux]  [k] kvm_emulate_hypercall
> > 
> > Out of these 19573 hypercalls, # of page encryption status hcalls are 19479,
> > so almost all hypercalls here are page encryption status hypercalls.
> 
> Oof.
> 
> > The above data indicates that there will be ~2% more Heavyweight VMEXITs
> > during SEV guest boot if we do page encryption status hypercalls 
> > pass-through to host userspace.
> > 
> > But, then Brijesh pointed out to me and highlighted that currently
> > OVMF is doing lot of VMEXITs because they don't use the DMA pool to 
> > minimize the C-bit toggles,
> > in other words, OVMF bounce buffer does page state change on every DMA 
> > allocate and free.
> > 
> > So here is the performance analysis after kernel and initrd have been
> > loaded into memory using grub and then starting perf just before booting 
> > the kernel.
> > 
> > These are the performance #'s after kernel and initrd have been loaded into 
> > memory, 
> > then perf is attached and kernel is booted : 
> > 
> > # Samples: 1M of event 'kvm:kvm_userspace_exit'
> > # Event count (approx.): 1081235
> > #
> > # Overhead  Trace output
> > #   
> > #
> > 99.77%  reason KVM_EXIT_IO (2)
> >  0.23%  reason KVM_EXIT_MMIO (6)
> > 
> > # Samples: 1K of event 'kvm:kvm_hypercall'
> > # Event count (approx.): 1279
> > #
> > 
> > So as the above data indicates, Linux is only making ~1K hypercalls,
> > compared to ~18K hypercalls made by OVMF in the above use case.
> > 
> > Does the above adds a prerequisite that OVMF needs to be optimized if 
> > and before hypercall pass-through can be done ? 
> 
> Disclaimer: my math could be totally wrong.
> 
> I doubt it's a hard requirement.  Assuming a conversative roundtrip time of 
> 50k
> cycles, those 18K hypercalls will add well under a 1/2 a second of boot time.
> If userspace can push the roundtrip time down to 10k cycles, the overhead is
> more like 50 milliseconds.
> 
> That being said, this does seem like a good OVMF cleanup, irrespective of this
> new hypercall.  I assume it's not cheap to convert a page between encrypted 
> and
> decrypted.
> 
> Thanks much for getting the numbers!

Considering the above data and guest boot time latencies 
(and potential issues with OVMF and optimizations required there),
do we have any consensus on whether we want to do page encryption
status hypercall passthrough or not ?

Thanks,
Ashish

Re: [RFC] KVM: x86: Support KVM VMs sharing SEV context

[Ashish Kalra]

>> > For additional context, we need a Migration Helper because SEV PSP
>> > migration is far too slow for our live migration on its own. Using an
>> > in-guest migrator lets us speed this up significantly.
>> 
>> We have the same problem here at IBM, hence the RFC referred to above.
>> 

I do believe that some of these alternative SEV live migration support
or Migration helper (MH) solutions will still use SEV PSP migration for 
migrating the MH itself, therefore the SEV live migration patches
(currently v10 posted upstream) still make sense and will be used.

Thanks,
Ashish

Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

On Thu, Feb 18, 2021 at 12:32:47PM -0600, Kalra, Ashish wrote:
> [AMD Public Use]
> 
> 
> -Original Message-
> From: Sean Christopherson  
> Sent: Tuesday, February 16, 2021 7:03 PM
> To: Kalra, Ashish 
> Cc: pbonz...@redhat.com; t...@linutronix.de; mi...@redhat.com; 
> h...@zytor.com; rkrc...@redhat.com; j...@8bytes.org; b...@suse.de; Lendacky, 
> Thomas ; x...@kernel.org; k...@vger.kernel.org; 
> linux-kernel@vger.kernel.org; srutherf...@google.com; 
> venu.busire...@oracle.com; Singh, Brijesh 
> Subject: Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST 
> ioctl
> 
> On Thu, Feb 04, 2021, Ashish Kalra wrote:
> > From: Brijesh Singh 
> > 
> > The ioctl is used to retrieve a guest's shared pages list.
> 
> >What's the performance hit to boot time if KVM_HC_PAGE_ENC_STATUS is passed 
> >through to userspace?  That way, userspace could manage the set of pages >in 
> >whatever data structure they want, and these get/set ioctls go away.
> 
> I will be more concerned about performance hit during guest DMA I/O if the 
> page encryption status hypercalls are passed through to user-space, 
> a lot of guest DMA I/O dynamically sets up pages for encryption and then 
> flips them at DMA completion, so guest I/O will surely take a performance 
> hit with this pass-through stuff.
> 

Here are some rough performance numbers comparing # of heavy-weight VMEXITs 
compared to # of hypercalls, 
during a SEV guest boot: (launch of a ubuntu 18.04 guest)

# ./perf record -e kvm:kvm_userspace_exit -e kvm:kvm_hypercall -a 
./qemu-system-x86_64 -enable-kvm -cpu host -machine q35 -smp 16,maxcpus=64 -m 
512M -drive 
if=pflash,format=raw,unit=0,file=/home/ashish/sev-migration/qemu-5.1.50/OVMF_CODE.fd,readonly
 -drive if=pflash,format=raw,unit=1,file=OVMF_VARS.fd -drive 
file=../ubuntu-18.04.qcow2,if=none,id=disk0,format=qcow2 -device 
virtio-scsi-pci,id=scsi,disable-legacy=on,iommu_platform=true -device 
scsi-hd,drive=disk0 -object 
sev-guest,id=sev0,cbitpos=47,reduced-phys-bits=1,policy=0x0 -machine 
memory-encryption=sev0 -trace events=/tmp/events -nographic -monitor pty 
-monitor unix:monitor-source,server,nowait -qmp 
unix:/tmp/qmp-sock,server,nowait -device 
virtio-rng-pci,disable-legacy=on,iommu_platform=true

...
...

root@diesel2540:/home/ashish/sev-migration/qemu-5.1.50# ./perf report
# To display the perf.data header info, please use --header/--header-only 
options.
#
#
# Total Lost Samples: 0
#
# Samples: 981K of event 'kvm:kvm_userspace_exit'
# Event count (approx.): 981021
#
# Overhead  Command  Shared Object Symbol
#   ...    ..
#
   100.00%  qemu-system-x86  [kernel.vmlinux]  [k] kvm_vcpu_ioctl


# Samples: 19K of event 'kvm:kvm_hypercall'
# Event count (approx.): 19573
#
# Overhead  Command  Shared Object Symbol
#   ...    .
#
   100.00%  qemu-system-x86  [kernel.vmlinux]  [k] kvm_emulate_hypercall

Out of these 19573 hypercalls, # of page encryption status hcalls are 19479,
so almost all hypercalls here are page encryption status hypercalls.

The above data indicates that there will be ~2% more Heavyweight VMEXITs
during SEV guest boot if we do page encryption status hypercalls 
pass-through to host userspace.

But, then Brijesh pointed out to me and highlighted that currently
OVMF is doing lot of VMEXITs because they don't use the DMA pool to minimize 
the C-bit toggles,
in other words, OVMF bounce buffer does page state change on every DMA allocate 
and free.

So here is the performance analysis after kernel and initrd have been
loaded into memory using grub and then starting perf just before booting the 
kernel.

These are the performance #'s after kernel and initrd have been loaded into 
memory, 
then perf is attached and kernel is booted : 

# Samples: 1M of event 'kvm:kvm_userspace_exit'
# Event count (approx.): 1081235
#
# Overhead  Trace output
#   
#
99.77%  reason KVM_EXIT_IO (2)
 0.23%  reason KVM_EXIT_MMIO (6)

# Samples: 1K of event 'kvm:kvm_hypercall'
# Event count (approx.): 1279
#

So as the above data indicates, Linux is only making ~1K hypercalls,
compared to ~18K hypercalls made by OVMF in the above use case.

Does the above adds a prerequisite that OVMF needs to be optimized if 
and before hypercall pass-through can be done ? 

Thanks,
Ashish

Re: [PATCH v10 12/16] KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature & Custom MSR.

[Ashish Kalra]

Hello Steve,

On Mon, Feb 08, 2021 at 02:50:14PM -0800, Steve Rutherford wrote:
> Hi Ashish,
> 
> On Sun, Feb 7, 2021 at 4:29 PM Ashish Kalra  wrote:
> >
> > Hello Steve,
> >
> > On Sat, Feb 06, 2021 at 01:56:46PM +, Ashish Kalra wrote:
> > > Hello Steve,
> > >
> > > On Sat, Feb 06, 2021 at 05:46:17AM +, Ashish Kalra wrote:
> > > > Hello Steve,
> > > >
> > > > Continued response to your queries, especially related to userspace
> > > > control of SEV live migration feature :
> > > >
> > > > On Fri, Feb 05, 2021 at 06:54:21PM -0800, Steve Rutherford wrote:
> > > > > On Thu, Feb 4, 2021 at 7:08 PM Ashish Kalra  
> > > > > wrote:
> > > > > >
> > > > > > Hello Steve,
> > > > > >
> > > > > > On Thu, Feb 04, 2021 at 04:56:35PM -0800, Steve Rutherford wrote:
> > > > > > > On Wed, Feb 3, 2021 at 4:39 PM Ashish Kalra 
> > > > > > >  wrote:
> > > > > > > >
> > > > > > > > From: Ashish Kalra 
> > > > > > > >
> > > > > > > > Add new KVM_FEATURE_SEV_LIVE_MIGRATION feature for guest to 
> > > > > > > > check
> > > > > > > > for host-side support for SEV live migration. Also add a new 
> > > > > > > > custom
> > > > > > > > MSR_KVM_SEV_LIVE_MIGRATION for guest to enable the SEV live 
> > > > > > > > migration
> > > > > > > > feature.
> > > > > > > >
> > > > > > > > Signed-off-by: Ashish Kalra 
> > > > > > > > ---
> > > > > > > >  Documentation/virt/kvm/cpuid.rst |  5 +
> > > > > > > >  Documentation/virt/kvm/msr.rst   | 12 
> > > > > > > >  arch/x86/include/uapi/asm/kvm_para.h |  4 
> > > > > > > >  arch/x86/kvm/svm/sev.c   | 13 +
> > > > > > > >  arch/x86/kvm/svm/svm.c   | 16 
> > > > > > > >  arch/x86/kvm/svm/svm.h   |  2 ++
> > > > > > > >  6 files changed, 52 insertions(+)
> > > > > > > >
> > > > > > > > diff --git a/Documentation/virt/kvm/cpuid.rst 
> > > > > > > > b/Documentation/virt/kvm/cpuid.rst
> > > > > > > > index cf62162d4be2..0bdb6cdb12d3 100644
> > > > > > > > --- a/Documentation/virt/kvm/cpuid.rst
> > > > > > > > +++ b/Documentation/virt/kvm/cpuid.rst
> > > > > > > > @@ -96,6 +96,11 @@ KVM_FEATURE_MSI_EXT_DEST_ID15
> > > > > > > >   guest checks this feature bit
> > > > > > > > before using 
> > > > > > > > extended destination
> > > > > > > > ID bits in MSI 
> > > > > > > > address bits 11-5.
> > > > > > > >
> > > > > > > > +KVM_FEATURE_SEV_LIVE_MIGRATION 16  guest checks 
> > > > > > > > this feature bit before
> > > > > > > > +   using the page 
> > > > > > > > encryption state
> > > > > > > > +   hypercall to 
> > > > > > > > notify the page state
> > > > > > > > +   change
> > > > > > > > +
> > > > > > > >  KVM_FEATURE_CLOCKSOURCE_STABLE_BIT 24  host will warn 
> > > > > > > > if no guest-side
> > > > > > > > per-cpu warps 
> > > > > > > > are expected in
> > > > > > > > kvmclock
> > > > > > > > diff --git a/Documentation/virt/kvm/msr.rst 
> > > > > > > > b/Documentation/virt/kvm/msr.rst
> > > > > > > > index e37a14c323d2..020245d16087 100644
> > > > > > > > --- a/Documentation/virt/kvm/msr.rst
> > > > > > > > +++ b/Documentation/virt/kvm/msr.rst
> > > > &g

Re: [PATCH v10 12/16] KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature & Custom MSR.

[Ashish Kalra]

Hello Steve, 

On Sat, Feb 06, 2021 at 01:56:46PM +, Ashish Kalra wrote:
> Hello Steve,
> 
> On Sat, Feb 06, 2021 at 05:46:17AM +0000, Ashish Kalra wrote:
> > Hello Steve,
> > 
> > Continued response to your queries, especially related to userspace
> > control of SEV live migration feature : 
> > 
> > On Fri, Feb 05, 2021 at 06:54:21PM -0800, Steve Rutherford wrote:
> > > On Thu, Feb 4, 2021 at 7:08 PM Ashish Kalra  wrote:
> > > >
> > > > Hello Steve,
> > > >
> > > > On Thu, Feb 04, 2021 at 04:56:35PM -0800, Steve Rutherford wrote:
> > > > > On Wed, Feb 3, 2021 at 4:39 PM Ashish Kalra  
> > > > > wrote:
> > > > > >
> > > > > > From: Ashish Kalra 
> > > > > >
> > > > > > Add new KVM_FEATURE_SEV_LIVE_MIGRATION feature for guest to check
> > > > > > for host-side support for SEV live migration. Also add a new custom
> > > > > > MSR_KVM_SEV_LIVE_MIGRATION for guest to enable the SEV live 
> > > > > > migration
> > > > > > feature.
> > > > > >
> > > > > > Signed-off-by: Ashish Kalra 
> > > > > > ---
> > > > > >  Documentation/virt/kvm/cpuid.rst |  5 +
> > > > > >  Documentation/virt/kvm/msr.rst   | 12 
> > > > > >  arch/x86/include/uapi/asm/kvm_para.h |  4 
> > > > > >  arch/x86/kvm/svm/sev.c   | 13 +
> > > > > >  arch/x86/kvm/svm/svm.c   | 16 
> > > > > >  arch/x86/kvm/svm/svm.h   |  2 ++
> > > > > >  6 files changed, 52 insertions(+)
> > > > > >
> > > > > > diff --git a/Documentation/virt/kvm/cpuid.rst 
> > > > > > b/Documentation/virt/kvm/cpuid.rst
> > > > > > index cf62162d4be2..0bdb6cdb12d3 100644
> > > > > > --- a/Documentation/virt/kvm/cpuid.rst
> > > > > > +++ b/Documentation/virt/kvm/cpuid.rst
> > > > > > @@ -96,6 +96,11 @@ KVM_FEATURE_MSI_EXT_DEST_ID15  
> > > > > > guest checks this feature bit
> > > > > > before using 
> > > > > > extended destination
> > > > > > ID bits in MSI 
> > > > > > address bits 11-5.
> > > > > >
> > > > > > +KVM_FEATURE_SEV_LIVE_MIGRATION 16  guest checks this 
> > > > > > feature bit before
> > > > > > +   using the page 
> > > > > > encryption state
> > > > > > +   hypercall to notify 
> > > > > > the page state
> > > > > > +   change
> > > > > > +
> > > > > >  KVM_FEATURE_CLOCKSOURCE_STABLE_BIT 24  host will warn if 
> > > > > > no guest-side
> > > > > > per-cpu warps are 
> > > > > > expected in
> > > > > > kvmclock
> > > > > > diff --git a/Documentation/virt/kvm/msr.rst 
> > > > > > b/Documentation/virt/kvm/msr.rst
> > > > > > index e37a14c323d2..020245d16087 100644
> > > > > > --- a/Documentation/virt/kvm/msr.rst
> > > > > > +++ b/Documentation/virt/kvm/msr.rst
> > > > > > @@ -376,3 +376,15 @@ data:
> > > > > > write '1' to bit 0 of the MSR, this causes the host to 
> > > > > > re-scan its queue
> > > > > > and check if there are more notifications pending. The MSR 
> > > > > > is available
> > > > > > if KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
> > > > > > +
> > > > > > +MSR_KVM_SEV_LIVE_MIGRATION:
> > > > > > +0x4b564d08
> > > > > > +
> > > > > > +   Control SEV Live Migration features.
> > > > > > +
> > > > > > +data:
> > > > > > +Bit 0 enables (1) or disables (0) host-side SEV Live 
> > > > > > Migration feature,
> > > > > > +in other words, this is guest

Re: [PATCH v10 12/16] KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature & Custom MSR.

[Ashish Kalra]

Hello Steve,

On Sat, Feb 06, 2021 at 05:46:17AM +, Ashish Kalra wrote:
> Hello Steve,
> 
> Continued response to your queries, especially related to userspace
> control of SEV live migration feature : 
> 
> On Fri, Feb 05, 2021 at 06:54:21PM -0800, Steve Rutherford wrote:
> > On Thu, Feb 4, 2021 at 7:08 PM Ashish Kalra  wrote:
> > >
> > > Hello Steve,
> > >
> > > On Thu, Feb 04, 2021 at 04:56:35PM -0800, Steve Rutherford wrote:
> > > > On Wed, Feb 3, 2021 at 4:39 PM Ashish Kalra  
> > > > wrote:
> > > > >
> > > > > From: Ashish Kalra 
> > > > >
> > > > > Add new KVM_FEATURE_SEV_LIVE_MIGRATION feature for guest to check
> > > > > for host-side support for SEV live migration. Also add a new custom
> > > > > MSR_KVM_SEV_LIVE_MIGRATION for guest to enable the SEV live migration
> > > > > feature.
> > > > >
> > > > > Signed-off-by: Ashish Kalra 
> > > > > ---
> > > > >  Documentation/virt/kvm/cpuid.rst |  5 +
> > > > >  Documentation/virt/kvm/msr.rst   | 12 
> > > > >  arch/x86/include/uapi/asm/kvm_para.h |  4 
> > > > >  arch/x86/kvm/svm/sev.c   | 13 +
> > > > >  arch/x86/kvm/svm/svm.c   | 16 
> > > > >  arch/x86/kvm/svm/svm.h   |  2 ++
> > > > >  6 files changed, 52 insertions(+)
> > > > >
> > > > > diff --git a/Documentation/virt/kvm/cpuid.rst 
> > > > > b/Documentation/virt/kvm/cpuid.rst
> > > > > index cf62162d4be2..0bdb6cdb12d3 100644
> > > > > --- a/Documentation/virt/kvm/cpuid.rst
> > > > > +++ b/Documentation/virt/kvm/cpuid.rst
> > > > > @@ -96,6 +96,11 @@ KVM_FEATURE_MSI_EXT_DEST_ID15  
> > > > > guest checks this feature bit
> > > > > before using extended 
> > > > > destination
> > > > > ID bits in MSI 
> > > > > address bits 11-5.
> > > > >
> > > > > +KVM_FEATURE_SEV_LIVE_MIGRATION 16  guest checks this 
> > > > > feature bit before
> > > > > +   using the page 
> > > > > encryption state
> > > > > +   hypercall to notify 
> > > > > the page state
> > > > > +   change
> > > > > +
> > > > >  KVM_FEATURE_CLOCKSOURCE_STABLE_BIT 24  host will warn if no 
> > > > > guest-side
> > > > > per-cpu warps are 
> > > > > expected in
> > > > > kvmclock
> > > > > diff --git a/Documentation/virt/kvm/msr.rst 
> > > > > b/Documentation/virt/kvm/msr.rst
> > > > > index e37a14c323d2..020245d16087 100644
> > > > > --- a/Documentation/virt/kvm/msr.rst
> > > > > +++ b/Documentation/virt/kvm/msr.rst
> > > > > @@ -376,3 +376,15 @@ data:
> > > > > write '1' to bit 0 of the MSR, this causes the host to 
> > > > > re-scan its queue
> > > > > and check if there are more notifications pending. The MSR is 
> > > > > available
> > > > > if KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
> > > > > +
> > > > > +MSR_KVM_SEV_LIVE_MIGRATION:
> > > > > +0x4b564d08
> > > > > +
> > > > > +   Control SEV Live Migration features.
> > > > > +
> > > > > +data:
> > > > > +Bit 0 enables (1) or disables (0) host-side SEV Live 
> > > > > Migration feature,
> > > > > +in other words, this is guest->host communication that it's 
> > > > > properly
> > > > > +handling the shared pages list.
> > > > > +
> > > > > +All other bits are reserved.
> > > > > diff --git a/arch/x86/include/uapi/asm/kvm_para.h 
> > > > > b/arch/x86/include/uapi/asm/kvm_para.h
> > > > > index 950afebfba88..f6bfa138874f 100644
> > > > > --- a/arch/x86/include/uapi/asm/kvm_para.h
> > > > > +++ b/arch/x86

Re: [PATCH v10 12/16] KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature & Custom MSR.

[Ashish Kalra]

Hello Steve,

Continued response to your queries, especially related to userspace
control of SEV live migration feature : 

On Fri, Feb 05, 2021 at 06:54:21PM -0800, Steve Rutherford wrote:
> On Thu, Feb 4, 2021 at 7:08 PM Ashish Kalra  wrote:
> >
> > Hello Steve,
> >
> > On Thu, Feb 04, 2021 at 04:56:35PM -0800, Steve Rutherford wrote:
> > > On Wed, Feb 3, 2021 at 4:39 PM Ashish Kalra  wrote:
> > > >
> > > > From: Ashish Kalra 
> > > >
> > > > Add new KVM_FEATURE_SEV_LIVE_MIGRATION feature for guest to check
> > > > for host-side support for SEV live migration. Also add a new custom
> > > > MSR_KVM_SEV_LIVE_MIGRATION for guest to enable the SEV live migration
> > > > feature.
> > > >
> > > > Signed-off-by: Ashish Kalra 
> > > > ---
> > > >  Documentation/virt/kvm/cpuid.rst |  5 +
> > > >  Documentation/virt/kvm/msr.rst   | 12 
> > > >  arch/x86/include/uapi/asm/kvm_para.h |  4 
> > > >  arch/x86/kvm/svm/sev.c   | 13 +
> > > >  arch/x86/kvm/svm/svm.c   | 16 
> > > >  arch/x86/kvm/svm/svm.h   |  2 ++
> > > >  6 files changed, 52 insertions(+)
> > > >
> > > > diff --git a/Documentation/virt/kvm/cpuid.rst 
> > > > b/Documentation/virt/kvm/cpuid.rst
> > > > index cf62162d4be2..0bdb6cdb12d3 100644
> > > > --- a/Documentation/virt/kvm/cpuid.rst
> > > > +++ b/Documentation/virt/kvm/cpuid.rst
> > > > @@ -96,6 +96,11 @@ KVM_FEATURE_MSI_EXT_DEST_ID15  guest 
> > > > checks this feature bit
> > > > before using extended 
> > > > destination
> > > > ID bits in MSI address 
> > > > bits 11-5.
> > > >
> > > > +KVM_FEATURE_SEV_LIVE_MIGRATION 16  guest checks this 
> > > > feature bit before
> > > > +   using the page 
> > > > encryption state
> > > > +   hypercall to notify the 
> > > > page state
> > > > +   change
> > > > +
> > > >  KVM_FEATURE_CLOCKSOURCE_STABLE_BIT 24  host will warn if no 
> > > > guest-side
> > > > per-cpu warps are 
> > > > expected in
> > > > kvmclock
> > > > diff --git a/Documentation/virt/kvm/msr.rst 
> > > > b/Documentation/virt/kvm/msr.rst
> > > > index e37a14c323d2..020245d16087 100644
> > > > --- a/Documentation/virt/kvm/msr.rst
> > > > +++ b/Documentation/virt/kvm/msr.rst
> > > > @@ -376,3 +376,15 @@ data:
> > > > write '1' to bit 0 of the MSR, this causes the host to re-scan 
> > > > its queue
> > > > and check if there are more notifications pending. The MSR is 
> > > > available
> > > > if KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
> > > > +
> > > > +MSR_KVM_SEV_LIVE_MIGRATION:
> > > > +0x4b564d08
> > > > +
> > > > +   Control SEV Live Migration features.
> > > > +
> > > > +data:
> > > > +Bit 0 enables (1) or disables (0) host-side SEV Live Migration 
> > > > feature,
> > > > +in other words, this is guest->host communication that it's 
> > > > properly
> > > > +handling the shared pages list.
> > > > +
> > > > +All other bits are reserved.
> > > > diff --git a/arch/x86/include/uapi/asm/kvm_para.h 
> > > > b/arch/x86/include/uapi/asm/kvm_para.h
> > > > index 950afebfba88..f6bfa138874f 100644
> > > > --- a/arch/x86/include/uapi/asm/kvm_para.h
> > > > +++ b/arch/x86/include/uapi/asm/kvm_para.h
> > > > @@ -33,6 +33,7 @@
> > > >  #define KVM_FEATURE_PV_SCHED_YIELD 13
> > > >  #define KVM_FEATURE_ASYNC_PF_INT   14
> > > >  #define KVM_FEATURE_MSI_EXT_DEST_ID15
> > > > +#define KVM_FEATURE_SEV_LIVE_MIGRATION 16
> > > >
> > > >  #define KVM_HINTS_REALTIME  0
> > > >
> > > > @@ -54,6 +55,7 @@
> > > >  #define MSR_KVM_POLL_CONTROL   0x4b564d05
>

Re: [PATCH v10 12/16] KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature & Custom MSR.

[Ashish Kalra]

Hello Steve,

Let me first answer those queries which i can do immediately ...

On Fri, Feb 05, 2021 at 06:54:21PM -0800, Steve Rutherford wrote:
> On Thu, Feb 4, 2021 at 7:08 PM Ashish Kalra  wrote:
> >
> > Hello Steve,
> >
> > On Thu, Feb 04, 2021 at 04:56:35PM -0800, Steve Rutherford wrote:
> > > On Wed, Feb 3, 2021 at 4:39 PM Ashish Kalra  wrote:
> > > >
> > > > From: Ashish Kalra 
> > > >
> > > > Add new KVM_FEATURE_SEV_LIVE_MIGRATION feature for guest to check
> > > > for host-side support for SEV live migration. Also add a new custom
> > > > MSR_KVM_SEV_LIVE_MIGRATION for guest to enable the SEV live migration
> > > > feature.
> > > >
> > > > Signed-off-by: Ashish Kalra 
> > > > ---
> > > >  Documentation/virt/kvm/cpuid.rst |  5 +
> > > >  Documentation/virt/kvm/msr.rst   | 12 
> > > >  arch/x86/include/uapi/asm/kvm_para.h |  4 
> > > >  arch/x86/kvm/svm/sev.c   | 13 +
> > > >  arch/x86/kvm/svm/svm.c   | 16 
> > > >  arch/x86/kvm/svm/svm.h   |  2 ++
> > > >  6 files changed, 52 insertions(+)
> > > >
> > > > diff --git a/Documentation/virt/kvm/cpuid.rst 
> > > > b/Documentation/virt/kvm/cpuid.rst
> > > > index cf62162d4be2..0bdb6cdb12d3 100644
> > > > --- a/Documentation/virt/kvm/cpuid.rst
> > > > +++ b/Documentation/virt/kvm/cpuid.rst
> > > > @@ -96,6 +96,11 @@ KVM_FEATURE_MSI_EXT_DEST_ID15  guest 
> > > > checks this feature bit
> > > > before using extended 
> > > > destination
> > > > ID bits in MSI address 
> > > > bits 11-5.
> > > >
> > > > +KVM_FEATURE_SEV_LIVE_MIGRATION 16  guest checks this 
> > > > feature bit before
> > > > +   using the page 
> > > > encryption state
> > > > +   hypercall to notify the 
> > > > page state
> > > > +   change
> > > > +
> > > >  KVM_FEATURE_CLOCKSOURCE_STABLE_BIT 24  host will warn if no 
> > > > guest-side
> > > > per-cpu warps are 
> > > > expected in
> > > > kvmclock
> > > > diff --git a/Documentation/virt/kvm/msr.rst 
> > > > b/Documentation/virt/kvm/msr.rst
> > > > index e37a14c323d2..020245d16087 100644
> > > > --- a/Documentation/virt/kvm/msr.rst
> > > > +++ b/Documentation/virt/kvm/msr.rst
> > > > @@ -376,3 +376,15 @@ data:
> > > > write '1' to bit 0 of the MSR, this causes the host to re-scan 
> > > > its queue
> > > > and check if there are more notifications pending. The MSR is 
> > > > available
> > > > if KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
> > > > +
> > > > +MSR_KVM_SEV_LIVE_MIGRATION:
> > > > +0x4b564d08
> > > > +
> > > > +   Control SEV Live Migration features.
> > > > +
> > > > +data:
> > > > +Bit 0 enables (1) or disables (0) host-side SEV Live Migration 
> > > > feature,
> > > > +in other words, this is guest->host communication that it's 
> > > > properly
> > > > +handling the shared pages list.
> > > > +
> > > > +All other bits are reserved.
> > > > diff --git a/arch/x86/include/uapi/asm/kvm_para.h 
> > > > b/arch/x86/include/uapi/asm/kvm_para.h
> > > > index 950afebfba88..f6bfa138874f 100644
> > > > --- a/arch/x86/include/uapi/asm/kvm_para.h
> > > > +++ b/arch/x86/include/uapi/asm/kvm_para.h
> > > > @@ -33,6 +33,7 @@
> > > >  #define KVM_FEATURE_PV_SCHED_YIELD 13
> > > >  #define KVM_FEATURE_ASYNC_PF_INT   14
> > > >  #define KVM_FEATURE_MSI_EXT_DEST_ID15
> > > > +#define KVM_FEATURE_SEV_LIVE_MIGRATION 16
> > > >
> > > >  #define KVM_HINTS_REALTIME  0
> > > >
> > > > @@ -54,6 +55,7 @@
> > > >  #define MSR_KVM_POLL_CONTROL   0x4b564d05
> > > >  #define MSR_KVM

Re: [PATCH v10 08/16] KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall

[Ashish Kalra]

Hello Steve,

On Thu, Feb 04, 2021 at 05:44:27PM -0800, Steve Rutherford wrote:
> On Wed, Feb 3, 2021 at 4:38 PM Ashish Kalra  wrote:
> >
> > From: Brijesh Singh 
> >
> > This hypercall is used by the SEV guest to notify a change in the page
> > encryption status to the hypervisor. The hypercall should be invoked
> > only when the encryption attribute is changed from encrypted -> decrypted
> > and vice versa. By default all guest pages are considered encrypted.
> >
> > The patch introduces a new shared pages list implemented as a
> > sorted linked list to track the shared/unencrypted regions marked by the
> > guest hypercall.
> >
> > Cc: Thomas Gleixner 
> > Cc: Ingo Molnar 
> > Cc: "H. Peter Anvin" 
> > Cc: Paolo Bonzini 
> > Cc: "Radim Krčmář" 
> > Cc: Joerg Roedel 
> > Cc: Borislav Petkov 
> > Cc: Tom Lendacky 
> > Cc: x...@kernel.org
> > Cc: k...@vger.kernel.org
> > Cc: linux-kernel@vger.kernel.org
> > Signed-off-by: Brijesh Singh 
> > Co-developed-by: Ashish Kalra 
> > Signed-off-by: Ashish Kalra 
> > ---
> >  Documentation/virt/kvm/hypercalls.rst |  15 +++
> >  arch/x86/include/asm/kvm_host.h   |   2 +
> >  arch/x86/kvm/svm/sev.c| 150 ++
> >  arch/x86/kvm/svm/svm.c|   2 +
> >  arch/x86/kvm/svm/svm.h|   5 +
> >  arch/x86/kvm/vmx/vmx.c|   1 +
> >  arch/x86/kvm/x86.c|   6 ++
> >  include/uapi/linux/kvm_para.h |   1 +
> >  8 files changed, 182 insertions(+)
> >
> > diff --git a/Documentation/virt/kvm/hypercalls.rst 
> > b/Documentation/virt/kvm/hypercalls.rst
> > index ed4fddd364ea..7aff0cebab7c 100644
> > --- a/Documentation/virt/kvm/hypercalls.rst
> > +++ b/Documentation/virt/kvm/hypercalls.rst
> > @@ -169,3 +169,18 @@ a0: destination APIC ID
> >
> >  :Usage example: When sending a call-function IPI-many to vCPUs, yield if
> > any of the IPI target vCPUs was preempted.
> > +
> > +
> > +8. KVM_HC_PAGE_ENC_STATUS
> > +-
> > +:Architecture: x86
> > +:Status: active
> > +:Purpose: Notify the encryption status changes in guest page table (SEV 
> > guest)
> > +
> > +a0: the guest physical address of the start page
> > +a1: the number of pages
> > +a2: encryption attribute
> > +
> > +   Where:
> > +   * 1: Encryption attribute is set
> > +   * 0: Encryption attribute is cleared
> > diff --git a/arch/x86/include/asm/kvm_host.h 
> > b/arch/x86/include/asm/kvm_host.h
> > index 3d6616f6f6ef..2da5f5e2a10e 100644
> > --- a/arch/x86/include/asm/kvm_host.h
> > +++ b/arch/x86/include/asm/kvm_host.h
> > @@ -1301,6 +1301,8 @@ struct kvm_x86_ops {
> > int (*complete_emulated_msr)(struct kvm_vcpu *vcpu, int err);
> >
> > void (*vcpu_deliver_sipi_vector)(struct kvm_vcpu *vcpu, u8 vector);
> > +   int (*page_enc_status_hc)(struct kvm *kvm, unsigned long gpa,
> > + unsigned long sz, unsigned long mode);
> >  };
> >
> >  struct kvm_x86_nested_ops {
> > diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
> > index 25eaf35ba51d..55c628df5155 100644
> > --- a/arch/x86/kvm/svm/sev.c
> > +++ b/arch/x86/kvm/svm/sev.c
> > @@ -45,6 +45,11 @@ struct enc_region {
> > unsigned long size;
> >  };
> >
> > +struct shared_region {
> > +   struct list_head list;
> > +   unsigned long gfn_start, gfn_end;
> > +};
> > +
> >  static int sev_flush_asids(void)
> >  {
> > int ret, error = 0;
> > @@ -196,6 +201,8 @@ static int sev_guest_init(struct kvm *kvm, struct 
> > kvm_sev_cmd *argp)
> > sev->active = true;
> > sev->asid = asid;
> > INIT_LIST_HEAD(>regions_list);
> > +   INIT_LIST_HEAD(>shared_pages_list);
> > +   sev->shared_pages_list_count = 0;
> >
> > return 0;
> >
> > @@ -1473,6 +1480,148 @@ static int sev_receive_finish(struct kvm *kvm, 
> > struct kvm_sev_cmd *argp)
> > return ret;
> >  }
> >
> > +static int remove_shared_region(unsigned long start, unsigned long end,
> > +   struct list_head *head)
> > +{
> > +   struct shared_region *pos;
> > +
> > +   list_for_each_entry(pos, head, list) {
> > +   if (pos->gfn_start == start &&
&g

Re: [PATCH v10 12/16] KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature & Custom MSR.

[Ashish Kalra]

Hello Steve,

On Thu, Feb 04, 2021 at 04:56:35PM -0800, Steve Rutherford wrote:
> On Wed, Feb 3, 2021 at 4:39 PM Ashish Kalra  wrote:
> >
> > From: Ashish Kalra 
> >
> > Add new KVM_FEATURE_SEV_LIVE_MIGRATION feature for guest to check
> > for host-side support for SEV live migration. Also add a new custom
> > MSR_KVM_SEV_LIVE_MIGRATION for guest to enable the SEV live migration
> > feature.
> >
> > Signed-off-by: Ashish Kalra 
> > ---
> >  Documentation/virt/kvm/cpuid.rst |  5 +
> >  Documentation/virt/kvm/msr.rst   | 12 
> >  arch/x86/include/uapi/asm/kvm_para.h |  4 
> >  arch/x86/kvm/svm/sev.c   | 13 +
> >  arch/x86/kvm/svm/svm.c   | 16 
> >  arch/x86/kvm/svm/svm.h   |  2 ++
> >  6 files changed, 52 insertions(+)
> >
> > diff --git a/Documentation/virt/kvm/cpuid.rst 
> > b/Documentation/virt/kvm/cpuid.rst
> > index cf62162d4be2..0bdb6cdb12d3 100644
> > --- a/Documentation/virt/kvm/cpuid.rst
> > +++ b/Documentation/virt/kvm/cpuid.rst
> > @@ -96,6 +96,11 @@ KVM_FEATURE_MSI_EXT_DEST_ID15  guest 
> > checks this feature bit
> > before using extended 
> > destination
> > ID bits in MSI address bits 
> > 11-5.
> >
> > +KVM_FEATURE_SEV_LIVE_MIGRATION 16  guest checks this feature 
> > bit before
> > +   using the page encryption 
> > state
> > +   hypercall to notify the 
> > page state
> > +   change
> > +
> >  KVM_FEATURE_CLOCKSOURCE_STABLE_BIT 24  host will warn if no 
> > guest-side
> > per-cpu warps are expected 
> > in
> > kvmclock
> > diff --git a/Documentation/virt/kvm/msr.rst b/Documentation/virt/kvm/msr.rst
> > index e37a14c323d2..020245d16087 100644
> > --- a/Documentation/virt/kvm/msr.rst
> > +++ b/Documentation/virt/kvm/msr.rst
> > @@ -376,3 +376,15 @@ data:
> > write '1' to bit 0 of the MSR, this causes the host to re-scan its 
> > queue
> > and check if there are more notifications pending. The MSR is 
> > available
> > if KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
> > +
> > +MSR_KVM_SEV_LIVE_MIGRATION:
> > +0x4b564d08
> > +
> > +   Control SEV Live Migration features.
> > +
> > +data:
> > +Bit 0 enables (1) or disables (0) host-side SEV Live Migration 
> > feature,
> > +in other words, this is guest->host communication that it's 
> > properly
> > +handling the shared pages list.
> > +
> > +All other bits are reserved.
> > diff --git a/arch/x86/include/uapi/asm/kvm_para.h 
> > b/arch/x86/include/uapi/asm/kvm_para.h
> > index 950afebfba88..f6bfa138874f 100644
> > --- a/arch/x86/include/uapi/asm/kvm_para.h
> > +++ b/arch/x86/include/uapi/asm/kvm_para.h
> > @@ -33,6 +33,7 @@
> >  #define KVM_FEATURE_PV_SCHED_YIELD 13
> >  #define KVM_FEATURE_ASYNC_PF_INT   14
> >  #define KVM_FEATURE_MSI_EXT_DEST_ID15
> > +#define KVM_FEATURE_SEV_LIVE_MIGRATION 16
> >
> >  #define KVM_HINTS_REALTIME  0
> >
> > @@ -54,6 +55,7 @@
> >  #define MSR_KVM_POLL_CONTROL   0x4b564d05
> >  #define MSR_KVM_ASYNC_PF_INT   0x4b564d06
> >  #define MSR_KVM_ASYNC_PF_ACK   0x4b564d07
> > +#define MSR_KVM_SEV_LIVE_MIGRATION 0x4b564d08
> >
> >  struct kvm_steal_time {
> > __u64 steal;
> > @@ -136,4 +138,6 @@ struct kvm_vcpu_pv_apf_data {
> >  #define KVM_PV_EOI_ENABLED KVM_PV_EOI_MASK
> >  #define KVM_PV_EOI_DISABLED 0x0
> >
> > +#define KVM_SEV_LIVE_MIGRATION_ENABLED BIT_ULL(0)
> > +
> >  #endif /* _UAPI_ASM_X86_KVM_PARA_H */
> > diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
> > index b0d324aed515..93f42b3d3e33 100644
> > --- a/arch/x86/kvm/svm/sev.c
> > +++ b/arch/x86/kvm/svm/sev.c
> > @@ -1627,6 +1627,16 @@ int svm_page_enc_status_hc(struct kvm *kvm, unsigned 
> > long gpa,
> > return ret;
> >  }
> >
> > +void sev_update_migration_flags(struct kvm *kvm, u64 data)
> > +{
> > +   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
> > +
> > +   if (!sev_guest(kvm))
> > +   return;
> 
> 

Re: [PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

Hello Tom,

On Thu, Feb 04, 2021 at 10:14:37AM -0600, Tom Lendacky wrote:
> On 2/3/21 6:39 PM, Ashish Kalra wrote:
> > From: Brijesh Singh 
> > 
> > The ioctl is used to retrieve a guest's shared pages list.
> > 
> 
> ...
> 
> > +int svm_get_shared_pages_list(struct kvm *kvm,
> > + struct kvm_shared_pages_list *list)
> > +{
> > +   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
> > +   struct shared_region_array_entry *array;
> > +   struct shared_region *pos;
> > +   int ret, nents = 0;
> > +   unsigned long sz;
> > +
> > +   if (!sev_guest(kvm))
> > +   return -ENOTTY;
> > +
> > +   if (!list->size)
> > +   return -EINVAL;
> > +
> > +   if (!sev->shared_pages_list_count)
> > +   return put_user(0, list->pnents);
> > +
> > +   sz = sev->shared_pages_list_count * sizeof(struct 
> > shared_region_array_entry);
> > +   if (sz > list->size)
> > +   return -E2BIG;
> > +
> > +   array = kmalloc(sz, GFP_KERNEL);
> > +   if (!array)
> > +   return -ENOMEM;
> > +
> > +   mutex_lock(>lock);
> 
> I think this lock needs to be taken before the memory size is calculated. If
> the list is expanded after obtaining the size and before taking the lock,
> you will run off the end of the array.
> 

Yes, as the page encryption status hcalls can happen simultaneously to
this ioctl, therefore this is an issue, so i will take the lock before
the memory size is calculated.

Thanks,
Ashish

> 
> > +   list_for_each_entry(pos, >shared_pages_list, list) {
> > +   array[nents].gfn_start = pos->gfn_start;
> > +   array[nents++].gfn_end = pos->gfn_end;
> > +   }
> > +   mutex_unlock(>lock);
> > +
> > +   ret = -EFAULT;
> > +   if (copy_to_user(list->buffer, array, sz))
> > +   goto out;
> > +   if (put_user(nents, list->pnents))
> > +   goto out;
> > +   ret = 0;
> > +out:
> > +   kfree(array);
> > +   return ret;
> > +}
> > +

[PATCH v10 12/16] KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature & Custom MSR.

[Ashish Kalra]

From: Ashish Kalra 

Add new KVM_FEATURE_SEV_LIVE_MIGRATION feature for guest to check
for host-side support for SEV live migration. Also add a new custom
MSR_KVM_SEV_LIVE_MIGRATION for guest to enable the SEV live migration
feature.

Signed-off-by: Ashish Kalra 
---
 Documentation/virt/kvm/cpuid.rst |  5 +
 Documentation/virt/kvm/msr.rst   | 12 
 arch/x86/include/uapi/asm/kvm_para.h |  4 
 arch/x86/kvm/svm/sev.c   | 13 +
 arch/x86/kvm/svm/svm.c   | 16 
 arch/x86/kvm/svm/svm.h   |  2 ++
 6 files changed, 52 insertions(+)

diff --git a/Documentation/virt/kvm/cpuid.rst b/Documentation/virt/kvm/cpuid.rst
index cf62162d4be2..0bdb6cdb12d3 100644
--- a/Documentation/virt/kvm/cpuid.rst
+++ b/Documentation/virt/kvm/cpuid.rst
@@ -96,6 +96,11 @@ KVM_FEATURE_MSI_EXT_DEST_ID15  guest checks 
this feature bit
before using extended 
destination
ID bits in MSI address bits 
11-5.
 
+KVM_FEATURE_SEV_LIVE_MIGRATION 16  guest checks this feature bit 
before
+   using the page encryption state
+   hypercall to notify the page 
state
+   change
+
 KVM_FEATURE_CLOCKSOURCE_STABLE_BIT 24  host will warn if no guest-side
per-cpu warps are expected in
kvmclock
diff --git a/Documentation/virt/kvm/msr.rst b/Documentation/virt/kvm/msr.rst
index e37a14c323d2..020245d16087 100644
--- a/Documentation/virt/kvm/msr.rst
+++ b/Documentation/virt/kvm/msr.rst
@@ -376,3 +376,15 @@ data:
write '1' to bit 0 of the MSR, this causes the host to re-scan its queue
and check if there are more notifications pending. The MSR is available
if KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
+
+MSR_KVM_SEV_LIVE_MIGRATION:
+0x4b564d08
+
+   Control SEV Live Migration features.
+
+data:
+Bit 0 enables (1) or disables (0) host-side SEV Live Migration feature,
+in other words, this is guest->host communication that it's properly
+handling the shared pages list.
+
+All other bits are reserved.
diff --git a/arch/x86/include/uapi/asm/kvm_para.h 
b/arch/x86/include/uapi/asm/kvm_para.h
index 950afebfba88..f6bfa138874f 100644
--- a/arch/x86/include/uapi/asm/kvm_para.h
+++ b/arch/x86/include/uapi/asm/kvm_para.h
@@ -33,6 +33,7 @@
 #define KVM_FEATURE_PV_SCHED_YIELD 13
 #define KVM_FEATURE_ASYNC_PF_INT   14
 #define KVM_FEATURE_MSI_EXT_DEST_ID15
+#define KVM_FEATURE_SEV_LIVE_MIGRATION 16
 
 #define KVM_HINTS_REALTIME  0
 
@@ -54,6 +55,7 @@
 #define MSR_KVM_POLL_CONTROL   0x4b564d05
 #define MSR_KVM_ASYNC_PF_INT   0x4b564d06
 #define MSR_KVM_ASYNC_PF_ACK   0x4b564d07
+#define MSR_KVM_SEV_LIVE_MIGRATION 0x4b564d08
 
 struct kvm_steal_time {
__u64 steal;
@@ -136,4 +138,6 @@ struct kvm_vcpu_pv_apf_data {
 #define KVM_PV_EOI_ENABLED KVM_PV_EOI_MASK
 #define KVM_PV_EOI_DISABLED 0x0
 
+#define KVM_SEV_LIVE_MIGRATION_ENABLED BIT_ULL(0)
+
 #endif /* _UAPI_ASM_X86_KVM_PARA_H */
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index b0d324aed515..93f42b3d3e33 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1627,6 +1627,16 @@ int svm_page_enc_status_hc(struct kvm *kvm, unsigned 
long gpa,
return ret;
 }
 
+void sev_update_migration_flags(struct kvm *kvm, u64 data)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+
+   if (!sev_guest(kvm))
+   return;
+
+   sev->live_migration_enabled = !!(data & KVM_SEV_LIVE_MIGRATION_ENABLED);
+}
+
 int svm_get_shared_pages_list(struct kvm *kvm,
  struct kvm_shared_pages_list *list)
 {
@@ -1639,6 +1649,9 @@ int svm_get_shared_pages_list(struct kvm *kvm,
if (!sev_guest(kvm))
return -ENOTTY;
 
+   if (!sev->live_migration_enabled)
+   return -EINVAL;
+
if (!list->size)
return -EINVAL;
 
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 58f89f83caab..43ea5061926f 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -2903,6 +2903,9 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct 
msr_data *msr)
svm->msr_decfg = data;
break;
}
+   case MSR_KVM_SEV_LIVE_MIGRATION:
+   sev_update_migration_flags(vcpu->kvm, data);
+   break;
case MSR_IA32_APICBASE:
if (kvm_vcpu_apicv_active(vcpu))
avic_update_vapic_bar(to_svm(vcpu), data);
@@ -3976,6 +3979,19 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu 
*vcpu)
vcpu->arch.cr3_lm_rsvd_bits &a

[PATCH v10 09/16] mm: x86: Invoke hypercall when page encryption status is changed

[Ashish Kalra]

From: Brijesh Singh 

Invoke a hypercall when a memory region is changed from encrypted ->
decrypted and vice versa. Hypervisor needs to know the page encryption
status during the guest migration.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Venu Busireddy 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 arch/x86/include/asm/paravirt.h   | 10 +
 arch/x86/include/asm/paravirt_types.h |  2 +
 arch/x86/kernel/paravirt.c|  1 +
 arch/x86/mm/mem_encrypt.c | 57 ++-
 arch/x86/mm/pat/set_memory.c  |  7 
 5 files changed, 76 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index f8dce11d2bc1..1265e1f5db5f 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -84,6 +84,12 @@ static inline void paravirt_arch_exit_mmap(struct mm_struct 
*mm)
PVOP_VCALL1(mmu.exit_mmap, mm);
 }
 
+static inline void page_encryption_changed(unsigned long vaddr, int npages,
+   bool enc)
+{
+   PVOP_VCALL3(mmu.page_encryption_changed, vaddr, npages, enc);
+}
+
 #ifdef CONFIG_PARAVIRT_XXL
 static inline void load_sp0(unsigned long sp0)
 {
@@ -829,6 +835,10 @@ static inline void paravirt_arch_dup_mmap(struct mm_struct 
*oldmm,
 static inline void paravirt_arch_exit_mmap(struct mm_struct *mm)
 {
 }
+
+static inline void page_encryption_changed(unsigned long vaddr, int npages, 
bool enc)
+{
+}
 #endif
 #endif /* __ASSEMBLY__ */
 #endif /* _ASM_X86_PARAVIRT_H */
diff --git a/arch/x86/include/asm/paravirt_types.h 
b/arch/x86/include/asm/paravirt_types.h
index b6b02b7c19cc..6a83821cf758 100644
--- a/arch/x86/include/asm/paravirt_types.h
+++ b/arch/x86/include/asm/paravirt_types.h
@@ -208,6 +208,8 @@ struct pv_mmu_ops {
 
/* Hook for intercepting the destruction of an mm_struct. */
void (*exit_mmap)(struct mm_struct *mm);
+   void (*page_encryption_changed)(unsigned long vaddr, int npages,
+   bool enc);
 
 #ifdef CONFIG_PARAVIRT_XXL
struct paravirt_callee_save read_cr2;
diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index 6c3407ba6ee9..52913356b6fa 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -340,6 +340,7 @@ struct paravirt_patch_template pv_ops = {
(void (*)(struct mmu_gather *, void *))tlb_remove_page,
 
.mmu.exit_mmap  = paravirt_nop,
+   .mmu.page_encryption_changed= paravirt_nop,
 
 #ifdef CONFIG_PARAVIRT_XXL
.mmu.read_cr2   = __PV_IS_CALLEE_SAVE(native_read_cr2),
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index c79e5736ab2b..dc17d14f9bcd 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -19,6 +19,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -29,6 +30,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "mm_internal.h"
 
@@ -229,6 +231,47 @@ void __init sev_setup_arch(void)
swiotlb_adjust_size(size);
 }
 
+static void set_memory_enc_dec_hypercall(unsigned long vaddr, int npages,
+   bool enc)
+{
+   unsigned long sz = npages << PAGE_SHIFT;
+   unsigned long vaddr_end, vaddr_next;
+
+   vaddr_end = vaddr + sz;
+
+   for (; vaddr < vaddr_end; vaddr = vaddr_next) {
+   int psize, pmask, level;
+   unsigned long pfn;
+   pte_t *kpte;
+
+   kpte = lookup_address(vaddr, );
+   if (!kpte || pte_none(*kpte))
+   return;
+
+   switch (level) {
+   case PG_LEVEL_4K:
+   pfn = pte_pfn(*kpte);
+   break;
+   case PG_LEVEL_2M:
+   pfn = pmd_pfn(*(pmd_t *)kpte);
+   break;
+   case PG_LEVEL_1G:
+   pfn = pud_pfn(*(pud_t *)kpte);
+   break;
+   default:
+   return;
+   }
+
+   psize = page_level_size(level);
+   pmask = page_level_mask(level);
+
+   kvm_sev_hypercall3(KVM_HC_PAGE_ENC_STATUS,
+  pfn << PAGE_SHIFT, psize >> PAGE_SHIFT, enc);
+
+   vaddr_next = (vaddr & pmask) + psize;
+   }
+}
+
 static void __init __set_clr_pte_enc(pte_t *kpte, int level, bool enc)
 {
pgprot_t old_prot, new_prot;
@@ -286,12 +329,13 @@ static void __init __set_clr_pte_enc(pte_t *kpte, int 
level, bool enc)
 static int __init early_set_memory_enc_dec(unsigned long vaddr,
 

[PATCH v10 16/16] KVM: SVM: Bypass DBG_DECRYPT API calls for unencrypted guest memory.

[Ashish Kalra]

From: Ashish Kalra 

For all unencrypted guest memory regions such as S/W IOTLB
bounce buffers and for guest regions marked as "__bss_decrypted",
ensure that DBG_DECRYPT API calls are bypassed.
The guest memory regions encryption status is referenced using the
shared pages list.

Signed-off-by: Ashish Kalra 
---
 arch/x86/kvm/svm/sev.c | 126 +
 1 file changed, 126 insertions(+)

diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 93f42b3d3e33..fa3fbbb73b33 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -888,6 +888,117 @@ static int __sev_dbg_encrypt_user(struct kvm *kvm, 
unsigned long paddr,
return ret;
 }
 
+static struct kvm_memory_slot *hva_to_memslot(struct kvm *kvm,
+ unsigned long hva)
+{
+   struct kvm_memslots *slots = kvm_memslots(kvm);
+   struct kvm_memory_slot *memslot;
+
+   kvm_for_each_memslot(memslot, slots) {
+   if (hva >= memslot->userspace_addr &&
+   hva < memslot->userspace_addr +
+ (memslot->npages << PAGE_SHIFT))
+   return memslot;
+   }
+
+   return NULL;
+}
+
+static bool hva_to_gfn(struct kvm *kvm, unsigned long hva, gfn_t *gfn)
+{
+   struct kvm_memory_slot *memslot;
+   gpa_t gpa_offset;
+
+   memslot = hva_to_memslot(kvm, hva);
+   if (!memslot)
+   return false;
+
+   gpa_offset = hva - memslot->userspace_addr;
+   *gfn = ((memslot->base_gfn << PAGE_SHIFT) + gpa_offset) >> PAGE_SHIFT;
+
+   return true;
+}
+
+static bool is_unencrypted_region(gfn_t gfn_start, gfn_t gfn_end,
+ struct list_head *head)
+{
+   struct shared_region *pos;
+
+   list_for_each_entry(pos, head, list)
+   if (gfn_start >= pos->gfn_start &&
+   gfn_end <= pos->gfn_end)
+   return true;
+
+   return false;
+}
+
+static int handle_unencrypted_region(struct kvm *kvm,
+unsigned long vaddr,
+unsigned long vaddr_end,
+unsigned long dst_vaddr,
+unsigned int size,
+bool *is_decrypted)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct page *page = NULL;
+   gfn_t gfn_start, gfn_end;
+   int len, s_off, d_off;
+   int srcu_idx;
+   int ret = 0;
+
+   /* ensure hva_to_gfn translations remain valid */
+   srcu_idx = srcu_read_lock(>srcu);
+
+   if (!hva_to_gfn(kvm, vaddr, _start)) {
+   srcu_read_unlock(>srcu, srcu_idx);
+   return -EINVAL;
+   }
+
+   if (!hva_to_gfn(kvm, vaddr_end, _end)) {
+   srcu_read_unlock(>srcu, srcu_idx);
+   return -EINVAL;
+   }
+
+   if (sev->shared_pages_list_count) {
+   if (is_unencrypted_region(gfn_start, gfn_end,
+ >shared_pages_list)) {
+   page = alloc_page(GFP_KERNEL);
+   if (!page) {
+   srcu_read_unlock(>srcu, srcu_idx);
+   return -ENOMEM;
+   }
+
+   /*
+* Since user buffer may not be page aligned, calculate 
the
+* offset within the page.
+*/
+   s_off = vaddr & ~PAGE_MASK;
+   d_off = dst_vaddr & ~PAGE_MASK;
+   len = min_t(size_t, (PAGE_SIZE - s_off), size);
+
+   if (copy_from_user(page_address(page),
+  (void __user *)(uintptr_t)vaddr, 
len)) {
+   __free_page(page);
+   srcu_read_unlock(>srcu, srcu_idx);
+   return -EFAULT;
+   }
+
+   if (copy_to_user((void __user *)(uintptr_t)dst_vaddr,
+page_address(page), len)) {
+   ret = -EFAULT;
+   }
+
+   __free_page(page);
+   srcu_read_unlock(>srcu, srcu_idx);
+   *is_decrypted = true;
+   return ret;
+   }
+   }
+   srcu_read_unlock(>srcu, srcu_idx);
+   *is_decrypted = false;
+   return ret;
+}
+
 static int sev_dbg_crypt(struct kvm *kvm, struct kvm_sev_cmd *argp, bool dec)
 {
unsigned long vaddr, vaddr_end, next_vaddr;
@@ -917,6 +1028,20 @@ static int sev_dbg_crypt(struct kvm *kvm, struct 
kvm_sev_cmd *argp, bool dec)
for (; vaddr < vaddr_end; 

[PATCH v10 15/16] KVM: x86: Add kexec support for SEV Live Migration.

[Ashish Kalra]

From: Ashish Kalra 

Reset the host's shared pages list related to kernel
specific page encryption status settings before we load a
new kernel by kexec. We cannot reset the complete
shared pages list here as we need to retain the
UEFI/OVMF firmware specific settings.

The host's shared pages list is maintained for the
guest to keep track of all unencrypted guest memory regions,
therefore we need to explicitly mark all shared pages as
encrypted again before rebooting into the new guest kernel.

Signed-off-by: Ashish Kalra 
---
 arch/x86/kernel/kvm.c | 28 
 1 file changed, 28 insertions(+)

diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index c4b8029c1442..d61156db7797 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -39,6 +39,7 @@
 #include 
 #include 
 #include 
+#include 
 
 DEFINE_STATIC_KEY_FALSE(kvm_async_pf_enabled);
 
@@ -384,6 +385,33 @@ static void kvm_pv_guest_cpu_reboot(void *unused)
 */
if (kvm_para_has_feature(KVM_FEATURE_PV_EOI))
wrmsrl(MSR_KVM_PV_EOI_EN, 0);
+   /*
+* Reset the host's shared pages list related to kernel
+* specific page encryption status settings before we load a
+* new kernel by kexec. NOTE: We cannot reset the complete
+* shared pages list here as we need to retain the
+* UEFI/OVMF firmware specific settings.
+*/
+   if (sev_live_migration_enabled & (smp_processor_id() == 0)) {
+   int i;
+   unsigned long nr_pages;
+
+   for (i = 0; i < e820_table->nr_entries; i++) {
+   struct e820_entry *entry = _table->entries[i];
+   unsigned long start_pfn;
+   unsigned long end_pfn;
+
+   if (entry->type != E820_TYPE_RAM)
+   continue;
+
+   start_pfn = entry->addr >> PAGE_SHIFT;
+   end_pfn = (entry->addr + entry->size) >> PAGE_SHIFT;
+   nr_pages = DIV_ROUND_UP(entry->size, PAGE_SIZE);
+
+   kvm_sev_hypercall3(KVM_HC_PAGE_ENC_STATUS,
+  entry->addr, nr_pages, 1);
+   }
+   }
kvm_pv_disable_apf();
kvm_disable_steal_time();
 }
-- 
2.17.1

[PATCH v10 14/16] KVM: x86: Add guest support for detecting and enabling SEV Live Migration feature.

[Ashish Kalra]

From: Ashish Kalra 

The guest support for detecting and enabling SEV Live migration
feature uses the following logic :

 - kvm_init_plaform() invokes check_kvm_sev_migration() which
   checks if its booted under the EFI

   - If not EFI,

 i) check for the KVM_FEATURE_CPUID

 ii) if CPUID reports that migration is supported, issue a wrmsrl()
 to enable the SEV live migration support

   - If EFI,

 i) check for the KVM_FEATURE_CPUID

 ii) If CPUID reports that migration is supported, read the UEFI variable 
which
 indicates OVMF support for live migration

 iii) the variable indicates live migration is supported, issue a wrmsrl() 
to
  enable the SEV live migration support

The EFI live migration check is done using a late_initcall() callback.

Also, ensure that _bss_decrypted section is marked as decrypted in the
shared pages list.

Signed-off-by: Ashish Kalra 
---
 arch/x86/include/asm/mem_encrypt.h |  8 +
 arch/x86/kernel/kvm.c  | 52 ++
 arch/x86/mm/mem_encrypt.c  | 41 +++
 3 files changed, 101 insertions(+)

diff --git a/arch/x86/include/asm/mem_encrypt.h 
b/arch/x86/include/asm/mem_encrypt.h
index 31c4df123aa0..19b77f3a62dc 100644
--- a/arch/x86/include/asm/mem_encrypt.h
+++ b/arch/x86/include/asm/mem_encrypt.h
@@ -21,6 +21,7 @@
 extern u64 sme_me_mask;
 extern u64 sev_status;
 extern bool sev_enabled;
+extern bool sev_live_migration_enabled;
 
 void sme_encrypt_execute(unsigned long encrypted_kernel_vaddr,
 unsigned long decrypted_kernel_vaddr,
@@ -44,8 +45,11 @@ void __init sme_enable(struct boot_params *bp);
 
 int __init early_set_memory_decrypted(unsigned long vaddr, unsigned long size);
 int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size);
+void __init early_set_mem_enc_dec_hypercall(unsigned long vaddr, int npages,
+   bool enc);
 
 void __init mem_encrypt_free_decrypted_mem(void);
+void __init check_kvm_sev_migration(void);
 
 /* Architecture __weak replacement functions */
 void __init mem_encrypt_init(void);
@@ -60,6 +64,7 @@ bool sev_es_active(void);
 #else  /* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define sme_me_mask0ULL
+#define sev_live_migration_enabled false
 
 static inline void __init sme_early_encrypt(resource_size_t paddr,
unsigned long size) { }
@@ -84,8 +89,11 @@ static inline int __init
 early_set_memory_decrypted(unsigned long vaddr, unsigned long size) { return 
0; }
 static inline int __init
 early_set_memory_encrypted(unsigned long vaddr, unsigned long size) { return 
0; }
+static inline void __init
+early_set_mem_enc_dec_hypercall(unsigned long vaddr, int npages, bool enc) {}
 
 static inline void mem_encrypt_free_decrypted_mem(void) { }
+static inline void check_kvm_sev_migration(void) { }
 
 #define __bss_decrypted
 
diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index 5e78e01ca3b4..c4b8029c1442 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -26,6 +26,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -429,6 +430,56 @@ static inline void __set_percpu_decrypted(void *ptr, 
unsigned long size)
early_set_memory_decrypted((unsigned long) ptr, size);
 }
 
+static int __init setup_kvm_sev_migration(void)
+{
+   efi_char16_t efi_sev_live_migration_enabled[] = 
L"SevLiveMigrationEnabled";
+   efi_guid_t efi_variable_guid = MEM_ENCRYPT_GUID;
+   efi_status_t status;
+   unsigned long size;
+   bool enabled;
+
+   /*
+* check_kvm_sev_migration() invoked via kvm_init_platform() before
+* this callback would have setup the indicator that live migration
+* feature is supported/enabled.
+*/
+   if (!sev_live_migration_enabled)
+   return 0;
+
+   if (!efi_enabled(EFI_RUNTIME_SERVICES)) {
+   pr_info("%s : EFI runtime services are not enabled\n", 
__func__);
+   return 0;
+   }
+
+   size = sizeof(enabled);
+
+   /* Get variable contents into buffer */
+   status = efi.get_variable(efi_sev_live_migration_enabled,
+ _variable_guid, NULL, , );
+
+   if (status == EFI_NOT_FOUND) {
+   pr_info("%s : EFI live migration variable not found\n", 
__func__);
+   return 0;
+   }
+
+   if (status != EFI_SUCCESS) {
+   pr_info("%s : EFI variable retrieval failed\n", __func__);
+   return 0;
+   }
+
+   if (enabled == 0) {
+   pr_info("%s: live migration disabled in EFI\n", __func__);
+   return 0;
+   }
+
+   pr_info("%s : live migration enabled in EFI\n", __func__);
+   wrmsrl(MSR_KVM_SEV_LIVE_MIGRATION, KVM_SEV_LIVE_MIGRATION_ENABLED);
+
+   return true;
+}
+
+late_

[PATCH v10 13/16] EFI: Introduce the new AMD Memory Encryption GUID.

[Ashish Kalra]

From: Ashish Kalra 

Introduce a new AMD Memory Encryption GUID which is currently
used for defining a new UEFI environment variable which indicates
UEFI/OVMF support for the SEV live migration feature. This variable
is setup when UEFI/OVMF detects host/hypervisor support for SEV
live migration and later this variable is read by the kernel using
EFI runtime services to verify if OVMF supports the live migration
feature.

Signed-off-by: Ashish Kalra 
---
 include/linux/efi.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/linux/efi.h b/include/linux/efi.h
index 763b816ba19c..ae47db882bee 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -362,6 +362,7 @@ void efi_native_runtime_setup(void);
 
 /* OEM GUIDs */
 #define DELLEMC_EFI_RCI2_TABLE_GUIDEFI_GUID(0x2d9f28a2, 0xa886, 
0x456a,  0x97, 0xa8, 0xf1, 0x1e, 0xf2, 0x4f, 0xf4, 0x55)
+#define MEM_ENCRYPT_GUID   EFI_GUID(0x0cf29b71, 0x9e51, 
0x433a,  0xa3, 0xb7, 0x81, 0xf3, 0xab, 0x16, 0xb8, 0x75)
 
 typedef struct {
efi_guid_t guid;
-- 
2.17.1

[PATCH v10 11/16] KVM: x86: Introduce KVM_SET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

From: Brijesh Singh 

The ioctl is used to setup the shared pages list for an
incoming guest.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Brijesh Singh 
Co-developed-by: Ashish Kalra 
Signed-off-by: Ashish Kalra 
---
 Documentation/virt/kvm/api.rst  | 20 +-
 arch/x86/include/asm/kvm_host.h |  2 +
 arch/x86/kvm/svm/sev.c  | 70 +
 arch/x86/kvm/svm/svm.c  |  1 +
 arch/x86/kvm/svm/svm.h  |  1 +
 arch/x86/kvm/x86.c  | 12 ++
 include/uapi/linux/kvm.h|  1 +
 7 files changed, 106 insertions(+), 1 deletion(-)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 59ef537c0cdd..efb4720733b4 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -4701,6 +4701,25 @@ This list can be used during the guest migration. If the 
page
 is private then the userspace need to use SEV migration commands to transmit
 the page.
 
+4.126 KVM_SET_SHARED_PAGES_LIST (vm ioctl)
+---
+
+:Capability: basic
+:Architectures: x86
+:Type: vm ioctl
+:Parameters: struct kvm_shared_pages_list (in/out)
+:Returns: 0 on success, -1 on error
+
+/* for KVM_SET_SHARED_PAGES_LIST */
+struct kvm_shared_pages_list {
+   int __user *pnents;
+   void __user *buffer;
+   __u32 size;
+};
+
+During the guest live migration the outgoing guest exports its unencrypted
+memory regions list, the KVM_SET_SHARED_PAGES_LIST can be used to build the
+shared/unencrypted regions list for an incoming guest.
 
 4.125 KVM_S390_PV_COMMAND
 -
@@ -4855,7 +4874,6 @@ into user space.
 If a vCPU is in running state while this ioctl is invoked, the vCPU may
 experience inconsistent filtering behavior on MSR accesses.
 
-
 5. The kvm_run structure
 
 
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index cd354d830e13..f05b812b69bd 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1305,6 +1305,8 @@ struct kvm_x86_ops {
  unsigned long sz, unsigned long mode);
int (*get_shared_pages_list)(struct kvm *kvm,
 struct kvm_shared_pages_list *list);
+   int (*set_shared_pages_list)(struct kvm *kvm,
+struct kvm_shared_pages_list *list);
 };
 
 struct kvm_x86_nested_ops {
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 701d74c8b15b..b0d324aed515 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1671,6 +1671,76 @@ int svm_get_shared_pages_list(struct kvm *kvm,
return ret;
 }
 
+int svm_set_shared_pages_list(struct kvm *kvm,
+ struct kvm_shared_pages_list *list)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct shared_region_array_entry *array;
+   struct shared_region *shrd_region;
+   int ret, nents, i;
+   unsigned long sz;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   if (get_user(nents, list->pnents))
+   return -EFAULT;
+
+   /* special case of resetting the shared pages list */
+   if (!list->buffer || !nents) {
+   struct shared_region *pos;
+
+   mutex_lock(>lock);
+   list_for_each_entry(pos, >shared_pages_list, list)
+   kfree(pos);
+   sev->shared_pages_list_count = 0;
+   mutex_unlock(>lock);
+
+   return 0;
+   }
+
+   sz = nents * sizeof(struct shared_region_array_entry);
+   array = kmalloc(sz, GFP_KERNEL);
+   if (!array)
+   return -ENOMEM;
+
+   ret = -EFAULT;
+   if (copy_from_user(array, list->buffer, sz))
+   goto out;
+
+   ret = 0;
+   mutex_lock(>lock);
+   for (i = 0; i < nents; i++) {
+   shrd_region = kzalloc(sizeof(*shrd_region), GFP_KERNEL_ACCOUNT);
+   if (!shrd_region) {
+   struct shared_region *pos;
+
+   /* Freeing previously allocated entries */
+   list_for_each_entry(pos,
+   >shared_pages_list,
+   list) {
+   kfree(pos);
+   }
+
+   mutex_unlock(>lock);
+   ret = -ENOMEM;
+   goto out;
+   }
+
+   shrd_region->gfn_start = array[i].gfn_start;
+   shrd_region->gfn_end = array[i].gfn_end;
+   list_add_tail(_region->list,
+  

[PATCH v10 10/16] KVM: x86: Introduce KVM_GET_SHARED_PAGES_LIST ioctl

[Ashish Kalra]

From: Brijesh Singh 

The ioctl is used to retrieve a guest's shared pages list.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Brijesh Singh 
Co-developed-by: Ashish Kalra 
Signed-off-by: Ashish Kalra 
---
 Documentation/virt/kvm/api.rst  | 24 
 arch/x86/include/asm/kvm_host.h |  2 ++
 arch/x86/kvm/svm/sev.c  | 49 +
 arch/x86/kvm/svm/svm.c  |  1 +
 arch/x86/kvm/svm/svm.h  |  1 +
 arch/x86/kvm/x86.c  | 12 
 include/uapi/linux/kvm.h|  9 ++
 7 files changed, 98 insertions(+)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 99ceb978c8b0..59ef537c0cdd 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -4677,6 +4677,30 @@ This ioctl resets VCPU registers and control structures 
according to
 the clear cpu reset definition in the POP. However, the cpu is not put
 into ESA mode. This reset is a superset of the initial reset.
 
+4.125 KVM_GET_SHARED_PAGES_LIST (vm ioctl)
+---
+
+:Capability: basic
+:Architectures: x86
+:Type: vm ioctl
+:Parameters: struct kvm_shared_pages_list (in/out)
+:Returns: 0 on success, -1 on error
+
+/* for KVM_GET_SHARED_PAGES_LIST */
+struct kvm_shared_pages_list {
+   int __user *pnents;
+   void __user *buffer;
+   __u32 size;
+};
+
+The encrypted VMs have the concept of private and shared pages. The private
+pages are encrypted with the guest-specific key, while the shared pages may
+be encrypted with the hypervisor key. The KVM_GET_SHARED_PAGES_LIST can
+be used to get guest's shared/unencrypted memory regions list.
+This list can be used during the guest migration. If the page
+is private then the userspace need to use SEV migration commands to transmit
+the page.
+
 
 4.125 KVM_S390_PV_COMMAND
 -
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 2da5f5e2a10e..cd354d830e13 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1303,6 +1303,8 @@ struct kvm_x86_ops {
void (*vcpu_deliver_sipi_vector)(struct kvm_vcpu *vcpu, u8 vector);
int (*page_enc_status_hc)(struct kvm *kvm, unsigned long gpa,
  unsigned long sz, unsigned long mode);
+   int (*get_shared_pages_list)(struct kvm *kvm,
+struct kvm_shared_pages_list *list);
 };
 
 struct kvm_x86_nested_ops {
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 55c628df5155..701d74c8b15b 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -50,6 +50,11 @@ struct shared_region {
unsigned long gfn_start, gfn_end;
 };
 
+struct shared_region_array_entry {
+   unsigned long gfn_start;
+   unsigned long gfn_end;
+};
+
 static int sev_flush_asids(void)
 {
int ret, error = 0;
@@ -1622,6 +1627,50 @@ int svm_page_enc_status_hc(struct kvm *kvm, unsigned 
long gpa,
return ret;
 }
 
+int svm_get_shared_pages_list(struct kvm *kvm,
+ struct kvm_shared_pages_list *list)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct shared_region_array_entry *array;
+   struct shared_region *pos;
+   int ret, nents = 0;
+   unsigned long sz;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   if (!list->size)
+   return -EINVAL;
+
+   if (!sev->shared_pages_list_count)
+   return put_user(0, list->pnents);
+
+   sz = sev->shared_pages_list_count * sizeof(struct 
shared_region_array_entry);
+   if (sz > list->size)
+   return -E2BIG;
+
+   array = kmalloc(sz, GFP_KERNEL);
+   if (!array)
+   return -ENOMEM;
+
+   mutex_lock(>lock);
+   list_for_each_entry(pos, >shared_pages_list, list) {
+   array[nents].gfn_start = pos->gfn_start;
+   array[nents++].gfn_end = pos->gfn_end;
+   }
+   mutex_unlock(>lock);
+
+   ret = -EFAULT;
+   if (copy_to_user(list->buffer, array, sz))
+   goto out;
+   if (put_user(nents, list->pnents))
+   goto out;
+   ret = 0;
+out:
+   kfree(array);
+   return ret;
+}
+
 int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
 {
struct kvm_sev_cmd sev_cmd;
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index bb249ec625fc..533ce47ff158 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -4538,6 +4538,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
.vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector,
 
.page_enc_stat

[PATCH v10 08/16] KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall

[Ashish Kalra]

From: Brijesh Singh 

This hypercall is used by the SEV guest to notify a change in the page
encryption status to the hypervisor. The hypercall should be invoked
only when the encryption attribute is changed from encrypted -> decrypted
and vice versa. By default all guest pages are considered encrypted.

The patch introduces a new shared pages list implemented as a
sorted linked list to track the shared/unencrypted regions marked by the
guest hypercall.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Brijesh Singh 
Co-developed-by: Ashish Kalra 
Signed-off-by: Ashish Kalra 
---
 Documentation/virt/kvm/hypercalls.rst |  15 +++
 arch/x86/include/asm/kvm_host.h   |   2 +
 arch/x86/kvm/svm/sev.c| 150 ++
 arch/x86/kvm/svm/svm.c|   2 +
 arch/x86/kvm/svm/svm.h|   5 +
 arch/x86/kvm/vmx/vmx.c|   1 +
 arch/x86/kvm/x86.c|   6 ++
 include/uapi/linux/kvm_para.h |   1 +
 8 files changed, 182 insertions(+)

diff --git a/Documentation/virt/kvm/hypercalls.rst 
b/Documentation/virt/kvm/hypercalls.rst
index ed4fddd364ea..7aff0cebab7c 100644
--- a/Documentation/virt/kvm/hypercalls.rst
+++ b/Documentation/virt/kvm/hypercalls.rst
@@ -169,3 +169,18 @@ a0: destination APIC ID
 
 :Usage example: When sending a call-function IPI-many to vCPUs, yield if
any of the IPI target vCPUs was preempted.
+
+
+8. KVM_HC_PAGE_ENC_STATUS
+-
+:Architecture: x86
+:Status: active
+:Purpose: Notify the encryption status changes in guest page table (SEV guest)
+
+a0: the guest physical address of the start page
+a1: the number of pages
+a2: encryption attribute
+
+   Where:
+   * 1: Encryption attribute is set
+   * 0: Encryption attribute is cleared
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 3d6616f6f6ef..2da5f5e2a10e 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1301,6 +1301,8 @@ struct kvm_x86_ops {
int (*complete_emulated_msr)(struct kvm_vcpu *vcpu, int err);
 
void (*vcpu_deliver_sipi_vector)(struct kvm_vcpu *vcpu, u8 vector);
+   int (*page_enc_status_hc)(struct kvm *kvm, unsigned long gpa,
+ unsigned long sz, unsigned long mode);
 };
 
 struct kvm_x86_nested_ops {
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 25eaf35ba51d..55c628df5155 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -45,6 +45,11 @@ struct enc_region {
unsigned long size;
 };
 
+struct shared_region {
+   struct list_head list;
+   unsigned long gfn_start, gfn_end;
+};
+
 static int sev_flush_asids(void)
 {
int ret, error = 0;
@@ -196,6 +201,8 @@ static int sev_guest_init(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
sev->active = true;
sev->asid = asid;
INIT_LIST_HEAD(>regions_list);
+   INIT_LIST_HEAD(>shared_pages_list);
+   sev->shared_pages_list_count = 0;
 
return 0;
 
@@ -1473,6 +1480,148 @@ static int sev_receive_finish(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+static int remove_shared_region(unsigned long start, unsigned long end,
+   struct list_head *head)
+{
+   struct shared_region *pos;
+
+   list_for_each_entry(pos, head, list) {
+   if (pos->gfn_start == start &&
+   pos->gfn_end == end) {
+   list_del(>list);
+   kfree(pos);
+   return -1;
+   } else if (start >= pos->gfn_start && end <= pos->gfn_end) {
+   if (start == pos->gfn_start)
+   pos->gfn_start = end + 1;
+   else if (end == pos->gfn_end)
+   pos->gfn_end = start - 1;
+   else {
+   /* Do a de-merge -- split linked list nodes */
+   unsigned long tmp;
+   struct shared_region *shrd_region;
+
+   tmp = pos->gfn_end;
+   pos->gfn_end = start-1;
+   shrd_region = kzalloc(sizeof(*shrd_region), 
GFP_KERNEL_ACCOUNT);
+   if (!shrd_region)
+   return -ENOMEM;
+   shrd_region->gfn_start = end + 1;
+   shrd_region->gfn_end = tmp;
+   list_add(_region->list, >list);
+   return 1;
+

[PATCH v10 06/16] KVM: SVM: Add KVM_SEV_RECEIVE_FINISH command

[Ashish Kalra]

From: Brijesh Singh 

The command finalize the guest receiving process and make the SEV guest
ready for the execution.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  8 +++
 arch/x86/kvm/svm/sev.c| 23 +++
 2 files changed, 31 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index da40be3d8bc2..1f7bbda1f971 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -375,6 +375,14 @@ Returns: 0 on success, -negative on error
 __u32 trans_len;
 };
 
+15. KVM_SEV_RECEIVE_FINISH
+
+
+After completion of the migration flow, the KVM_SEV_RECEIVE_FINISH command can 
be
+issued by the hypervisor to make the guest ready for execution.
+
+Returns: 0 on success, -negative on error
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 73d5dbb72a65..25eaf35ba51d 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1453,6 +1453,26 @@ static int sev_receive_update_data(struct kvm *kvm, 
struct kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_receive_finish(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_receive_finish *data;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL);
+   if (!data)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_RECEIVE_FINISH, data, >error);
+
+   kfree(data);
+   return ret;
+}
+
 int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
 {
struct kvm_sev_cmd sev_cmd;
@@ -1518,6 +1538,9 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
case KVM_SEV_RECEIVE_UPDATE_DATA:
r = sev_receive_update_data(kvm, _cmd);
break;
+   case KVM_SEV_RECEIVE_FINISH:
+   r = sev_receive_finish(kvm, _cmd);
+   break;
default:
r = -EINVAL;
goto out;
-- 
2.17.1

[PATCH v10 07/16] KVM: x86: Add AMD SEV specific Hypercall3

[Ashish Kalra]

From: Brijesh Singh 

KVM hypercall framework relies on alternative framework to patch the
VMCALL -> VMMCALL on AMD platform. If a hypercall is made before
apply_alternative() is called then it defaults to VMCALL. The approach
works fine on non SEV guest. A VMCALL would causes #UD, and hypervisor
will be able to decode the instruction and do the right things. But
when SEV is active, guest memory is encrypted with guest key and
hypervisor will not be able to decode the instruction bytes.

Add SEV specific hypercall3, it unconditionally uses VMMCALL. The hypercall
will be used by the SEV guest to notify encrypted pages to the hypervisor.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Reviewed-by: Venu Busireddy 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 arch/x86/include/asm/kvm_para.h | 12 
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 338119852512..bc1b11d057fc 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -85,6 +85,18 @@ static inline long kvm_hypercall4(unsigned int nr, unsigned 
long p1,
return ret;
 }
 
+static inline long kvm_sev_hypercall3(unsigned int nr, unsigned long p1,
+ unsigned long p2, unsigned long p3)
+{
+   long ret;
+
+   asm volatile("vmmcall"
+: "=a"(ret)
+: "a"(nr), "b"(p1), "c"(p2), "d"(p3)
+: "memory");
+   return ret;
+}
+
 #ifdef CONFIG_KVM_GUEST
 bool kvm_para_available(void);
 unsigned int kvm_arch_para_features(void);
-- 
2.17.1

[PATCH v10 05/16] KVM: SVM: Add KVM_SEV_RECEIVE_UPDATE_DATA command

[Ashish Kalra]

From: Brijesh Singh 

The command is used for copying the incoming buffer into the
SEV guest memory space.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst| 24 ++
 arch/x86/kvm/svm/sev.c| 79 +++
 include/uapi/linux/kvm.h  |  9 +++
 3 files changed, 112 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index 079ac5ac2459..da40be3d8bc2 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -351,6 +351,30 @@ On success, the 'handle' field contains a new handle and 
on error, a negative va
 
 For more details, see SEV spec Section 6.12.
 
+14. KVM_SEV_RECEIVE_UPDATE_DATA
+
+
+The KVM_SEV_RECEIVE_UPDATE_DATA command can be used by the hypervisor to copy
+the incoming buffers into the guest memory region with encryption context
+created during the KVM_SEV_RECEIVE_START.
+
+Parameters (in): struct kvm_sev_receive_update_data
+
+Returns: 0 on success, -negative on error
+
+::
+
+struct kvm_sev_launch_receive_update_data {
+__u64 hdr_uaddr;/* userspace address containing the 
packet header */
+__u32 hdr_len;
+
+__u64 guest_uaddr;  /* the destination guest memory region 
*/
+__u32 guest_len;
+
+__u64 trans_uaddr;  /* the incoming buffer memory region  
*/
+__u32 trans_len;
+};
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index ec0d573cb09a..73d5dbb72a65 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1377,6 +1377,82 @@ static int sev_receive_start(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_receive_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct kvm_sev_receive_update_data params;
+   struct sev_data_receive_update_data *data;
+   void *hdr = NULL, *trans = NULL;
+   struct page **guest_page;
+   unsigned long n;
+   int ret, offset;
+
+   if (!sev_guest(kvm))
+   return -EINVAL;
+
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_receive_update_data)))
+   return -EFAULT;
+
+   if (!params.hdr_uaddr || !params.hdr_len ||
+   !params.guest_uaddr || !params.guest_len ||
+   !params.trans_uaddr || !params.trans_len)
+   return -EINVAL;
+
+   /* Check if we are crossing the page boundary */
+   offset = params.guest_uaddr & (PAGE_SIZE - 1);
+   if ((params.guest_len + offset > PAGE_SIZE))
+   return -EINVAL;
+
+   hdr = psp_copy_user_blob(params.hdr_uaddr, params.hdr_len);
+   if (IS_ERR(hdr))
+   return PTR_ERR(hdr);
+
+   trans = psp_copy_user_blob(params.trans_uaddr, params.trans_len);
+   if (IS_ERR(trans)) {
+   ret = PTR_ERR(trans);
+   goto e_free_hdr;
+   }
+
+   ret = -ENOMEM;
+   data = kzalloc(sizeof(*data), GFP_KERNEL);
+   if (!data)
+   goto e_free_trans;
+
+   data->hdr_address = __psp_pa(hdr);
+   data->hdr_len = params.hdr_len;
+   data->trans_address = __psp_pa(trans);
+   data->trans_len = params.trans_len;
+
+   /* Pin guest memory */
+   ret = -EFAULT;
+   guest_page = sev_pin_memory(kvm, params.guest_uaddr & PAGE_MASK,
+   PAGE_SIZE, , 0);
+   if (!guest_page)
+   goto e_free;
+
+   /* The RECEIVE_UPDATE_DATA command requires C-bit to be always set. */
+   data->guest_address = (page_to_pfn(guest_page[0]) << PAGE_SHIFT) +
+   offset;
+   data->guest_address |= sev_me_mask;
+   data->guest_len = params.guest_len;
+   data->handle = sev->handle;
+
+   ret = sev_issue_cmd(kvm, SEV_CMD_RECEIVE_UPDATE_DATA, data,
+   >error);
+
+   sev_unpin_memory(kvm, guest_page, n);
+
+e_free:
+   kfree(data);
+e_free_trans:
+   kfree(trans);
+e_free_hdr:
+   kfree(hdr);
+
+   return ret;
+}
+
 int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
 {
struct kvm_sev_cmd sev_cmd;
@@ -1439,6 +1515,9 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
case KVM_SEV_RECEIVE_START:
r = sev_receive_start(kvm, _cmd);
break;
+   case KVM_SEV_R

[PATCH v10 02/16] KVM: SVM: Add KVM_SEND_UPDATE_DATA command

[Ashish Kalra]

From: Brijesh Singh 

The command is used for encrypting the guest memory region using the encryption
context created with KVM_SEV_SEND_START.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by : Steve Rutherford 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  24 
 arch/x86/kvm/svm/sev.c| 122 ++
 include/uapi/linux/kvm.h  |   9 ++
 3 files changed, 155 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index 9f9896b72d36..8bed1d801558 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -290,6 +290,30 @@ Returns: 0 on success, -negative on error
 __u32 session_len;
 };
 
+11. KVM_SEV_SEND_UPDATE_DATA
+
+
+The KVM_SEV_SEND_UPDATE_DATA command can be used by the hypervisor to encrypt 
the
+outgoing guest memory region with the encryption context creating using
+KVM_SEV_SEND_START.
+
+Parameters (in): struct kvm_sev_send_update_data
+
+Returns: 0 on success, -negative on error
+
+::
+
+struct kvm_sev_launch_send_update_data {
+__u64 hdr_uaddr;/* userspace address containing the 
packet header */
+__u32 hdr_len;
+
+__u64 guest_uaddr;  /* the source memory region to be 
encrypted */
+__u32 guest_len;
+
+__u64 trans_uaddr;  /* the destition memory region  */
+__u32 trans_len;
+};
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 3026c7fd2ffc..98e46ae1cba3 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -33,6 +33,7 @@ static DECLARE_RWSEM(sev_deactivate_lock);
 static DEFINE_MUTEX(sev_bitmap_lock);
 unsigned int max_sev_asid;
 static unsigned int min_sev_asid;
+static unsigned long sev_me_mask;
 static unsigned long *sev_asid_bitmap;
 static unsigned long *sev_reclaim_asid_bitmap;
 
@@ -1161,6 +1162,123 @@ static int sev_send_start(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+/* Userspace wants to query either header or trans length. */
+static int
+__sev_send_update_data_query_lengths(struct kvm *kvm, struct kvm_sev_cmd *argp,
+struct kvm_sev_send_update_data *params)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_update_data *data;
+   int ret;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL_ACCOUNT);
+   if (!data)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_SEND_UPDATE_DATA, data, >error);
+
+   params->hdr_len = data->hdr_len;
+   params->trans_len = data->trans_len;
+
+   if (copy_to_user((void __user *)(uintptr_t)argp->data, params,
+sizeof(struct kvm_sev_send_update_data)))
+   ret = -EFAULT;
+
+   kfree(data);
+   return ret;
+}
+
+static int sev_send_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_update_data *data;
+   struct kvm_sev_send_update_data params;
+   void *hdr, *trans_data;
+   struct page **guest_page;
+   unsigned long n;
+   int ret, offset;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_send_update_data)))
+   return -EFAULT;
+
+   /* userspace wants to query either header or trans length */
+   if (!params.trans_len || !params.hdr_len)
+   return __sev_send_update_data_query_lengths(kvm, argp, );
+
+   if (!params.trans_uaddr || !params.guest_uaddr ||
+   !params.guest_len || !params.hdr_uaddr)
+   return -EINVAL;
+
+   /* Check if we are crossing the page boundary */
+   offset = params.guest_uaddr & (PAGE_SIZE - 1);
+   if ((params.guest_len + offset > PAGE_SIZE))
+   return -EINVAL;
+
+   /* Pin guest memory */
+   guest_page = sev_pin_memory(kvm, params.guest_uaddr & PAGE_MASK,
+   PAGE_SIZE, , 0);
+   if (!guest_page)
+   return -EFAULT;
+
+   /* allocate memory for header and transport buffer */
+   ret = -ENOMEM;
+   hdr = kmalloc(params.hdr_len, GFP_KERNEL_ACCOUNT);
+   if (!hdr)
+   goto e_unpin;
+
+   trans_data = kmalloc(params.trans_len, GFP_KERNEL_AC

[PATCH v10 04/16] KVM: SVM: Add support for KVM_SEV_RECEIVE_START command

[Ashish Kalra]

From: Brijesh Singh 

The command is used to create the encryption context for an incoming
SEV guest. The encryption context can be later used by the hypervisor
to import the incoming data into the SEV guest memory space.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst| 29 +++
 arch/x86/kvm/svm/sev.c| 81 +++
 include/uapi/linux/kvm.h  |  9 +++
 3 files changed, 119 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index 0da0c199efa8..079ac5ac2459 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -322,6 +322,35 @@ issued by the hypervisor to delete the encryption context.
 
 Returns: 0 on success, -negative on error
 
+13. KVM_SEV_RECEIVE_START
+
+
+The KVM_SEV_RECEIVE_START command is used for creating the memory encryption
+context for an incoming SEV guest. To create the encryption context, the user 
must
+provide a guest policy, the platform public Diffie-Hellman (PDH) key and 
session
+information.
+
+Parameters: struct  kvm_sev_receive_start (in/out)
+
+Returns: 0 on success, -negative on error
+
+::
+
+struct kvm_sev_receive_start {
+__u32 handle;   /* if zero then firmware creates a new 
handle */
+__u32 policy;   /* guest's policy */
+
+__u64 pdh_uaddr;/* userspace address pointing to the 
PDH key */
+__u32 pdh_len;
+
+__u64 session_uaddr;/* userspace address which points to 
the guest session information */
+__u32 session_len;
+};
+
+On success, the 'handle' field contains a new handle and on error, a negative 
value.
+
+For more details, see SEV spec Section 6.12.
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 0d117b1e6491..ec0d573cb09a 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1299,6 +1299,84 @@ static int sev_send_finish(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_receive_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_receive_start *start;
+   struct kvm_sev_receive_start params;
+   int *error = >error;
+   void *session_data;
+   void *pdh_data;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   /* Get parameter from the userspace */
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_receive_start)))
+   return -EFAULT;
+
+   /* some sanity checks */
+   if (!params.pdh_uaddr || !params.pdh_len ||
+   !params.session_uaddr || !params.session_len)
+   return -EINVAL;
+
+   pdh_data = psp_copy_user_blob(params.pdh_uaddr, params.pdh_len);
+   if (IS_ERR(pdh_data))
+   return PTR_ERR(pdh_data);
+
+   session_data = psp_copy_user_blob(params.session_uaddr,
+   params.session_len);
+   if (IS_ERR(session_data)) {
+   ret = PTR_ERR(session_data);
+   goto e_free_pdh;
+   }
+
+   ret = -ENOMEM;
+   start = kzalloc(sizeof(*start), GFP_KERNEL);
+   if (!start)
+   goto e_free_session;
+
+   start->handle = params.handle;
+   start->policy = params.policy;
+   start->pdh_cert_address = __psp_pa(pdh_data);
+   start->pdh_cert_len = params.pdh_len;
+   start->session_address = __psp_pa(session_data);
+   start->session_len = params.session_len;
+
+   /* create memory encryption context */
+   ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_RECEIVE_START, start,
+   error);
+   if (ret)
+   goto e_free;
+
+   /* Bind ASID to this guest */
+   ret = sev_bind_asid(kvm, start->handle, error);
+   if (ret)
+   goto e_free;
+
+   params.handle = start->handle;
+   if (copy_to_user((void __user *)(uintptr_t)argp->data,
+, sizeof(struct kvm_sev_receive_start))) {
+   ret = -EFAULT;
+   sev_unbind_asid(kvm, start->handle);
+   goto e_free;
+   }
+
+   sev->handle = start->handle;
+   sev->fd = argp->sev_fd;
+
+e_free:
+   kfree(start);
+e_free_session:
+   kfree(session_data);
+e_free_pdh:
+   kfree(pdh_d

[PATCH v10 03/16] KVM: SVM: Add KVM_SEV_SEND_FINISH command

[Ashish Kalra]

From: Brijesh Singh 

The command is used to finailize the encryption context created with
KVM_SEV_SEND_START command.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  8 +++
 arch/x86/kvm/svm/sev.c| 23 +++
 2 files changed, 31 insertions(+)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index 8bed1d801558..0da0c199efa8 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -314,6 +314,14 @@ Returns: 0 on success, -negative on error
 __u32 trans_len;
 };
 
+12. KVM_SEV_SEND_FINISH
+
+
+After completion of the migration flow, the KVM_SEV_SEND_FINISH command can be
+issued by the hypervisor to delete the encryption context.
+
+Returns: 0 on success, -negative on error
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 98e46ae1cba3..0d117b1e6491 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1279,6 +1279,26 @@ static int sev_send_update_data(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+static int sev_send_finish(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_finish *data;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL);
+   if (!data)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_SEND_FINISH, data, >error);
+
+   kfree(data);
+   return ret;
+}
+
 int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
 {
struct kvm_sev_cmd sev_cmd;
@@ -1335,6 +1355,9 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp)
case KVM_SEV_SEND_UPDATE_DATA:
r = sev_send_update_data(kvm, _cmd);
break;
+   case KVM_SEV_SEND_FINISH:
+   r = sev_send_finish(kvm, _cmd);
+   break;
default:
r = -EINVAL;
goto out;
-- 
2.17.1

[PATCH v10 01/16] KVM: SVM: Add KVM_SEV SEND_START command

[Ashish Kalra]

From: Brijesh Singh 

The command is used to create an outgoing SEV guest encryption context.

Cc: Thomas Gleixner 
Cc: Ingo Molnar 
Cc: "H. Peter Anvin" 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Joerg Roedel 
Cc: Borislav Petkov 
Cc: Tom Lendacky 
Cc: x...@kernel.org
Cc: k...@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Steve Rutherford 
Reviewed-by: Venu Busireddy 
Signed-off-by: Brijesh Singh 
Signed-off-by: Ashish Kalra 
---
 .../virt/kvm/amd-memory-encryption.rst|  27 
 arch/x86/kvm/svm/sev.c| 125 ++
 include/linux/psp-sev.h   |   8 +-
 include/uapi/linux/kvm.h  |  12 ++
 4 files changed, 168 insertions(+), 4 deletions(-)

diff --git a/Documentation/virt/kvm/amd-memory-encryption.rst 
b/Documentation/virt/kvm/amd-memory-encryption.rst
index 09a8f2a34e39..9f9896b72d36 100644
--- a/Documentation/virt/kvm/amd-memory-encryption.rst
+++ b/Documentation/virt/kvm/amd-memory-encryption.rst
@@ -263,6 +263,33 @@ Returns: 0 on success, -negative on error
 __u32 trans_len;
 };
 
+10. KVM_SEV_SEND_START
+--
+
+The KVM_SEV_SEND_START command can be used by the hypervisor to create an
+outgoing guest encryption context.
+
+Parameters (in): struct kvm_sev_send_start
+
+Returns: 0 on success, -negative on error
+
+::
+struct kvm_sev_send_start {
+__u32 policy; /* guest policy */
+
+__u64 pdh_cert_uaddr; /* platform Diffie-Hellman 
certificate */
+__u32 pdh_cert_len;
+
+__u64 plat_certs_uaddr;/* platform certificate chain */
+__u32 plat_certs_len;
+
+__u64 amd_certs_uaddr;/* AMD certificate */
+__u32 amd_certs_len;
+
+__u64 session_uaddr;  /* Guest session information */
+__u32 session_len;
+};
+
 References
 ==
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index ac652bc476ae..3026c7fd2ffc 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1039,6 +1039,128 @@ static int sev_launch_secret(struct kvm *kvm, struct 
kvm_sev_cmd *argp)
return ret;
 }
 
+/* Userspace wants to query session length. */
+static int
+__sev_send_start_query_session_length(struct kvm *kvm, struct kvm_sev_cmd 
*argp,
+ struct kvm_sev_send_start *params)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_start *data;
+   int ret;
+
+   data = kzalloc(sizeof(*data), GFP_KERNEL_ACCOUNT);
+   if (data == NULL)
+   return -ENOMEM;
+
+   data->handle = sev->handle;
+   ret = sev_issue_cmd(kvm, SEV_CMD_SEND_START, data, >error);
+
+   params->session_len = data->session_len;
+   if (copy_to_user((void __user *)(uintptr_t)argp->data, params,
+   sizeof(struct kvm_sev_send_start)))
+   ret = -EFAULT;
+
+   kfree(data);
+   return ret;
+}
+
+static int sev_send_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+   struct kvm_sev_info *sev = _kvm_svm(kvm)->sev_info;
+   struct sev_data_send_start *data;
+   struct kvm_sev_send_start params;
+   void *amd_certs, *session_data;
+   void *pdh_cert, *plat_certs;
+   int ret;
+
+   if (!sev_guest(kvm))
+   return -ENOTTY;
+
+   if (copy_from_user(, (void __user *)(uintptr_t)argp->data,
+   sizeof(struct kvm_sev_send_start)))
+   return -EFAULT;
+
+   /* if session_len is zero, userspace wants to query the session length 
*/
+   if (!params.session_len)
+   return __sev_send_start_query_session_length(kvm, argp,
+   );
+
+   /* some sanity checks */
+   if (!params.pdh_cert_uaddr || !params.pdh_cert_len ||
+   !params.session_uaddr || params.session_len > SEV_FW_BLOB_MAX_SIZE)
+   return -EINVAL;
+
+   /* allocate the memory to hold the session data blob */
+   session_data = kmalloc(params.session_len, GFP_KERNEL_ACCOUNT);
+   if (!session_data)
+   return -ENOMEM;
+
+   /* copy the certificate blobs from userspace */
+   pdh_cert = psp_copy_user_blob(params.pdh_cert_uaddr,
+   params.pdh_cert_len);
+   if (IS_ERR(pdh_cert)) {
+   ret = PTR_ERR(pdh_cert);
+   goto e_free_session;
+   }
+
+   plat_certs = psp_copy_user_blob(params.plat_certs_uaddr,
+   params.plat_certs_len);
+   if (IS_ERR(plat_certs)) {
+   ret = PTR_ERR(plat_certs);
+   goto e_free_pdh;
+   }
+
+   amd_certs = psp_copy_user_blob(params.amd_certs_uaddr,
+   params.amd_certs_len);
+