Re: [PATCH v3 04/14] KVM: s390: device attribute to set AP interpretive execution

[Tony Krowiak]

On 03/15/2018 01:56 PM, Pierre Morel wrote:
On 15/03/2018 18:21, Tony Krowiak wrote:
On 03/15/2018 11:45 AM, Pierre Morel wrote:
On 15/03/2018 16:26, Tony Krowiak wrote:
On 03/15/2018 09:00 AM, Pierre Morel wrote:
On 14/03/2018 22:57, Halil Pasic wrote:

On 03/14/2018 07:25 PM, Tony Krowiak wrote:
The VFIO AP device model exploits interpretive execution of AP
instructions (APIE) to provide guests passthrough access to AP
devices. This patch introduces a new device attribute in the
KVM_S390_VM_CRYPTO device attribute group to set APIE from
the VFIO AP device defined on the guest.

Signed-off-by: Tony Krowiak <akrow...@linux.vnet.ibm.com>
---
[..]

diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index a60c45b..bc46b67 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -815,6 +815,19 @@ static int kvm_s390_vm_set_crypto(struct 
kvm *kvm, struct kvm_device_attr *attr)
sizeof(kvm->arch.crypto.crycb->dea_wrapping_key_mask));
          VM_EVENT(kvm, 3, "%s", "DISABLE: DEA keywrapping 
support");
          break;
+    case KVM_S390_VM_CRYPTO_INTERPRET_AP:
+        if (attr->addr) {
+            if (!test_kvm_cpu_feat(kvm, KVM_S390_VM_CPU_FEAT_AP))
Unlock mutex before returning?

Maybe flip conditions (don't allow manipulating apie if feature 
not there).
Clearing the anyways clear apie if feature not there ain't too 
bad, but
rejecting the operation appears nicer to me.

+                return -EOPNOTSUPP;
+            kvm->arch.crypto.apie = 1;
+            VM_EVENT(kvm, 3, "%s",
+                 "ENABLE: AP interpretive execution");
+        } else {
+            kvm->arch.crypto.apie = 0;
+            VM_EVENT(kvm, 3, "%s",
+                 "DISABLE: AP interpretive execution");
+        }
+        break;
      default:
          mutex_unlock(&kvm->lock);
          return -ENXIO;
I wonder how the loop after this switch works for 
KVM_S390_VM_CRYPTO_INTERPRET_AP:

         kvm_for_each_vcpu(i, vcpu, kvm) {
                 kvm_s390_vcpu_crypto_setup(vcpu);
                 exit_sie(vcpu);
         }

 From not doing something like for KVM_S390_VM_CRYPTO_INTERPRET_AP

         if (kvm->created_vcpus) {
                 mutex_unlock(&kvm->lock);
                 return -EBUSY;
and from the aforementioned loop I guess ECA.28 can be changed
for a running guest.

If there are running vcpus when KVM_S390_VM_CRYPTO_INTERPRET_AP is
changed (set) these will be taken out of SIE by exit_sie(). Then 
for the
corresponding threads the control probably goes to QEMU (the 
emulator in
the userspace). And it puts that vcpu back into the SIE, and then 
that
cpu starts acting according to the new ECA.28 value. While other 
vcpus
may still work with the old value of ECA.28.

I'm not saying what I describe above is necessarily something 
broken.
But I would like to have it explained, why is it OK -- provided I 
did not
make any errors in my reasoning (assumptions included).

Can you help me understand this code?

Regards,
Halil

[..]


I have the same concerns as Halil.

We do not need to change the virtulization type
(hardware/software) on the fly for the current use case.

Couldn't we delay this until we have one and in between only make 
the vCPU hotplug clean?

We only need to let the door open for the day we have such a use 
case.
Are you suggesting this code be removed? If so, then where and 
under what conditions would
you suggest setting ECA.28 given you objected to setting it based 
on whether the
AP feature is installed?

I would only call kvm_s390_vcpu_crypto_setup() from inside 
kvm_arch_vcpu_init()
as it is already.
It is not called from kvm_arch_vcpu_init(), it is called from 
kvm_arch_vcpu_setup(). 

hum, sorry for this.
However, the idea pertains, not to call this function from inside an 
ioctl changing crypto parameters, but only during vcpu creation.
Unfortunately, the ioctl does not get called until after the vcpus are 
created (see my comments below)



Also,
this loop was already here, I did not put it in. Assuming whomever 
put it there did so
for a reason, it is not my place to remove it. According to a trace I 
ran, the calls to this
function occur after the vcpus are created. Consequently, the 
kvm_s390_vcpu_crypto_setup()
function would not be called without the loop and neither the key 
wrapping support nor the
ECA_APIE would be configured in the vcpu's SIE descriptor.

If you have a better idea for where/how to set this flag, I'm all
ears. It would be nice if it could be set before the vcpus are 
created, but I haven't
found a good candidate. I suspect that the loop was put in to make 
sure that all vcpus
get updated regardless of whether they are running or not, but I 
don't know what happens
after a vcpu is kicked out of SIE. I suspect, as Halil surmised, that 
QEMU
restores the vcpus to SIE. This would seemingly cause the 
kvm_arch_vcpu_setup() to get
called at which time the ECA_APIE value as well as the key wrapping 
values will get set.
If somebody has knowledge of the flow here, please feel free to pitch 
in.





Pierre