Re: [PATCH] docs: Describe protected virtualization guest setup

[Boris Fiuczynski]

On 4/29/20 3:29 PM, Daniel P. Berrangé wrote:

On Tue, Apr 28, 2020 at 05:58:02PM +0200, Boris Fiuczynski wrote:

From: Viktor Mihajlovski 

Protected virtualization/IBM Secure Execution for Linux protects
guest memory and state from the host.

Add some basic information about technology and a brief guide
on setting up secure guests with libvirt.

Signed-off-by: Viktor Mihajlovski 
Reviewed-by: Boris Fiuczynski 
Reviewed-by: Paulo de Rezende Pinatti 
---
  docs/kbase.html.in  |   3 +
  docs/kbase/protected_virtualization.rst | 188 


I'd suggest calling this  s390_protected_virt.rst

Done




diff --git a/docs/kbase.html.in b/docs/kbase.html.in
index c586e0f676..05a3239224 100644
--- a/docs/kbase.html.in
+++ b/docs/kbase.html.in
@@ -14,6 +14,9 @@
  Secure usage
  Secure usage of the libvirt APIs
  
+Protected virtualization


"s390 Protected virtualization"  as the title


As discussed changed to "Protected virtualization on s390".




+Running secure guests with IBM Secure Execution


s/secure guests/secure s390 guests/


Done




+
  Launch security
  Securely launching VMs with AMD SEV
  
diff --git a/docs/kbase/protected_virtualization.rst b/docs/kbase/protected_virtualization.rst

new file mode 100644
index 00..48f2add14e
--- /dev/null
+++ b/docs/kbase/protected_virtualization.rst
@@ -0,0 +1,188 @@
+
+Protected Virtualization


s/^/s390/


As discussed changed to "Protected virtualization on s390".




+



+Host Requirements
+=
+
+IBM Secure Execution for Linux has some hardware and firmware
+requirements. The system hardware must be an IBM z15 (or newer),
+or an IBM LinuxONE III (or newer).
+
+It is also necessary that the IBM Secure Execution feature is
+enabled for that system. Check for facility '158', e.g.
+
+::
+
+   $ grep facilities /proc/cpuinfo | grep 158


I'd suggest that "virt-host-validate" be probing for this
so we can just tell them to run that command.


Extending virt-host-validate can be done.
This documentation follows the same bare bone approach as is used in 
docs/kbase/launch_security_sev.rst.

Would it be OK to explain both with and without libvirt means?

I will have a look at virt-host-validate once the cache invalidation is 
ready.





+The kernel must include the protected virtualization support
+which can be verified by checking for the presence of directory
+``/sys/firmware/uv``. It will only be present when both the
+hardware and the kernel support are available.
+
+Finally, the host operating system must donate some memory to
+the ultravisor needed to store memory security information.
+This is achieved by specifying the following kernel command
+line parameter to the host boot configuration
+
+::
+
+   prot_virt=1
+
+
+Guest Requirements
+==
+
+Guest Boot
+--
+
+To start a guest in protected virtualization secure mode, the
+boot image must have been prepared first with the program
+``genprotimg`` using the correct public key for this host.
+``genprotimg`` is part of the package ``s390-tools``, or
+``s390-utils``, depending on the Linux distribution being used.
+It can also be found at
+``_
+
+The guests have to be configured to use the host CPU model, which
+must contain the ``unpack`` facility indicating ultravisor guest support.
+
+With the following command it's possible to check whether the host
+CPU model satisfies the requirement
+
+::
+
+   $ virsh domcapabilities | grep unpack
+
+which should return
+
+::
+
+   
+
+If the check fails despite the host system actually supporting
+protected virtualization guests, this can be caused by a stale
+libvirt capabilities cache. To recover, run the following
+commands
+
+::
+
+   $ systemctl stop libvirtd
+   $ rm /var/cache/libvirt/qemu/capabilities/*.xml
+   $ systemctl start libvirtd


If this is truly needed that it is a bug in libvirt - we're missing
something in the cache invalidation logic.


Yes, as I wrote to your other mail... we are working on it.




Regards,
Daniel




--
Mit freundlichen Grüßen/Kind regards
   Boris Fiuczynski

IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Gregor Pillen
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

Re: [PATCH] docs: Describe protected virtualization guest setup

[Daniel P . Berrangé]

On Wed, Apr 29, 2020 at 05:50:58PM +0200, Boris Fiuczynski wrote:
> On 4/29/20 5:15 PM, Daniel P. Berrangé wrote:
> > On Wed, Apr 29, 2020 at 05:08:31PM +0200, Viktor Mihajlovski wrote:
> > > 
> > > 
> > > On 4/29/20 3:29 PM, Daniel P. Berrangé wrote:
> > > > On Tue, Apr 28, 2020 at 05:58:02PM +0200, Boris Fiuczynski wrote:
> > > > > From: Viktor Mihajlovski 
> > > > > 
> > > > > Protected virtualization/IBM Secure Execution for Linux protects
> > > > > guest memory and state from the host.
> > > > > 
> > > > > Add some basic information about technology and a brief guide
> > > > > on setting up secure guests with libvirt.
> > > > > 
> > > > > Signed-off-by: Viktor Mihajlovski 
> > > > > Reviewed-by: Boris Fiuczynski 
> > > > > Reviewed-by: Paulo de Rezende Pinatti 
> > > > > ---
> > > > >docs/kbase.html.in  |   3 +
> > > > >docs/kbase/protected_virtualization.rst | 188 
> > > > > 
> > > > 
> > > > I'd suggest calling this  s390_protected_virt.rst
> > > We can do that.
> > > > 
> > > > > diff --git a/docs/kbase.html.in b/docs/kbase.html.in
> > > > > index c586e0f676..05a3239224 100644
> > > > > --- a/docs/kbase.html.in
> > > > > +++ b/docs/kbase.html.in
> > > > > @@ -14,6 +14,9 @@
> > > > >Secure usage
> > > > >Secure usage of the libvirt APIs
> > > > > +Protected 
> > > > > virtualization
> > > > 
> > > > "s390 Protected virtualization"  as the title
> > > > 
> > > The terminology that was used in the KVM upstream code is simply protected
> > > virtualization without a prefix, so I'd avoid creating a new denomination 
> > > in
> > > libvirt.
> > 
> > Putting an "s390" prefix on this isn't inventing new terminology - it is
> > just making it obvious to users what target it applies to.
> 
> Daniel & Viktor: Would calling it "Protected virtualization on s390" serve
> both of you?

Sure.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [PATCH] docs: Describe protected virtualization guest setup

[Viktor Mihajlovski]

On 4/29/20 5:50 PM, Boris Fiuczynski wrote:

On 4/29/20 5:15 PM, Daniel P. Berrangé wrote:

On Wed, Apr 29, 2020 at 05:08:31PM +0200, Viktor Mihajlovski wrote:



On 4/29/20 3:29 PM, Daniel P. Berrangé wrote:

On Tue, Apr 28, 2020 at 05:58:02PM +0200, Boris Fiuczynski wrote:

From: Viktor Mihajlovski 

Protected virtualization/IBM Secure Execution for Linux protects
guest memory and state from the host.

Add some basic information about technology and a brief guide
on setting up secure guests with libvirt.

Signed-off-by: Viktor Mihajlovski 
Reviewed-by: Boris Fiuczynski 
Reviewed-by: Paulo de Rezende Pinatti 
---
   docs/kbase.html.in  |   3 +
   docs/kbase/protected_virtualization.rst | 188 



I'd suggest calling this  s390_protected_virt.rst

We can do that.



diff --git a/docs/kbase.html.in b/docs/kbase.html.in
index c586e0f676..05a3239224 100644
--- a/docs/kbase.html.in
+++ b/docs/kbase.html.in
@@ -14,6 +14,9 @@
   Secure usage
   Secure usage of the libvirt APIs
+    href="kbase/protected_virtualization.html">Protected 
virtualization


"s390 Protected virtualization"  as the title

The terminology that was used in the KVM upstream code is simply 
protected
virtualization without a prefix, so I'd avoid creating a new 
denomination in

libvirt.


Putting an "s390" prefix on this isn't inventing new terminology - it is
just making it obvious to users what target it applies to.


Daniel & Viktor: Would calling it "Protected virtualization on s390" 
serve both of you?



I can live with that.



Alternatively we could use the (unmodified) marketing name "IBM Secure
Execution for Linux" here and below in the RST and reverse the "also 
known

as" sentence in the overview.


That's even worse IMHO. Just put an s390 prefix on the current text.

Regards,
Daniel






--
Kind Regards,
   Viktor

Re: [PATCH] docs: Describe protected virtualization guest setup

[Boris Fiuczynski]

On 4/29/20 5:15 PM, Daniel P. Berrangé wrote:

On Wed, Apr 29, 2020 at 05:08:31PM +0200, Viktor Mihajlovski wrote:



On 4/29/20 3:29 PM, Daniel P. Berrangé wrote:

On Tue, Apr 28, 2020 at 05:58:02PM +0200, Boris Fiuczynski wrote:

From: Viktor Mihajlovski 

Protected virtualization/IBM Secure Execution for Linux protects
guest memory and state from the host.

Add some basic information about technology and a brief guide
on setting up secure guests with libvirt.

Signed-off-by: Viktor Mihajlovski 
Reviewed-by: Boris Fiuczynski 
Reviewed-by: Paulo de Rezende Pinatti 
---
   docs/kbase.html.in  |   3 +
   docs/kbase/protected_virtualization.rst | 188 


I'd suggest calling this  s390_protected_virt.rst

We can do that.



diff --git a/docs/kbase.html.in b/docs/kbase.html.in
index c586e0f676..05a3239224 100644
--- a/docs/kbase.html.in
+++ b/docs/kbase.html.in
@@ -14,6 +14,9 @@
   Secure usage
   Secure usage of the libvirt APIs
+Protected 
virtualization


"s390 Protected virtualization"  as the title


The terminology that was used in the KVM upstream code is simply protected
virtualization without a prefix, so I'd avoid creating a new denomination in
libvirt.


Putting an "s390" prefix on this isn't inventing new terminology - it is
just making it obvious to users what target it applies to.


Daniel & Viktor: Would calling it "Protected virtualization on s390" 
serve both of you?





Alternatively we could use the (unmodified) marketing name "IBM Secure
Execution for Linux" here and below in the RST and reverse the "also known
as" sentence in the overview.


That's even worse IMHO. Just put an s390 prefix on the current text.

Regards,
Daniel




--
Mit freundlichen Grüßen/Kind regards
   Boris Fiuczynski

IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Gregor Pillen
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

Re: [PATCH] docs: Describe protected virtualization guest setup

[Daniel P . Berrangé]

On Wed, Apr 29, 2020 at 05:08:31PM +0200, Viktor Mihajlovski wrote:
> 
> 
> On 4/29/20 3:29 PM, Daniel P. Berrangé wrote:
> > On Tue, Apr 28, 2020 at 05:58:02PM +0200, Boris Fiuczynski wrote:
> > > From: Viktor Mihajlovski 
> > > 
> > > Protected virtualization/IBM Secure Execution for Linux protects
> > > guest memory and state from the host.
> > > 
> > > Add some basic information about technology and a brief guide
> > > on setting up secure guests with libvirt.
> > > 
> > > Signed-off-by: Viktor Mihajlovski 
> > > Reviewed-by: Boris Fiuczynski 
> > > Reviewed-by: Paulo de Rezende Pinatti 
> > > ---
> > >   docs/kbase.html.in  |   3 +
> > >   docs/kbase/protected_virtualization.rst | 188 
> > 
> > I'd suggest calling this  s390_protected_virt.rst
> We can do that.
> > 
> > > diff --git a/docs/kbase.html.in b/docs/kbase.html.in
> > > index c586e0f676..05a3239224 100644
> > > --- a/docs/kbase.html.in
> > > +++ b/docs/kbase.html.in
> > > @@ -14,6 +14,9 @@
> > >   Secure usage
> > >   Secure usage of the libvirt APIs
> > > +Protected 
> > > virtualization
> > 
> > "s390 Protected virtualization"  as the title
> > 
> The terminology that was used in the KVM upstream code is simply protected
> virtualization without a prefix, so I'd avoid creating a new denomination in
> libvirt.

Putting an "s390" prefix on this isn't inventing new terminology - it is
just making it obvious to users what target it applies to.

> Alternatively we could use the (unmodified) marketing name "IBM Secure
> Execution for Linux" here and below in the RST and reverse the "also known
> as" sentence in the overview.

That's even worse IMHO. Just put an s390 prefix on the current text.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [PATCH] docs: Describe protected virtualization guest setup

[Viktor Mihajlovski]

On 4/29/20 3:29 PM, Daniel P. Berrangé wrote:

On Tue, Apr 28, 2020 at 05:58:02PM +0200, Boris Fiuczynski wrote:

From: Viktor Mihajlovski 

Protected virtualization/IBM Secure Execution for Linux protects
guest memory and state from the host.

Add some basic information about technology and a brief guide
on setting up secure guests with libvirt.

Signed-off-by: Viktor Mihajlovski 
Reviewed-by: Boris Fiuczynski 
Reviewed-by: Paulo de Rezende Pinatti 
---
  docs/kbase.html.in  |   3 +
  docs/kbase/protected_virtualization.rst | 188 


I'd suggest calling this  s390_protected_virt.rst

We can do that.



diff --git a/docs/kbase.html.in b/docs/kbase.html.in
index c586e0f676..05a3239224 100644
--- a/docs/kbase.html.in
+++ b/docs/kbase.html.in
@@ -14,6 +14,9 @@
  Secure usage
  Secure usage of the libvirt APIs
  
+Protected virtualization


"s390 Protected virtualization"  as the title

The terminology that was used in the KVM upstream code is simply 
protected virtualization without a prefix, so I'd avoid creating a new 
denomination in libvirt.
Alternatively we could use the (unmodified) marketing name "IBM Secure 
Execution for Linux" here and below in the RST and reverse the "also 
known as" sentence in the overview.

+Running secure guests with IBM Secure Execution


s/secure guests/secure s390 guests/OK


+
  Launch security
  Securely launching VMs with AMD SEV
  
diff --git a/docs/kbase/protected_virtualization.rst b/docs/kbase/protected_virtualization.rst

new file mode 100644
index 00..48f2add14e
--- /dev/null
+++ b/docs/kbase/protected_virtualization.rst
@@ -0,0 +1,188 @@
+
+Protected Virtualization


s/^/s390/


see above

[...]
--
Kind Regards,
   Viktor

Re: [PATCH] docs: Describe protected virtualization guest setup

[Christian Borntraeger]

On 29.04.20 16:09, Christian Borntraeger wrote:
> 
> 
> On 29.04.20 15:25, Daniel P. Berrangé wrote:
>> On Wed, Apr 29, 2020 at 10:19:20AM -0300, Daniel Henrique Barboza wrote:
>>>
>>>
>>> On 4/28/20 12:58 PM, Boris Fiuczynski wrote:
 From: Viktor Mihajlovski 

>>>
>>> [...]
 +
 +If the check fails despite the host system actually supporting
 +protected virtualization guests, this can be caused by a stale
 +libvirt capabilities cache. To recover, run the following
 +commands
 +
 +::
 +
 +   $ systemctl stop libvirtd
 +   $ rm /var/cache/libvirt/qemu/capabilities/*.xml
 +   $ systemctl start libvirtd
 +
 +
>>>
>>>
>>> Why isn't Libvirt re-fetching the capabilities after host changes that 
>>> affects
>>> KVM capabilities? I see that we're following up QEMU timestamps to detect
>>> if the binary changes, which is sensible, but what about /dev/kvm? Shouldn't
>>> we refresh domain capabilities every time following a host reboot?
>>
>> Caching of capabilities was done precisely  to avoid refreshing on every boot
>> because it resulted in slow startup for apps using libvirt after boot.
> 
> Caching beyond the life time of /dev/kvm seems broken from a kernel 
> perspective.
> It is totally valid to load a new version of the kvm module with new 
> capabilities.
> 
> I am curious, Why did it take so long? we should - on normal system - only 
> refresh _one_ binary as most binaries are just TCG.
> 
> As Boris said, we are going to provide yet another check (besides the nested
> thing.
> 
> But in general I think invalidating the cache for that one and only binary
> that provides KVM after a change of /dev/kvm seems like the proper solution.
> 



Looking into that, I found a handler for the "nested" module parameter. Now: We 
also
have a hpage parameter that decides if you can use large pages or not in the 
host.
Do we need to check that as well or is that something that libvirt does not 
care 
about?
This parameter will go away at a later point in time though as soon as we have 
added
the code to have hpage and nested at the same time. 

So given the shakiness of these things, the time stamp of /dev/kvm really seems 
to be the best way to go forward long term. Would be really interesting to 
understand
the conditions where you saw the long startup times due to rescanning.

Re: [PATCH] docs: Describe protected virtualization guest setup

[Christian Borntraeger]

On 29.04.20 15:25, Daniel P. Berrangé wrote:
> On Wed, Apr 29, 2020 at 10:19:20AM -0300, Daniel Henrique Barboza wrote:
>>
>>
>> On 4/28/20 12:58 PM, Boris Fiuczynski wrote:
>>> From: Viktor Mihajlovski 
>>>
>>
>> [...]
>>> +
>>> +If the check fails despite the host system actually supporting
>>> +protected virtualization guests, this can be caused by a stale
>>> +libvirt capabilities cache. To recover, run the following
>>> +commands
>>> +
>>> +::
>>> +
>>> +   $ systemctl stop libvirtd
>>> +   $ rm /var/cache/libvirt/qemu/capabilities/*.xml
>>> +   $ systemctl start libvirtd
>>> +
>>> +
>>
>>
>> Why isn't Libvirt re-fetching the capabilities after host changes that 
>> affects
>> KVM capabilities? I see that we're following up QEMU timestamps to detect
>> if the binary changes, which is sensible, but what about /dev/kvm? Shouldn't
>> we refresh domain capabilities every time following a host reboot?
> 
> Caching of capabilities was done precisely  to avoid refreshing on every boot
> because it resulted in slow startup for apps using libvirt after boot.

Caching beyond the life time of /dev/kvm seems broken from a kernel perspective.
It is totally valid to load a new version of the kvm module with new 
capabilities.

I am curious, Why did it take so long? we should - on normal system - only 
refresh _one_ binary as most binaries are just TCG.

As Boris said, we are going to provide yet another check (besides the nested
thing.

But in general I think invalidating the cache for that one and only binary
that provides KVM after a change of /dev/kvm seems like the proper solution.


> 
> We look for specific features that change as a way to indicate a refresh
> is needed.  If there's a need to delete the capabilities manually that
> indicates we're missing some feature when deciding whether the cache is
> stale.

Re: [PATCH] docs: Describe protected virtualization guest setup

[Daniel P . Berrangé]

On Tue, Apr 28, 2020 at 05:58:02PM +0200, Boris Fiuczynski wrote:
> From: Viktor Mihajlovski 
> 
> Protected virtualization/IBM Secure Execution for Linux protects
> guest memory and state from the host.
> 
> Add some basic information about technology and a brief guide
> on setting up secure guests with libvirt.
> 
> Signed-off-by: Viktor Mihajlovski 
> Reviewed-by: Boris Fiuczynski 
> Reviewed-by: Paulo de Rezende Pinatti 
> ---
>  docs/kbase.html.in  |   3 +
>  docs/kbase/protected_virtualization.rst | 188 

I'd suggest calling this  s390_protected_virt.rst

> diff --git a/docs/kbase.html.in b/docs/kbase.html.in
> index c586e0f676..05a3239224 100644
> --- a/docs/kbase.html.in
> +++ b/docs/kbase.html.in
> @@ -14,6 +14,9 @@
>  Secure usage
>  Secure usage of the libvirt APIs
>  
> +Protected 
> virtualization

"s390 Protected virtualization"  as the title

> +Running secure guests with IBM Secure Execution

s/secure guests/secure s390 guests/

> +
>  Launch security
>  Securely launching VMs with AMD SEV
>  
> diff --git a/docs/kbase/protected_virtualization.rst 
> b/docs/kbase/protected_virtualization.rst
> new file mode 100644
> index 00..48f2add14e
> --- /dev/null
> +++ b/docs/kbase/protected_virtualization.rst
> @@ -0,0 +1,188 @@
> +
> +Protected Virtualization

s/^/s390/

> +

> +Host Requirements
> +=
> +
> +IBM Secure Execution for Linux has some hardware and firmware
> +requirements. The system hardware must be an IBM z15 (or newer),
> +or an IBM LinuxONE III (or newer).
> +
> +It is also necessary that the IBM Secure Execution feature is
> +enabled for that system. Check for facility '158', e.g.
> +
> +::
> +
> +   $ grep facilities /proc/cpuinfo | grep 158

I'd suggest that "virt-host-validate" be probing for this
so we can just tell them to run that command.

> +The kernel must include the protected virtualization support
> +which can be verified by checking for the presence of directory
> +``/sys/firmware/uv``. It will only be present when both the
> +hardware and the kernel support are available.
> +
> +Finally, the host operating system must donate some memory to
> +the ultravisor needed to store memory security information.
> +This is achieved by specifying the following kernel command
> +line parameter to the host boot configuration
> +
> +::
> +
> +   prot_virt=1
> +
> +
> +Guest Requirements
> +==
> +
> +Guest Boot
> +--
> +
> +To start a guest in protected virtualization secure mode, the
> +boot image must have been prepared first with the program
> +``genprotimg`` using the correct public key for this host.
> +``genprotimg`` is part of the package ``s390-tools``, or
> +``s390-utils``, depending on the Linux distribution being used.
> +It can also be found at
> +``_
> +
> +The guests have to be configured to use the host CPU model, which
> +must contain the ``unpack`` facility indicating ultravisor guest support.
> +
> +With the following command it's possible to check whether the host
> +CPU model satisfies the requirement
> +
> +::
> +
> +   $ virsh domcapabilities | grep unpack
> +
> +which should return
> +
> +::
> +
> +   
> +
> +If the check fails despite the host system actually supporting
> +protected virtualization guests, this can be caused by a stale
> +libvirt capabilities cache. To recover, run the following
> +commands
> +
> +::
> +
> +   $ systemctl stop libvirtd
> +   $ rm /var/cache/libvirt/qemu/capabilities/*.xml
> +   $ systemctl start libvirtd

If this is truly needed that it is a bug in libvirt - we're missing
something in the cache invalidation logic.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [PATCH] docs: Describe protected virtualization guest setup

[Boris Fiuczynski]

On 4/29/20 3:25 PM, Daniel P. Berrangé wrote:

On Wed, Apr 29, 2020 at 10:19:20AM -0300, Daniel Henrique Barboza wrote:



On 4/28/20 12:58 PM, Boris Fiuczynski wrote:

From: Viktor Mihajlovski 



[...]

+
+If the check fails despite the host system actually supporting
+protected virtualization guests, this can be caused by a stale
+libvirt capabilities cache. To recover, run the following
+commands
+
+::
+
+   $ systemctl stop libvirtd
+   $ rm /var/cache/libvirt/qemu/capabilities/*.xml
+   $ systemctl start libvirtd
+
+



Why isn't Libvirt re-fetching the capabilities after host changes that affects
KVM capabilities? I see that we're following up QEMU timestamps to detect
if the binary changes, which is sensible, but what about /dev/kvm? Shouldn't
we refresh domain capabilities every time following a host reboot?


Caching of capabilities was done precisely  to avoid refreshing on every boot
because it resulted in slow startup for apps using libvirt after boot.

We look for specific features that change as a way to indicate a refresh
is needed.  If there's a need to delete the capabilities manually that
indicates we're missing some feature when deciding whether the cache is
stale.

Regards,
Daniel



Daniel's,
we will provide a patch serie proposing code for such caps cache 
invalidation triggers for IBM Secure Execution as well as for AMD SEV.

Afterwards we can change the documentation as well.

--
Mit freundlichen Grüßen/Kind regards
   Boris Fiuczynski

IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Gregor Pillen
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

Re: [PATCH] docs: Describe protected virtualization guest setup

[Daniel P . Berrangé]

On Wed, Apr 29, 2020 at 10:19:20AM -0300, Daniel Henrique Barboza wrote:
> 
> 
> On 4/28/20 12:58 PM, Boris Fiuczynski wrote:
> > From: Viktor Mihajlovski 
> > 
> 
> [...]
> > +
> > +If the check fails despite the host system actually supporting
> > +protected virtualization guests, this can be caused by a stale
> > +libvirt capabilities cache. To recover, run the following
> > +commands
> > +
> > +::
> > +
> > +   $ systemctl stop libvirtd
> > +   $ rm /var/cache/libvirt/qemu/capabilities/*.xml
> > +   $ systemctl start libvirtd
> > +
> > +
> 
> 
> Why isn't Libvirt re-fetching the capabilities after host changes that affects
> KVM capabilities? I see that we're following up QEMU timestamps to detect
> if the binary changes, which is sensible, but what about /dev/kvm? Shouldn't
> we refresh domain capabilities every time following a host reboot?

Caching of capabilities was done precisely  to avoid refreshing on every boot
because it resulted in slow startup for apps using libvirt after boot.

We look for specific features that change as a way to indicate a refresh
is needed.  If there's a need to delete the capabilities manually that
indicates we're missing some feature when deciding whether the cache is
stale.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [PATCH] docs: Describe protected virtualization guest setup

[Daniel Henrique Barboza]

On 4/28/20 12:58 PM, Boris Fiuczynski wrote:

From: Viktor Mihajlovski 



[...]

+
+If the check fails despite the host system actually supporting
+protected virtualization guests, this can be caused by a stale
+libvirt capabilities cache. To recover, run the following
+commands
+
+::
+
+   $ systemctl stop libvirtd
+   $ rm /var/cache/libvirt/qemu/capabilities/*.xml
+   $ systemctl start libvirtd
+
+



Why isn't Libvirt re-fetching the capabilities after host changes that affects
KVM capabilities? I see that we're following up QEMU timestamps to detect
if the binary changes, which is sensible, but what about /dev/kvm? Shouldn't
we refresh domain capabilities every time following a host reboot?

IMHO this is a discussion worth having before making this sort of workaround
an official part of the feature.



Thanks,


DHB

Re: [PATCH] docs: Describe protected virtualization guest setup

[Viktor Mihajlovski]

On 4/28/20 5:58 PM, Boris Fiuczynski wrote:

From: Viktor Mihajlovski 

Protected virtualization/IBM Secure Execution for Linux protects
guest memory and state from the host.

Add some basic information about technology and a brief guide
on setting up secure guests with libvirt.

Signed-off-by: Viktor Mihajlovski 
Reviewed-by: Boris Fiuczynski 

   ^-- Should be changed to s-o-b before applying.

Reviewed-by: Paulo de Rezende Pinatti 
---
  docs/kbase.html.in  |   3 +
  docs/kbase/protected_virtualization.rst | 188 
  2 files changed, 191 insertions(+)
  create mode 100644 docs/kbase/protected_virtualization.rst

[...]
--
Kind Regards,
   Viktor

[PATCH] docs: Describe protected virtualization guest setup

[Boris Fiuczynski]

From: Viktor Mihajlovski 

Protected virtualization/IBM Secure Execution for Linux protects
guest memory and state from the host.

Add some basic information about technology and a brief guide
on setting up secure guests with libvirt.

Signed-off-by: Viktor Mihajlovski 
Reviewed-by: Boris Fiuczynski 
Reviewed-by: Paulo de Rezende Pinatti 
---
 docs/kbase.html.in  |   3 +
 docs/kbase/protected_virtualization.rst | 188 
 2 files changed, 191 insertions(+)
 create mode 100644 docs/kbase/protected_virtualization.rst

diff --git a/docs/kbase.html.in b/docs/kbase.html.in
index c586e0f676..05a3239224 100644
--- a/docs/kbase.html.in
+++ b/docs/kbase.html.in
@@ -14,6 +14,9 @@
 Secure usage
 Secure usage of the libvirt APIs
 
+Protected 
virtualization
+Running secure guests with IBM Secure Execution
+
 Launch security
 Securely launching VMs with AMD SEV
 
diff --git a/docs/kbase/protected_virtualization.rst 
b/docs/kbase/protected_virtualization.rst
new file mode 100644
index 00..48f2add14e
--- /dev/null
+++ b/docs/kbase/protected_virtualization.rst
@@ -0,0 +1,188 @@
+
+Protected Virtualization
+
+
+.. contents::
+
+Overview
+
+
+Protected virtualization, also known as IBM Secure Execution is a
+hardware-based privacy protection technology for s390x (IBM Z).
+It allows to execute virtual machines such that the host system
+has no access to a VM's state and memory contents.
+
+Unlike other similar technologies, the memory of a running guest
+is not encrypted but protected by hardware access controls, which
+may only be manipulated by trusted system firmware, called
+ultravisor.
+
+For the cases where the host needs access to guest memory (e.g. for
+paging), it can request pages to be exported to it. The exported page
+will be encrypted with a unique key for the running guest by the
+ultravisor. The ultravisor also computes an integrity value for
+the page, and stores it in a special table, together with the page
+index and a counter. This way it can verify the integrity of
+the page content upon re-import into the guest.
+
+In other cases it may be necessary for a guest to grant the host access
+to dedicated memory regions (e.g. for I/O). The guest can request
+that the ultravisor removes the memory protection from individual
+pages, so that they can be shared with the host. Likewise, the
+guest can undo the sharing.
+
+A secure guest will initially start in a regular non-protected VM.
+The start-up is controlled by a small bootstrap program loaded
+into memory together with encrypted operating system components and
+a control structure (the PV header).
+The operating system components (e.g. Linux kernel, initial RAM
+file system, kernel parameters) are encrypted and integrity
+protected. The component encryption keys and integrity values are
+stored in the PV header.
+The PV header is wrapped with a public key belonging to a specific
+system (in fact it can be wrapped with multiple such keys). The
+matching private key is only accessible by trusted hardware and
+firmware in that specific system.
+Consequently, such a secure guest boot image can only be run on the
+systems it has been prepared for. Its contents can't be decrypted
+without access to the private key and it can't be modified as
+it is integrity protected.
+
+Host Requirements
+=
+
+IBM Secure Execution for Linux has some hardware and firmware
+requirements. The system hardware must be an IBM z15 (or newer),
+or an IBM LinuxONE III (or newer).
+
+It is also necessary that the IBM Secure Execution feature is
+enabled for that system. Check for facility '158', e.g.
+
+::
+
+   $ grep facilities /proc/cpuinfo | grep 158
+
+The kernel must include the protected virtualization support
+which can be verified by checking for the presence of directory
+``/sys/firmware/uv``. It will only be present when both the
+hardware and the kernel support are available.
+
+Finally, the host operating system must donate some memory to
+the ultravisor needed to store memory security information.
+This is achieved by specifying the following kernel command
+line parameter to the host boot configuration
+
+::
+
+   prot_virt=1
+
+
+Guest Requirements
+==
+
+Guest Boot
+--
+
+To start a guest in protected virtualization secure mode, the
+boot image must have been prepared first with the program
+``genprotimg`` using the correct public key for this host.
+``genprotimg`` is part of the package ``s390-tools``, or
+``s390-utils``, depending on the Linux distribution being used.
+It can also be found at
+``_
+
+The guests have to be configured to use the host CPU model, which
+must contain the ``unpack`` facility indicating ultravisor guest support.
+
+With the following command it's possible to check whether the host
+CP