Re: [Qemu-devel] [PATCH 03/11] spapr: add option vector handling in CAS-generated resets

[David Gibson]

On Wed, Oct 12, 2016 at 06:13:51PM -0500, Michael Roth wrote:
> In some cases, ibm,client-architecture-support calls can fail. This
> could happen in the current code for situations where the modified
> device tree segment exceeds the buffer size provided by the guest
> via the call parameters. In these cases, QEMU will reset, allowing
> an opportunity to regenerate the device tree from scratch via
> boot-time handling. There are potentially other scenarios as well,
> not currently reachable in the current code, but possible in theory,
> such as cases where device-tree properties or nodes need to be removed.
> 
> We currently don't handle either of these properly for option vector
> capabilities however. Instead of carrying the negotiated capability
> beyond the reset and creating the boot-time device tree accordingly,
> we start from scratch, generating the same boot-time device tree as we
> did prior to the CAS-generated and the same device tree updates as we
> did before. This could (in theory) cause us to get stuck in a reset
> loop. This hasn't been observed, but depending on the extensiveness
> of CAS-induced device tree updates in the future, could eventually
> become an issue.
> 
> Address this by pulling capability-related device tree
> updates resulting from CAS calls into a common routine,
> spapr_populate_cas_updates(), and adding an sPAPROptionVector*
> parameter that allows us to test for newly-negotiated capabilities.
> We invoke it as follows:
> 
> 1) When ibm,client-architecture-support gets called, we
>call spapr_populate_cas_updates() with the set of capabilities
>added since the previous call to ibm,client-architecture-support.
>For the initial boot, or a system reset generated by something
>other than the CAS call itself, this set will consist of *all*
>options supported both the platform and the guest. For calls
>to ibm,client-architecture-support immediately after a CAS-induced
>reset, we call spapr_populate_cas_updates() with only the set
>of capabilities added since the previous call, since the other
>capabilities will have already been addressed by the boot-time
>device-tree this time around. In the unlikely event that
>capabilities are *removed* since the previous CAS, we will
>generate a CAS-induced reset. In the unlikely event that we
>cannot fit the device-tree updates into the buffer provided
>by the guest, well generate a CAS-induced reset.
> 
> 2) When a CAS update results in the need to reset the machine and
>include the updates in the boot-time device tree, we call the
>spapr_populate_cas_updates() using the full set of negotiated
>capabilities as part of the reset path. At initial boot, or after
>a reset generated by something other than the CAS call itself,
>this set will be empty, resulting in what should be the same
>boot-time device-tree as we generated prior to this patch. For
>CAS-induced reset, this routine will be called with the full set of
>capabilities negotiated by the platform/guest in the previous
>CAS call, which should result in CAS updates from previous call
>being accounted for in the initial boot-time device tree.
> 
> Signed-off-by: Michael Roth 

Reviewed-by: David Gibson 

I suspect HPT resizing is also going to need actual CAS reboots
(rather than just adjusting the DT), so it's handy you've implemented
that here.

> ---
>  hw/ppc/spapr.c | 43 ++-
>  hw/ppc/spapr_hcall.c   | 22 ++
>  include/hw/ppc/spapr.h |  4 +++-
>  3 files changed, 55 insertions(+), 14 deletions(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 934d6b2..460c7a8 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -854,13 +854,28 @@ out:
>  return ret;
>  }
>  
> +static int spapr_populate_cas_updates(sPAPRMachineState *spapr, void *fdt,
> +  sPAPROptionVector *ov5_updates)
> +{
> +sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(spapr);
> +int ret = 0;
> +
> +/* Generate ibm,dynamic-reconfiguration-memory node if required */
> +if (spapr_ovec_test(ov5_updates, OV5_DRCONF_MEMORY)) {
> +g_assert(smc->dr_lmb_enabled);
> +ret = spapr_populate_drconf_memory(spapr, fdt);
> +}
> +
> +return ret;
> +}
> +
>  int spapr_h_cas_compose_response(sPAPRMachineState *spapr,
>   target_ulong addr, target_ulong size,
> - bool cpu_update)
> + bool cpu_update,
> + sPAPROptionVector *ov5_updates)
>  {
>  void *fdt, *fdt_skel;
>  sPAPRDeviceTreeUpdateHeader hdr = { .version_id = 1 };
> -sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
>  
>  size -= sizeof(hdr);
>  
> @@ -879,11 +894,7 @@ int 

[Qemu-devel] [PATCH 03/11] spapr: add option vector handling in CAS-generated resets

[Michael Roth]

In some cases, ibm,client-architecture-support calls can fail. This
could happen in the current code for situations where the modified
device tree segment exceeds the buffer size provided by the guest
via the call parameters. In these cases, QEMU will reset, allowing
an opportunity to regenerate the device tree from scratch via
boot-time handling. There are potentially other scenarios as well,
not currently reachable in the current code, but possible in theory,
such as cases where device-tree properties or nodes need to be removed.

We currently don't handle either of these properly for option vector
capabilities however. Instead of carrying the negotiated capability
beyond the reset and creating the boot-time device tree accordingly,
we start from scratch, generating the same boot-time device tree as we
did prior to the CAS-generated and the same device tree updates as we
did before. This could (in theory) cause us to get stuck in a reset
loop. This hasn't been observed, but depending on the extensiveness
of CAS-induced device tree updates in the future, could eventually
become an issue.

Address this by pulling capability-related device tree
updates resulting from CAS calls into a common routine,
spapr_populate_cas_updates(), and adding an sPAPROptionVector*
parameter that allows us to test for newly-negotiated capabilities.
We invoke it as follows:

1) When ibm,client-architecture-support gets called, we
   call spapr_populate_cas_updates() with the set of capabilities
   added since the previous call to ibm,client-architecture-support.
   For the initial boot, or a system reset generated by something
   other than the CAS call itself, this set will consist of *all*
   options supported both the platform and the guest. For calls
   to ibm,client-architecture-support immediately after a CAS-induced
   reset, we call spapr_populate_cas_updates() with only the set
   of capabilities added since the previous call, since the other
   capabilities will have already been addressed by the boot-time
   device-tree this time around. In the unlikely event that
   capabilities are *removed* since the previous CAS, we will
   generate a CAS-induced reset. In the unlikely event that we
   cannot fit the device-tree updates into the buffer provided
   by the guest, well generate a CAS-induced reset.

2) When a CAS update results in the need to reset the machine and
   include the updates in the boot-time device tree, we call the
   spapr_populate_cas_updates() using the full set of negotiated
   capabilities as part of the reset path. At initial boot, or after
   a reset generated by something other than the CAS call itself,
   this set will be empty, resulting in what should be the same
   boot-time device-tree as we generated prior to this patch. For
   CAS-induced reset, this routine will be called with the full set of
   capabilities negotiated by the platform/guest in the previous
   CAS call, which should result in CAS updates from previous call
   being accounted for in the initial boot-time device tree.

Signed-off-by: Michael Roth 
---
 hw/ppc/spapr.c | 43 ++-
 hw/ppc/spapr_hcall.c   | 22 ++
 include/hw/ppc/spapr.h |  4 +++-
 3 files changed, 55 insertions(+), 14 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 934d6b2..460c7a8 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -854,13 +854,28 @@ out:
 return ret;
 }
 
+static int spapr_populate_cas_updates(sPAPRMachineState *spapr, void *fdt,
+  sPAPROptionVector *ov5_updates)
+{
+sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(spapr);
+int ret = 0;
+
+/* Generate ibm,dynamic-reconfiguration-memory node if required */
+if (spapr_ovec_test(ov5_updates, OV5_DRCONF_MEMORY)) {
+g_assert(smc->dr_lmb_enabled);
+ret = spapr_populate_drconf_memory(spapr, fdt);
+}
+
+return ret;
+}
+
 int spapr_h_cas_compose_response(sPAPRMachineState *spapr,
  target_ulong addr, target_ulong size,
- bool cpu_update)
+ bool cpu_update,
+ sPAPROptionVector *ov5_updates)
 {
 void *fdt, *fdt_skel;
 sPAPRDeviceTreeUpdateHeader hdr = { .version_id = 1 };
-sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(qdev_get_machine());
 
 size -= sizeof(hdr);
 
@@ -879,11 +894,7 @@ int spapr_h_cas_compose_response(sPAPRMachineState *spapr,
 _FDT((spapr_fixup_cpu_dt(fdt, spapr)));
 }
 
-/* Generate ibm,dynamic-reconfiguration-memory node if required */
-if (spapr_ovec_test(spapr->ov5_cas, OV5_DRCONF_MEMORY)) {
-g_assert(smc->dr_lmb_enabled);
-_FDT((spapr_populate_drconf_memory(spapr, fdt)));
-}
+spapr_populate_cas_updates(spapr, fdt, ov5_updates);
 
 /* Pack resulting tree */
 _FDT((fdt_pack(fdt)));
@@ -904,7 +915,8 @@