Re: [PATCH v5 0/4] introduction of migration_version attribute for VFIO live migration

[Yan Zhao]

On Tue, Jun 02, 2020 at 09:55:28PM -0600, Alex Williamson wrote:
> On Tue, 2 Jun 2020 23:19:48 -0400
> Yan Zhao  wrote:
> 
> > On Tue, Jun 02, 2020 at 04:55:27PM -0600, Alex Williamson wrote:
> > > On Wed, 29 Apr 2020 20:39:50 -0400
> > > Yan Zhao  wrote:
> > >   
> > > > On Wed, Apr 29, 2020 at 05:48:44PM +0800, Dr. David Alan Gilbert wrote:
> > > >   
> > > > > > > > > > > > > > > > > An mdev type is meant to define a software 
> > > > > > > > > > > > > > > > > compatible interface, so in
> > > > > > > > > > > > > > > > > the case of mdev->mdev migration, doesn't 
> > > > > > > > > > > > > > > > > migrating to a different type
> > > > > > > > > > > > > > > > > fail the most basic of compatibility tests 
> > > > > > > > > > > > > > > > > that we expect userspace to
> > > > > > > > > > > > > > > > > perform?  IOW, if two mdev types are 
> > > > > > > > > > > > > > > > > migration compatible, it seems a
> > > > > > > > > > > > > > > > > prerequisite to that is that they provide the 
> > > > > > > > > > > > > > > > > same software interface,
> > > > > > > > > > > > > > > > > which means they should be the same mdev type.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > In the hybrid cases of mdev->phys or 
> > > > > > > > > > > > > > > > > phys->mdev, how does a
> > > > > > > > > > > > > > > > management
> > > > > > > > > > > > > > > > > tool begin to even guess what might be 
> > > > > > > > > > > > > > > > > compatible?  Are we expecting
> > > > > > > > > > > > > > > > > libvirt to probe ever device with this 
> > > > > > > > > > > > > > > > > attribute in the system?  Is
> > > > > > > > > > > > > > > > > there going to be a new class hierarchy 
> > > > > > > > > > > > > > > > > created to enumerate all
> > > > > > > > > > > > > > > > > possible migrate-able devices?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > yes, management tool needs to guess and test 
> > > > > > > > > > > > > > > > migration compatible
> > > > > > > > > > > > > > > > between two devices. But I think it's not the 
> > > > > > > > > > > > > > > > problem only for
> > > > > > > > > > > > > > > > mdev->phys or phys->mdev. even for mdev->mdev, 
> > > > > > > > > > > > > > > > management tool needs
> > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > first assume that the two mdevs have the same 
> > > > > > > > > > > > > > > > type of parent devices
> > > > > > > > > > > > > > > > (e.g.their pciids are equal). otherwise, it's 
> > > > > > > > > > > > > > > > still enumerating
> > > > > > > > > > > > > > > > possibilities.
> > > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > on the other hand, for two mdevs,
> > > > > > > > > > > > > > > > mdev1 from pdev1, its mdev_type is 1/2 of pdev1;
> > > > > > > > > > > > > > > > mdev2 from pdev2, its mdev_type is 1/4 of pdev2;
> > > > > > > > > > > > > > > > if pdev2 is exactly 2 times of pdev1, why not 
> > > > > > > > > > > > > > > > allow migration between
> > > > > > > > > > > > > > > > mdev1 <-> mdev2.
> > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > How could the manage tool figure out that 1/2 of 
> > > > > > > > > > > > > > > pdev1 is equivalent 
> > > > > > > > > > > > > > > to 1/4 of pdev2? If we really want to allow such 
> > > > > > > > > > > > > > > thing happen, the best
> > > > > > > > > > > > > > > choice is to report the same mdev type on both 
> > > > > > > > > > > > > > > pdev1 and pdev2.
> > > > > > > > > > > > > > I think that's exactly the value of this 
> > > > > > > > > > > > > > migration_version interface.
> > > > > > > > > > > > > > the management tool can take advantage of this 
> > > > > > > > > > > > > > interface to know if two
> > > > > > > > > > > > > > devices are migration compatible, no matter they 
> > > > > > > > > > > > > > are mdevs, non-mdevs,
> > > > > > > > > > > > > > or mix.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > as I know, (please correct me if not right), 
> > > > > > > > > > > > > > current libvirt still
> > > > > > > > > > > > > > requires manually generating mdev devices, and it 
> > > > > > > > > > > > > > just duplicates src vm
> > > > > > > > > > > > > > configuration to the target vm.
> > > > > > > > > > > > > > for libvirt, currently it's always phys->phys and 
> > > > > > > > > > > > > > mdev->mdev (and of the
> > > > > > > > > > > > > > same mdev type).
> > > > > > > > > > > > > > But it does not justify that hybrid cases should 
> > > > > > > > > > > > > > not be allowed. otherwise,
> > > > > > > > > > > > > > why do we need to introduce this migration_version 
> > > > > > > > > > > > > > interface and leave
> > > > > > > > > > > > > > the judgement of migration compatibility to vendor 
> > > > > > > > > > > > > > driver? why not simply
> > > > > > > > > > > > > > set the criteria to something like "pciids of 
> > > > > > > > > > > > > > parent devices are equal,
> > > > > > > > > > > > > > and mdev types are equal" ?
> > > > > 

Re: [PATCH] libxl: Normalize MAC address in device conf when hotplugging a netdev

[Laine Stump]

On 5/28/20 1:09 PM, Jim Fehlig wrote:

Similar to commit 6c17606b7cc, normalize the MAC addresses in persistent
and live device config to avoid a different MAC address for the device
once the VM is rebooted and persistent config takes affect.



Well...


This has a bigger change than commit 6c17606b7cc! :-)


For those who don't feel like digging back through the git blame 
history, commit 6c17606b7cc just added a call to 
qemuDomainAttachDeviceLiveAndConfigHomogenize() in the QEMU driver 
version of AttachDeviceFlags() to make the MAC addresses of the 
*already-existing* "live" and "config" copies of the device object match 
each other - qemu's function had previously been changed back in commit 
55ce6564 to create separate copies of the device object for config and 
live (although it still had to save an extra pointer to the config copy, 
which was being consumed by the the qemuDomainAttachDeviceConfig(), with 
the pointer NULLed



The libxl driver hadn't gotten that change though, so up until now it's 
only had a single device object pointer, and although it separately 
parsed the XML for live and config, this was done sequentially into the 
same object, so the two never existed at the same time. Since both 
objects must exist at the same time to copy anything from one to the 
other (without "extra steps"), this new patch is effectively the libxl 
equivalent of commit 55ce6564 and commit 6c17606b7cc put together.


That was a long prelude to "maybe mention commit 55ce6564 in the commit 
log as well" :-)





Signed-off-by: Jim Fehlig 
---
  src/libxl/libxl_driver.c | 56 +---
  1 file changed, 46 insertions(+), 10 deletions(-)

diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c
index 63ec0a2188..a80bc3fe3a 100644
--- a/src/libxl/libxl_driver.c
+++ b/src/libxl/libxl_driver.c
@@ -4096,6 +4096,31 @@ libxlDomainUpdateDeviceConfig(virDomainDefPtr vmdef, 
virDomainDeviceDefPtr dev)
  }
  
  
+static void

+libxlDomainAttachDeviceNormalize(const virDomainDeviceDef *devConf,



What? You didn't like my verb in the QEMU version? :-)


(I can't remember why I picked Homogenize instead of Normalize, but I 
must have had some reason. Maybe I was just thinking back to the dairy 
farm next door to my childhood home (true story!))



At any rate, it all looks good


Reviewed-by: Laine Stump 



+ virDomainDeviceDefPtr devLive)
+{
+/*
+ * Fixup anything that needs to be identical in the live and
+ * config versions of DeviceDef, but might not be. Do this by
+ * changing the contents of devLive. This is done after all
+ * post-parse tweaks and validation, so be very careful about what
+ * changes are made.
+ */
+
+/* MAC address should be identical in both DeviceDefs, but if it
+ * wasn't specified in the XML, and was instead autogenerated, it
+ * will be different for the two since they are each the result of
+ * a separate parser call. If it *was* specified, it will already
+ * be the same, so copying does no harm.
+ */
+
+if (devConf->type == VIR_DOMAIN_DEVICE_NET)
+virMacAddrSet(>data.net->mac, >data.net->mac);
+
+}
+
+
  static int
  libxlDomainAttachDeviceFlags(virDomainPtr dom, const char *xml,
   unsigned int flags)
@@ -4104,7 +4129,9 @@ libxlDomainAttachDeviceFlags(virDomainPtr dom, const char 
*xml,
  libxlDriverConfigPtr cfg = libxlDriverConfigGet(driver);
  virDomainObjPtr vm = NULL;
  virDomainDefPtr vmdef = NULL;
-virDomainDeviceDefPtr dev = NULL;
+virDomainDeviceDefPtr devConf = NULL;
+virDomainDeviceDef devConfSave = { 0 };
+virDomainDeviceDefPtr devLive = NULL;
  int ret = -1;
  
  virCheckFlags(VIR_DOMAIN_DEVICE_MODIFY_LIVE |

@@ -4123,28 +4150,36 @@ libxlDomainAttachDeviceFlags(virDomainPtr dom, const 
char *xml,
  goto endjob;
  
  if (flags & VIR_DOMAIN_DEVICE_MODIFY_CONFIG) {

-if (!(dev = virDomainDeviceDefParse(xml, vm->def,
-driver->xmlopt, NULL,
-VIR_DOMAIN_DEF_PARSE_INACTIVE)))
+if (!(devConf = virDomainDeviceDefParse(xml, vm->def,
+driver->xmlopt, NULL,
+
VIR_DOMAIN_DEF_PARSE_INACTIVE)))
  goto endjob;
  
  /* Make a copy for updated domain. */

  if (!(vmdef = virDomainObjCopyPersistentDef(vm, driver->xmlopt, 
NULL)))
  goto endjob;
  
-if (libxlDomainAttachDeviceConfig(vmdef, dev) < 0)

+/*
+ * devConf will be NULLed out by
+ * libxlDomainAttachDeviceConfig(), so save it for later use by
+ * libxlDomainAttachDeviceNormalize()
+ */
+devConfSave = *devConf;
+
+if (libxlDomainAttachDeviceConfig(vmdef, devConf) < 0)
  goto endjob;
  }
  
  if (flags & 

Re: [PATCH v5 0/4] introduction of migration_version attribute for VFIO live migration

[Alex Williamson]

On Tue, 2 Jun 2020 23:19:48 -0400
Yan Zhao  wrote:

> On Tue, Jun 02, 2020 at 04:55:27PM -0600, Alex Williamson wrote:
> > On Wed, 29 Apr 2020 20:39:50 -0400
> > Yan Zhao  wrote:
> >   
> > > On Wed, Apr 29, 2020 at 05:48:44PM +0800, Dr. David Alan Gilbert wrote:
> > >   
> > > > > > > > > > > > > > > > An mdev type is meant to define a software 
> > > > > > > > > > > > > > > > compatible interface, so in
> > > > > > > > > > > > > > > > the case of mdev->mdev migration, doesn't 
> > > > > > > > > > > > > > > > migrating to a different type
> > > > > > > > > > > > > > > > fail the most basic of compatibility tests that 
> > > > > > > > > > > > > > > > we expect userspace to
> > > > > > > > > > > > > > > > perform?  IOW, if two mdev types are migration 
> > > > > > > > > > > > > > > > compatible, it seems a
> > > > > > > > > > > > > > > > prerequisite to that is that they provide the 
> > > > > > > > > > > > > > > > same software interface,
> > > > > > > > > > > > > > > > which means they should be the same mdev type.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > In the hybrid cases of mdev->phys or 
> > > > > > > > > > > > > > > > phys->mdev, how does a
> > > > > > > > > > > > > > > management
> > > > > > > > > > > > > > > > tool begin to even guess what might be 
> > > > > > > > > > > > > > > > compatible?  Are we expecting
> > > > > > > > > > > > > > > > libvirt to probe ever device with this 
> > > > > > > > > > > > > > > > attribute in the system?  Is
> > > > > > > > > > > > > > > > there going to be a new class hierarchy created 
> > > > > > > > > > > > > > > > to enumerate all
> > > > > > > > > > > > > > > > possible migrate-able devices?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > yes, management tool needs to guess and test 
> > > > > > > > > > > > > > > migration compatible
> > > > > > > > > > > > > > > between two devices. But I think it's not the 
> > > > > > > > > > > > > > > problem only for
> > > > > > > > > > > > > > > mdev->phys or phys->mdev. even for mdev->mdev, 
> > > > > > > > > > > > > > > management tool needs
> > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > first assume that the two mdevs have the same 
> > > > > > > > > > > > > > > type of parent devices
> > > > > > > > > > > > > > > (e.g.their pciids are equal). otherwise, it's 
> > > > > > > > > > > > > > > still enumerating
> > > > > > > > > > > > > > > possibilities.
> > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > on the other hand, for two mdevs,
> > > > > > > > > > > > > > > mdev1 from pdev1, its mdev_type is 1/2 of pdev1;
> > > > > > > > > > > > > > > mdev2 from pdev2, its mdev_type is 1/4 of pdev2;
> > > > > > > > > > > > > > > if pdev2 is exactly 2 times of pdev1, why not 
> > > > > > > > > > > > > > > allow migration between
> > > > > > > > > > > > > > > mdev1 <-> mdev2.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > How could the manage tool figure out that 1/2 of 
> > > > > > > > > > > > > > pdev1 is equivalent 
> > > > > > > > > > > > > > to 1/4 of pdev2? If we really want to allow such 
> > > > > > > > > > > > > > thing happen, the best
> > > > > > > > > > > > > > choice is to report the same mdev type on both 
> > > > > > > > > > > > > > pdev1 and pdev2.
> > > > > > > > > > > > > I think that's exactly the value of this 
> > > > > > > > > > > > > migration_version interface.
> > > > > > > > > > > > > the management tool can take advantage of this 
> > > > > > > > > > > > > interface to know if two
> > > > > > > > > > > > > devices are migration compatible, no matter they are 
> > > > > > > > > > > > > mdevs, non-mdevs,
> > > > > > > > > > > > > or mix.
> > > > > > > > > > > > > 
> > > > > > > > > > > > > as I know, (please correct me if not right), current 
> > > > > > > > > > > > > libvirt still
> > > > > > > > > > > > > requires manually generating mdev devices, and it 
> > > > > > > > > > > > > just duplicates src vm
> > > > > > > > > > > > > configuration to the target vm.
> > > > > > > > > > > > > for libvirt, currently it's always phys->phys and 
> > > > > > > > > > > > > mdev->mdev (and of the
> > > > > > > > > > > > > same mdev type).
> > > > > > > > > > > > > But it does not justify that hybrid cases should not 
> > > > > > > > > > > > > be allowed. otherwise,
> > > > > > > > > > > > > why do we need to introduce this migration_version 
> > > > > > > > > > > > > interface and leave
> > > > > > > > > > > > > the judgement of migration compatibility to vendor 
> > > > > > > > > > > > > driver? why not simply
> > > > > > > > > > > > > set the criteria to something like "pciids of parent 
> > > > > > > > > > > > > devices are equal,
> > > > > > > > > > > > > and mdev types are equal" ?
> > > > > > > > > > > > > 
> > > > > > > > > > > > > 
> > > > > > > > > > > > > > btw mdev<->phys just brings trouble to upper stack 
> > > > > > > > > > > > > > as Alex pointed out. 
> > > > > > > > > > > > > could you help me 

Re: [PATCH v5 0/4] introduction of migration_version attribute for VFIO live migration

[Yan Zhao]

On Tue, Jun 02, 2020 at 04:55:27PM -0600, Alex Williamson wrote:
> On Wed, 29 Apr 2020 20:39:50 -0400
> Yan Zhao  wrote:
> 
> > On Wed, Apr 29, 2020 at 05:48:44PM +0800, Dr. David Alan Gilbert wrote:
> > 
> > > > > > > > > > > > > > > An mdev type is meant to define a software 
> > > > > > > > > > > > > > > compatible interface, so in
> > > > > > > > > > > > > > > the case of mdev->mdev migration, doesn't 
> > > > > > > > > > > > > > > migrating to a different type
> > > > > > > > > > > > > > > fail the most basic of compatibility tests that 
> > > > > > > > > > > > > > > we expect userspace to
> > > > > > > > > > > > > > > perform?  IOW, if two mdev types are migration 
> > > > > > > > > > > > > > > compatible, it seems a
> > > > > > > > > > > > > > > prerequisite to that is that they provide the 
> > > > > > > > > > > > > > > same software interface,
> > > > > > > > > > > > > > > which means they should be the same mdev type.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > In the hybrid cases of mdev->phys or phys->mdev, 
> > > > > > > > > > > > > > > how does a  
> > > > > > > > > > > > > > management  
> > > > > > > > > > > > > > > tool begin to even guess what might be 
> > > > > > > > > > > > > > > compatible?  Are we expecting
> > > > > > > > > > > > > > > libvirt to probe ever device with this attribute 
> > > > > > > > > > > > > > > in the system?  Is
> > > > > > > > > > > > > > > there going to be a new class hierarchy created 
> > > > > > > > > > > > > > > to enumerate all
> > > > > > > > > > > > > > > possible migrate-able devices?
> > > > > > > > > > > > > > >  
> > > > > > > > > > > > > > yes, management tool needs to guess and test 
> > > > > > > > > > > > > > migration compatible
> > > > > > > > > > > > > > between two devices. But I think it's not the 
> > > > > > > > > > > > > > problem only for
> > > > > > > > > > > > > > mdev->phys or phys->mdev. even for mdev->mdev, 
> > > > > > > > > > > > > > management tool needs
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > first assume that the two mdevs have the same type 
> > > > > > > > > > > > > > of parent devices
> > > > > > > > > > > > > > (e.g.their pciids are equal). otherwise, it's still 
> > > > > > > > > > > > > > enumerating
> > > > > > > > > > > > > > possibilities.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > on the other hand, for two mdevs,
> > > > > > > > > > > > > > mdev1 from pdev1, its mdev_type is 1/2 of pdev1;
> > > > > > > > > > > > > > mdev2 from pdev2, its mdev_type is 1/4 of pdev2;
> > > > > > > > > > > > > > if pdev2 is exactly 2 times of pdev1, why not allow 
> > > > > > > > > > > > > > migration between
> > > > > > > > > > > > > > mdev1 <-> mdev2.  
> > > > > > > > > > > > > 
> > > > > > > > > > > > > How could the manage tool figure out that 1/2 of 
> > > > > > > > > > > > > pdev1 is equivalent 
> > > > > > > > > > > > > to 1/4 of pdev2? If we really want to allow such 
> > > > > > > > > > > > > thing happen, the best
> > > > > > > > > > > > > choice is to report the same mdev type on both pdev1 
> > > > > > > > > > > > > and pdev2.  
> > > > > > > > > > > > I think that's exactly the value of this 
> > > > > > > > > > > > migration_version interface.
> > > > > > > > > > > > the management tool can take advantage of this 
> > > > > > > > > > > > interface to know if two
> > > > > > > > > > > > devices are migration compatible, no matter they are 
> > > > > > > > > > > > mdevs, non-mdevs,
> > > > > > > > > > > > or mix.
> > > > > > > > > > > > 
> > > > > > > > > > > > as I know, (please correct me if not right), current 
> > > > > > > > > > > > libvirt still
> > > > > > > > > > > > requires manually generating mdev devices, and it just 
> > > > > > > > > > > > duplicates src vm
> > > > > > > > > > > > configuration to the target vm.
> > > > > > > > > > > > for libvirt, currently it's always phys->phys and 
> > > > > > > > > > > > mdev->mdev (and of the
> > > > > > > > > > > > same mdev type).
> > > > > > > > > > > > But it does not justify that hybrid cases should not be 
> > > > > > > > > > > > allowed. otherwise,
> > > > > > > > > > > > why do we need to introduce this migration_version 
> > > > > > > > > > > > interface and leave
> > > > > > > > > > > > the judgement of migration compatibility to vendor 
> > > > > > > > > > > > driver? why not simply
> > > > > > > > > > > > set the criteria to something like "pciids of parent 
> > > > > > > > > > > > devices are equal,
> > > > > > > > > > > > and mdev types are equal" ?
> > > > > > > > > > > > 
> > > > > > > > > > > >   
> > > > > > > > > > > > > btw mdev<->phys just brings trouble to upper stack as 
> > > > > > > > > > > > > Alex pointed out.   
> > > > > > > > > > > > could you help me understand why it will bring trouble 
> > > > > > > > > > > > to upper stack?
> > > > > > > > > > > > 
> > > > > > > > > > > > I think it just needs to read src migration_version 
> > > > > > > > > > > > under src dev node,
> > > > > > > > > > 

Re: [PATCH v5 0/4] introduction of migration_version attribute for VFIO live migration

[Alex Williamson]

On Wed, 29 Apr 2020 20:39:50 -0400
Yan Zhao  wrote:

> On Wed, Apr 29, 2020 at 05:48:44PM +0800, Dr. David Alan Gilbert wrote:
> 
> > > > > > > > > > > > > > An mdev type is meant to define a software 
> > > > > > > > > > > > > > compatible interface, so in
> > > > > > > > > > > > > > the case of mdev->mdev migration, doesn't migrating 
> > > > > > > > > > > > > > to a different type
> > > > > > > > > > > > > > fail the most basic of compatibility tests that we 
> > > > > > > > > > > > > > expect userspace to
> > > > > > > > > > > > > > perform?  IOW, if two mdev types are migration 
> > > > > > > > > > > > > > compatible, it seems a
> > > > > > > > > > > > > > prerequisite to that is that they provide the same 
> > > > > > > > > > > > > > software interface,
> > > > > > > > > > > > > > which means they should be the same mdev type.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > In the hybrid cases of mdev->phys or phys->mdev, 
> > > > > > > > > > > > > > how does a  
> > > > > > > > > > > > > management  
> > > > > > > > > > > > > > tool begin to even guess what might be compatible?  
> > > > > > > > > > > > > > Are we expecting
> > > > > > > > > > > > > > libvirt to probe ever device with this attribute in 
> > > > > > > > > > > > > > the system?  Is
> > > > > > > > > > > > > > there going to be a new class hierarchy created to 
> > > > > > > > > > > > > > enumerate all
> > > > > > > > > > > > > > possible migrate-able devices?
> > > > > > > > > > > > > >  
> > > > > > > > > > > > > yes, management tool needs to guess and test 
> > > > > > > > > > > > > migration compatible
> > > > > > > > > > > > > between two devices. But I think it's not the problem 
> > > > > > > > > > > > > only for
> > > > > > > > > > > > > mdev->phys or phys->mdev. even for mdev->mdev, 
> > > > > > > > > > > > > management tool needs
> > > > > > > > > > > > > to
> > > > > > > > > > > > > first assume that the two mdevs have the same type of 
> > > > > > > > > > > > > parent devices
> > > > > > > > > > > > > (e.g.their pciids are equal). otherwise, it's still 
> > > > > > > > > > > > > enumerating
> > > > > > > > > > > > > possibilities.
> > > > > > > > > > > > > 
> > > > > > > > > > > > > on the other hand, for two mdevs,
> > > > > > > > > > > > > mdev1 from pdev1, its mdev_type is 1/2 of pdev1;
> > > > > > > > > > > > > mdev2 from pdev2, its mdev_type is 1/4 of pdev2;
> > > > > > > > > > > > > if pdev2 is exactly 2 times of pdev1, why not allow 
> > > > > > > > > > > > > migration between
> > > > > > > > > > > > > mdev1 <-> mdev2.  
> > > > > > > > > > > > 
> > > > > > > > > > > > How could the manage tool figure out that 1/2 of pdev1 
> > > > > > > > > > > > is equivalent 
> > > > > > > > > > > > to 1/4 of pdev2? If we really want to allow such thing 
> > > > > > > > > > > > happen, the best
> > > > > > > > > > > > choice is to report the same mdev type on both pdev1 
> > > > > > > > > > > > and pdev2.  
> > > > > > > > > > > I think that's exactly the value of this 
> > > > > > > > > > > migration_version interface.
> > > > > > > > > > > the management tool can take advantage of this interface 
> > > > > > > > > > > to know if two
> > > > > > > > > > > devices are migration compatible, no matter they are 
> > > > > > > > > > > mdevs, non-mdevs,
> > > > > > > > > > > or mix.
> > > > > > > > > > > 
> > > > > > > > > > > as I know, (please correct me if not right), current 
> > > > > > > > > > > libvirt still
> > > > > > > > > > > requires manually generating mdev devices, and it just 
> > > > > > > > > > > duplicates src vm
> > > > > > > > > > > configuration to the target vm.
> > > > > > > > > > > for libvirt, currently it's always phys->phys and 
> > > > > > > > > > > mdev->mdev (and of the
> > > > > > > > > > > same mdev type).
> > > > > > > > > > > But it does not justify that hybrid cases should not be 
> > > > > > > > > > > allowed. otherwise,
> > > > > > > > > > > why do we need to introduce this migration_version 
> > > > > > > > > > > interface and leave
> > > > > > > > > > > the judgement of migration compatibility to vendor 
> > > > > > > > > > > driver? why not simply
> > > > > > > > > > > set the criteria to something like "pciids of parent 
> > > > > > > > > > > devices are equal,
> > > > > > > > > > > and mdev types are equal" ?
> > > > > > > > > > > 
> > > > > > > > > > >   
> > > > > > > > > > > > btw mdev<->phys just brings trouble to upper stack as 
> > > > > > > > > > > > Alex pointed out.   
> > > > > > > > > > > could you help me understand why it will bring trouble to 
> > > > > > > > > > > upper stack?
> > > > > > > > > > > 
> > > > > > > > > > > I think it just needs to read src migration_version under 
> > > > > > > > > > > src dev node,
> > > > > > > > > > > and test it in target migration version under target dev 
> > > > > > > > > > > node. 
> > > > > > > > > > > 
> > > > > > > > > > > after all, through this interface we just help the upper 
> > > > > > > > > > > layer
> > > > > > > > > > > 

[PATCH] autogen.sh: Restore --no-git (avoid git submodule update)

[Ian Jackson]

Prior to 2621d48f005a "gnulib: delete all gnulib integration",
one could pass ./autogen.sh --no-git to prevent the libvirt build
system from running git submodule update.

This feature is needed by systems like the Xen Project CI which want
to explicitly control the revisions of every tree.  These will
typically arrange to initialise the submodules check out the right
version of everything, and then expect the build system not to mess
with it any more.

Despite to the old documentation comments referring only to gnulib,
the --no-git feature is required not only because of gnulib but also
because of the other submodule, src/keycodemapdb.

(And in any case, even if it were no longer required because all the
submodules were removed, it ought ideally to have been retained as a
no-op for compaibility reasons.)

So restore the --no-git feature.

Because of the way the argument parsing of autogen.sh works, it is
easiest to recognise this option only if it comes first.  This works
for the Xen Project CI, which has always passed this option first.

If something else is using this option (and hasn't introduced a
different workaround in the meantime), not in the first position,
then perhaps a more sophisticated approach will be needed.  But I
think this will do for now.

Signed-off-by: Ian Jackson 
---
 autogen.sh | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/autogen.sh b/autogen.sh
index 4e1bbceb0a..1c98273452 100755
--- a/autogen.sh
+++ b/autogen.sh
@@ -1,5 +1,10 @@
 #!/bin/sh
 # Run this to generate all the initial makefiles, etc.
+#
+# THe following options must come first.  All other or subsequent
+# arguments are passed to configure:
+#   --no-git   skip `git submodule update --init`
+
 test -n "$srcdir" || srcdir=$(dirname "$0")
 test -n "$srcdir" || srcdir=.
 
@@ -13,7 +18,11 @@ cd "$srcdir"
 exit 1
 }
 
-git submodule update --init || exit 1
+if [ "x$1" = x--no-git ]; then
+   shift
+else
+   git submodule update --init || exit 1
+fi
 
 autoreconf --verbose --force --install || exit 1
 
-- 
2.11.0

[libvirt PATCH v2 1/4] ci: Use variables to build image names

[Andrea Bolognani]

This removes a lot of repetition and makes the configuration much
easier to read.

Signed-off-by: Andrea Bolognani 
Reviewed-by: Daniel P. Berrangé 
---
 .gitlab-ci.yml | 79 --
 1 file changed, 57 insertions(+), 22 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 149334ed6f..35895a4931 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -18,6 +18,7 @@ stages:
 # Default native build jobs that are always run
 .native_build_default_job_template: _build_default_job_definition
   stage: native_build
+  image: quay.io/libvirt/buildenv-libvirt-$NAME:latest
   cache:
 paths:
   - ccache/
@@ -42,6 +43,7 @@ stages:
 # Default cross build jobs that are always run
 .cross_build_default_job_template: _build_default_job_definition
   stage: cross_build
+  image: quay.io/libvirt/buildenv-libvirt-$NAME-cross-$CROSS:latest
   cache:
 paths:
   - ccache/
@@ -67,94 +69,127 @@ stages:
 
 x64-debian-9:
   <<: *native_build_extra_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-debian-9:latest
+  variables:
+NAME: debian-9
 
 x64-debian-10:
   <<: *native_build_default_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-debian-10:latest
+  variables:
+NAME: debian-10
 
 x64-debian-sid:
   <<: *native_build_extra_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-debian-sid:latest
+  variables:
+NAME: debian-sid
 
 x64-centos-7:
   <<: *native_build_default_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-centos-7:latest
+  variables:
+NAME: centos-7
 
 x64-centos-8:
   <<: *native_build_extra_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-centos-8:latest
+  variables:
+NAME: centos-8
 
 x64-fedora-31:
   <<: *native_build_extra_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-fedora-31:latest
+  variables:
+NAME: fedora-31
 
 x64-fedora-32:
   <<: *native_build_default_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-fedora-32:latest
+  variables:
+NAME: fedora-32
 
 x64-fedora-rawhide:
   <<: *native_build_default_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-fedora-rawhide:latest
+  variables:
+NAME: fedora-rawhide
 
 x64-opensuse-151:
   <<: *native_build_default_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-opensuse-151:latest
+  variables:
+NAME: opensuse-151
 
 x64-ubuntu-1804:
   <<: *native_build_extra_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-ubuntu-1804:latest
+  variables:
+NAME: ubuntu-1804
 
 x64-ubuntu-2004:
   <<: *native_build_default_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-ubuntu-2004:latest
+  variables:
+NAME: ubuntu-2004
 
 
 # Cross compiled build jobs
 
 armv6l-debian-9:
   <<: *cross_build_extra_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-debian-9-cross-armv6l:latest
+  variables:
+NAME: debian-9
+CROSS: armv6l
 
 mips64el-debian-9:
   <<: *cross_build_extra_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-debian-9-cross-mips64el:latest
+  variables:
+NAME: debian-9
+CROSS: mips64el
 
 mips-debian-9:
   <<: *cross_build_extra_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-debian-9-cross-mips:latest
+  variables:
+NAME: debian-9
+CROSS: mips
 
 aarch64-debian-10:
   <<: *cross_build_extra_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-debian-10-cross-aarch64:latest
+  variables:
+NAME: debian-10
+CROSS: aarch64
 
 ppc64le-debian-10:
   <<: *cross_build_extra_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-debian-10-cross-ppc64le:latest
+  variables:
+NAME: debian-10
+CROSS: ppc64le
 
 s390x-debian-10:
   <<: *cross_build_default_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-debian-10-cross-s390x:latest
+  variables:
+NAME: debian-10
+CROSS: s390x
 
 armv7l-debian-sid:
   <<: *cross_build_default_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-debian-sid-cross-armv7l:latest
+  variables:
+NAME: debian-sid
+CROSS: armv7l
 
 i686-debian-sid:
   <<: *cross_build_extra_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-debian-sid-cross-i686:latest
+  variables:
+NAME: debian-sid
+CROSS: i686
 
 mipsel-debian-sid:
   <<: *cross_build_extra_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-debian-sid-cross-mipsel:latest
+  variables:
+NAME: debian-sid
+CROSS: mipsel
 
 mingw32-fedora-rawhide:
   <<: *cross_build_default_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-fedora-rawhide-cross-mingw32:latest
+  variables:
+NAME: fedora-rawhide
+CROSS: mingw32
 
 mingw64-fedora-rawhide:
   <<: *cross_build_default_job_definition
-  image: quay.io/libvirt/buildenv-libvirt-fedora-rawhide-cross-mingw64:latest
+  variables:
+NAME: fedora-rawhide
+CROSS: mingw64
 
 
 # This artifact published by this job is downloaded by libvirt.org to
-- 
2.25.4

[libvirt PATCH v2 4/4] ci: Update build system integration

[Andrea Bolognani]

The ci-* targets need to know where our container images are stored
and how they are called to work, so now that we use the GitLab
container registry instead of Quay some changes are necessary.

Signed-off-by: Andrea Bolognani 
---
 ci/Makefile   |  6 +++---
 ci/list-images.sh | 24 ++--
 2 files changed, 9 insertions(+), 21 deletions(-)

diff --git a/ci/Makefile b/ci/Makefile
index bc1dac11e3..a6a65bdc0f 100644
--- a/ci/Makefile
+++ b/ci/Makefile
@@ -50,7 +50,7 @@ CI_BUILD_SCRIPT = $(CI_ROOTDIR)/build.sh
 # Location of the container images we're going to pull
 # Can be useful to overridde to use a locally built
 # image instead
-CI_IMAGE_PREFIX = quay.io/libvirt/buildenv-libvirt-
+CI_IMAGE_PREFIX = registry.gitlab.com/libvirt/libvirt/ci-
 
 # The default tag is ':latest' but if the container
 # repo above uses different conventions this can override it
@@ -243,11 +243,11 @@ ci-list-images:
@echo
@echo "Available x86 container images:"
@echo
-   @sh list-images.sh "$(CI_ENGINE)" "$(CI_IMAGE_PREFIX)" | grep -v cross
+   @sh list-images.sh "$(CI_IMAGE_PREFIX)" | grep -v cross
@echo
@echo "Available cross-compiler container images:"
@echo
-   @sh list-images.sh "$(CI_ENGINE)" "$(CI_IMAGE_PREFIX)" | grep cross
+   @sh list-images.sh "$(CI_IMAGE_PREFIX)" | grep cross
@echo
 
 ci-help:
diff --git a/ci/list-images.sh b/ci/list-images.sh
index 35efdb6982..b85b132253 100644
--- a/ci/list-images.sh
+++ b/ci/list-images.sh
@@ -1,26 +1,14 @@
 #!/bin/sh
 
-engine="$1"
-prefix="$2"
+prefix="${1##registry.gitlab.com/}"
 
-do_podman() {
-# Podman freaks out if the search term ends with a dash, which ours
-# by default does, so let's strip it. The repository name is the
-# second field in the output, and it already starts with the registry
-podman search --limit 100 "${prefix%-}" | while read _ repo _; do
-echo "$repo"
-done
-}
+PROJECT_ID=192693
 
-do_docker() {
-# Docker doesn't include the registry name in the output, so we have
-# to add it. The repository name is the first field in the output
-registry="${prefix%%/*}"
-docker search --limit 100 "$prefix" | while read repo _; do
-echo "$registry/$repo"
-done
+all_repos() {
+  curl -s 
"https://gitlab.com/api/v4/projects/$PROJECT_ID/registry/repositories?per_page=100;
 \
+| tr , '\n' | grep '"path":' | sed 's,"path":",,g;s,"$,,g'
 }
 
-"do_$engine" | grep "^$prefix" | sed "s,^$prefix,,g" | while read repo; do
+all_repos | grep "^$prefix" | sed "s,^$prefix,,g" | while read repo; do
 echo "$repo"
 done | sort -u
-- 
2.25.4

[libvirt PATCH v2 0/4] ci: Use GitLab container registry

[Andrea Bolognani]

Changes from [v1]:

* only build containers necessary for extra jobs when said jobs
  are actually going to run;

* rename container build jobs to '$arch-$os-container';

* rename 'other' stage to 'preliminary_checks' and move it before
  native builds;

* simplify build system integration.


[v1] https://www.redhat.com/archives/libvir-list/2020-May/msg01183.html

Andrea Bolognani (4):
  ci: Use variables to build image names
  ci: Add 'preliminary_checks' stage
  ci: Use GitLab container registry
  ci: Update build system integration

 .gitlab-ci.yml| 325 --
 ci/Makefile   |   6 +-
 ci/containers/README.rst  |  14 +
 ci/containers/ci-centos-7.Dockerfile  | 137 
 ci/containers/ci-centos-8.Dockerfile  | 108 ++
 .../ci-debian-10-cross-aarch64.Dockerfile | 122 +++
 .../ci-debian-10-cross-armv6l.Dockerfile  | 120 +++
 .../ci-debian-10-cross-armv7l.Dockerfile  | 121 +++
 .../ci-debian-10-cross-i686.Dockerfile| 121 +++
 .../ci-debian-10-cross-mips.Dockerfile| 121 +++
 .../ci-debian-10-cross-mips64el.Dockerfile| 121 +++
 .../ci-debian-10-cross-mipsel.Dockerfile  | 121 +++
 .../ci-debian-10-cross-ppc64le.Dockerfile | 121 +++
 .../ci-debian-10-cross-s390x.Dockerfile   | 121 +++
 ci/containers/ci-debian-10.Dockerfile | 112 ++
 .../ci-debian-9-cross-aarch64.Dockerfile  | 126 +++
 .../ci-debian-9-cross-armv6l.Dockerfile   | 124 +++
 .../ci-debian-9-cross-armv7l.Dockerfile   | 125 +++
 .../ci-debian-9-cross-mips.Dockerfile | 125 +++
 .../ci-debian-9-cross-mips64el.Dockerfile | 125 +++
 .../ci-debian-9-cross-mipsel.Dockerfile   | 125 +++
 .../ci-debian-9-cross-ppc64le.Dockerfile  | 125 +++
 .../ci-debian-9-cross-s390x.Dockerfile| 125 +++
 ci/containers/ci-debian-9.Dockerfile  | 116 +++
 .../ci-debian-sid-cross-aarch64.Dockerfile| 122 +++
 .../ci-debian-sid-cross-armv6l.Dockerfile | 120 +++
 .../ci-debian-sid-cross-armv7l.Dockerfile | 121 +++
 .../ci-debian-sid-cross-i686.Dockerfile   | 121 +++
 .../ci-debian-sid-cross-mips.Dockerfile   | 121 +++
 .../ci-debian-sid-cross-mips64el.Dockerfile   | 121 +++
 .../ci-debian-sid-cross-mipsel.Dockerfile | 120 +++
 .../ci-debian-sid-cross-ppc64le.Dockerfile| 121 +++
 .../ci-debian-sid-cross-s390x.Dockerfile  | 121 +++
 ci/containers/ci-debian-sid.Dockerfile| 112 ++
 ci/containers/ci-fedora-31.Dockerfile | 109 ++
 ci/containers/ci-fedora-32.Dockerfile | 109 ++
 ...ci-fedora-rawhide-cross-mingw32.Dockerfile | 129 +++
 ...ci-fedora-rawhide-cross-mingw64.Dockerfile | 129 +++
 ci/containers/ci-fedora-rawhide.Dockerfile| 110 ++
 ci/containers/ci-opensuse-151.Dockerfile  | 109 ++
 ci/containers/ci-ubuntu-1804.Dockerfile   | 117 +++
 ci/containers/ci-ubuntu-2004.Dockerfile   | 113 ++
 ci/containers/refresh |  43 +++
 ci/list-images.sh |  24 +-
 44 files changed, 5051 insertions(+), 48 deletions(-)
 create mode 100644 ci/containers/README.rst
 create mode 100644 ci/containers/ci-centos-7.Dockerfile
 create mode 100644 ci/containers/ci-centos-8.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-aarch64.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-armv6l.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-armv7l.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-i686.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-mips.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-mips64el.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-mipsel.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-ppc64le.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-s390x.Dockerfile
 create mode 100644 ci/containers/ci-debian-10.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-aarch64.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-armv6l.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-armv7l.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-mips.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-mips64el.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-mipsel.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-ppc64le.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-s390x.Dockerfile
 create mode 100644 ci/containers/ci-debian-9.Dockerfile
 create mode 100644 ci/containers/ci-debian-sid-cross-aarch64.Dockerfile
 create mode 100644 ci/containers/ci-debian-sid-cross-armv6l.Dockerfile
 create mode 100644 ci/containers/ci-debian-sid-cross-armv7l.Dockerfile
 create mode 100644 

[libvirt PATCH v2 3/4] ci: Use GitLab container registry

[Andrea Bolognani]

Instead of using pre-built containers hosted on Quay, build
containers as part of the GitLab CI pipeline and upload them to the
GitLab container registry for later use.

This will not significantly slow down builds, because containers are
only rebuilt when the corresponding Dockerfile has been modified.

Signed-off-by: Andrea Bolognani 
---
 .gitlab-ci.yml| 245 +-
 ci/containers/README.rst  |  14 +
 ci/containers/ci-centos-7.Dockerfile  | 137 ++
 ci/containers/ci-centos-8.Dockerfile  | 108 
 .../ci-debian-10-cross-aarch64.Dockerfile | 122 +
 .../ci-debian-10-cross-armv6l.Dockerfile  | 120 +
 .../ci-debian-10-cross-armv7l.Dockerfile  | 121 +
 .../ci-debian-10-cross-i686.Dockerfile| 121 +
 .../ci-debian-10-cross-mips.Dockerfile| 121 +
 .../ci-debian-10-cross-mips64el.Dockerfile| 121 +
 .../ci-debian-10-cross-mipsel.Dockerfile  | 121 +
 .../ci-debian-10-cross-ppc64le.Dockerfile | 121 +
 .../ci-debian-10-cross-s390x.Dockerfile   | 121 +
 ci/containers/ci-debian-10.Dockerfile | 112 
 .../ci-debian-9-cross-aarch64.Dockerfile  | 126 +
 .../ci-debian-9-cross-armv6l.Dockerfile   | 124 +
 .../ci-debian-9-cross-armv7l.Dockerfile   | 125 +
 .../ci-debian-9-cross-mips.Dockerfile | 125 +
 .../ci-debian-9-cross-mips64el.Dockerfile | 125 +
 .../ci-debian-9-cross-mipsel.Dockerfile   | 125 +
 .../ci-debian-9-cross-ppc64le.Dockerfile  | 125 +
 .../ci-debian-9-cross-s390x.Dockerfile| 125 +
 ci/containers/ci-debian-9.Dockerfile  | 116 +
 .../ci-debian-sid-cross-aarch64.Dockerfile| 122 +
 .../ci-debian-sid-cross-armv6l.Dockerfile | 120 +
 .../ci-debian-sid-cross-armv7l.Dockerfile | 121 +
 .../ci-debian-sid-cross-i686.Dockerfile   | 121 +
 .../ci-debian-sid-cross-mips.Dockerfile   | 121 +
 .../ci-debian-sid-cross-mips64el.Dockerfile   | 121 +
 .../ci-debian-sid-cross-mipsel.Dockerfile | 120 +
 .../ci-debian-sid-cross-ppc64le.Dockerfile| 121 +
 .../ci-debian-sid-cross-s390x.Dockerfile  | 121 +
 ci/containers/ci-debian-sid.Dockerfile| 112 
 ci/containers/ci-fedora-31.Dockerfile | 109 
 ci/containers/ci-fedora-32.Dockerfile | 109 
 ...ci-fedora-rawhide-cross-mingw32.Dockerfile | 129 +
 ...ci-fedora-rawhide-cross-mingw64.Dockerfile | 129 +
 ci/containers/ci-fedora-rawhide.Dockerfile| 110 
 ci/containers/ci-opensuse-151.Dockerfile  | 109 
 ci/containers/ci-ubuntu-1804.Dockerfile   | 117 +
 ci/containers/ci-ubuntu-2004.Dockerfile   | 113 
 ci/containers/refresh |  43 +++
 42 files changed, 4984 insertions(+), 5 deletions(-)
 create mode 100644 ci/containers/README.rst
 create mode 100644 ci/containers/ci-centos-7.Dockerfile
 create mode 100644 ci/containers/ci-centos-8.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-aarch64.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-armv6l.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-armv7l.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-i686.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-mips.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-mips64el.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-mipsel.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-ppc64le.Dockerfile
 create mode 100644 ci/containers/ci-debian-10-cross-s390x.Dockerfile
 create mode 100644 ci/containers/ci-debian-10.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-aarch64.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-armv6l.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-armv7l.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-mips.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-mips64el.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-mipsel.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-ppc64le.Dockerfile
 create mode 100644 ci/containers/ci-debian-9-cross-s390x.Dockerfile
 create mode 100644 ci/containers/ci-debian-9.Dockerfile
 create mode 100644 ci/containers/ci-debian-sid-cross-aarch64.Dockerfile
 create mode 100644 ci/containers/ci-debian-sid-cross-armv6l.Dockerfile
 create mode 100644 ci/containers/ci-debian-sid-cross-armv7l.Dockerfile
 create mode 100644 ci/containers/ci-debian-sid-cross-i686.Dockerfile
 create mode 100644 ci/containers/ci-debian-sid-cross-mips.Dockerfile
 create mode 100644 ci/containers/ci-debian-sid-cross-mips64el.Dockerfile
 create mode 100644 

[libvirt PATCH v2 2/4] ci: Add 'preliminary_checks' stage

[Andrea Bolognani]

We're going to build container images as part of the CI pipeline
soon, which means that we need to move all jobs that run in a
container image which is not provided by an external project (such
as the one that we use for DCO checking) later in the pipeline or
they will fail.

Signed-off-by: Andrea Bolognani 
---
 .gitlab-ci.yml | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 35895a4931..a8b16c6dfd 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -3,6 +3,7 @@ variables:
 
 stages:
   - prebuild
+  - preliminary_checks
   - native_build
   - cross_build
 
@@ -196,7 +197,7 @@ mingw64-fedora-rawhide:
 # be deployed to the web root:
 #
https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=website
 website:
-  stage: prebuild
+  stage: preliminary_checks
   before_script:
 - *script_variables
   script:
@@ -218,7 +219,7 @@ website:
 
 
 codestyle:
-  stage: prebuild
+  stage: preliminary_checks
   before_script:
 - *script_variables
   script:
-- 
2.25.4

[libvirt PATCH 4/5] news: Convert to reStructuredText

[Andrea Bolognani]

Instead of storing release notes as XML and then converting them
to HTML and ASCII at build time using XSLT and a custom script,
we can use reStructuredText as both the source and ASCII
representation and generate HTML from it using the same tooling
we already use for the rest of the documentation.

Signed-off-by: Andrea Bolognani 
---
 Makefile.am  |   22 +-
 NEWS.rst | 3362 +++
 docs/Makefile.am |   27 +-
 docs/libvirt.css |   15 -
 docs/news-ascii.xsl  |   95 -
 docs/news-html.xsl   |  106 -
 docs/news.rng|   72 -
 docs/news.xml| 5473 --
 scripts/reformat-news.py |  102 -
 tests/virschematest.c|2 -
 10 files changed, 3374 insertions(+), 5902 deletions(-)
 create mode 100644 NEWS.rst
 delete mode 100644 docs/news-ascii.xsl
 delete mode 100644 docs/news-html.xsl
 delete mode 100644 docs/news.rng
 delete mode 100644 docs/news.xml
 delete mode 100755 scripts/reformat-news.py

diff --git a/Makefile.am b/Makefile.am
index d56deeb080..3b93170bc6 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -46,6 +46,7 @@ EXTRA_DIST = \
   README.rst \
   AUTHORS.in \
   CONTRIBUTING.rst \
+  NEWS.rst \
   scripts/apibuild.py \
   scripts/augeas-gentest.py \
   build-aux/check-spacing.pl \
@@ -69,7 +70,6 @@ EXTRA_DIST = \
   scripts/minimize-po.py \
   scripts/mock-noinline.py \
   scripts/prohibit-duplicate-header.py \
-  scripts/reformat-news.py \
   scripts/test-wrap-argv.py \
   build-aux/syntax-check.mk \
   build-aux/useless-if-before-free \
@@ -83,26 +83,6 @@ EXTRA_DIST = \
 pkgconfigdir = $(libdir)/pkgconfig
 pkgconfig_DATA = libvirt.pc libvirt-qemu.pc libvirt-lxc.pc libvirt-admin.pc
 
-NEWS: \
- $(srcdir)/docs/news.xml \
- $(srcdir)/docs/news-ascii.xsl \
- $(top_srcdir)/scripts/reformat-news.py
-   $(AM_V_GEN) \
-   if [ -x $(XSLTPROC) ]; then \
- $(XSLTPROC) --nonet \
-   $(srcdir)/docs/news-ascii.xsl \
-   $(srcdir)/docs/news.xml \
- >$@-tmp \
-   || { rm -f $@-tmp; exit 1; }; \
- $(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/reformat-news.py $@-tmp 
>$@ \
-   || { rm -f $@-tmp; exit 1; }; \
- rm -f $@-tmp; \
-   fi
-EXTRA_DIST += \
-   $(srcdir)/docs/news.xml \
-   $(srcdir)/docs/news-ascii.xsl \
-   $(NULL)
-
 rpm: clean
@(unset CDPATH ; $(MAKE) dist && rpmbuild -ta $(distdir).tar.xz)
 
diff --git a/NEWS.rst b/NEWS.rst
new file mode 100644
index 00..128b899b88
--- /dev/null
+++ b/NEWS.rst
@@ -0,0 +1,3362 @@
+
+libvirt releases
+
+
+This is the list of official releases for libvirt, along with an overview of
+the changes introduced by each of them.
+
+For a more fine-grained view, use the `git log`_.
+
+
+v6.5.0 (unreleased)
+===
+
+* **New features**
+
+* **Improvements**
+
+* **Bug fixes**
+
+
+v6.4.0 (2020-06-02)
+===
+
+* **New features**
+
+  - qemu: Add support for pvscsi controllers
+
+pvscsi is the VMware paravirtualized SCSI controller, which has been
+supported in QEMU for a number of years.
+
+  - cpu: Report model information for ARM CPUs
+
+``virsh capabilities`` will now include information about the host CPU when
+run on ARM machines.
+
+* **Improvements**
+
+  - qemu: stricter validation for disk type='lun'
+
+The 'lun' type is meant for SCSI command passthrough, which can't be
+achieved if qemu's block layer features are used. Disk type='lun' is now
+allowed only when the format is 'raw' and no other block layer features are
+requested.
+
+* **Bug fixes**
+
+  - qemu: fixed regression in network device hotplug with new qemu versions
+
+Starting from QEMU-5.0 it's required to conform to strict schema when
+hotplugging network devices. Libvirt didn't conform to the schema so in
+versions prior to 6.4.0 network device hotplug fails in certain cases. This
+version fixes it and adds stricter testing to prevent further issues.
+
+  - remote: Look up libxl driver correctly
+
+This makes ``xen://`` connection URIs usable in split daemon mode.
+
+  - systemd: Start libvirtd after firewalld/iptables services
+
+This solves an issue where iptables rules and chains created by libvirtd
+would get removed by a service started after it.
+
+  - network: Re-create iptables chains on firewalld restart
+
+firewalld resets all iptables rules and chains on restart, and this
+includes deleting those created by libvirt.
+
[...]
+
+v2.5.0 (2016-12-04)
+===
+
+* **New features**
+
+  - shmem: Add support for additional models
+
+The shmem device can now utilize QEMU's ivshmem-plain and ivshmem-doorbell,
+more modern versions of ivshmem.
+
+  - vbox: Add VirtualBox 5.1 support
+
+  - libssh: New transport
+
+The new libssh transport allows one to connect to a running libvirtd via
+SSH, using the libssh 

[libvirt PATCH 5/5] news: Add information about old releases

[Andrea Bolognani]

Until libvirt 2.5.0 we didn't have a real process for release
notes in place, and we just published the list of commits that
had made it into each release, dividing them into categories that
mostly matched the sections we use today. Those documents haven't
been relevant for years, but they're still in the git repository
and collectively take up almost 2 MiB of disk space.

Let's import the only valuable piece of information they contain,
the release date for each libvirt versions, into the current
document and then drop them for good.

Signed-off-by: Andrea Bolognani 
---
 NEWS.rst   |  721 +++-
 docs/news-2005.html.in |   28 -
 docs/news-2006.html.in |  354 
 docs/news-2007.html.in |  534 --
 docs/news-2008.html.in |  580 ---
 docs/news-2009.html.in | 1603 -
 docs/news-2010.html.in | 2218 
 docs/news-2011.html.in | 3314 ---
 docs/news-2012.html.in | 3012 
 docs/news-2013.html.in | 3675 ---
 docs/news-2014.html.in | 3418 
 docs/news-2015.html.in | 2864 --
 docs/news-2016.html.in | 3740 
 13 files changed, 718 insertions(+), 25343 deletions(-)
 delete mode 100644 docs/news-2005.html.in
 delete mode 100644 docs/news-2006.html.in
 delete mode 100644 docs/news-2007.html.in
 delete mode 100644 docs/news-2008.html.in
 delete mode 100644 docs/news-2009.html.in
 delete mode 100644 docs/news-2010.html.in
 delete mode 100644 docs/news-2011.html.in
 delete mode 100644 docs/news-2012.html.in
 delete mode 100644 docs/news-2013.html.in
 delete mode 100644 docs/news-2014.html.in
 delete mode 100644 docs/news-2015.html.in
 delete mode 100644 docs/news-2016.html.in

diff --git a/NEWS.rst b/NEWS.rst
index 128b899b88..82b93d5a44 100644
--- a/NEWS.rst
+++ b/NEWS.rst
@@ -3354,9 +3354,724 @@ v2.5.0 (2016-12-04)
   - Fix compilation on macOS
 
 
-Older libvirt releases didn't have proper release notes: if you are interested
-in changes between them, you should check out docs/news-\*.html or the full git
-log (see instructions in ChangeLog).
+v2.4.0 (2016-11-01)
+===
+
+No release notes.
+
[...]
+
+v0.0.1 (2005-12-19)
+===
+
+No release notes.
 
 
 .. _git log: https://gitlab.com/libvirt/libvirt/-/commits/master
diff --git a/docs/news-2005.html.in b/docs/news-2005.html.in
deleted file mode 100644
index 8d6dc150d3..00
--- a/docs/news-2005.html.in
+++ /dev/null
@@ -1,28 +0,0 @@
-
-
-http://www.w3.org/1999/xhtml;>
-  
-
-  
-  
-Releases (2005)
-Here is the list of official releases made during the year 2005.
-
-It is also possible to just use
-the GIT version or snapshot,
-contact the mailing list and check
-the https://libvirt.org/git/?p=libvirt.git;a=log;>GIT log
-to gauge progress.
-
-
-0.0.1: Dec 19 2005
-
-  Features:
-  First release,
-  Basic management of existing Xen domains,
-  Minimal autogenerated Python bindings
-  
-
-
-  
-
diff --git a/docs/news-2006.html.in b/docs/news-2006.html.in
deleted file mode 100644
index 7463d7bad3..00
--- a/docs/news-2006.html.in
+++ /dev/null
@@ -1,354 +0,0 @@
-
-
-http://www.w3.org/1999/xhtml;>
-  
-
-  
-  
-Releases (2006)
-Here is the list of official releases made during the year 2006.
-A similar list for 2005 is also available.
-
-It is also possible to just use
-the GIT version or snapshot,
-contact the mailing list and check
-the https://libvirt.org/git/?p=libvirt.git;a=log;>GIT log
-to gauge progress.
-
-
[...]
-
-  
-
diff --git a/docs/news-2007.html.in b/docs/news-2007.html.in
deleted file mode 100644
index 14099936fd..00
--- a/docs/news-2007.html.in
+++ /dev/null
@@ -1,534 +0,0 @@
-
-
-http://www.w3.org/1999/xhtml;>
-  
-
-  
-  
-Releases (2007)
-Here is the list of official releases made during the year 2007.
-A similar list for 2006 is also available.
-
-It is also possible to just use
-the GIT version or snapshot,
-contact the mailing list and check
-the https://libvirt.org/git/?p=libvirt.git;a=log;>GIT log
-to gauge progress.
-
-
[...]
-
-  
-
diff --git a/docs/news-2008.html.in b/docs/news-2008.html.in
deleted file mode 100644
index 28f3b98d82..00
--- a/docs/news-2008.html.in
+++ /dev/null
@@ -1,580 +0,0 @@
-
-
-http://www.w3.org/1999/xhtml;>
-  
-
-  
-  
-Releases (2008)
-Here is the list of official releases made during the year 2008.
-A similar list for 2007 is also available.
-
-It is also possible to just use
-the GIT version or snapshot,
-contact the mailing list and check
-the https://libvirt.org/git/?p=libvirt.git;a=log;>GIT log
-to gauge progress.
-
-
[...]
-
-  
-
diff --git a/docs/news-2009.html.in b/docs/news-2009.html.in

[libvirt PATCH 0/5] news: Convert to reStructuredText

[Andrea Bolognani]

Rationale in patch 4/5, in case that diffstat is not enough of a
selling point for you ;)

Some of the patches in this series are __heavily__ snipped.

browse: https://gitlab.com/abologna/libvirt/-/tree/news-convert-cleanup
git fetch: https://gitlab.com/abologna/libvirt.git news-convert-cleanup

Andrea Bolognani (5):
  docs: Fix dot_rst_html_in definition
  news: Point to GitLab for full git log
  news: Output reStructuredText for the ASCII version
  news: Convert to reStructuredText
  news: Add information about old releases

 Makefile.am  |   22 +-
 NEWS.rst | 4077 
 docs/Makefile.am |   27 +-
 docs/libvirt.css |   15 -
 docs/news-2005.html.in   |   28 -
 docs/news-2006.html.in   |  354 ---
 docs/news-2007.html.in   |  534 
 docs/news-2008.html.in   |  580 
 docs/news-2009.html.in   | 1603 ---
 docs/news-2010.html.in   | 2218 ---
 docs/news-2011.html.in   | 3314 ---
 docs/news-2012.html.in   | 3012 -
 docs/news-2013.html.in   | 3675 -
 docs/news-2014.html.in   | 3418 
 docs/news-2015.html.in   | 2864 
 docs/news-2016.html.in   | 3740 --
 docs/news-ascii.xsl  |   69 -
 docs/news-html.xsl   |  106 -
 docs/news.rng|   72 -
 docs/news.xml| 5473 --
 scripts/reformat-news.py |  102 -
 tests/virschematest.c|2 -
 22 files changed, 4089 insertions(+), 31216 deletions(-)
 create mode 100644 NEWS.rst
 delete mode 100644 docs/news-2005.html.in
 delete mode 100644 docs/news-2006.html.in
 delete mode 100644 docs/news-2007.html.in
 delete mode 100644 docs/news-2008.html.in
 delete mode 100644 docs/news-2009.html.in
 delete mode 100644 docs/news-2010.html.in
 delete mode 100644 docs/news-2011.html.in
 delete mode 100644 docs/news-2012.html.in
 delete mode 100644 docs/news-2013.html.in
 delete mode 100644 docs/news-2014.html.in
 delete mode 100644 docs/news-2015.html.in
 delete mode 100644 docs/news-2016.html.in
 delete mode 100644 docs/news-ascii.xsl
 delete mode 100644 docs/news-html.xsl
 delete mode 100644 docs/news.rng
 delete mode 100644 docs/news.xml
 delete mode 100755 scripts/reformat-news.py

-- 
2.25.4

[libvirt PATCH 1/5] docs: Fix dot_rst_html_in definition

[Andrea Bolognani]

As the name clearly implies, it's supposed to list the .html.in
files that are generated from .rst files, but it mistakenly lists
the corresponding .html files instead.

Signed-off-by: Andrea Bolognani 
---
 docs/Makefile.am | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/Makefile.am b/docs/Makefile.am
index ce3d296b19..ba538e55a0 100644
--- a/docs/Makefile.am
+++ b/docs/Makefile.am
@@ -325,7 +325,7 @@ dot_html_in = \
 dot_rst = \
   $(notdir $(wildcard $(srcdir)/*.rst))
 dot_rst_html_in = \
-  $(dot_rst:%.rst=%.html)
+  $(dot_rst:%.rst=%.html.in)
 dot_html = \
   $(dot_html_generated_in:%.html.in=%.html) \
   $(dot_html_in:%.html.in=%.html) \
-- 
2.25.4

[libvirt PATCH 3/5] news: Output reStructuredText for the ASCII version

[Andrea Bolognani]

The ASCII output our scripts produce is already very close to
reStructuredText, and with just a few extra tweaks we can get
almost all of the way there.

Signed-off-by: Andrea Bolognani 
---
 docs/news-ascii.xsl  | 40 +---
 scripts/reformat-news.py | 12 ++--
 2 files changed, 39 insertions(+), 13 deletions(-)

diff --git a/docs/news-ascii.xsl b/docs/news-ascii.xsl
index 8dacae934c..9f6c61a3c6 100644
--- a/docs/news-ascii.xsl
+++ b/docs/news-ascii.xsl
@@ -8,36 +8,48 @@
 
   
   
-libvirt releases
+
+libvirt releases
 
+
+This is the list of official releases for libvirt, along with an overview of
+the changes introduced by each of them.
+
+For a more fine-grained view, use the `git log`_.
 
 
 
-==
+
 Older libvirt releases didn't have proper release notes: if you are interested
-in changes between them, you should check out docs/news-*.html or the full git
+in changes between them, you should check out docs/news-\*.html or the full git
 log (see instructions in ChangeLog).
+
+
+.. _git log: https://gitlab.com/libvirt/libvirt/-/commits/master
 
   
 
   
   
 
-# 
+
+
 
  (
 
 )
+===
 
+
 
   
 
   
   
 
-* 
+* **
 
-
+**
 
 
   
@@ -60,10 +72,24 @@ log (see instructions in ChangeLog).
 
   
   
+
+
 | 
-
+
 
 
   
 
+  
+  
+
+  
+
+  
+  
+ ``
+
+`` 
+  
+
 
diff --git a/scripts/reformat-news.py b/scripts/reformat-news.py
index 7bc752d821..532a83803d 100755
--- a/scripts/reformat-news.py
+++ b/scripts/reformat-news.py
@@ -67,16 +67,16 @@ def reformat(line):
 indent = 2
 # Section
 elif marker == '*':
-initial_indent = 2
-indent = 4
+initial_indent = 0
+indent = 2
 # Change summary
 elif marker == '-':
-initial_indent = 4
-indent = 6
+initial_indent = 2
+indent = 4
 # Change description
 elif marker == '|':
-initial_indent = 8
-indent = 8
+initial_indent = 4
+indent = 4
 # In this one case, the marker should not ultimately show
 # up in the output file, so we strip it before moving on
 line = line[1:]
-- 
2.25.4

[libvirt PATCH 2/5] news: Point to GitLab for full git log

[Andrea Bolognani]

The primary git repository is the one on GitLab these days.

Signed-off-by: Andrea Bolognani 
---
 docs/news-html.xsl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/news-html.xsl b/docs/news-html.xsl
index 6bf8bca073..ec031c6d15 100644
--- a/docs/news-html.xsl
+++ b/docs/news-html.xsl
@@ -27,7 +27,7 @@
 This is the list of official releases for libvirt, along with an
 overview of the changes introduced by each of them.
 For a more fine-grained view, use the
-https://libvirt.org/git/?p=libvirt.git;a=log;>git log.
+https://gitlab.com/libvirt/libvirt/-/commits/master;>git 
log.
 
 
 Older libvirt releases didn't have proper release notes,
-- 
2.25.4

[PATCH v6 05/10] conf, qemu, security, tests: introducing 'def->tpms' array

[Daniel Henrique Barboza]

A TPM Proxy device can coexist with a regular TPM, but the
current domain definition supports only a single TPM device
in the 'tpm' pointer. This patch replaces this existing pointer
in the domain definition to an array of TPM devices.

All files that references the old pointer were adapted to
handle the new array instead. virDomainDefParseXML() TPM related
code was adapted to handle the parsing of an extra TPM device.
TPM validations after this new scenario will be updated in
the next patch.

Tested-by: Satheesh Rajendran 
Reviewed-by: Stefan Berger 
Signed-off-by: Daniel Henrique Barboza 
---
 src/conf/domain_audit.c |  4 +--
 src/conf/domain_conf.c  | 50 +++
 src/conf/domain_conf.h  |  6 +++-
 src/qemu/qemu_alias.c   |  4 +--
 src/qemu/qemu_cgroup.c  | 10 ---
 src/qemu/qemu_command.c | 34 ++---
 src/qemu/qemu_domain.c  | 31 +---
 src/qemu/qemu_domain_address.c  | 11 ---
 src/qemu/qemu_extdevice.c   | 18 +++-
 src/qemu/qemu_tpm.c | 52 +
 src/security/security_dac.c |  8 ++---
 src/security/security_selinux.c | 32 
 src/security/virt-aa-helper.c   |  9 --
 tests/qemuxml2argvtest.c| 13 +
 14 files changed, 186 insertions(+), 96 deletions(-)

diff --git a/src/conf/domain_audit.c b/src/conf/domain_audit.c
index 1b0abb21a0..8bc6633af4 100644
--- a/src/conf/domain_audit.c
+++ b/src/conf/domain_audit.c
@@ -821,8 +821,8 @@ virDomainAuditStart(virDomainObjPtr vm, const char *reason, 
bool success)
 for (i = 0; i < vm->def->nrngs; i++)
 virDomainAuditRNG(vm, NULL, vm->def->rngs[i], "start", true);
 
-if (vm->def->tpm)
-virDomainAuditTPM(vm, vm->def->tpm, "start", true);
+for (i = 0; i < vm->def->ntpms; i++)
+virDomainAuditTPM(vm, vm->def->tpms[i], "start", true);
 
 for (i = 0; i < vm->def->nshmems; i++)
 virDomainAuditShmem(vm, vm->def->shmems[i], "start", true);
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 1406cf079e..899f994e23 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -1165,6 +1165,7 @@ VIR_ENUM_IMPL(virDomainTPMModel,
   "tpm-tis",
   "tpm-crb",
   "tpm-spapr",
+  "spapr-tpm-proxy",
 );
 
 VIR_ENUM_IMPL(virDomainTPMBackend,
@@ -3480,7 +3481,9 @@ void virDomainDefFree(virDomainDefPtr def)
 virDomainMemoryDefFree(def->mems[i]);
 VIR_FREE(def->mems);
 
-virDomainTPMDefFree(def->tpm);
+for (i = 0; i < def->ntpms; i++)
+virDomainTPMDefFree(def->tpms[i]);
+VIR_FREE(def->tpms);
 
 for (i = 0; i < def->npanics; i++)
 virDomainPanicDefFree(def->panics[i]);
@@ -4313,10 +4316,10 @@ virDomainDeviceInfoIterateInternal(virDomainDefPtr def,
 if ((rc = cb(def, , >shmems[i]->info, opaque)) != 0)
 return rc;
 }
-if (def->tpm) {
-device.type = VIR_DOMAIN_DEVICE_TPM;
-device.data.tpm = def->tpm;
-if ((rc = cb(def, , >tpm->info, opaque)) != 0)
+device.type = VIR_DOMAIN_DEVICE_TPM;
+for (i = 0; i < def->ntpms; i++) {
+device.data.tpm = def->tpms[i];
+if ((rc = cb(def, , >tpms[i]->info, opaque)) != 0)
 return rc;
 }
 device.type = VIR_DOMAIN_DEVICE_PANIC;
@@ -21971,15 +21974,23 @@ virDomainDefParseXML(xmlDocPtr xml,
 if ((n = virXPathNodeSet("./devices/tpm", ctxt, )) < 0)
 goto error;
 
-if (n > 1) {
+if (n > 2) {
 virReportError(VIR_ERR_XML_ERROR, "%s",
-   _("only a single TPM device is supported"));
+   _("a maximum of two TPM devices is supported, one of "
+ "them being a TPM Proxy device"));
 goto error;
 }
 
-if (n > 0) {
-if (!(def->tpm = virDomainTPMDefParseXML(xmlopt, nodes[0], ctxt, 
flags)))
+if (n && VIR_ALLOC_N(def->tpms, n) < 0)
+goto error;
+
+for (i = 0; i < n; i++) {
+virDomainTPMDefPtr tpm = virDomainTPMDefParseXML(xmlopt, nodes[i],
+ ctxt, flags);
+if (!tpm)
 goto error;
+
+def->tpms[def->ntpms++] = tpm;
 }
 VIR_FREE(nodes);
 
@@ -24341,16 +24352,19 @@ virDomainDefCheckABIStabilityFlags(virDomainDefPtr 
src,
 goto error;
 }
 
-if (src->tpm && dst->tpm) {
-if (!virDomainTPMDefCheckABIStability(src->tpm, dst->tpm))
-goto error;
-} else if (src->tpm || dst->tpm) {
-virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
-   _("Either both target and source domains or none of "
- "them must have TPM device present"));
+if (src->ntpms != dst->ntpms) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("Target domain TPM device count %zu "
+ "does not 

[PATCH v6 00/10] Introducing TPM Proxy device support for PPC64

[Daniel Henrique Barboza]

No functional changes made from the previous version. Just a
rebase and a trivial conflict fix in news.xml after Libvirt
6.4.0 release.

changes in v6:
- fixed news.xml conflict after 6.4.0 release, moved the change
to 6.5.0
- added Stefan's r-b on patches 4, 5 and 6
- added Satheesh's tested-by tag on all patches

Gitlab tree: https://gitlab.com/danielhb/libvirt/tree/spapr_tpm_proxy_v6

v5 link: https://www.redhat.com/archives/libvir-list/2020-May/msg00997.html
v4 link: https://www.redhat.com/archives/libvir-list/2020-May/msg00814.html
v3 link: https://www.redhat.com/archives/libvir-list/2020-May/msg00642.html
v2 link: https://www.redhat.com/archives/libvir-list/2020-May/msg00604.html
v1 link: https://www.redhat.com/archives/libvir-list/2020-May/msg00351.html

Daniel Henrique Barboza (10):
  docs: documentation and schema for the new TPM Proxy model
  qemu: Extend QEMU capabilities with 'spapr-tpm-proxy'
  qemu_extdevice.c: remove unneeded 'ret' variable
  qemu_tpm, security, tests: change 'switch' clauses for 'if'
  conf, qemu, security, tests: introducing 'def->tpms' array
  qemu: add validations after TPM Proxy model introduction
  tests: add XML schema tests for the TPM Proxy device
  qemu: build command line for the TPM Proxy device
  tests/qemuxml2argvtest.c: add TPM Proxy command line tests
  docs/news.xml: update for the new TPM Proxy device

 docs/formatdomain.html.in | 19 -
 docs/news.xml | 17 +
 docs/schemas/domaincommon.rng |  1 +
 src/conf/domain_audit.c   |  4 +-
 src/conf/domain_conf.c| 50 +++-
 src/conf/domain_conf.h|  6 +-
 src/qemu/qemu_alias.c |  9 ++-
 src/qemu/qemu_capabilities.c  |  4 +
 src/qemu/qemu_capabilities.h  |  3 +
 src/qemu/qemu_cgroup.c| 10 ++-
 src/qemu/qemu_command.c   | 59 +++---
 src/qemu/qemu_domain.c| 68 ++---
 src/qemu/qemu_domain_address.c| 11 ++-
 src/qemu/qemu_extdevice.c | 24 +++---
 src/qemu/qemu_tpm.c   | 76 +--
 src/qemu/qemu_validate.c  | 19 +
 src/security/security_dac.c   |  8 +-
 src/security/security_selinux.c   | 44 +--
 src/security/virt-aa-helper.c | 14 ++--
 .../qemucapabilitiesdata/caps_4.2.0.ppc64.xml |  1 +
 .../qemucapabilitiesdata/caps_5.0.0.ppc64.xml |  1 +
 tests/qemuxml2argvdata/ppc64-tpm-double.xml   | 34 +
 .../ppc64-tpmproxy-double.xml | 38 ++
 .../ppc64-tpmproxy-single.ppc64-latest.args   | 34 +
 .../ppc64-tpmproxy-single.xml | 33 
 .../ppc64-tpmproxy-with-tpm.ppc64-latest.args | 37 +
 .../ppc64-tpmproxy-with-tpm.xml   | 36 +
 tests/qemuxml2argvtest.c  | 33 +---
 .../ppc64-tpmproxy-single.ppc64-latest.xml| 42 ++
 .../ppc64-tpmproxy-with-tpm.ppc64-latest.xml  | 46 +++
 tests/qemuxml2xmltest.c   |  2 +
 31 files changed, 631 insertions(+), 152 deletions(-)
 create mode 100644 tests/qemuxml2argvdata/ppc64-tpm-double.xml
 create mode 100644 tests/qemuxml2argvdata/ppc64-tpmproxy-double.xml
 create mode 100644 
tests/qemuxml2argvdata/ppc64-tpmproxy-single.ppc64-latest.args
 create mode 100644 tests/qemuxml2argvdata/ppc64-tpmproxy-single.xml
 create mode 100644 
tests/qemuxml2argvdata/ppc64-tpmproxy-with-tpm.ppc64-latest.args
 create mode 100644 tests/qemuxml2argvdata/ppc64-tpmproxy-with-tpm.xml
 create mode 100644 
tests/qemuxml2xmloutdata/ppc64-tpmproxy-single.ppc64-latest.xml
 create mode 100644 
tests/qemuxml2xmloutdata/ppc64-tpmproxy-with-tpm.ppc64-latest.xml

-- 
2.26.2

[PATCH v6 06/10] qemu: add validations after TPM Proxy model introduction

[Daniel Henrique Barboza]

Previous patch handled the conversion of def->tpm to the
array def->tpms and the XML parsing logic. This patch handles
the validations needed to ensure the intended behavior.

The existing qemuValidateDomainDeviceDefTPM() function was updated
to guarantee that the VIR_DOMAIN_TPM_MODEL_SPAPR_PROXY model is
exclusive to PPC64 guests and to the VIR_DOMAIN_TPM_TYPE_PASSTHROUGH
backend.

A new function called qemuDomainDefTPMsPostParse() was added to guarantee
that the following combinations in the same domain are valid:

- a single TPM device
- a single TPM Proxy device
- a single TPM + single TPM Proxy devices

And these combinations in the same domain are NOT valid:

- 2 or more TPM devices
- 2 or more TPM Proxy devices

Tested-by: Satheesh Rajendran 
Reviewed-by: Stefan Berger 
Signed-off-by: Daniel Henrique Barboza 
---
 src/qemu/qemu_domain.c   | 37 +
 src/qemu/qemu_validate.c | 19 +++
 2 files changed, 56 insertions(+)

diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index 3173fc3a3b..b1b50f05d4 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -4953,6 +4953,40 @@ qemuDomainDefTsegPostParse(virDomainDefPtr def,
 }
 
 
+static int
+qemuDomainDefTPMsPostParse(virDomainDefPtr def)
+{
+virDomainTPMDefPtr proxyTPM = NULL;
+virDomainTPMDefPtr regularTPM = NULL;
+size_t i;
+
+if (def->ntpms < 2)
+return 0;
+
+for (i = 0; i < def->ntpms; i++) {
+virDomainTPMDefPtr tpm = def->tpms[i];
+
+if (tpm->model == VIR_DOMAIN_TPM_MODEL_SPAPR_PROXY) {
+if (proxyTPM) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
+   _("only a single TPM Proxy device is 
supported"));
+return -1;
+} else {
+proxyTPM = tpm;
+}
+} else if (regularTPM) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
+   _("only a single TPM non-proxy device is 
supported"));
+return -1;
+} else {
+regularTPM = tpm;
+}
+}
+
+return 0;
+}
+
+
 static int
 qemuDomainDefPostParseBasic(virDomainDefPtr def,
 void *opaque G_GNUC_UNUSED)
@@ -5039,6 +5073,9 @@ qemuDomainDefPostParse(virDomainDefPtr def,
 if (qemuDomainDefTsegPostParse(def, qemuCaps) < 0)
 return -1;
 
+if (qemuDomainDefTPMsPostParse(def) < 0)
+return -1;
+
 return 0;
 }
 
diff --git a/src/qemu/qemu_validate.c b/src/qemu/qemu_validate.c
index 584d1375b8..28e02ebefc 100644
--- a/src/qemu/qemu_validate.c
+++ b/src/qemu/qemu_validate.c
@@ -3623,6 +3623,25 @@ qemuValidateDomainDeviceDefTPM(virDomainTPMDef *tpm,
 case VIR_DOMAIN_TPM_MODEL_SPAPR:
 flag = QEMU_CAPS_DEVICE_TPM_SPAPR;
 break;
+case VIR_DOMAIN_TPM_MODEL_SPAPR_PROXY:
+if (!ARCH_IS_PPC64(def->os.arch)) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("TPM Proxy model %s is only available for "
+ "PPC64 guests"),
+  virDomainTPMModelTypeToString(tpm->model));
+return -1;
+}
+
+/* TPM Proxy devices have 'passthrough' backend */
+if (tpm->type != VIR_DOMAIN_TPM_TYPE_PASSTHROUGH) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("TPM Proxy model %s requires "
+ "'Passthrough' backend"),
+virDomainTPMModelTypeToString(tpm->model));
+}
+
+flag = QEMU_CAPS_DEVICE_SPAPR_TPM_PROXY;
+break;
 case VIR_DOMAIN_TPM_MODEL_LAST:
 default:
 virReportEnumRangeError(virDomainTPMModel, tpm->model);
-- 
2.26.2

[PATCH v6 09/10] tests/qemuxml2argvtest.c: add TPM Proxy command line tests

[Daniel Henrique Barboza]

Add tests for both supported scenarios: a single TPM Proxy and
a TPM Proxy with a regular TPM device in the same domain.

Tested-by: Satheesh Rajendran 
Reviewed-by: Stefan Berger 
Signed-off-by: Daniel Henrique Barboza 
---
 .../ppc64-tpmproxy-single.ppc64-latest.args   | 34 +
 .../ppc64-tpmproxy-with-tpm.ppc64-latest.args | 37 +++
 tests/qemuxml2argvtest.c  |  3 ++
 3 files changed, 74 insertions(+)
 create mode 100644 
tests/qemuxml2argvdata/ppc64-tpmproxy-single.ppc64-latest.args
 create mode 100644 
tests/qemuxml2argvdata/ppc64-tpmproxy-with-tpm.ppc64-latest.args

diff --git a/tests/qemuxml2argvdata/ppc64-tpmproxy-single.ppc64-latest.args 
b/tests/qemuxml2argvdata/ppc64-tpmproxy-single.ppc64-latest.args
new file mode 100644
index 00..f606cee16b
--- /dev/null
+++ b/tests/qemuxml2argvdata/ppc64-tpmproxy-single.ppc64-latest.args
@@ -0,0 +1,34 @@
+LC_ALL=C \
+PATH=/bin \
+HOME=/tmp/lib/domain--1-QEMUGuest1 \
+USER=test \
+LOGNAME=test \
+XDG_DATA_HOME=/tmp/lib/domain--1-QEMUGuest1/.local/share \
+XDG_CACHE_HOME=/tmp/lib/domain--1-QEMUGuest1/.cache \
+XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
+QEMU_AUDIO_DRV=none \
+/usr/bin/qemu-system-ppc64 \
+-name guest=QEMUGuest1,debug-threads=on \
+-S \
+-object secret,id=masterKey0,format=raw,\
+file=/tmp/lib/domain--1-QEMUGuest1/master-key.aes \
+-machine pseries,accel=tcg,usb=off,dump-guest-core=off \
+-cpu POWER9 \
+-m 256 \
+-overcommit mem-lock=off \
+-smp 1,sockets=1,cores=1,threads=1 \
+-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \
+-display none \
+-no-user-config \
+-nodefaults \
+-chardev socket,id=charmonitor,fd=1729,server,nowait \
+-mon chardev=charmonitor,id=monitor,mode=control \
+-rtc base=utc \
+-no-shutdown \
+-boot strict=on \
+-device pci-ohci,id=usb,bus=pci.0,addr=0x1 \
+-device spapr-tpm-proxy,id=tpmproxy0,host-path=/dev/tpmrm0 \
+-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 \
+-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,\
+resourcecontrol=deny \
+-msg timestamp=on
diff --git a/tests/qemuxml2argvdata/ppc64-tpmproxy-with-tpm.ppc64-latest.args 
b/tests/qemuxml2argvdata/ppc64-tpmproxy-with-tpm.ppc64-latest.args
new file mode 100644
index 00..83eb58ae19
--- /dev/null
+++ b/tests/qemuxml2argvdata/ppc64-tpmproxy-with-tpm.ppc64-latest.args
@@ -0,0 +1,37 @@
+LC_ALL=C \
+PATH=/bin \
+HOME=/tmp/lib/domain--1-QEMUGuest1 \
+USER=test \
+LOGNAME=test \
+XDG_DATA_HOME=/tmp/lib/domain--1-QEMUGuest1/.local/share \
+XDG_CACHE_HOME=/tmp/lib/domain--1-QEMUGuest1/.cache \
+XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
+QEMU_AUDIO_DRV=none \
+/usr/bin/qemu-system-ppc64 \
+-name guest=QEMUGuest1,debug-threads=on \
+-S \
+-object secret,id=masterKey0,format=raw,\
+file=/tmp/lib/domain--1-QEMUGuest1/master-key.aes \
+-machine pseries,accel=tcg,usb=off,dump-guest-core=off \
+-cpu POWER9 \
+-m 256 \
+-overcommit mem-lock=off \
+-smp 1,sockets=1,cores=1,threads=1 \
+-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \
+-display none \
+-no-user-config \
+-nodefaults \
+-chardev socket,id=charmonitor,fd=1729,server,nowait \
+-mon chardev=charmonitor,id=monitor,mode=control \
+-rtc base=utc \
+-no-shutdown \
+-boot strict=on \
+-device pci-ohci,id=usb,bus=pci.0,addr=0x1 \
+-device spapr-tpm-proxy,id=tpmproxy0,host-path=/dev/tpmrm0 \
+-tpmdev emulator,id=tpm-tpm0,chardev=chrtpm \
+-chardev socket,id=chrtpm,path=/dev/test \
+-device tpm-spapr,tpmdev=tpm-tpm0,id=tpm0,reg=0x4000 \
+-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 \
+-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,\
+resourcecontrol=deny \
+-msg timestamp=on
diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c
index 5feb964537..83a051ed21 100644
--- a/tests/qemuxml2argvtest.c
+++ b/tests/qemuxml2argvtest.c
@@ -3076,6 +3076,9 @@ mymain(void)
 QEMU_CAPS_DEVICE_TPM_PASSTHROUGH,
 QEMU_CAPS_DEVICE_SPAPR_TPM_PROXY);
 
+DO_TEST_CAPS_LATEST_PPC64("ppc64-tpmproxy-single");
+DO_TEST_CAPS_LATEST_PPC64("ppc64-tpmproxy-with-tpm");
+
 DO_TEST("aarch64-usb-controller-qemu-xhci",
 QEMU_CAPS_OBJECT_GPEX,
 QEMU_CAPS_NEC_USB_XHCI,
-- 
2.26.2

[PATCH v6 03/10] qemu_extdevice.c: remove unneeded 'ret' variable

[Daniel Henrique Barboza]

qemuExtDevicesInitPaths() does not need 'ret'.

Tested-by: Satheesh Rajendran 
Reviewed-by: Stefan Berger 
Signed-off-by: Daniel Henrique Barboza 
---
 src/qemu/qemu_extdevice.c | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/src/qemu/qemu_extdevice.c b/src/qemu/qemu_extdevice.c
index dc188e6042..8f1bd9311e 100644
--- a/src/qemu/qemu_extdevice.c
+++ b/src/qemu/qemu_extdevice.c
@@ -73,12 +73,10 @@ static int
 qemuExtDevicesInitPaths(virQEMUDriverPtr driver,
 virDomainDefPtr def)
 {
-int ret = 0;
-
 if (def->tpm)
-ret = qemuExtTPMInitPaths(driver, def);
+return qemuExtTPMInitPaths(driver, def);
 
-return ret;
+return 0;
 }
 
 
-- 
2.26.2

[PATCH v6 04/10] qemu_tpm, security, tests: change 'switch' clauses for 'if'

[Daniel Henrique Barboza]

This trivial rework is aimed to reduce the amount of line changes
made by the next patch, when 'def->tpm' will become a 'def->tpms'
array.

Instead of using a 'switch' where only the VIR_DOMAIN_TPM_TYPE_EMULATOR
label does something, use an 'if' clause instead.

Tested-by: Satheesh Rajendran 
Reviewed-by: Stefan Berger 
Signed-off-by: Daniel Henrique Barboza 
---
 src/qemu/qemu_tpm.c | 44 +
 src/security/security_selinux.c | 16 ++--
 src/security/virt-aa-helper.c   |  7 +-
 tests/qemuxml2argvtest.c|  7 +-
 4 files changed, 10 insertions(+), 64 deletions(-)

diff --git a/src/qemu/qemu_tpm.c b/src/qemu/qemu_tpm.c
index fe567f440c..afec0e5328 100644
--- a/src/qemu/qemu_tpm.c
+++ b/src/qemu/qemu_tpm.c
@@ -680,14 +680,9 @@ qemuExtTPMInitPaths(virQEMUDriverPtr driver,
 {
 g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver);
 
-switch (def->tpm->type) {
-case VIR_DOMAIN_TPM_TYPE_EMULATOR:
+if (def->tpm->type == VIR_DOMAIN_TPM_TYPE_EMULATOR)
 return qemuTPMEmulatorInitPaths(def->tpm, cfg->swtpmStorageDir,
 def->uuid);
-case VIR_DOMAIN_TPM_TYPE_PASSTHROUGH:
-case VIR_DOMAIN_TPM_TYPE_LAST:
-break;
-}
 
 return 0;
 }
@@ -700,8 +695,7 @@ qemuExtTPMPrepareHost(virQEMUDriverPtr driver,
 g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver);
 g_autofree char *shortName = NULL;
 
-switch (def->tpm->type) {
-case VIR_DOMAIN_TPM_TYPE_EMULATOR:
+if (def->tpm->type == VIR_DOMAIN_TPM_TYPE_EMULATOR) {
 shortName = virDomainDefGetShortName(def);
 if (!shortName)
 return -1;
@@ -711,9 +705,6 @@ qemuExtTPMPrepareHost(virQEMUDriverPtr driver,
   cfg->swtpm_group,
   cfg->swtpmStateDir, cfg->user,
   shortName);
-case VIR_DOMAIN_TPM_TYPE_PASSTHROUGH:
-case VIR_DOMAIN_TPM_TYPE_LAST:
-break;
 }
 
 return 0;
@@ -723,15 +714,8 @@ qemuExtTPMPrepareHost(virQEMUDriverPtr driver,
 void
 qemuExtTPMCleanupHost(virDomainDefPtr def)
 {
-switch (def->tpm->type) {
-case VIR_DOMAIN_TPM_TYPE_EMULATOR:
+if (def->tpm->type == VIR_DOMAIN_TPM_TYPE_EMULATOR)
 qemuTPMDeleteEmulatorStorage(def->tpm);
-break;
-case VIR_DOMAIN_TPM_TYPE_PASSTHROUGH:
-case VIR_DOMAIN_TPM_TYPE_LAST:
-/* nothing to do */
-break;
-}
 }
 
 
@@ -825,15 +809,9 @@ qemuExtTPMStart(virQEMUDriverPtr driver,
 {
 virDomainTPMDefPtr tpm = vm->def->tpm;
 
-switch (tpm->type) {
-case VIR_DOMAIN_TPM_TYPE_EMULATOR:
+if (tpm->type == VIR_DOMAIN_TPM_TYPE_EMULATOR)
 return qemuExtTPMStartEmulator(driver, vm, incomingMigration);
 
-case VIR_DOMAIN_TPM_TYPE_PASSTHROUGH:
-case VIR_DOMAIN_TPM_TYPE_LAST:
-break;
-}
-
 return 0;
 }
 
@@ -845,18 +823,13 @@ qemuExtTPMStop(virQEMUDriverPtr driver,
 g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver);
 g_autofree char *shortName = NULL;
 
-switch (vm->def->tpm->type) {
-case VIR_DOMAIN_TPM_TYPE_EMULATOR:
+if (vm->def->tpm->type == VIR_DOMAIN_TPM_TYPE_EMULATOR) {
 shortName = virDomainDefGetShortName(vm->def);
 if (!shortName)
 return;
 
 qemuTPMEmulatorStop(cfg->swtpmStateDir, shortName);
 qemuSecurityCleanupTPMEmulator(driver, vm);
-break;
-case VIR_DOMAIN_TPM_TYPE_PASSTHROUGH:
-case VIR_DOMAIN_TPM_TYPE_LAST:
-break;
 }
 
 return;
@@ -873,8 +846,7 @@ qemuExtTPMSetupCgroup(virQEMUDriverPtr driver,
 int rc;
 pid_t pid;
 
-switch (def->tpm->type) {
-case VIR_DOMAIN_TPM_TYPE_EMULATOR:
+if (def->tpm->type == VIR_DOMAIN_TPM_TYPE_EMULATOR) {
 shortName = virDomainDefGetShortName(def);
 if (!shortName)
 return -1;
@@ -886,10 +858,6 @@ qemuExtTPMSetupCgroup(virQEMUDriverPtr driver,
 }
 if (virCgroupAddProcess(cgroup, pid) < 0)
 return -1;
-break;
-case VIR_DOMAIN_TPM_TYPE_PASSTHROUGH:
-case VIR_DOMAIN_TPM_TYPE_LAST:
-break;
 }
 
 return 0;
diff --git a/src/security/security_selinux.c b/src/security/security_selinux.c
index 7bb7c2b7b1..c4c0504b56 100644
--- a/src/security/security_selinux.c
+++ b/src/security/security_selinux.c
@@ -3499,10 +3499,7 @@ virSecuritySELinuxSetTPMLabels(virSecurityManagerPtr mgr,
 if (seclabel == NULL)
 return 0;
 
-switch (def->tpm->type) {
-case VIR_DOMAIN_TPM_TYPE_PASSTHROUGH:
-break;
-case VIR_DOMAIN_TPM_TYPE_EMULATOR:
+if (def->tpm->type == VIR_DOMAIN_TPM_TYPE_EMULATOR) {
 ret = virSecuritySELinuxSetFileLabels(
 mgr, def->tpm->data.emulator.storagepath,
 seclabel);
@@ -3510,9 +3507,6 @@ virSecuritySELinuxSetTPMLabels(virSecurityManagerPtr mgr,
 ret = 

[PATCH v6 08/10] qemu: build command line for the TPM Proxy device

[Daniel Henrique Barboza]

This patch wraps it up all the wiring done in previous patches,
enabling a PPC64 guest to launch a guest using a TPM Proxy
device.

Note that device validation is already being done in qemu_validate.c,
qemuValidateDomainDeviceDefTPM(), on domain define time. We don't
need to verify QEMU capabilities for this device again inside
qemu_command.c.

Tested-by: Satheesh Rajendran 
Reviewed-by: Stefan Berger 
Signed-off-by: Daniel Henrique Barboza 
---
 src/qemu/qemu_alias.c   |  5 -
 src/qemu/qemu_command.c | 27 ++-
 2 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/src/qemu/qemu_alias.c b/src/qemu/qemu_alias.c
index 85fdb85940..bb7145d630 100644
--- a/src/qemu/qemu_alias.c
+++ b/src/qemu/qemu_alias.c
@@ -408,7 +408,10 @@ qemuAssignDeviceTPMAlias(virDomainTPMDefPtr tpm,
 if (tpm->info.alias)
 return 0;
 
-tpm->info.alias = g_strdup_printf("tpm%d", idx);
+if (tpm->model == VIR_DOMAIN_TPM_MODEL_SPAPR_PROXY)
+tpm->info.alias = g_strdup_printf("tpmproxy%d", idx);
+else
+tpm->info.alias = g_strdup_printf("tpm%d", idx);
 return 0;
 }
 
diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index 59caff08f2..ab3828ec57 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -9039,6 +9039,26 @@ qemuBuildTPMCommandLine(virCommandPtr cmd,
 }
 
 
+static int
+qemuBuildTPMProxyCommandLine(virCommandPtr cmd,
+ virDomainTPMDefPtr tpm)
+{
+g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER;
+const char *filePath = NULL;
+
+filePath = tpm->data.passthrough.source.data.file.path;
+
+virCommandAddArg(cmd, "-device");
+virBufferAsprintf(, "%s,id=%s,host-path=",
+  virDomainTPMModelTypeToString(tpm->model),
+  tpm->info.alias);
+virQEMUBuildBufferEscapeComma(, filePath);
+virCommandAddArgBuffer(cmd, );
+
+return 0;
+}
+
+
 static int
 qemuBuildTPMsCommandLine(virCommandPtr cmd,
  const virDomainDef *def,
@@ -9047,8 +9067,13 @@ qemuBuildTPMsCommandLine(virCommandPtr cmd,
 size_t i;
 
 for (i = 0; i < def->ntpms; i++) {
-if (qemuBuildTPMCommandLine(cmd, def, def->tpms[i], qemuCaps) < 0)
+if (def->tpms[i]->model == VIR_DOMAIN_TPM_MODEL_SPAPR_PROXY) {
+if (qemuBuildTPMProxyCommandLine(cmd, def->tpms[i]) < 0)
+return -1;
+} else if (qemuBuildTPMCommandLine(cmd, def,
+   def->tpms[i], qemuCaps) < 0) {
 return -1;
+}
 }
 
 return 0;
-- 
2.26.2

[PATCH v6 07/10] tests: add XML schema tests for the TPM Proxy device

[Daniel Henrique Barboza]

This tests aims to exercise how a TPM Proxy device can be
added in the domain, either alone or with a regular TPM
device. It also ensures that we do not allow bogus scenarios
to slip by.

Tested-by: Satheesh Rajendran 
Reviewed-by: Stefan Berger 
Signed-off-by: Daniel Henrique Barboza 
---
 tests/qemuxml2argvdata/ppc64-tpm-double.xml   | 34 ++
 .../ppc64-tpmproxy-double.xml | 38 +++
 .../ppc64-tpmproxy-single.xml | 33 +
 .../ppc64-tpmproxy-with-tpm.xml   | 36 +++
 tests/qemuxml2argvtest.c  | 12 +
 .../ppc64-tpmproxy-single.ppc64-latest.xml| 42 +
 .../ppc64-tpmproxy-with-tpm.ppc64-latest.xml  | 46 +++
 tests/qemuxml2xmltest.c   |  2 +
 8 files changed, 243 insertions(+)
 create mode 100644 tests/qemuxml2argvdata/ppc64-tpm-double.xml
 create mode 100644 tests/qemuxml2argvdata/ppc64-tpmproxy-double.xml
 create mode 100644 tests/qemuxml2argvdata/ppc64-tpmproxy-single.xml
 create mode 100644 tests/qemuxml2argvdata/ppc64-tpmproxy-with-tpm.xml
 create mode 100644 
tests/qemuxml2xmloutdata/ppc64-tpmproxy-single.ppc64-latest.xml
 create mode 100644 
tests/qemuxml2xmloutdata/ppc64-tpmproxy-with-tpm.ppc64-latest.xml

diff --git a/tests/qemuxml2argvdata/ppc64-tpm-double.xml 
b/tests/qemuxml2argvdata/ppc64-tpm-double.xml
new file mode 100644
index 00..8730547a4d
--- /dev/null
+++ b/tests/qemuxml2argvdata/ppc64-tpm-double.xml
@@ -0,0 +1,34 @@
+
+  QEMUGuest1
+  c7a5fdbd-edaf-9455-926a-d65c16db1809
+  219100
+  219100
+  1
+  
+hvm
+  
+  
+
+
+
+  
+  
+  destroy
+  restart
+  restart
+  
+/usr/bin/qemu-system-ppc64
+
+  
+
+
+  
+
+  
+  
+
+
+  
+
+  
+
diff --git a/tests/qemuxml2argvdata/ppc64-tpmproxy-double.xml 
b/tests/qemuxml2argvdata/ppc64-tpmproxy-double.xml
new file mode 100644
index 00..12abda509e
--- /dev/null
+++ b/tests/qemuxml2argvdata/ppc64-tpmproxy-double.xml
@@ -0,0 +1,38 @@
+
+  QEMUGuest1
+  c7a5fdbd-edaf-9455-926a-d65c16db1809
+  219100
+  219100
+  1
+  
+hvm
+  
+  
+
+
+
+  
+  
+  destroy
+  restart
+  restart
+  
+/usr/bin/qemu-system-ppc64
+
+  
+
+
+  
+
+  
+
+
+  
+
+  
+
+
+  
+
+  
+
diff --git a/tests/qemuxml2argvdata/ppc64-tpmproxy-single.xml 
b/tests/qemuxml2argvdata/ppc64-tpmproxy-single.xml
new file mode 100644
index 00..729a2cdf28
--- /dev/null
+++ b/tests/qemuxml2argvdata/ppc64-tpmproxy-single.xml
@@ -0,0 +1,33 @@
+
+  QEMUGuest1
+  c7a5fdbd-edaf-9455-926a-d65c16db1809
+  219100
+  219100
+  1
+  
+hvm
+  
+  
+
+
+
+  
+  
+  destroy
+  restart
+  restart
+  
+/usr/bin/qemu-system-ppc64
+
+  
+
+
+  
+
+  
+
+
+  
+
+  
+
diff --git a/tests/qemuxml2argvdata/ppc64-tpmproxy-with-tpm.xml 
b/tests/qemuxml2argvdata/ppc64-tpmproxy-with-tpm.xml
new file mode 100644
index 00..a61ec9845c
--- /dev/null
+++ b/tests/qemuxml2argvdata/ppc64-tpmproxy-with-tpm.xml
@@ -0,0 +1,36 @@
+
+  QEMUGuest1
+  c7a5fdbd-edaf-9455-926a-d65c16db1809
+  219100
+  219100
+  1
+  
+hvm
+  
+  
+
+
+
+  
+  
+  destroy
+  restart
+  restart
+  
+/usr/bin/qemu-system-ppc64
+
+  
+
+
+
+
+  
+
+
+  
+
+
+  
+
+  
+
diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c
index 5a4ac187ef..5feb964537 100644
--- a/tests/qemuxml2argvtest.c
+++ b/tests/qemuxml2argvtest.c
@@ -3064,6 +3064,18 @@ mymain(void)
  QEMU_CAPS_NEC_USB_XHCI,
  QEMU_CAPS_DEVICE_QEMU_XHCI);
 
+DO_TEST_PARSE_ERROR("ppc64-tpmproxy-double",
+QEMU_CAPS_DEVICE_SPAPR_PCI_HOST_BRIDGE,
+QEMU_CAPS_PCI_OHCI,
+QEMU_CAPS_DEVICE_TPM_PASSTHROUGH,
+QEMU_CAPS_DEVICE_SPAPR_TPM_PROXY);
+
+DO_TEST_PARSE_ERROR("ppc64-tpm-double",
+QEMU_CAPS_DEVICE_SPAPR_PCI_HOST_BRIDGE,
+QEMU_CAPS_PCI_OHCI,
+QEMU_CAPS_DEVICE_TPM_PASSTHROUGH,
+QEMU_CAPS_DEVICE_SPAPR_TPM_PROXY);
+
 DO_TEST("aarch64-usb-controller-qemu-xhci",
 QEMU_CAPS_OBJECT_GPEX,
 QEMU_CAPS_NEC_USB_XHCI,
diff --git a/tests/qemuxml2xmloutdata/ppc64-tpmproxy-single.ppc64-latest.xml 
b/tests/qemuxml2xmloutdata/ppc64-tpmproxy-single.ppc64-latest.xml
new file mode 100644
index 00..4e0e5f24b8
--- /dev/null
+++ b/tests/qemuxml2xmloutdata/ppc64-tpmproxy-single.ppc64-latest.xml
@@ -0,0 +1,42 @@
+
+  QEMUGuest1
+  c7a5fdbd-edaf-9455-926a-d65c16db1809
+  219100
+  219100
+  1
+  
+hvm
+
+  
+  
+
+
+
+  
+  
+POWER9
+  
+  
+  destroy
+  restart
+  restart
+  
+/usr/bin/qemu-system-ppc64
+
+  
+
+
+  
+  

[PATCH v6 10/10] docs/news.xml: update for the new TPM Proxy device

[Daniel Henrique Barboza]

Tested-by: Satheesh Rajendran 
Reviewed-by: Stefan Berger 
Signed-off-by: Daniel Henrique Barboza 
---
 docs/news.xml | 17 +
 1 file changed, 17 insertions(+)

diff --git a/docs/news.xml b/docs/news.xml
index 18d96af048..fc41e53d1d 100644
--- a/docs/news.xml
+++ b/docs/news.xml
@@ -44,6 +44,23 @@
 
   
 
+  
+
+  qemu: add TPM Proxy device support
+
+
+  libvirt can now create guests using a new device type called
+  "TPM Proxy". The TPM Proxy connects to a TPM Resource Manager
+  present in the host, enabling the guest to run in secure virtual
+  machine mode with the help of an Ultravisor. Adding a TPM Proxy to
+  a pSeries guest brings no security benefits unless the guest is
+  running on a PPC64 host that has Ultravisor and TPM Resource Manager
+  support. Only one TPM Proxy is allowed per guest. A guest using
+  a TPM Proxy device can instantiate another TPM device at the same
+  time. This device is supported only for pSeries guests via the new
+  'spapr-tpm-proxy' model of the TPM 'passthrough' backend.
+
+  
 
 
 
-- 
2.26.2

[PATCH v6 02/10] qemu: Extend QEMU capabilities with 'spapr-tpm-proxy'

[Daniel Henrique Barboza]

Expose the TPM Proxy support for PPC64 guests by creating a new
cap called QEMU_CAPS_DEVICE_SPAPR_TPM_PROXY.

This device is part of the machinery the guest need to orchestrate
with the PPC64 Ultravisor the transition to the Secure VM (SVM)
mode. Inside QEMU, this device will be used with the H_TPM_COMM
hypercall to connect with the TPM Resource Manager, enabling
the guest to open and close TPM sessions with the host TPM.

Tested-by: Satheesh Rajendran 
Reviewed-by: Stefan Berger 
Reviewed-by: Ján Tomko 
Signed-off-by: Daniel Henrique Barboza 
---
 src/qemu/qemu_capabilities.c| 4 
 src/qemu/qemu_capabilities.h| 3 +++
 tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 +
 tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 1 +
 4 files changed, 9 insertions(+)

diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c
index f12769635a..1b40735b6c 100644
--- a/src/qemu/qemu_capabilities.c
+++ b/src/qemu/qemu_capabilities.c
@@ -582,6 +582,9 @@ VIR_ENUM_IMPL(virQEMUCaps,
   "tcg",
   "virtio-blk-pci.scsi.default.disabled",
   "pvscsi",
+
+  /* 370 */
+  "spapr-tpm-proxy",
 );
 
 
@@ -1306,6 +1309,7 @@ struct virQEMUCapsStringFlags virQEMUCapsObjectTypes[] = {
 { "vhost-user-fs-device", QEMU_CAPS_DEVICE_VHOST_USER_FS },
 { "tcg-accel", QEMU_CAPS_TCG },
 { "pvscsi", QEMU_CAPS_SCSI_PVSCSI },
+{ "spapr-tpm-proxy", QEMU_CAPS_DEVICE_SPAPR_TPM_PROXY },
 };
 
 
diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h
index 076ecad0f7..478cd2398f 100644
--- a/src/qemu/qemu_capabilities.h
+++ b/src/qemu/qemu_capabilities.h
@@ -564,6 +564,9 @@ typedef enum { /* virQEMUCapsFlags grouping marker for 
syntax-check */
 QEMU_CAPS_VIRTIO_BLK_SCSI_DEFAULT_DISABLED, /* virtio-blk-pci.scsi 
disabled by default */
 QEMU_CAPS_SCSI_PVSCSI, /* -device pvscsi */
 
+/* 370 */
+QEMU_CAPS_DEVICE_SPAPR_TPM_PROXY, /* -device spapr-tpm-proxy */
+
 QEMU_CAPS_LAST /* this must always be the last item */
 } virQEMUCapsFlags;
 
diff --git a/tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml 
b/tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml
index eeec181f62..b6b226da51 100644
--- a/tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml
+++ b/tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml
@@ -185,6 +185,7 @@
   
   
   
+  
   4001050
   0
   42900242
diff --git a/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml 
b/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml
index 69e5b4c8fa..ee5ab416aa 100644
--- a/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml
+++ b/tests/qemucapabilitiesdata/caps_5.0.0.ppc64.xml
@@ -201,6 +201,7 @@
   
   
   
+  
   500
   0
   42900241
-- 
2.26.2

[PATCH v6 01/10] docs: documentation and schema for the new TPM Proxy model

[Daniel Henrique Barboza]

QEMU 4.1.0 introduced a new device type called TPM Proxy, currently
implemented by PPC64 guests via a new virtual device called
'spapr-tpm-proxy' (see QEMU 0fb6bd073230 for more info).

The TPM Proxy device interacts with a TPM Resource Manager, a host
device capable of multiplexing the host TPM with multiple processes.
This allows multiple guests to access some TPM features at the
same time. Note that this mode of operation does not provide
full TPM features to be available for the guest - for that case
the guest still needs to assign a vTPM device (tpm-spapr for
PPC64 guests). Although redundant, there is currently no technical
limitation for a guest to assign both a vTPM and a TPM Proxy at the
same time.

This patch adds documentation and schema for a new TPM model
type called 'spapr-tpm-proxy' that creates this new TPM Proxy
device. This model is valid only for the 'passthrough' backend.
An example of a TPM Proxy device connected to a TPM Resource Manager
'/dev/tpmrm0' will look like this:


  

  


Tested-by: Satheesh Rajendran 
Reviewed-by: Stefan Berger 
Signed-off-by: Daniel Henrique Barboza 
---
 docs/formatdomain.html.in | 19 ++-
 docs/schemas/domaincommon.rng |  1 +
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
index 33cec1e6dd..769f01bf27 100644
--- a/docs/formatdomain.html.in
+++ b/docs/formatdomain.html.in
@@ -8798,6 +8798,18 @@ qemu-kvm -net nic,model=? /dev/null
   backend device is a TPM 2.0. Since 6.1.0,
   pSeries guests on PPC64 are supported and the default is
   tpm-spapr.
+
+  Since 6.4.0, a new model called
+  spapr-tpm-proxy was added for pSeries guests. This model
+  only works with the passthrough backend. It creates a
+  TPM Proxy device that communicates with an existing TPM Resource 
Manager
+  in the host, for example /dev/tpmrm0, enabling the 
guest to
+  run in secure virtual machine mode with the help of an Ultravisor. 
Adding
+  a TPM Proxy to a pSeries guest brings no security benefits unless 
the guest
+  is running on a PPC64 host that has an Ultravisor and a TPM Resource 
Manager.
+  Only one TPM Proxy device is allowed per guest, but a TPM Proxy 
device can
+  be added together with
+  other TPM devices.
 
   
   backend
@@ -8810,7 +8822,7 @@ qemu-kvm -net nic,model=? /dev/null
   passthrough
   
 
-  Use the host's TPM device.
+  Use the host's TPM or TPM Resource Manager device.
 
 
   This backend type requires exclusive access to a TPM device on
@@ -8818,6 +8830,11 @@ qemu-kvm -net nic,model=? /dev/null
   qualified file name is specified by path attribute of the
   source element. If no file name is specified then
   /dev/tpm0 is automatically used.
+
+  Since 6.4.0, when choosing the
+  spapr-tpm-proxy model, the file name specified is
+  expected to be a TPM Resource Manager device, e.g.
+  /dev/tpmrm0.
 
   
 
diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng
index 6727cd743b..b13cd2b330 100644
--- a/docs/schemas/domaincommon.rng
+++ b/docs/schemas/domaincommon.rng
@@ -4618,6 +4618,7 @@
 tpm-tis
 tpm-crb
 tpm-spapr
+spapr-tpm-proxy
   
 
   
-- 
2.26.2

Re: [PATCH] virsysinfo: Parse OEM strings

[Michal Privoznik]

On 6/2/20 12:56 PM, Daniel P. Berrangé wrote:

On Tue, Jun 02, 2020 at 12:53:18PM +0200, Michal Privoznik wrote:

Setting OEM strings for a domain was introduced in
v4.1.0-rc1~315. However, any application that wanted to use them
(e.g. to point to an URL where a config file is stored) had to
'dmidecode -u --oem-string N' (where N is index of the string).
Well, we can expose them under our  XML and if the
domain is running Libvirt inside it can be obtained using
virConnectGetSysinfo() API.

Signed-off-by: Michal Privoznik 
---
  src/util/virsysinfo.c   | 60 -
  tests/sysinfodata/x86sysinfo.data   | 10 +
  tests/sysinfodata/x86sysinfo.expect | 10 +
  3 files changed, 79 insertions(+), 1 deletion(-)



diff --git a/tests/sysinfodata/x86sysinfo.data 
b/tests/sysinfodata/x86sysinfo.data
index 426261041d..5615e144fb 100644
--- a/tests/sysinfodata/x86sysinfo.data
+++ b/tests/sysinfodata/x86sysinfo.data
@@ -81,3 +81,13 @@ Memory Device
Serial Number: 29057112
Asset Tag: 0839
Part Number: IMSH2GS13A1F1C-10F
+
+OEM Strings
+String 1: Default string
+String 2: Default string
+String 3: MIAMI
+String 4: Default string
+String 5: F
+String 6: F
+String 7: F
+String 8: Default string


What does dmidecode do for escaping (if anything) if I set a string
value of

 "Ha ha ha try parsing\nString 3: this correctly\nString 
4:then"


This is what I put into domain XML:

  

  Hello
  World
  Ha ha ha try parsing\n
  String 3: this correctly\n
  String 4:then
  This is, more tricky value=escaped

  

This is how it appeared on the cmd line:

-smbios 'type=11,value=Hello,value=World,value=Ha ha ha try parsing\n
  String 3: this correctly\n
  String 4:then,value=This is,, more tricky value=escaped' \


Now, inside the guest, plain dmidecode replaces \n with a dot:

# dmidecode -t 11
Handle 0x0E00, DMI type 11, 5 bytes
OEM Strings
String 1: Hello
String 2: World
String 3: Ha ha ha try parsing\n.  String 3: this 
correctly\n.  String 4:then

String 4: This is, more tricky value=escaped



But if I tell it to not decode entries, then it displays the string 
correctly:


# dmidecode -u --oem-string 3
Ha ha ha try parsing\n
  String 3: this correctly\n
  String 4:then


So maybe we should use the latter instead? We could use 'dmidecode -u 
--oem-string count' to get the number of strings and then iterate 
through each one of them.


Michal

Re: [libvirt PATCH 3/5] ci: Use GitLab container registry

[Daniel P . Berrangé]

On Tue, Jun 02, 2020 at 02:45:30PM +0200, Andrea Bolognani wrote:
> On Tue, 2020-06-02 at 12:23 +0100, Daniel P. Berrangé wrote:
> > On Tue, Jun 02, 2020 at 01:10:08PM +0200, Andrea Bolognani wrote:
> > > On Tue, 2020-06-02 at 11:33 +0100, Daniel P. Berrangé wrote:
> > > > I don't think we should be building container images that we're not 
> > > > going
> > > > to be using in any of the jobs, as it can only ever slow down the build
> > > > overall.
> > > 
> > > These same containers are also available for use outside of CI, eg.
> > > with 'make ci-build', so I think we should keep building them.
> > 
> > That only needs them built on the master branch of the main repo
> > though, not every branch in every fork
> 
> Fair enough. So what you're suggesting is something like
> 
>   .container_optional_job_template: _optional_job_definition
> <<: *container_job_definition
> allow_failure: true
> except:
>   variables:
> - $CI_PROJECT_NAMESPACE != libvirt
> only:
>   - master
> 
> correct?

Perhaps just matching what we do with extra builds:

  only:
- master
- /^ci-full-.*$/

so users can still get the full set of builds in their fork if they push
to certain branch.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [libvirt PATCH 3/5] ci: Use GitLab container registry

[Andrea Bolognani]

On Tue, 2020-06-02 at 12:23 +0100, Daniel P. Berrangé wrote:
> On Tue, Jun 02, 2020 at 01:10:08PM +0200, Andrea Bolognani wrote:
> > On Tue, 2020-06-02 at 11:33 +0100, Daniel P. Berrangé wrote:
> > > I don't think we should be building container images that we're not going
> > > to be using in any of the jobs, as it can only ever slow down the build
> > > overall.
> > 
> > These same containers are also available for use outside of CI, eg.
> > with 'make ci-build', so I think we should keep building them.
> 
> That only needs them built on the master branch of the main repo
> though, not every branch in every fork

Fair enough. So what you're suggesting is something like

  .container_optional_job_template: _optional_job_definition
<<: *container_job_definition
allow_failure: true
except:
  variables:
- $CI_PROJECT_NAMESPACE != libvirt
only:
  - master

correct?

> > As for slowing down builds, that still only applies to the first
> > build after Dockerfiles are updated, so I don't think it ultimately
> > matters very much.
> 
> I'd expect a rebiuld if the distro base image changes which could
> be fairly often for the rawhide like distros.

This advantage might be cancelled out by the fact that only a limited
number of shared runners is available, so if for example we have
access to 5 runners, whether we run 6 or 10 jobs will make no
difference in terms of total pipeline completion time.

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: libvirt 6.4.0 is available

[Andrea Bolognani]

On Tue, 2020-06-02 at 12:14 +0200, Daniel Veillard wrote:
>   I made the release earlier, it is tagged in git and signed source
> tarball and rpm are available at the usual place:
> 
>https://libvirt.org/sources/

Can we stop building source RPMs entirely please? We have already
agreed that they're pretty much pointless and that we're not going
to carry them over when we start publishing releases on GitLab.

> There isn't that much detail about the release in the news file

Unfortunately

  https://www.redhat.com/archives/libvir-list/2020-June/msg00024.html

missed the release by an hour or so O:-)

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [libvirt PATCH 4/5] ci: Update build system integration

[Daniel P . Berrangé]

On Tue, Jun 02, 2020 at 01:22:50PM +0200, Andrea Bolognani wrote:
> On Tue, 2020-06-02 at 11:36 +0100, Daniel P. Berrangé wrote:
> > On Fri, May 29, 2020 at 03:00:43PM +0200, Andrea Bolognani wrote:
> > > +# Registry where container images are stored
> > > +CI_IMAGE_REGISTRY = registry.gitlab.com
> > > +
> > >  # Location of the container images we're going to pull
> > >  # Can be useful to overridde to use a locally built
> > >  # image instead
> > > -CI_IMAGE_PREFIX = quay.io/libvirt/buildenv-libvirt-
> > > +CI_IMAGE_PREFIX = libvirt/libvirt/ci-
> > 
> [...]
> > 
> > >  ci-run-command@%: ci-prepare-tree
> > > - $(CI_ENGINE) run $(CI_ENGINE_ARGS) $(CI_IMAGE_PREFIX)$*$(CI_IMAGE_TAG) \
> > > + image=; \
> > > + if test "$(CI_IMAGE_REGISTRY)"; then \
> > 
> > What condition is this expected to be testing ?
> 
> The case where someone built a (possibly custom) image locally and
> wants to use it with something like
> 
>   $ make ci-build@centos-8 \
>  CI_IMAGE_REGISTRY= \
>  CI_IMAGE_PREFIX=my-
> 
> This usage scenario is explicity called out in the comment for the
> CI_IMAGE_PREFIX variable.

This needs to be "test -n" then to validate non-zero length variable


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [libvirt PATCH 5/5] ci: Improve CI_IMAGE_TAG handling

[Andrea Bolognani]

On Tue, 2020-06-02 at 11:37 +0100, Daniel P. Berrangé wrote:
> On Fri, May 29, 2020 at 03:00:44PM +0200, Andrea Bolognani wrote:
> > -# The default tag is ':latest' but if the container
> > +# The default tag is 'latest' but if the container
> >  # repo above uses different conventions this can override it
> > -CI_IMAGE_TAG = :latest
> > +CI_IMAGE_TAG = latest
> >  
> >  # We delete the virtual root after completion, set
> >  # to 0 if you need to keep it around for debugging
> > @@ -220,7 +220,10 @@ ci-run-command@%: ci-prepare-tree
> > if test "$(CI_IMAGE_REGISTRY)"; then \
> > image="$${image}$(CI_IMAGE_REGISTRY)/"; \
> > fi; \
> > -   image="$${image}$(CI_IMAGE_PREFIX)$*$(CI_IMAGE_TAG)"; \
> > +   image="$${image}$(CI_IMAGE_PREFIX)$*"; \
> > +   if test "$(CI_IMAGE_TAG)"; then \
> > +   image="$${image}:$(CI_IMAGE_TAG)"; \
> > +   fi; \
> 
> Again, I'm not seeing what this test is for

It was intended to account for the possibility of the user passing
an empty CI_IMAGE_TAG, but now that I think about it a bit more
that would be quite a silly thing to do and erroring out in that
case is perfectly fine. I'll drop this commit.

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [libvirt PATCH 4/5] ci: Update build system integration

[Andrea Bolognani]

On Tue, 2020-06-02 at 11:36 +0100, Daniel P. Berrangé wrote:
> On Fri, May 29, 2020 at 03:00:43PM +0200, Andrea Bolognani wrote:
> > +# Registry where container images are stored
> > +CI_IMAGE_REGISTRY = registry.gitlab.com
> > +
> >  # Location of the container images we're going to pull
> >  # Can be useful to overridde to use a locally built
> >  # image instead
> > -CI_IMAGE_PREFIX = quay.io/libvirt/buildenv-libvirt-
> > +CI_IMAGE_PREFIX = libvirt/libvirt/ci-
> 
[...]
> 
> >  ci-run-command@%: ci-prepare-tree
> > -   $(CI_ENGINE) run $(CI_ENGINE_ARGS) $(CI_IMAGE_PREFIX)$*$(CI_IMAGE_TAG) \
> > +   image=; \
> > +   if test "$(CI_IMAGE_REGISTRY)"; then \
> 
> What condition is this expected to be testing ?

The case where someone built a (possibly custom) image locally and
wants to use it with something like

  $ make ci-build@centos-8 \
 CI_IMAGE_REGISTRY= \
 CI_IMAGE_PREFIX=my-

This usage scenario is explicity called out in the comment for the
CI_IMAGE_PREFIX variable.

But, I just realized I can avoid introducing CI_IMAGE_PREFIX by
changing the code slightly in ci/list-images.sh, so I'll do that
instead and avoid the extra complexity.

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [libvirt PATCH 3/5] ci: Use GitLab container registry

[Daniel P . Berrangé]

On Tue, Jun 02, 2020 at 01:10:08PM +0200, Andrea Bolognani wrote:
> On Tue, 2020-06-02 at 11:33 +0100, Daniel P. Berrangé wrote:
> > On Fri, May 29, 2020 at 03:00:42PM +0200, Andrea Bolognani wrote:
> > > +# We build many containers which can be useful to debug problems but are 
> > > not
> > > +# needed for the pipeline itself to complete: those sometimes fail, and 
> > > when
> > > +# that happens it's mostly because of temporary issues with Debian sid. 
> > > We
> > > +# don't want those failures to affect the overall pipeline status
> > > +.container_optional_job_template: _optional_job_definition
> > > +  <<: *container_job_definition
> > > +  allow_failure: true
> > 
> > I don't think we should be building container images that we're not going
> > to be using in any of the jobs, as it can only ever slow down the build
> > overall.
> 
> These same containers are also available for use outside of CI, eg.
> with 'make ci-build', so I think we should keep building them.

That only needs them built on the master branch of the main repo
though, not every branch in every fork

> As for slowing down builds, that still only applies to the first
> build after Dockerfiles are updated, so I don't think it ultimately
> matters very much.

I'd expect a rebiuld if the distro base image changes which could
be fairly often for the rawhide like distros.

> 
> > > +# Container build jobs
> > > +
> > > +centos-7-container:
> > 
> > IMHO we should name these to match the build  job. eg arch, then distro
> > 
> >x64-centos-7-container
> 
> Okay.
> 
> > > +debian-9-cross-armv6l-container:
> > > +  <<: *container_job_definition
> > > +  variables:
> > > +NAME: debian-9-cross-armv6l
> > 
> > This container, and many others are only used by the "extra" build
> > jobs, so should be subject to the same filtering.
> 
> Okay, even though as we discussed separately the whole idea of
> splitting jobs between regular and extra might be more trouble than
> it's worth and be more confusing than helpful.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [libvirt PATCH 3/5] ci: Use GitLab container registry

[Andrea Bolognani]

On Tue, 2020-06-02 at 11:33 +0100, Daniel P. Berrangé wrote:
> On Fri, May 29, 2020 at 03:00:42PM +0200, Andrea Bolognani wrote:
> > +# We build many containers which can be useful to debug problems but are 
> > not
> > +# needed for the pipeline itself to complete: those sometimes fail, and 
> > when
> > +# that happens it's mostly because of temporary issues with Debian sid. We
> > +# don't want those failures to affect the overall pipeline status
> > +.container_optional_job_template: _optional_job_definition
> > +  <<: *container_job_definition
> > +  allow_failure: true
> 
> I don't think we should be building container images that we're not going
> to be using in any of the jobs, as it can only ever slow down the build
> overall.

These same containers are also available for use outside of CI, eg.
with 'make ci-build', so I think we should keep building them.

As for slowing down builds, that still only applies to the first
build after Dockerfiles are updated, so I don't think it ultimately
matters very much.

> > +# Container build jobs
> > +
> > +centos-7-container:
> 
> IMHO we should name these to match the build  job. eg arch, then distro
> 
>x64-centos-7-container

Okay.

> > +debian-9-cross-armv6l-container:
> > +  <<: *container_job_definition
> > +  variables:
> > +NAME: debian-9-cross-armv6l
> 
> This container, and many others are only used by the "extra" build
> jobs, so should be subject to the same filtering.

Okay, even though as we discussed separately the whole idea of
splitting jobs between regular and extra might be more trouble than
it's worth and be more confusing than helpful.

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [libvirt PATCH 2/5] ci: Add 'other' stage

[Andrea Bolognani]

On Tue, 2020-06-02 at 11:26 +0100, Daniel P. Berrangé wrote:
> On Fri, May 29, 2020 at 03:00:41PM +0200, Andrea Bolognani wrote:
> > +++ b/.gitlab-ci.yml
> > @@ -5,6 +5,7 @@ stages:
> >- prebuild
> >- native_build
> >- cross_build
> > +  - other
> 
> Can we keep this before the native_build, as I wanted to see quick
> reports of code style mistakes.

Sure, that should work as long as container builds are still
performed earlier.

Do you have a good name for the stage in mind? O:-)

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [PATCH] virsysinfo: Parse OEM strings

[Daniel P . Berrangé]

On Tue, Jun 02, 2020 at 12:53:18PM +0200, Michal Privoznik wrote:
> Setting OEM strings for a domain was introduced in
> v4.1.0-rc1~315. However, any application that wanted to use them
> (e.g. to point to an URL where a config file is stored) had to
> 'dmidecode -u --oem-string N' (where N is index of the string).
> Well, we can expose them under our  XML and if the
> domain is running Libvirt inside it can be obtained using
> virConnectGetSysinfo() API.
> 
> Signed-off-by: Michal Privoznik 
> ---
>  src/util/virsysinfo.c   | 60 -
>  tests/sysinfodata/x86sysinfo.data   | 10 +
>  tests/sysinfodata/x86sysinfo.expect | 10 +
>  3 files changed, 79 insertions(+), 1 deletion(-)

> diff --git a/tests/sysinfodata/x86sysinfo.data 
> b/tests/sysinfodata/x86sysinfo.data
> index 426261041d..5615e144fb 100644
> --- a/tests/sysinfodata/x86sysinfo.data
> +++ b/tests/sysinfodata/x86sysinfo.data
> @@ -81,3 +81,13 @@ Memory Device
>   Serial Number: 29057112
>   Asset Tag: 0839
>   Part Number: IMSH2GS13A1F1C-10F
> +
> +OEM Strings
> +String 1: Default string
> +String 2: Default string
> +String 3: MIAMI
> +String 4: Default string
> +String 5: F
> +String 6: F
> +String 7: F
> +String 8: Default string

What does dmidecode do for escaping (if anything) if I set a string
value of

"Ha ha ha try parsing\nString 3: this correctly\nString 
4:then"

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

[PATCH] virsysinfo: Parse OEM strings

[Michal Privoznik]

Setting OEM strings for a domain was introduced in
v4.1.0-rc1~315. However, any application that wanted to use them
(e.g. to point to an URL where a config file is stored) had to
'dmidecode -u --oem-string N' (where N is index of the string).
Well, we can expose them under our  XML and if the
domain is running Libvirt inside it can be obtained using
virConnectGetSysinfo() API.

Signed-off-by: Michal Privoznik 
---
 src/util/virsysinfo.c   | 60 -
 tests/sysinfodata/x86sysinfo.data   | 10 +
 tests/sysinfodata/x86sysinfo.expect | 10 +
 3 files changed, 79 insertions(+), 1 deletion(-)

diff --git a/src/util/virsysinfo.c b/src/util/virsysinfo.c
index 41f4d1cff9..94f7ca8811 100644
--- a/src/util/virsysinfo.c
+++ b/src/util/virsysinfo.c
@@ -919,6 +919,61 @@ virSysinfoParseX86Chassis(const char *base,
 }
 
 
+static int
+virSysinfoParseOEMStrings(const char *base,
+  virSysinfoOEMStringsDefPtr *stringsRet)
+{
+virSysinfoOEMStringsDefPtr strings = NULL;
+int ret = -1;
+const char *cur;
+
+if (!(cur = strstr(base, "OEM Strings")))
+return 0;
+
+if (VIR_ALLOC(strings) < 0)
+return -1;
+
+while ((cur = strstr(cur, "String "))) {
+char *eol;
+
+cur += 7;
+
+if (!(eol = strchr(cur, '\n'))) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("Malformed output of dmidecode"));
+goto cleanup;
+}
+
+while (g_ascii_isdigit(*cur))
+cur++;
+
+if (*cur != ':') {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("Malformed output of dmidecode"));
+goto cleanup;
+}
+
+cur += 2;
+
+virSkipSpacesBackwards(cur, );
+if (!eol)
+continue;
+
+if (VIR_EXPAND_N(strings->values, strings->nvalues, 1) < 0)
+goto cleanup;
+
+strings->values[strings->nvalues - 1] = g_strndup(cur, eol - cur);
+}
+
+*stringsRet = g_steal_pointer();
+ret = 0;
+
+ cleanup:
+virSysinfoOEMStringsDefFree(strings);
+return ret;
+}
+
+
 static int
 virSysinfoParseX86Processor(const char *base, virSysinfoDefPtr ret)
 {
@@ -1132,7 +1187,7 @@ virSysinfoReadDMI(void)
 return NULL;
 }
 
-cmd = virCommandNewArgList(path, "-q", "-t", "0,1,2,3,4,17", NULL);
+cmd = virCommandNewArgList(path, "-q", "-t", "0,1,2,3,4,11,17", NULL);
 VIR_FREE(path);
 virCommandSetOutputBuffer(cmd, );
 if (virCommandRun(cmd, NULL) < 0)
@@ -1155,6 +1210,9 @@ virSysinfoReadDMI(void)
 if (virSysinfoParseX86Chassis(outbuf, >chassis) < 0)
 goto error;
 
+if (virSysinfoParseOEMStrings(outbuf, >oemStrings) < 0)
+goto error;
+
 ret->nprocessor = 0;
 ret->processor = NULL;
 if (virSysinfoParseX86Processor(outbuf, ret) < 0)
diff --git a/tests/sysinfodata/x86sysinfo.data 
b/tests/sysinfodata/x86sysinfo.data
index 426261041d..5615e144fb 100644
--- a/tests/sysinfodata/x86sysinfo.data
+++ b/tests/sysinfodata/x86sysinfo.data
@@ -81,3 +81,13 @@ Memory Device
Serial Number: 29057112
Asset Tag: 0839
Part Number: IMSH2GS13A1F1C-10F
+
+OEM Strings
+String 1: Default string
+String 2: Default string
+String 3: MIAMI
+String 4: Default string
+String 5: F
+String 6: F
+String 7: F
+String 8: Default string
diff --git a/tests/sysinfodata/x86sysinfo.expect 
b/tests/sysinfodata/x86sysinfo.expect
index fcdd790cbd..118cc4277e 100644
--- a/tests/sysinfodata/x86sysinfo.expect
+++ b/tests/sysinfodata/x86sysinfo.expect
@@ -50,4 +50,14 @@
 29057112
 IMSH2GS13A1F1C-10F
   
+  
+Default string
+Default string
+MIAMI
+Default string
+F
+F
+F
+Default string
+  
 
-- 
2.26.2

Re: [libvirt PATCH 4/5] ci: Update build system integration

[Daniel P . Berrangé]

On Fri, May 29, 2020 at 03:00:43PM +0200, Andrea Bolognani wrote:
> The ci-* targets need to know where our container images are stored
> and how they are called to work, so now that we use the GitLab
> container registry instead of Quay some changes are necessary.
> 
> Signed-off-by: Andrea Bolognani 
> ---
>  ci/Makefile   | 16 
>  ci/list-images.sh | 24 ++--
>  2 files changed, 18 insertions(+), 22 deletions(-)
> 
> diff --git a/ci/Makefile b/ci/Makefile
> index bc1dac11e3..e1a5faaba6 100644
> --- a/ci/Makefile
> +++ b/ci/Makefile
> @@ -47,10 +47,13 @@ CI_PREPARE_SCRIPT = $(CI_ROOTDIR)/prepare.sh
>  # Script containing build instructions
>  CI_BUILD_SCRIPT = $(CI_ROOTDIR)/build.sh
>  
> +# Registry where container images are stored
> +CI_IMAGE_REGISTRY = registry.gitlab.com
> +
>  # Location of the container images we're going to pull
>  # Can be useful to overridde to use a locally built
>  # image instead
> -CI_IMAGE_PREFIX = quay.io/libvirt/buildenv-libvirt-
> +CI_IMAGE_PREFIX = libvirt/libvirt/ci-
>  
>  # The default tag is ':latest' but if the container
>  # repo above uses different conventions this can override it
> @@ -213,7 +216,12 @@ ci-prepare-tree: ci-check-engine
>   fi
>  
>  ci-run-command@%: ci-prepare-tree
> - $(CI_ENGINE) run $(CI_ENGINE_ARGS) $(CI_IMAGE_PREFIX)$*$(CI_IMAGE_TAG) \
> + image=; \
> + if test "$(CI_IMAGE_REGISTRY)"; then \

What condition is this expected to be testing ?

> + image="$${image}$(CI_IMAGE_REGISTRY)/"; \
> + fi; \
> + image="$${image}$(CI_IMAGE_PREFIX)$*$(CI_IMAGE_TAG)"; \
> + $(CI_ENGINE) run $(CI_ENGINE_ARGS) "$$image" \
>   /bin/bash -c ' \
>   $(CI_USER_HOME)/prepare || exit 1; \
>   sudo \

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [libvirt PATCH 5/5] ci: Improve CI_IMAGE_TAG handling

[Daniel P . Berrangé]

On Fri, May 29, 2020 at 03:00:44PM +0200, Andrea Bolognani wrote:
> Since we're already building the full container image reference
> dynamically at this point, we can finally get rid of the annoying
> requirement to include ":" in CI_IMAGE_TAG.
> 
> Signed-off-by: Andrea Bolognani 
> ---
>  ci/Makefile | 9 ++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/ci/Makefile b/ci/Makefile
> index e1a5faaba6..96e4d62611 100644
> --- a/ci/Makefile
> +++ b/ci/Makefile
> @@ -55,9 +55,9 @@ CI_IMAGE_REGISTRY = registry.gitlab.com
>  # image instead
>  CI_IMAGE_PREFIX = libvirt/libvirt/ci-
>  
> -# The default tag is ':latest' but if the container
> +# The default tag is 'latest' but if the container
>  # repo above uses different conventions this can override it
> -CI_IMAGE_TAG = :latest
> +CI_IMAGE_TAG = latest
>  
>  # We delete the virtual root after completion, set
>  # to 0 if you need to keep it around for debugging
> @@ -220,7 +220,10 @@ ci-run-command@%: ci-prepare-tree
>   if test "$(CI_IMAGE_REGISTRY)"; then \
>   image="$${image}$(CI_IMAGE_REGISTRY)/"; \
>   fi; \
> - image="$${image}$(CI_IMAGE_PREFIX)$*$(CI_IMAGE_TAG)"; \
> + image="$${image}$(CI_IMAGE_PREFIX)$*"; \
> + if test "$(CI_IMAGE_TAG)"; then \
> + image="$${image}:$(CI_IMAGE_TAG)"; \
> + fi; \

Again, I'm not seeing what this test is for

>   $(CI_ENGINE) run $(CI_ENGINE_ARGS) "$$image" \
>   /bin/bash -c ' \
>   $(CI_USER_HOME)/prepare || exit 1; \
> -- 
> 2.25.4
> 

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [libvirt PATCH 3/5] ci: Use GitLab container registry

[Daniel P . Berrangé]

On Fri, May 29, 2020 at 03:00:42PM +0200, Andrea Bolognani wrote:
> Instead of using pre-built containers hosted on Quay, build
> containers as part of the GitLab CI pipeline and upload them to the
> GitLab container registry for later use.
> 
> This will not significantly slow down builds, because containers are
> only rebuilt when the corresponding Dockerfile has been modified.
> 
> Signed-off-by: Andrea Bolognani 
> ---
>  .gitlab-ci.yml| 234 +-
>  ci/containers/README.rst  |  14 ++
>  ci/containers/ci-centos-7.Dockerfile  | 137 ++
>  ci/containers/ci-centos-8.Dockerfile  | 108 
>  .../ci-debian-10-cross-aarch64.Dockerfile | 122 +
>  .../ci-debian-10-cross-armv6l.Dockerfile  | 120 +
>  .../ci-debian-10-cross-armv7l.Dockerfile  | 121 +
>  .../ci-debian-10-cross-i686.Dockerfile| 121 +
>  .../ci-debian-10-cross-mips.Dockerfile| 121 +
>  .../ci-debian-10-cross-mips64el.Dockerfile| 121 +
>  .../ci-debian-10-cross-mipsel.Dockerfile  | 121 +
>  .../ci-debian-10-cross-ppc64le.Dockerfile | 121 +
>  .../ci-debian-10-cross-s390x.Dockerfile   | 121 +
>  ci/containers/ci-debian-10.Dockerfile | 112 +
>  .../ci-debian-9-cross-aarch64.Dockerfile  | 126 ++
>  .../ci-debian-9-cross-armv6l.Dockerfile   | 124 ++
>  .../ci-debian-9-cross-armv7l.Dockerfile   | 125 ++
>  .../ci-debian-9-cross-mips.Dockerfile | 125 ++
>  .../ci-debian-9-cross-mips64el.Dockerfile | 125 ++
>  .../ci-debian-9-cross-mipsel.Dockerfile   | 125 ++
>  .../ci-debian-9-cross-ppc64le.Dockerfile  | 125 ++
>  .../ci-debian-9-cross-s390x.Dockerfile| 125 ++
>  ci/containers/ci-debian-9.Dockerfile  | 116 +
>  .../ci-debian-sid-cross-aarch64.Dockerfile| 122 +
>  .../ci-debian-sid-cross-armv6l.Dockerfile | 120 +
>  .../ci-debian-sid-cross-armv7l.Dockerfile | 121 +
>  .../ci-debian-sid-cross-i686.Dockerfile   | 121 +
>  .../ci-debian-sid-cross-mips.Dockerfile   | 121 +
>  .../ci-debian-sid-cross-mips64el.Dockerfile   | 121 +
>  .../ci-debian-sid-cross-mipsel.Dockerfile | 120 +
>  .../ci-debian-sid-cross-ppc64le.Dockerfile| 121 +
>  .../ci-debian-sid-cross-s390x.Dockerfile  | 121 +
>  ci/containers/ci-debian-sid.Dockerfile| 112 +
>  ci/containers/ci-fedora-31.Dockerfile | 109 
>  ci/containers/ci-fedora-32.Dockerfile | 109 
>  ...ci-fedora-rawhide-cross-mingw32.Dockerfile | 129 ++
>  ...ci-fedora-rawhide-cross-mingw64.Dockerfile | 129 ++
>  ci/containers/ci-fedora-rawhide.Dockerfile| 110 
>  ci/containers/ci-opensuse-151.Dockerfile  | 109 
>  ci/containers/ci-ubuntu-1804.Dockerfile   | 117 +
>  ci/containers/ci-ubuntu-2004.Dockerfile   | 113 +
>  ci/containers/refresh |  43 
>  42 files changed, 4973 insertions(+), 5 deletions(-)
>  create mode 100644 ci/containers/README.rst
>  create mode 100644 ci/containers/ci-centos-7.Dockerfile
>  create mode 100644 ci/containers/ci-centos-8.Dockerfile
>  create mode 100644 ci/containers/ci-debian-10-cross-aarch64.Dockerfile
>  create mode 100644 ci/containers/ci-debian-10-cross-armv6l.Dockerfile
>  create mode 100644 ci/containers/ci-debian-10-cross-armv7l.Dockerfile
>  create mode 100644 ci/containers/ci-debian-10-cross-i686.Dockerfile
>  create mode 100644 ci/containers/ci-debian-10-cross-mips.Dockerfile
>  create mode 100644 ci/containers/ci-debian-10-cross-mips64el.Dockerfile
>  create mode 100644 ci/containers/ci-debian-10-cross-mipsel.Dockerfile
>  create mode 100644 ci/containers/ci-debian-10-cross-ppc64le.Dockerfile
>  create mode 100644 ci/containers/ci-debian-10-cross-s390x.Dockerfile
>  create mode 100644 ci/containers/ci-debian-10.Dockerfile
>  create mode 100644 ci/containers/ci-debian-9-cross-aarch64.Dockerfile
>  create mode 100644 ci/containers/ci-debian-9-cross-armv6l.Dockerfile
>  create mode 100644 ci/containers/ci-debian-9-cross-armv7l.Dockerfile
>  create mode 100644 ci/containers/ci-debian-9-cross-mips.Dockerfile
>  create mode 100644 ci/containers/ci-debian-9-cross-mips64el.Dockerfile
>  create mode 100644 ci/containers/ci-debian-9-cross-mipsel.Dockerfile
>  create mode 100644 ci/containers/ci-debian-9-cross-ppc64le.Dockerfile
>  create mode 100644 ci/containers/ci-debian-9-cross-s390x.Dockerfile
>  create mode 100644 ci/containers/ci-debian-9.Dockerfile
>  create mode 100644 ci/containers/ci-debian-sid-cross-aarch64.Dockerfile
>  create mode 100644 ci/containers/ci-debian-sid-cross-armv6l.Dockerfile
>  create mode 100644 ci/containers/ci-debian-sid-cross-armv7l.Dockerfile
>  create mode 100644 

Re: [libvirt PATCH 2/5] ci: Add 'other' stage

[Daniel P . Berrangé]

On Fri, May 29, 2020 at 03:00:41PM +0200, Andrea Bolognani wrote:
> We're going to build container images as part of the CI pipeline
> soon, which means that we need to move all jobs that run in a
> container image which is not provided by an external project (such
> as the one that we use for DCO checking) later in the pipeline or
> they will fail.
> 
> Signed-off-by: Andrea Bolognani 
> ---
>  .gitlab-ci.yml | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 35895a4931..8a5b3372de 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -5,6 +5,7 @@ stages:
>- prebuild
>- native_build
>- cross_build
> +  - other

Can we keep this before the native_build, as I wanted to see quick
reports of code style mistakes.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [libvirt PATCH 1/5] ci: Use variables to build image names

[Daniel P . Berrangé]

On Fri, May 29, 2020 at 03:00:40PM +0200, Andrea Bolognani wrote:
> This removes a lot of repetition and makes the configuration much
> easier to read.
> 
> Signed-off-by: Andrea Bolognani 
> ---
>  .gitlab-ci.yml | 79 --
>  1 file changed, 57 insertions(+), 22 deletions(-)

Reviewed-by: Daniel P. Berrangé 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

libvirt 6.4.0 is available

[Daniel Veillard]

  I made the release earlier, it is tagged in git and signed source
tarball and rpm are available at the usual place:

   https://libvirt.org/sources/

I didn't push an RC2 this week end as there was no code change from RC1
only a bug fix made its way yesterday. I also pushed a python binding
6.4.0 release that can be found at:

   https://libvirt.org/sources/python/

There isn't that much detail about the release in the news file,
obviously a decent amount of bug fixes and improvements though:

Improvements:

- qemu: stricter validation for disk type='lun'
  The 'lun' type is meant for SCSI command passthrough, which can't be
  achieved if qemu's block layer features are used. Disk type='lun' is
  now allowed only when the format is 'raw' and no other block layer
  features are requested.

Bug fixes:

- qemu: fixed regression in network device hotplug with new qemu versions
  Starting from QEMU-5.0 it's required to conform to strict schema when
  hotplugging network devices. Libvirt didn't conform to the schema so in
  versions prior to 6.4.0 network device hotplug fails in certain cases.
  This version fixes it and adds stricter testing to prevent further
  issues.

 Thanks everybody for your help with this release,

   enjoy and stay safe !

Daniel

-- 
Daniel Veillard  | Red Hat Developers Tools http://developer.redhat.com/
veill...@redhat.com  | libxml Gnome XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | virtualization library  http://libvirt.org/

Re: [PATCH 0/3] Rmove support of tftp storage protocol

[Max Reitz]

On 22.05.20 04:07, Han Han wrote:
> 
> 
> On Thu, May 21, 2020 at 5:21 PM Daniel P. Berrangé  > wrote:
> 
> On Thu, May 21, 2020 at 04:05:47PM +0800, Han Han wrote:
> > On Thu, May 21, 2020 at 2:44 PM Peter Krempa  > wrote:
> >
> > > On Thu, May 21, 2020 at 14:08:17 +0800, Han Han wrote:
> > > > Let's drop tftp support since it is ususabled before or after
> QEMU v2.8
> > > > [1].
> > > >
> > > > [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1372143
> > > >
> > > >
> > > > Han Han (3):
> > > >   docs: Remove tftp protocol support from docs
> > > >   src: Remove tftp network storage support
> > > >   news: Remove support for tftp storage protocol
> > >
> > > NACK series:
> > >
> > > 1) qemu 2.8 is still supported by libvirt
> > >
> > However, the tftp is actually unusable for all versions of qemu,
> while the
> > formatdomain doc
> > states that it is optional protocol
> 
> Do you have any more information on this. I'd be surprised if it was
> /always/ unusuable, as opposed to initially working and then getting
> broken in some particular version of QEMU.
> 
> I tested on qemu-kvm-1.5.3-174.el7.x86_64 curl-7.29.0-58.el7.x86_64
> tftp-server-5.2-22.el7.x86_64, it doesn't work, either:
> ➜  ~ curl tftp://127.0.0.1/img  -o /tmp/tftp      
>                          
>   % Total    % Received % Xferd  Average Speed   Time    Time     Time
>  Current
>                                  Dload  Upload   Total   Spent    Left
>  Speed
> 100  100M  100  100M    0     0  20.6M      0  0:00:04  0:00:04 --:--:--
> 20.6M
> 100  100M  100  100M    0     0  20.6M      0  0:00:04  0:00:04 --:--:--
> 20.6M
> ➜  ~ qemu-img create -f qcow2 -b 'json:{"file.driver":"tftp",
> "file.url":"tftp://127.0.0.1/img "}'
> /var/lib/libvirt/images/tftp.img
> Formatting '/var/lib/libvirt/images/tftp.img', fmt=qcow2 size=104857600
> backing_file='json:{"file.driver":"tftp",
> "file.url":"tftp://127.0.0.1/img "}'
> encryption=off cluster_size=65536 lazy_refcounts=off
> ^C
> 
> ➜  ~ qemu-img info 'json:{"file.driver":"tftp",
> "file.url":"tftp://127.0.0.1/img "}'
> ^C
> 
> The qemu-img command above is stuck and never returns.
> 
> Max, is the tftp unusable for all versions of QEMU? Could you please
> give more info on that?

I have no idea.  I’ve never tested it.

Max



signature.asc
Description: OpenPGP digital signature

[libvirt PATCH] maint: Post-release version bump to 6.5.0

[Andrea Bolognani]

Signed-off-by: Andrea Bolognani 
---
Pushed as trivial.

 configure.ac  | 2 +-
 docs/news.xml | 8 
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index 73dc9f841c..6c8ac2f04b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -16,7 +16,7 @@ dnl You should have received a copy of the GNU Lesser General 
Public
 dnl License along with this library.  If not, see
 dnl .
 
-AC_INIT([libvirt], [6.4.0], [libvir-list@redhat.com], [], 
[https://libvirt.org])
+AC_INIT([libvirt], [6.5.0], [libvir-list@redhat.com], [], 
[https://libvirt.org])
 
 if test $srcdir = "."
 then
diff --git a/docs/news.xml b/docs/news.xml
index f040d18f96..18d96af048 100644
--- a/docs/news.xml
+++ b/docs/news.xml
@@ -42,6 +42,14 @@
  -->
 
 
+  
+
+
+
+
+
+
+  
   
 
   
-- 
2.25.4

Re: [libvirt PATCH] news: Update for libvirt 6.4.0

[Michal Privoznik]

On 6/2/20 10:24 AM, Andrea Bolognani wrote:

Signed-off-by: Andrea Bolognani 
---
  docs/news.xml | 45 +
  1 file changed, 45 insertions(+)


Reviewed-by: Michal Privoznik 

Michal

[libvirt PATCH] gitdm: Add missing entries

[Andrea Bolognani]

A few new individuals have contributed to libvirt since the last
time the gitdm configuration was updated.

Signed-off-by: Andrea Bolognani 
---
 docs/gitdm/groups/unaffiliated | 4 
 1 file changed, 4 insertions(+)

diff --git a/docs/gitdm/groups/unaffiliated b/docs/gitdm/groups/unaffiliated
index a58731a683..6e9de7084a 100644
--- a/docs/gitdm/groups/unaffiliated
+++ b/docs/gitdm/groups/unaffiliated
@@ -14,6 +14,7 @@ gmx.de
 googlemail.com
 hotmail.com
 mail.ru
+outlook.com
 pobox.com
 protonmail.com
 riseup.net
@@ -39,6 +40,8 @@ bi...@bigon.be
 bugzilla.redhat.si...@arlott.org
 car...@cardoe.com
 char...@dyfis.net
+c...@cky.nz
+cont...@puzio.waw.pl
 d.herrendoer...@herrendoerfer.name
 d...@danny.cz
 de...@fobos.de
@@ -53,6 +56,7 @@ iba...@adinet.com.uy
 igo...@moomers.org
 in...@nafets.de
 intrig...@boum.org
+j...@ristioja.ee
 james...@cowgill.org.uk
 ja...@shubin.ca
 jas...@humppa.nl
-- 
2.25.4

Re: [PATCH 5/5] qemu_validate.c: revert NUMA CPU range user warning

[Igor Mammedov]

On Mon, 1 Jun 2020 17:14:20 -0300
Daniel Henrique Barboza  wrote:

> On 6/1/20 4:40 PM, Peter Krempa wrote:
> > On Mon, Jun 01, 2020 at 14:50:41 -0300, Daniel Henrique Barboza wrote:  
> >> Now that we have the auto-fill code in place, and with proper documentation
> >> to let the user know that (1) we will auto-fill the NUMA cpus up to the
> >> number to maximum VCPUs number if QEMU supports it and (2) the user
> >> is advised to always supply a complete NUMA topology, this warning
> >> is unneeded.
> >>
> >> This reverts commit 38d2e033686b5cc274f8f55075ce1985b71e329a.  
> > 
> > Since we already have the validation in place for some time now I think
> > we should just keep it. The auto-filling would be a useful hack to work
> > around if config breaks, but judged by itself it's of questionable
> > benefit.  
> 
> That's a good point. I agree that removing the message after being in place
> for this long is more trouble than it's worth.
> 
> > 
> > Specifically users might end up with a topology which they didn't
> > expect. Reasoning is basically the same as with qemu. Any default
> > behaviour here is a policy decision and it might not suit all uses.
> >  
> 
>   
> An ideal situation would be QEMU to never accept incomplete NUMA topologies
> in the first place.
At least with your series I can safely drop deprecated incomplete NUMA 
topologies
on QEMU side (which were producing warnings for a while)

> 
> Given that this wasn't the case and now there might be a plethora of guests
> running with goofy topologies all around, the already existing warning
> message + this auto-fill hack + documentation mentioning that users should
> avoid these topologies is a fine solution from Libvirt side, in my
> estimation.
> 
> 
> Thanks,
> 
> 
> DHB
> 

Re: [PATCH v2 1/7] util: introduce a parser for kernel cmdline arguments

[Bjoern Walk]

Paulo de Rezende Pinatti  [2020-05-29, 12:10PM +0200]:
> Introduce two utility functions to parse a kernel command
> line string according to the kernel code parsing rules in
> order to enable the caller to perform operations such as
> verifying whether certain argument=value combinations are
> present or retrieving an argument's value.
> 
> Signed-off-by: Paulo de Rezende Pinatti 
> Signed-off-by: Boris Fiuczynski 
> ---
>  src/libvirt_private.syms |   2 +
>  src/util/virutil.c   | 169 +++
>  src/util/virutil.h   |  17 
>  tests/utiltest.c | 141 
>  4 files changed, 329 insertions(+)

Reviewed-by: Bjoern Walk 


signature.asc
Description: PGP signature

[PATCH] numa: forbid '-numa node, mem' for 5.1 and newer machine types

[Igor Mammedov]

Deprecation period is run out and it's a time to flip the switch
introduced by cd5ff8333a.  Disable legacy option for new machine
types (since 5.1) and amend documentation.

'-numa node,memdev' shall be used instead of disabled option
with new machine types.

Signed-off-by: Igor Mammedov 
---
 - rebased on top of current master
 - move compat mode from 4.2 to 5.0

CC: peter.mayd...@linaro.org
CC: ehabk...@redhat.com
CC: marcel.apfelb...@gmail.com
CC: m...@redhat.com
CC: pbonz...@redhat.com
CC: r...@twiddle.net
CC: da...@gibson.dropbear.id.au
CC: libvir-list@redhat.com
CC: qemu-...@nongnu.org
CC: qemu-...@nongnu.org
---
 docs/system/deprecated.rst | 17 -
 hw/arm/virt.c  |  2 +-
 hw/core/numa.c |  6 ++
 hw/i386/pc.c   |  1 -
 hw/i386/pc_piix.c  |  1 +
 hw/i386/pc_q35.c   |  1 +
 hw/ppc/spapr.c |  2 +-
 qemu-options.hx|  9 +
 8 files changed, 15 insertions(+), 24 deletions(-)

diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst
index f0061f94aa..57edc075c2 100644
--- a/docs/system/deprecated.rst
+++ b/docs/system/deprecated.rst
@@ -101,23 +101,6 @@ error in the future.
 The ``-realtime mlock=on|off`` argument has been replaced by the
 ``-overcommit mem-lock=on|off`` argument.
 
-``-numa node,mem=``\ *size* (since 4.1)
-'''
-
-The parameter ``mem`` of ``-numa node`` is used to assign a part of
-guest RAM to a NUMA node. But when using it, it's impossible to manage 
specified
-RAM chunk on the host side (like bind it to a host node, setting bind policy, 
...),
-so guest end-ups with the fake NUMA configuration with suboptiomal performance.
-However since 2014 there is an alternative way to assign RAM to a NUMA node
-using parameter ``memdev``, which does the same as ``mem`` and adds
-means to actualy manage node RAM on the host side. Use parameter ``memdev``
-with *memory-backend-ram* backend as an replacement for parameter ``mem``
-to achieve the same fake NUMA effect or a properly configured
-*memory-backend-file* backend to actually benefit from NUMA configuration.
-In future new machine versions will not accept the option but it will still
-work with old machine types. User can check QAPI schema to see if the legacy
-option is supported by looking at MachineInfo::numa-mem-supported property.
-
 ``-numa`` node (without memory specified) (since 4.1)
 '
 
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 37462a6f78..063d4703f7 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -2262,7 +2262,6 @@ static void virt_machine_class_init(ObjectClass *oc, void 
*data)
 hc->pre_plug = virt_machine_device_pre_plug_cb;
 hc->plug = virt_machine_device_plug_cb;
 hc->unplug_request = virt_machine_device_unplug_request_cb;
-mc->numa_mem_supported = true;
 mc->nvdimm_supported = true;
 mc->auto_enable_numa_with_memhp = true;
 mc->default_ram_id = "mach-virt.ram";
@@ -2375,6 +2374,7 @@ DEFINE_VIRT_MACHINE_AS_LATEST(5, 1)
 static void virt_machine_5_0_options(MachineClass *mc)
 {
 virt_machine_5_1_options(mc);
+mc->numa_mem_supported = true;
 }
 DEFINE_VIRT_MACHINE(5, 0)
 
diff --git a/hw/core/numa.c b/hw/core/numa.c
index 316bc50d75..05be412e59 100644
--- a/hw/core/numa.c
+++ b/hw/core/numa.c
@@ -117,6 +117,12 @@ static void parse_numa_node(MachineState *ms, 
NumaNodeOptions *node,
 }
 
 if (node->has_mem) {
+if (!mc->numa_mem_supported) {
+error_setg(errp, "Parameter -numa node,mem is not supported by 
this"
+  " machine type. Use -numa node,memdev instead");
+return;
+}
+
 numa_info[nodenr].node_mem = node->mem;
 if (!qtest_enabled()) {
 warn_report("Parameter -numa node,mem is deprecated,"
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 2128f3d6fe..a86136069c 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1960,7 +1960,6 @@ static void pc_machine_class_init(ObjectClass *oc, void 
*data)
 hc->unplug = pc_machine_device_unplug_cb;
 mc->default_cpu_type = TARGET_DEFAULT_CPU_TYPE;
 mc->nvdimm_supported = true;
-mc->numa_mem_supported = true;
 mc->default_ram_id = "pc.ram";
 
 object_class_property_add(oc, PC_MACHINE_DEVMEM_REGION_SIZE, "int",
diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index f66e1d73ce..8955436efa 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -438,6 +438,7 @@ static void pc_i440fx_5_0_machine_options(MachineClass *m)
 pc_i440fx_5_1_machine_options(m);
 m->alias = NULL;
 m->is_default = false;
+m->numa_mem_supported = true;
 compat_props_add(m->compat_props, hw_compat_5_0, hw_compat_5_0_len);
 compat_props_add(m->compat_props, pc_compat_5_0, pc_compat_5_0_len);
 }
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 4ba8ac8774..af6b32adb7 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -366,6 +366,7 @@ 

[libvirt PATCH] news: Update for libvirt 6.4.0

[Andrea Bolognani]

Signed-off-by: Andrea Bolognani 
---
 docs/news.xml | 45 +
 1 file changed, 45 insertions(+)

diff --git a/docs/news.xml b/docs/news.xml
index 104babbae7..956af3036b 100644
--- a/docs/news.xml
+++ b/docs/news.xml
@@ -44,6 +44,24 @@
 
   
 
+  
+
+  qemu: Add support for pvscsi controllers
+
+
+  pvscsi is the VMware paravirtualized SCSI controller, which has been
+  supported in QEMU for a number of years.
+
+  
+  
+
+  cpu: Report model information for ARM CPUs
+
+
+  virsh capabilities will now include information about
+  the host CPU when run on ARM machines.
+
+  
 
 
   
@@ -71,6 +89,33 @@
   further issues.
 
   
+  
+
+  remote: Look up libxl driver correctly
+
+
+  This makes xen:// connection URIs usable in split daemon
+  mode.
+
+  
+  
+
+  systemd: Start libvirtd after firewalld/iptables services
+
+
+  This solves an issue where iptables rules and chains created by
+  libvirtd would get removed by a service started after it.
+
+  
+  
+
+  network: Re-create iptables chains on firewalld restart
+
+
+  firewalld resets all iptables rules and chains on restart, and this
+  includes deleting those created by libvirt.
+
+  
 
   
   
-- 
2.25.4

Re: [PATCH libvirt v1 0/6] Fix ZPCI address auto-generation on s390

[Shalini Chellathurai Saroja]

On 6/1/20 6:49 PM, Andrea Bolognani wrote:

On Wed, 2020-05-20 at 13:32 +0200, Boris Fiuczynski wrote:

If I do not misinterpret Dan's last mail he agrees that specifying uid=0
should be considered as error.
How do we proceed from here? Shalini series still can be applied on
master cleanly. Do you want her to resend the series anyway?

I checked again and it still applies cleanly, so no need to resend.
And Dan seemed to be fine with the changes after all, so I'm going
to go ahead and review them this week.

ok, thank you.