[PATCH 2/2] qemu: Add testing for locking option

[Masayoshi Mizuma]

From: Masayoshi Mizuma 

Add testing for locking option for qemu disk source.
Test three switches for locking: 'auto', 'on' and 'off'.

Signed-off-by: Masayoshi Mizuma 
---
 .../disk-file-locking-auto.x86_64-latest.args | 43 +++
 .../disk-file-locking-auto.xml| 27 
 .../disk-file-locking-off.x86_64-latest.args  | 43 +++
 .../disk-file-locking-off.xml | 27 
 .../disk-file-locking-on.x86_64-latest.args   | 43 +++
 .../qemuxml2argvdata/disk-file-locking-on.xml | 27 
 tests/qemuxml2argvtest.c  |  3 ++
 7 files changed, 213 insertions(+)
 create mode 100644 
tests/qemuxml2argvdata/disk-file-locking-auto.x86_64-latest.args
 create mode 100644 tests/qemuxml2argvdata/disk-file-locking-auto.xml
 create mode 100644 
tests/qemuxml2argvdata/disk-file-locking-off.x86_64-latest.args
 create mode 100644 tests/qemuxml2argvdata/disk-file-locking-off.xml
 create mode 100644 
tests/qemuxml2argvdata/disk-file-locking-on.x86_64-latest.args
 create mode 100644 tests/qemuxml2argvdata/disk-file-locking-on.xml

diff --git a/tests/qemuxml2argvdata/disk-file-locking-auto.x86_64-latest.args 
b/tests/qemuxml2argvdata/disk-file-locking-auto.x86_64-latest.args
new file mode 100644
index 00..4abe38515c
--- /dev/null
+++ b/tests/qemuxml2argvdata/disk-file-locking-auto.x86_64-latest.args
@@ -0,0 +1,43 @@
+LC_ALL=C \
+PATH=/bin \
+HOME=/tmp/lib/domain--1-QEMUGuest1 \
+USER=test \
+LOGNAME=test \
+XDG_DATA_HOME=/tmp/lib/domain--1-QEMUGuest1/.local/share \
+XDG_CACHE_HOME=/tmp/lib/domain--1-QEMUGuest1/.cache \
+XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
+QEMU_AUDIO_DRV=none \
+/usr/bin/qemu-system-i386 \
+-name guest=QEMUGuest1,debug-threads=on \
+-S \
+-object secret,id=masterKey0,format=raw,\
+file=/tmp/lib/domain--1-QEMUGuest1/master-key.aes \
+-machine pc,accel=tcg,usb=off,dump-guest-core=off,memory-backend=pc.ram \
+-cpu qemu64 \
+-m 214 \
+-object memory-backend-ram,id=pc.ram,size=224395264 \
+-overcommit mem-lock=off \
+-smp 1,sockets=1,cores=1,threads=1 \
+-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \
+-display none \
+-no-user-config \
+-nodefaults \
+-chardev socket,id=charmonitor,fd=1729,server,nowait \
+-mon chardev=charmonitor,id=monitor,mode=control \
+-rtc base=utc \
+-no-shutdown \
+-no-acpi \
+-boot strict=on \
+-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 \
+-blockdev '{"driver":"file","filename":"/tmp/QEMUGuest1.img","locking":"auto",\
+"node-name":"libvirt-1-storage","cache":{"direct":true,"no-flush":false},\
+"auto-read-only":true,"discard":"unmap"}' \
+-blockdev '{"node-name":"libvirt-1-format","read-only":false,\
+"cache":{"direct":true,"no-flush":false},"driver":"qcow2",\
+"file":"libvirt-1-storage"}' \
+-device 
ide-hd,bus=ide.0,unit=0,drive=libvirt-1-format,id=ide0-0-0,bootindex=1,\
+write-cache=on \
+-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x2 \
+-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,\
+resourcecontrol=deny \
+-msg timestamp=on
diff --git a/tests/qemuxml2argvdata/disk-file-locking-auto.xml 
b/tests/qemuxml2argvdata/disk-file-locking-auto.xml
new file mode 100644
index 00..9ae9372c01
--- /dev/null
+++ b/tests/qemuxml2argvdata/disk-file-locking-auto.xml
@@ -0,0 +1,27 @@
+
+  QEMUGuest1
+  c7a5fdbd-edaf-9455-926a-d65c16db1809
+  219100
+  219100
+  1
+  
+hvm
+
+  
+  
+  destroy
+  restart
+  destroy
+  
+/usr/bin/qemu-system-i386
+
+  
+  
+  
+  
+
+
+
+
+  
+
diff --git a/tests/qemuxml2argvdata/disk-file-locking-off.x86_64-latest.args 
b/tests/qemuxml2argvdata/disk-file-locking-off.x86_64-latest.args
new file mode 100644
index 00..33786b46cd
--- /dev/null
+++ b/tests/qemuxml2argvdata/disk-file-locking-off.x86_64-latest.args
@@ -0,0 +1,43 @@
+LC_ALL=C \
+PATH=/bin \
+HOME=/tmp/lib/domain--1-QEMUGuest1 \
+USER=test \
+LOGNAME=test \
+XDG_DATA_HOME=/tmp/lib/domain--1-QEMUGuest1/.local/share \
+XDG_CACHE_HOME=/tmp/lib/domain--1-QEMUGuest1/.cache \
+XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
+QEMU_AUDIO_DRV=none \
+/usr/bin/qemu-system-i386 \
+-name guest=QEMUGuest1,debug-threads=on \
+-S \
+-object secret,id=masterKey0,format=raw,\
+file=/tmp/lib/domain--1-QEMUGuest1/master-key.aes \
+-machine pc,accel=tcg,usb=off,dump-guest-core=off,memory-backend=pc.ram \
+-cpu qemu64 \
+-m 214 \
+-object memory-backend-ram,id=pc.ram,size=224395264 \
+-overcommit mem-lock=off \
+-smp 1,sockets=1,cores=1,threads=1 \
+-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \
+-display none \
+-no-user-config \
+-nodefaults \
+-chardev socket,id=charmonitor,fd=1729,server,nowait \
+-mon chardev=charmonitor,id=monitor,mode=control \
+-rtc base=utc \
+-no-shutdown \
+-no-acpi \
+-boot strict=on \
+-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 \
+-blockdev '{"driver":"file","filename":"/tmp/QEMUGuest1.img","locking":"off",\

[PATCH 1/2] qemu: introduce locking option for disk source of qemu

[Masayoshi Mizuma]

From: Masayoshi Mizuma 

Introduce locking option for disk source of qemu.
It may be useful to avoid file lock issues.
locking option supports three switches; 'auto', 'on' and 'off'.
The default behaivor will work if locking option isn't set.

Example of the usage:

  




  

Signed-off-by: Masayoshi Mizuma 
---
 docs/schemas/domaincommon.rng  |  9 +
 src/conf/domain_conf.c |  8 
 src/conf/storage_source_conf.c |  9 +
 src/conf/storage_source_conf.h | 14 ++
 src/libvirt_private.syms   |  1 +
 src/qemu/qemu_block.c  |  5 +
 6 files changed, 46 insertions(+)

diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng
index a4bddcf132..d33d853f31 100644
--- a/docs/schemas/domaincommon.rng
+++ b/docs/schemas/domaincommon.rng
@@ -1676,6 +1676,15 @@
 
   
 
+
+  
+
+  auto
+  on
+  off
+
+  
+
   
 
   
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index dab4f10326..067ffa877b 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -8540,6 +8540,7 @@ virDomainStorageSourceParse(xmlNodePtr node,
 {
 VIR_XPATH_NODE_AUTORESTORE(ctxt)
 xmlNodePtr tmp;
+char *locking;
 
 ctxt->node = node;
 
@@ -8606,6 +8607,9 @@ virDomainStorageSourceParse(xmlNodePtr node,
 return -1;
 }
 
+if ((locking = virXMLPropString(node, "locking")))
+src->locking = virStorageFileLockingTypeFromString(locking);
+
 return 0;
 }
 
@@ -24102,6 +24106,10 @@ virDomainDiskSourceFormat(virBufferPtr buf,
 return -1;
 }
 
+if (src->locking != VIR_STORAGE_FILE_LOCKING_DEFAULT)
+virBufferEscapeString(, " locking='%s'",
+  virStorageFileLockingTypeToString(src->locking));
+
 virDomainDiskSourceFormatSlices(, src);
 
 if (src->type != VIR_STORAGE_TYPE_NETWORK)
diff --git a/src/conf/storage_source_conf.c b/src/conf/storage_source_conf.c
index dab5e855f5..3ac0c7f75b 100644
--- a/src/conf/storage_source_conf.c
+++ b/src/conf/storage_source_conf.c
@@ -49,6 +49,15 @@ VIR_ENUM_IMPL(virStorage,
 );
 
 
+VIR_ENUM_IMPL(virStorageFileLocking,
+  VIR_STORAGE_FILE_LOCKING_LAST,
+  "default",
+  "auto",
+  "on",
+  "off",
+);
+
+
 VIR_ENUM_IMPL(virStorageFileFormat,
   VIR_STORAGE_FILE_LAST,
   "none",
diff --git a/src/conf/storage_source_conf.h b/src/conf/storage_source_conf.h
index e66ccdedef..6f5b165504 100644
--- a/src/conf/storage_source_conf.h
+++ b/src/conf/storage_source_conf.h
@@ -82,6 +82,18 @@ typedef enum {
 VIR_ENUM_DECL(virStorageFileFormat);
 
 
+typedef enum {
+VIR_STORAGE_FILE_LOCKING_DEFAULT = 0,
+VIR_STORAGE_FILE_LOCKING_AUTO,
+VIR_STORAGE_FILE_LOCKING_ON,
+VIR_STORAGE_FILE_LOCKING_OFF,
+
+VIR_STORAGE_FILE_LOCKING_LAST,
+} virStorageFileLocking;
+
+VIR_ENUM_DECL(virStorageFileLocking);
+
+
 typedef enum {
 VIR_STORAGE_FILE_FEATURE_LAZY_REFCOUNTS = 0,
 
@@ -394,6 +406,8 @@ struct _virStorageSource {
 char *nfs_group;
 uid_t nfs_uid;
 gid_t nfs_gid;
+
+int locking; /* enum virStorageFileLocking */
 };
 
 G_DEFINE_AUTOPTR_CLEANUP_FUNC(virStorageSource, virObjectUnref);
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index fbaf16704b..c72d2161b2 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -1068,6 +1068,7 @@ virStorageFileFeatureTypeFromString;
 virStorageFileFeatureTypeToString;
 virStorageFileFormatTypeFromString;
 virStorageFileFormatTypeToString;
+virStorageFileLockingTypeToString;
 virStorageNetHostDefClear;
 virStorageNetHostDefCopy;
 virStorageNetHostDefFree;
diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c
index d845a3312d..d7eec16ab6 100644
--- a/src/qemu/qemu_block.c
+++ b/src/qemu/qemu_block.c
@@ -1015,6 +1015,7 @@ qemuBlockStorageSourceGetFileProps(virStorageSourcePtr 
src,
 {
 const char *iomode = NULL;
 const char *prManagerAlias = NULL;
+const char *locking = NULL;
 virJSONValuePtr ret = NULL;
 
 if (!onlytarget) {
@@ -1023,12 +1024,16 @@ qemuBlockStorageSourceGetFileProps(virStorageSourcePtr 
src,
 
 if (src->iomode != VIR_DOMAIN_DISK_IO_DEFAULT)
 iomode = virDomainDiskIoTypeToString(src->iomode);
+
+if (src->locking != VIR_STORAGE_FILE_LOCKING_DEFAULT)
+locking = virStorageFileLockingTypeToString(src->locking);
 }
 
 ignore_value(virJSONValueObjectCreate(,
   "s:filename", src->path,
   "S:aio", iomode,
   "S:pr-manager", prManagerAlias,
+  "S:locking", locking,
   NULL) < 0);
 return ret;
 }
-- 
2.27.0

[RFC PATCH 7/7] qemu: Add virtio disks as sharable transient disks

[Masayoshi Mizuma]

From: Masayoshi Mizuma 

Add virtio disks to be sharable transient disks.

Signed-off-by: Masayoshi Mizuma 
---
 src/qemu/qemu_hotplug.c | 13 -
 src/qemu/qemu_process.c |  2 +-
 2 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c
index 5d0445538d..fc6ca028e3 100644
--- a/src/qemu/qemu_hotplug.c
+++ b/src/qemu/qemu_hotplug.c
@@ -814,7 +814,9 @@ qemuDomainAttachDiskGeneric(virQEMUDriverPtr driver,
 static int
 qemuDomainAttachVirtioDiskDevice(virQEMUDriverPtr driver,
  virDomainObjPtr vm,
- virDomainDiskDefPtr disk)
+ virDomainDiskDefPtr disk,
+ int bootindex,
+ qemuDomainAsyncJob asyncJob)
 {
 virDomainDeviceDef dev = { VIR_DOMAIN_DEVICE_DISK, { .disk = disk } };
 bool releaseaddr = false;
@@ -823,8 +825,8 @@ qemuDomainAttachVirtioDiskDevice(virQEMUDriverPtr driver,
 if (qemuDomainEnsureVirtioAddress(, vm, , disk->dst) < 0)
 return -1;
 
-if ((rv = qemuDomainAttachDiskGeneric(driver, vm, disk, 0,
-  QEMU_ASYNC_JOB_NONE)) < 0) {
+if ((rv = qemuDomainAttachDiskGeneric(driver, vm, disk, bootindex,
+  asyncJob)) < 0) {
 if (rv == -1 && releaseaddr)
 qemuDomainReleaseDeviceAddress(vm, >info);
 
@@ -1033,7 +1035,7 @@ static int
 qemuDomainAttachDeviceDiskLiveInternal(virQEMUDriverPtr driver,
virDomainObjPtr vm,
virDomainDeviceDefPtr dev,
-   qemuDomainAsyncJob asyncJob 
G_GNUC_UNUSED)
+   qemuDomainAsyncJob asyncJob)
 {
 size_t i;
 virDomainDiskDefPtr disk = dev->data.disk;
@@ -1080,7 +1082,8 @@ qemuDomainAttachDeviceDiskLiveInternal(virQEMUDriverPtr 
driver,
 break;
 
 case VIR_DOMAIN_DISK_BUS_VIRTIO:
-ret = qemuDomainAttachVirtioDiskDevice(driver, vm, disk);
+ret = qemuDomainAttachVirtioDiskDevice(driver, vm, disk,
+   disk->info.bootIndex, asyncJob);
 break;
 
 case VIR_DOMAIN_DISK_BUS_SCSI:
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index 2e2d1c6fea..55577632c5 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -6907,7 +6907,7 @@ qemuCheckTransientDiskSharable(virDomainObjPtr vm)
 for (i = 0; i < vm->def->ndisks; i++) {
 virDomainDiskDefPtr disk = vm->def->disks[i];
 
-if (disk->transient && disk->bus != VIR_DOMAIN_DISK_BUS_LAST)
+if (disk->transient && disk->bus != VIR_DOMAIN_DISK_BUS_VIRTIO)
  return;
 }
 
-- 
2.27.0

[RFC PATCH 4/7] qemu_hotplug: Add bootindex argument to qemuDomainAttachDiskGeneric

[Masayoshi Mizuma]

From: Masayoshi Mizuma 

Add bootindex argument to qemuDomainAttachDiskGeneric() so that
qemu can detect the boot index for the disks which are hot-added
before CPUs start.

Signed-off-by: Masayoshi Mizuma 
---
 src/qemu/qemu_hotplug.c | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c
index b989652533..a2535949b7 100644
--- a/src/qemu/qemu_hotplug.c
+++ b/src/qemu/qemu_hotplug.c
@@ -694,6 +694,7 @@ static int
 qemuDomainAttachDiskGeneric(virQEMUDriverPtr driver,
 virDomainObjPtr vm,
 virDomainDiskDefPtr disk,
+unsigned int bootindex,
 qemuDomainAsyncJob asyncJob)
 {
 g_autoptr(qemuBlockStorageSourceChainData) data = NULL;
@@ -732,7 +733,7 @@ qemuDomainAttachDiskGeneric(virQEMUDriverPtr driver,
 goto cleanup;
 }
 
-if (!(devstr = qemuBuildDiskDeviceStr(vm->def, disk, 0, priv->qemuCaps)))
+if (!(devstr = qemuBuildDiskDeviceStr(vm->def, disk, bootindex, 
priv->qemuCaps)))
 goto cleanup;
 
 if (VIR_REALLOC_N(vm->def->disks, vm->def->ndisks + 1) < 0)
@@ -822,7 +823,8 @@ qemuDomainAttachVirtioDiskDevice(virQEMUDriverPtr driver,
 if (qemuDomainEnsureVirtioAddress(, vm, , disk->dst) < 0)
 return -1;
 
-if ((rv = qemuDomainAttachDiskGeneric(driver, vm, disk, 
QEMU_ASYNC_JOB_NONE)) < 0) {
+if ((rv = qemuDomainAttachDiskGeneric(driver, vm, disk, 0,
+  QEMU_ASYNC_JOB_NONE)) < 0) {
 if (rv == -1 && releaseaddr)
 qemuDomainReleaseDeviceAddress(vm, >info);
 
@@ -1001,7 +1003,7 @@ qemuDomainAttachSCSIDisk(virQEMUDriverPtr driver,
 return -1;
 }
 
-if (qemuDomainAttachDiskGeneric(driver, vm, disk, QEMU_ASYNC_JOB_NONE) < 0)
+if (qemuDomainAttachDiskGeneric(driver, vm, disk, 0, QEMU_ASYNC_JOB_NONE) 
< 0)
 return -1;
 
 return 0;
@@ -1018,7 +1020,7 @@ qemuDomainAttachUSBMassStorageDevice(virQEMUDriverPtr 
driver,
 if (virDomainUSBAddressEnsure(priv->usbaddrs, >info) < 0)
 return -1;
 
-if (qemuDomainAttachDiskGeneric(driver, vm, disk, QEMU_ASYNC_JOB_NONE) < 
0) {
+if (qemuDomainAttachDiskGeneric(driver, vm, disk, 0, QEMU_ASYNC_JOB_NONE) 
< 0) {
 virDomainUSBAddressRelease(priv->usbaddrs, >info);
 return -1;
 }
-- 
2.27.0

[RFC PATCH 2/7] qemu_hotplug: Add asyncJob argument to qemuDomainAttachDeviceDiskLiveInternal

[Masayoshi Mizuma]

From: Masayoshi Mizuma 

Add asynJob argument to qemuDomainAttachDeviceDiskLiveInternal() so that it can
be used before CPUs start. To avoid a compile warning, G_GNUC_UNUSED is added
here. A later patch will remove G_GNUC_UNUSED.

Signed-off-by: Masayoshi Mizuma 
---
 src/qemu/qemu_hotplug.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c
index 609e9d1a8a..a61899d53a 100644
--- a/src/qemu/qemu_hotplug.c
+++ b/src/qemu/qemu_hotplug.c
@@ -1030,7 +1030,8 @@ qemuDomainAttachUSBMassStorageDevice(virQEMUDriverPtr 
driver,
 static int
 qemuDomainAttachDeviceDiskLiveInternal(virQEMUDriverPtr driver,
virDomainObjPtr vm,
-   virDomainDeviceDefPtr dev)
+   virDomainDeviceDefPtr dev,
+   qemuDomainAsyncJob asyncJob 
G_GNUC_UNUSED)
 {
 size_t i;
 virDomainDiskDefPtr disk = dev->data.disk;
@@ -1138,7 +1139,7 @@ qemuDomainAttachDeviceDiskLive(virQEMUDriverPtr driver,
 return 0;
 }
 
-return qemuDomainAttachDeviceDiskLiveInternal(driver, vm, dev);
+return qemuDomainAttachDeviceDiskLiveInternal(driver, vm, dev, 
QEMU_ASYNC_JOB_NONE);
 }
 
 
-- 
2.27.0

[RFC PATCH 5/7] qemu_hotplug: make disk sharable

[Masayoshi Mizuma]

From: Masayoshi Mizuma 

Add qemuHotplugCreateDisksTransient() to make  disk
sharable.
The procedure is followings.
First, create the overlay disk with the original disk is set as the
backingStore. Then, blockdev-del the StorageProgs and FormatProgs of
the disk. That's because to fix the bootindex of the disk.
Lastly, device_add the disks.

Signed-off-by: Masayoshi Mizuma 
---
 src/qemu/qemu_command.c |   3 +
 src/qemu/qemu_hotplug.c | 285 
 src/qemu/qemu_hotplug.h |   3 +
 3 files changed, 291 insertions(+)

diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index 1ec302d4ac..81a27703c5 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -2139,6 +2139,9 @@ qemuBuildDisksCommandLine(virCommandPtr cmd,
 bootCD = 0;
 break;
 case VIR_DOMAIN_DISK_DEVICE_DISK:
+/* to use bootindex later for transient disk */
+disk->info.bootIndex = bootDisk;
+G_GNUC_FALLTHROUGH;
 case VIR_DOMAIN_DISK_DEVICE_LUN:
 bootindex = bootDisk;
 bootDisk = 0;
diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c
index a2535949b7..5d0445538d 100644
--- a/src/qemu/qemu_hotplug.c
+++ b/src/qemu/qemu_hotplug.c
@@ -6710,3 +6710,288 @@ qemuDomainSetVcpuInternal(virQEMUDriverPtr driver,
 virBitmapFree(livevcpus);
 return ret;
 }
+
+struct _qemuHotplugTransientDiskContext {
+virDomainDeviceDefPtr trandev;
+virDomainDiskDefPtr *domdisk;
+size_t ndd;
+};
+
+typedef struct  _qemuHotplugTransientDiskContext  
qemuHotplugTransientDiskContext;
+typedef qemuHotplugTransientDiskContext *qemuHotplugTransientDiskContextPtr;
+
+static qemuHotplugTransientDiskContextPtr
+qemuHotplugTransientDiskContextNew(size_t ndisks)
+{
+qemuHotplugTransientDiskContextPtr ret = 
g_new0(qemuHotplugTransientDiskContext, 1);
+
+ret->trandev = g_new0(virDomainDeviceDef, ndisks);
+ret->domdisk = g_new0(virDomainDiskDefPtr, ndisks);
+
+return ret;
+}
+
+static void
+qemuHotplugTransientDiskCleanup(virDomainDeviceDefPtr data,
+virDomainDiskDefPtr *domdisk)
+{
+VIR_FREE(data);
+VIR_FREE(domdisk);
+
+return;
+}
+
+static void
+qemuHotplugTransientDiskContextCleanup(qemuHotplugTransientDiskContextPtr 
hptctxt)
+{
+if (!hptctxt)
+return;
+
+qemuHotplugTransientDiskCleanup(hptctxt->trandev, hptctxt->domdisk);
+
+g_free(hptctxt);
+}
+
+G_DEFINE_AUTOPTR_CLEANUP_FUNC(qemuHotplugTransientDiskContext, 
qemuHotplugTransientDiskContextCleanup);
+
+static int
+qemuHotplugDiskPrepareOneBlockdev(virQEMUDriverPtr driver,
+  virDomainObjPtr vm,
+  virQEMUDriverConfigPtr cfg,
+  virDomainDiskDefPtr domdisk,
+  virDomainDiskDefPtr trandisk,
+  GHashTable *blockNamedNodeData,
+  qemuDomainAsyncJob asyncJob)
+{
+qemuDomainObjPrivatePtr priv = vm->privateData;
+g_autoptr(qemuBlockStorageSourceChainData) data = NULL;
+g_autoptr(virStorageSource) terminator = NULL;
+
+terminator = virStorageSourceNew();
+
+if (qemuDomainPrepareStorageSourceBlockdev(trandisk, trandisk->src,
+   priv, cfg) < 0)
+return -1;
+
+if (!(data = 
qemuBuildStorageSourceChainAttachPrepareBlockdevTop(trandisk->src,
+ 
terminator,
+ 
priv->qemuCaps)))
+return -1;
+
+if (qemuBlockStorageSourceCreateDetectSize(blockNamedNodeData,
+   trandisk->src, domdisk->src) < 
0)
+   return -1;
+
+if (qemuBlockStorageSourceCreate(vm, trandisk->src, domdisk->src,
+NULL, data->srcdata[0],
+asyncJob) < 0)
+   return -1;
+
+/* blockdev-del the transient disk src. The disk is blockdev-add'ed
+ * while the disk is hot-added */
+if (qemuBlockStorageSourceDetachOneBlockdev(driver, vm,
+asyncJob, trandisk->src) < 0)
+   return -1;
+
+return 0;
+}
+
+static int
+qemuHotplugDiskTransientPrepareOne(virDomainObjPtr vm,
+   virQEMUDriverConfigPtr cfg,
+   virDomainDiskDefPtr domdisk,
+   virDomainDiskDefPtr trandisk,
+   GHashTable *blockNamedNodeData,
+   qemuDomainAsyncJob asyncJob)
+{
+qemuDomainObjPrivatePtr priv = vm->privateData;
+virQEMUDriverPtr driver = priv->driver;
+bool supportsCreate;
+
+if (qemuDomainStorageSourceValidateDepth(trandisk->src, 1, trandisk->dst) 

[RFC PATCH 6/7] qemu: Add check whether the transient disks are sharable

[Masayoshi Mizuma]

From: Masayoshi Mizuma 

Check whether the transient disks are shareable or not.
If followings are true, the transient disks are shareable.

 - qemu has blockdev and hotplug feature
 - the all disk bus support hot-plug

Signed-off-by: Masayoshi Mizuma 
---
 src/qemu/qemu_command.c | 17 +
 src/qemu/qemu_domain.h  |  3 +++
 src/qemu/qemu_process.c | 38 +-
 3 files changed, 53 insertions(+), 5 deletions(-)

diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index 81a27703c5..d5958f46ef 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -2063,10 +2063,14 @@ qemuBuildDiskCommandLine(virCommandPtr cmd,
  const virDomainDef *def,
  virDomainDiskDefPtr disk,
  virQEMUCapsPtr qemuCaps,
- unsigned int bootindex)
+ unsigned int bootindex,
+ qemuDomainObjPrivatePtr priv)
 {
 g_autofree char *optstr = NULL;
 
+if ((disk->transient) && (priv->TransientDiskSharable))
+disk->src->readonly = true;
+
 if (qemuBuildDiskSourceCommandLine(cmd, disk, qemuCaps) < 0)
 return -1;
 
@@ -2084,6 +2088,10 @@ qemuBuildDiskCommandLine(virCommandPtr cmd,
 if (qemuCommandAddExtDevice(cmd, >info) < 0)
 return -1;
 
+/* All disks are hot-added later if TransientDiskSharable is true */
+if (priv->TransientDiskSharable)
+return 0;
+
 virCommandAddArg(cmd, "-device");
 
 if (!(optstr = qemuBuildDiskDeviceStr(def, disk, bootindex,
@@ -2098,7 +2106,8 @@ qemuBuildDiskCommandLine(virCommandPtr cmd,
 static int
 qemuBuildDisksCommandLine(virCommandPtr cmd,
   const virDomainDef *def,
-  virQEMUCapsPtr qemuCaps)
+  virQEMUCapsPtr qemuCaps,
+  qemuDomainObjPrivatePtr priv)
 {
 size_t i;
 unsigned int bootCD = 0;
@@ -2154,7 +2163,7 @@ qemuBuildDisksCommandLine(virCommandPtr cmd,
 if (disk->device == VIR_DOMAIN_DISK_DEVICE_FLOPPY)
 bootindex = 0;
 
-if (qemuBuildDiskCommandLine(cmd, def, disk, qemuCaps, bootindex) < 0)
+if (qemuBuildDiskCommandLine(cmd, def, disk, qemuCaps, bootindex, 
priv) < 0)
 return -1;
 }
 
@@ -9935,7 +9944,7 @@ qemuBuildCommandLine(virQEMUDriverPtr driver,
   VIR_DOMAIN_CONTROLLER_TYPE_CCID) 
< 0)
 return NULL;
 
-if (qemuBuildDisksCommandLine(cmd, def, qemuCaps) < 0)
+if (qemuBuildDisksCommandLine(cmd, def, qemuCaps, priv) < 0)
 return NULL;
 
 if (qemuBuildFilesystemCommandLine(cmd, def, qemuCaps, priv) < 0)
diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h
index 154339ef8f..37b050def4 100644
--- a/src/qemu/qemu_domain.h
+++ b/src/qemu/qemu_domain.h
@@ -267,6 +267,9 @@ struct _qemuDomainObjPrivate {
 /* prevent deletion of  disk overlay files between startup and
  * succesful setup of the overlays */
 bool inhibitDiskTransientDelete;
+
+/* True if the all transient disks are sharable */
+bool TransientDiskSharable;
 };
 
 #define QEMU_DOMAIN_PRIVATE(vm) \
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index f87a3c0f60..2e2d1c6fea 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -6892,6 +6892,40 @@ qemuProcessEnablePerf(virDomainObjPtr vm)
 return 0;
 }
 
+static void
+qemuCheckTransientDiskSharable(virDomainObjPtr vm)
+{
+qemuDomainObjPrivatePtr priv = vm->privateData;
+bool hotplug = virQEMUCapsGet(priv->qemuCaps, 
QEMU_CAPS_PCIE_ROOT_PORT_HOTPLUG);
+size_t i;
+
+priv->TransientDiskSharable = false;
+
+if (!hotplug)
+return;
+
+for (i = 0; i < vm->def->ndisks; i++) {
+virDomainDiskDefPtr disk = vm->def->disks[i];
+
+if (disk->transient && disk->bus != VIR_DOMAIN_DISK_BUS_LAST)
+ return;
+}
+
+priv->TransientDiskSharable = true;
+}
+
+static int
+qemuProcessCreateDisksTransient(virDomainObjPtr vm,
+qemuDomainAsyncJob asyncJob)
+{
+qemuDomainObjPrivatePtr priv = vm->privateData;
+
+if (priv->TransientDiskSharable)
+return qemuHotplugCreateDisksTransient(vm, asyncJob);
+else
+return qemuSnapshotCreateDisksTransient(vm, asyncJob);
+}
+
 
 /**
  * qemuProcessLaunch:
@@ -6982,6 +7016,8 @@ qemuProcessLaunch(virConnectPtr conn,
 incoming != NULL) < 0)
 goto cleanup;
 
+qemuCheckTransientDiskSharable(vm);
+
 VIR_DEBUG("Building emulator command line");
 if (!(cmd = qemuBuildCommandLine(driver,
  qemuDomainLogContextGetManager(logCtxt),
@@ -7228,7 +7264,7 @@ qemuProcessLaunch(virConnectPtr conn,
 goto cleanup;
 
 VIR_DEBUG("Setting up transient disk");
-if (qemuSnapshotCreateDisksTransient(vm, asyncJob) < 0)
+if 

[RFC PATCH 3/7] qemu_hotplug: Add asynJob argument to qemuDomainRemoveDiskDevice

[Masayoshi Mizuma]

From: Masayoshi Mizuma 

Add asynJob argument to qemuDomainRemoveDiskDevice() so that it can
be used before CPUs start.

Signed-off-by: Masayoshi Mizuma 
---
 src/qemu/qemu_hotplug.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c
index a61899d53a..b989652533 100644
--- a/src/qemu/qemu_hotplug.c
+++ b/src/qemu/qemu_hotplug.c
@@ -4302,7 +4302,8 @@ static bool qemuIsMultiFunctionDevice(virDomainDefPtr def,
 static int
 qemuDomainRemoveDiskDevice(virQEMUDriverPtr driver,
virDomainObjPtr vm,
-   virDomainDiskDefPtr disk)
+   virDomainDiskDefPtr disk,
+   qemuDomainAsyncJob asyncJob)
 {
 qemuDomainDiskPrivatePtr diskPriv = QEMU_DOMAIN_DISK_PRIVATE(disk);
 g_autoptr(qemuBlockStorageSourceChainData) diskBackend = NULL;
@@ -4347,7 +4348,8 @@ qemuDomainRemoveDiskDevice(virQEMUDriverPtr driver,
 }
 }
 
-qemuDomainObjEnterMonitor(driver, vm);
+if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) < 0)
+goto cleanup;
 
 if (corAlias)
 ignore_value(qemuMonitorBlockdevDel(priv->mon, corAlias));
@@ -5093,7 +5095,8 @@ qemuDomainRemoveDevice(virQEMUDriverPtr driver,
  * into this function.
  */
 case VIR_DOMAIN_DEVICE_DISK:
-if (qemuDomainRemoveDiskDevice(driver, vm, dev->data.disk) < 0)
+if (qemuDomainRemoveDiskDevice(driver, vm, dev->data.disk,
+   QEMU_ASYNC_JOB_NONE) < 0)
 return -1;
 break;
 case VIR_DOMAIN_DEVICE_CONTROLLER:
-- 
2.27.0

[RFC PATCH 0/7] To make disk sharable

[Masayoshi Mizuma]

From: Masayoshi Mizuma 

This patch series is RFC to implement to make  disk sharable.

Currently,  disk option for qemu uses blockdev-snapshot QMP
command with overlay.
In that case, qemu holds the write lock to the original disk, so we cannot
share the original disks with the other qemu guests.

This patch series tries to implement to make the disks, which have
 disk option, sharable by using disk hot-plug.

First, create the overlay disk with the original disk is set as the 
backingStore.
Then, blockdev-del the StorageProgs and FormatProgs of the disk. That's
because to fix the bootindex of the disk.
Lastly, device_add the disks and CPUs start.

The sharable transient disk has the same limitation as the current 
 disk option, and also has the following limitations:

  - qemu needs to have hotplug feature
  - All the disk bus is virtio

Masayoshi Mizuma (7):
  qemu_hotplug: Add asynJob argument to qemuDomainAttachDiskGeneric
  qemu_hotplug: Add asyncJob argument to
qemuDomainAttachDeviceDiskLiveInternal
  qemu_hotplug: Add asynJob argument to qemuDomainRemoveDiskDevice
  qemu_hotplug: Add bootindex argument to qemuDomainAttachDiskGeneric
  qemu_hotplug: make  disk sharable
  qemu: Add check whether the transient disks are sharable
  qemu: Add virtio disks as sharable transient disks

 src/qemu/qemu_command.c |  20 ++-
 src/qemu/qemu_domain.h  |   3 +
 src/qemu/qemu_hotplug.c | 322 ++--
 src/qemu/qemu_hotplug.h |   3 +
 src/qemu/qemu_process.c |  38 -
 5 files changed, 368 insertions(+), 18 deletions(-)

-- 
2.27.0

[RFC PATCH 1/7] qemu_hotplug: Add asynJob argument to qemuDomainAttachDiskGeneric

[Masayoshi Mizuma]

From: Masayoshi Mizuma 

Add asynJob argument to qemuDomainAttachDiskGeneric() so that it can
be used before CPUs start.

Signed-off-by: Masayoshi Mizuma 
---
 src/qemu/qemu_hotplug.c | 12 +++-
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c
index 882e5d2384..609e9d1a8a 100644
--- a/src/qemu/qemu_hotplug.c
+++ b/src/qemu/qemu_hotplug.c
@@ -693,7 +693,8 @@ qemuDomainChangeEjectableMedia(virQEMUDriverPtr driver,
 static int
 qemuDomainAttachDiskGeneric(virQEMUDriverPtr driver,
 virDomainObjPtr vm,
-virDomainDiskDefPtr disk)
+virDomainDiskDefPtr disk,
+qemuDomainAsyncJob asyncJob)
 {
 g_autoptr(qemuBlockStorageSourceChainData) data = NULL;
 int ret = -1;
@@ -740,7 +741,8 @@ qemuDomainAttachDiskGeneric(virQEMUDriverPtr driver,
 if (qemuHotplugAttachManagedPR(driver, vm, disk->src, QEMU_ASYNC_JOB_NONE) 
< 0)
 goto cleanup;
 
-qemuDomainObjEnterMonitor(driver, vm);
+if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) < 0)
+goto cleanup;
 
 if (qemuBlockStorageSourceChainAttach(priv->mon, data) < 0)
 goto exit_monitor;
@@ -820,7 +822,7 @@ qemuDomainAttachVirtioDiskDevice(virQEMUDriverPtr driver,
 if (qemuDomainEnsureVirtioAddress(, vm, , disk->dst) < 0)
 return -1;
 
-if ((rv = qemuDomainAttachDiskGeneric(driver, vm, disk)) < 0) {
+if ((rv = qemuDomainAttachDiskGeneric(driver, vm, disk, 
QEMU_ASYNC_JOB_NONE)) < 0) {
 if (rv == -1 && releaseaddr)
 qemuDomainReleaseDeviceAddress(vm, >info);
 
@@ -999,7 +1001,7 @@ qemuDomainAttachSCSIDisk(virQEMUDriverPtr driver,
 return -1;
 }
 
-if (qemuDomainAttachDiskGeneric(driver, vm, disk) < 0)
+if (qemuDomainAttachDiskGeneric(driver, vm, disk, QEMU_ASYNC_JOB_NONE) < 0)
 return -1;
 
 return 0;
@@ -1016,7 +1018,7 @@ qemuDomainAttachUSBMassStorageDevice(virQEMUDriverPtr 
driver,
 if (virDomainUSBAddressEnsure(priv->usbaddrs, >info) < 0)
 return -1;
 
-if (qemuDomainAttachDiskGeneric(driver, vm, disk) < 0) {
+if (qemuDomainAttachDiskGeneric(driver, vm, disk, QEMU_ASYNC_JOB_NONE) < 
0) {
 virDomainUSBAddressRelease(priv->usbaddrs, >info);
 return -1;
 }
-- 
2.27.0

[PATCH 12/12] news: implement new Hyper-V APIs

[Matt Coleman]

Signed-off-by: Matt Coleman 
---
 NEWS.rst | 12 
 1 file changed, 12 insertions(+)

diff --git a/NEWS.rst b/NEWS.rst
index 7a2d6649b4..cc8ac4e6eb 100644
--- a/NEWS.rst
+++ b/NEWS.rst
@@ -18,6 +18,18 @@ v7.1.0 (unreleased)
 The virtio-pmem is a virtio variant of NVDIMM and just like NVDIMM
 virtio-pmem also allows accessing host pages bypassing guest page cache.
 
+  * hyperv: implement new APIs
+
+The ``virDomainUndefine()``, ``virDomainUndefineFlags()``,
+``virDomainDefineXML()``, ``virDomainAttachDevice()``, and
+``virDomainAttachDeviceFlags()``, ``virConnectListAllNetworks()``,
+``virConnectNumOfNetworks()``, ``virNetworkLookupByName()``,
+``virNetworkLookupByUUID()``, ``virConnectNumOfDefinedNetworks()``,
+``virConnectListDefinedNetworks()``, ``virNetworkGetAutostart()``,
+``virNetworkIsActive()``, ``virNetworkIsPersistent()``,
+``virNetworkGetXMLDesc()``, and ``virDomainScreenshot()``, APIs have been
+implemented in the Hyper-V driver.
+
 * **Improvements**
 
 * **Bug fixes**
-- 
2.30.0

[PATCH 11/12] hyperv: provide a more detailed error message for WSMan faults

[Matt Coleman]

Signed-off-by: Matt Coleman 
---
 src/hyperv/hyperv_wmi.c | 15 ---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/src/hyperv/hyperv_wmi.c b/src/hyperv/hyperv_wmi.c
index 4c1bd5e0d2..c14ff0e64a 100644
--- a/src/hyperv/hyperv_wmi.c
+++ b/src/hyperv/hyperv_wmi.c
@@ -789,9 +789,18 @@ hypervInvokeMethod(hypervPrivate *priv,
 
 returnValue = ws_xml_get_xpath_value(response, returnValue_xpath);
 if (!returnValue) {
-virReportError(VIR_ERR_INTERNAL_ERROR,
-   _("Could not get return value for %s invocation"),
-   params->method);
+g_autofree char *faultReason_xpath = 
g_strdup("/s:Envelope/s:Body/s:Fault/s:Reason/s:Text");
+g_autofree char *faultReason = ws_xml_get_xpath_value(response, 
faultReason_xpath);
+
+if (faultReason)
+virReportError(VIR_ERR_INTERNAL_ERROR,
+   _("WS-Management fault during %s invocation: %s"),
+   params->method, faultReason);
+else
+virReportError(VIR_ERR_INTERNAL_ERROR,
+   _("Could not get return value for %s invocation"),
+   params->method);
+
 return -1;
 }
 
-- 
2.30.0

[PATCH 10/12] hyperv: implement domainScreenshot

[Matt Coleman]

Signed-off-by: Matt Coleman 
---
 src/hyperv/hyperv_driver.c| 207 ++
 src/hyperv/hyperv_wmi_classes.h   |  21 +++
 src/hyperv/hyperv_wmi_generator.input | 163 
 3 files changed, 391 insertions(+)

diff --git a/src/hyperv/hyperv_driver.c b/src/hyperv/hyperv_driver.c
index 22cfe6657d..0fe01f13b5 100644
--- a/src/hyperv/hyperv_driver.c
+++ b/src/hyperv/hyperv_driver.c
@@ -22,6 +22,8 @@
 
 #include 
 
+#include 
+
 #include "internal.h"
 #include "datatypes.h"
 #include "virdomainobjlist.h"
@@ -38,6 +40,8 @@
 #include "virstring.h"
 #include "virkeycode.h"
 #include "domain_conf.h"
+#include "virfdstream.h"
+#include "virfile.h"
 
 #define VIR_FROM_THIS VIR_FROM_HYPERV
 
@@ -294,6 +298,73 @@ hypervCapsInit(hypervPrivate *priv)
 return NULL;
 }
 
+
+static int
+hypervGetVideoResolution(hypervPrivate *priv,
+ char *vm_uuid,
+ int *xRes,
+ int *yRes,
+ bool fallback)
+{
+g_auto(virBuffer) query = VIR_BUFFER_INITIALIZER;
+g_autoptr(Msvm_S3DisplayController) s3Display = NULL;
+g_autoptr(Msvm_SyntheticDisplayController) synthetic = NULL;
+g_autoptr(Msvm_VideoHead) heads = NULL;
+const char *wmiClass = NULL;
+char *deviceId = NULL;
+g_autofree char *enabledStateString = NULL;
+
+if (fallback) {
+wmiClass = "Msvm_S3DisplayController";
+
+virBufferEscapeSQL(,
+   MSVM_S3DISPLAYCONTROLLER_WQL_SELECT "WHERE 
SystemName = '%s'",
+   vm_uuid);
+
+if (hypervGetWmiClass(Msvm_S3DisplayController, ) < 0 || 
!s3Display)
+return -1;
+
+deviceId = s3Display->data->DeviceID;
+} else {
+wmiClass = "Msvm_SyntheticDisplayController";
+
+virBufferEscapeSQL(,
+   MSVM_SYNTHETICDISPLAYCONTROLLER_WQL_SELECT "WHERE 
SystemName = '%s'",
+   vm_uuid);
+
+if (hypervGetWmiClass(Msvm_SyntheticDisplayController, ) < 0 
|| !synthetic)
+return -1;
+
+deviceId = synthetic->data->DeviceID;
+}
+
+virBufferFreeAndReset();
+
+virBufferAsprintf(,
+  "ASSOCIATORS OF {%s."
+  "CreationClassName='%s',"
+  "DeviceID='%s',"
+  "SystemCreationClassName='Msvm_ComputerSystem',"
+  "SystemName='%s'"
+  "} WHERE AssocClass = Msvm_VideoHeadOnController "
+  "ResultClass = Msvm_VideoHead",
+  wmiClass, wmiClass, deviceId, vm_uuid);
+
+if (hypervGetWmiClass(Msvm_VideoHead, ) < 0)
+return -1;
+
+enabledStateString = g_strdup_printf("%d", 
CIM_ENABLEDLOGICALELEMENT_ENABLEDSTATE_ENABLED);
+if (heads && STREQ(heads->data->EnabledState, enabledStateString)) {
+*xRes = heads->data->CurrentHorizontalResolution;
+*yRes = heads->data->CurrentVerticalResolution;
+
+return 0;
+}
+
+return -1;
+}
+
+
 /*
  * Virtual device functions
  */
@@ -2322,6 +2393,141 @@ hypervDomainGetState(virDomainPtr domain, int *state, 
int *reason,
 }
 
 
+static char *
+hypervDomainScreenshot(virDomainPtr domain,
+   virStreamPtr stream,
+   unsigned int screen G_GNUC_UNUSED,
+   unsigned int flags)
+{
+char uuid_string[VIR_UUID_STRING_BUFLEN];
+hypervPrivate *priv = domain->conn->privateData;
+g_auto(virBuffer) query = VIR_BUFFER_INITIALIZER;
+g_autoptr(hypervInvokeParamsList) params = NULL;
+g_auto(WsXmlDocH) ret_doc = NULL;
+int xRes = 640;
+int yRes = 480;
+g_autofree char *width = NULL;
+g_autofree char *height = NULL;
+g_autofree char *imageDataText = NULL;
+g_autofree unsigned char *imageDataBuffer = NULL;
+size_t imageDataBufferSize;
+const char *temporaryDirectory = NULL;
+g_autofree char *temporaryFile = NULL;
+g_autofree uint8_t *ppmBuffer = NULL;
+char *result = NULL;
+const char *xpath = 
"/s:Envelope/s:Body/p:GetVirtualSystemThumbnailImage_OUTPUT/p:ImageData";
+int pixelCount;
+int pixelByteCount;
+size_t i = 0;
+int fd = -1;
+bool unlinkTemporaryFile = false;
+
+virCheckFlags(0, NULL);
+
+virUUIDFormat(domain->uuid, uuid_string);
+
+/* Hyper-V Generation 1 VMs use two video heads:
+ *  - S3DisplayController is used for early boot screens.
+ *  - SyntheticDisplayController takes over when the guest OS initializes 
its video driver.
+ *
+ * This attempts to get the resolution from the SyntheticDisplayController 
first.
+ * If that fails, it falls back to S3DisplayController. */
+if (hypervGetVideoResolution(priv, uuid_string, , , false) < 0) {
+if (hypervGetVideoResolution(priv, uuid_string, , , true) < 
0)
+goto cleanup;
+}
+
+/* prepare params */
+params = 

[PATCH 09/12] hyperv: implement networkGetXMLDesc

[Matt Coleman]

Co-authored-by: Dawid Zamirski 
Signed-off-by: Matt Coleman 
---
 src/hyperv/hyperv_network_driver.c | 24 
 1 file changed, 24 insertions(+)

diff --git a/src/hyperv/hyperv_network_driver.c 
b/src/hyperv/hyperv_network_driver.c
index d6407ac591..93ef01c9aa 100644
--- a/src/hyperv/hyperv_network_driver.c
+++ b/src/hyperv/hyperv_network_driver.c
@@ -180,6 +180,29 @@ hypervNetworkLookupByName(virConnectPtr conn, const char 
*name)
 }
 
 
+static char *
+hypervNetworkGetXMLDesc(virNetworkPtr network, unsigned int flags)
+{
+g_autoptr(virNetwork) hypervNetwork = NULL;
+g_autoptr(virNetworkDef) def = NULL;
+
+def = g_new0(virNetworkDef, 1);
+
+hypervNetwork = hypervNetworkLookupByUUID(network->conn, network->uuid);
+if (!hypervNetwork)
+return NULL;
+
+memcpy(def->uuid, network->uuid, VIR_UUID_BUFLEN);
+def->uuid_specified = true;
+
+def->name = g_strdup(hypervNetwork->name);
+
+def->forward.type = VIR_NETWORK_FORWARD_NONE;
+
+return virNetworkDefFormat(def, NULL, flags);
+}
+
+
 static int
 hypervNetworkGetAutostart(virNetworkPtr network G_GNUC_UNUSED, int *autostart)
 {
@@ -212,6 +235,7 @@ virNetworkDriver hypervNetworkDriver = {
 .connectListAllNetworks = hypervConnectListAllNetworks, /* 7.1.0 */
 .networkLookupByUUID = hypervNetworkLookupByUUID, /* 7.1.0 */
 .networkLookupByName = hypervNetworkLookupByName, /* 7.1.0 */
+.networkGetXMLDesc = hypervNetworkGetXMLDesc, /* 7.1.0 */
 .networkGetAutostart = hypervNetworkGetAutostart, /* 7.1.0 */
 .networkIsActive = hypervNetworkIsActive, /* 7.1.0 */
 .networkIsPersistent = hypervNetworkIsPersistent, /* 7.1.0 */
-- 
2.30.0

[PATCH 07/12] hyperv: implement connectNumOfDefinedNetworks and connectListDefinedNetworks

[Matt Coleman]

Co-authored-by: Dawid Zamirski 
Signed-off-by: Matt Coleman 
---
 src/hyperv/hyperv_network_driver.c | 20 
 1 file changed, 20 insertions(+)

diff --git a/src/hyperv/hyperv_network_driver.c 
b/src/hyperv/hyperv_network_driver.c
index 25755f2525..f85dea31fe 100644
--- a/src/hyperv/hyperv_network_driver.c
+++ b/src/hyperv/hyperv_network_driver.c
@@ -76,6 +76,24 @@ hypervNetworkLookup(virConnectPtr conn, const char 
*property, const char *value)
  * Exported API functions
  */
 
+static int
+hypervConnectNumOfDefinedNetworks(virConnectPtr conn G_GNUC_UNUSED)
+{
+/* Hyper-V networks are always active */
+return 0;
+}
+
+
+static int
+hypervConnectListDefinedNetworks(virConnectPtr conn G_GNUC_UNUSED,
+ char **const names G_GNUC_UNUSED,
+ int maxnames G_GNUC_UNUSED)
+{
+/* Hyper-V networks are always active */
+return 0;
+}
+
+
 #define MATCH(FLAG) (flags & (FLAG))
 static int
 hypervConnectListAllNetworks(virConnectPtr conn,
@@ -164,6 +182,8 @@ hypervNetworkLookupByName(virConnectPtr conn, const char 
*name)
 
 virNetworkDriver hypervNetworkDriver = {
 .connectNumOfNetworks = hypervConnectNumOfNetworks, /* 7.1.0 */
+.connectNumOfDefinedNetworks = hypervConnectNumOfDefinedNetworks, /* 7.1.0 
*/
+.connectListDefinedNetworks = hypervConnectListDefinedNetworks, /* 7.1.0 */
 .connectListAllNetworks = hypervConnectListAllNetworks, /* 7.1.0 */
 .networkLookupByUUID = hypervNetworkLookupByUUID, /* 7.1.0 */
 .networkLookupByName = hypervNetworkLookupByName, /* 7.1.0 */
-- 
2.30.0

[PATCH 08/12] hyperv: implement networkGetAutostart, networkIsActive, and networkIsPersistent

[Matt Coleman]

Co-authored-by: Dawid Zamirski 
Signed-off-by: Matt Coleman 
---
 src/hyperv/hyperv_network_driver.c | 28 
 1 file changed, 28 insertions(+)

diff --git a/src/hyperv/hyperv_network_driver.c 
b/src/hyperv/hyperv_network_driver.c
index f85dea31fe..d6407ac591 100644
--- a/src/hyperv/hyperv_network_driver.c
+++ b/src/hyperv/hyperv_network_driver.c
@@ -180,6 +180,31 @@ hypervNetworkLookupByName(virConnectPtr conn, const char 
*name)
 }
 
 
+static int
+hypervNetworkGetAutostart(virNetworkPtr network G_GNUC_UNUSED, int *autostart)
+{
+/* Hyper-V networks are always active */
+*autostart = 1;
+return 0;
+}
+
+
+static int
+hypervNetworkIsActive(virNetworkPtr network G_GNUC_UNUSED)
+{
+/* Hyper-V networks are always active */
+return 1;
+}
+
+
+static int
+hypervNetworkIsPersistent(virNetworkPtr network G_GNUC_UNUSED)
+{
+/* Hyper-V networks are always persistent */
+return 1;
+}
+
+
 virNetworkDriver hypervNetworkDriver = {
 .connectNumOfNetworks = hypervConnectNumOfNetworks, /* 7.1.0 */
 .connectNumOfDefinedNetworks = hypervConnectNumOfDefinedNetworks, /* 7.1.0 
*/
@@ -187,4 +212,7 @@ virNetworkDriver hypervNetworkDriver = {
 .connectListAllNetworks = hypervConnectListAllNetworks, /* 7.1.0 */
 .networkLookupByUUID = hypervNetworkLookupByUUID, /* 7.1.0 */
 .networkLookupByName = hypervNetworkLookupByName, /* 7.1.0 */
+.networkGetAutostart = hypervNetworkGetAutostart, /* 7.1.0 */
+.networkIsActive = hypervNetworkIsActive, /* 7.1.0 */
+.networkIsPersistent = hypervNetworkIsPersistent, /* 7.1.0 */
 };
-- 
2.30.0

[PATCH 06/12] hyperv: implement networkLookupByName and networkLookupByUUID

[Matt Coleman]

Signed-off-by: Matt Coleman 
---
 src/hyperv/hyperv_network_driver.c | 43 ++
 1 file changed, 43 insertions(+)

diff --git a/src/hyperv/hyperv_network_driver.c 
b/src/hyperv/hyperv_network_driver.c
index 3931e548f5..25755f2525 100644
--- a/src/hyperv/hyperv_network_driver.c
+++ b/src/hyperv/hyperv_network_driver.c
@@ -49,6 +49,29 @@ hypervMsvmVirtualSwitchToNetwork(virConnectPtr conn, 
Msvm_VirtualEthernetSwitch
 }
 
 
+static virNetworkPtr
+hypervNetworkLookup(virConnectPtr conn, const char *property, const char 
*value)
+{
+hypervPrivate *priv = conn->privateData;
+g_auto(virBuffer) query = VIR_BUFFER_INITIALIZER;
+g_autoptr(Msvm_VirtualEthernetSwitch) virtualSwitch = NULL;
+
+virBufferAsprintf(, MSVM_VIRTUALETHERNETSWITCH_WQL_SELECT "WHERE 
%s", property);
+virBufferEscapeSQL(, " = '%s'", value);
+
+if (hypervGetWmiClass(Msvm_VirtualEthernetSwitch, ) < 0)
+return NULL;
+
+if (!virtualSwitch) {
+virReportError(VIR_ERR_NO_NETWORK,
+   _("No network found with property '%s' = '%s'"), 
property, value);
+return NULL;
+}
+
+return hypervMsvmVirtualSwitchToNetwork(conn, virtualSwitch);
+}
+
+
 /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
  * Exported API functions
  */
@@ -121,7 +144,27 @@ hypervConnectNumOfNetworks(virConnectPtr conn)
 }
 
 
+static virNetworkPtr
+hypervNetworkLookupByUUID(virConnectPtr conn, const unsigned char *uuid)
+{
+char uuid_string[VIR_UUID_STRING_BUFLEN];
+
+virUUIDFormat(uuid, uuid_string);
+
+return hypervNetworkLookup(conn, "Name", uuid_string);
+}
+
+
+static virNetworkPtr
+hypervNetworkLookupByName(virConnectPtr conn, const char *name)
+{
+return hypervNetworkLookup(conn, "ElementName", name);
+}
+
+
 virNetworkDriver hypervNetworkDriver = {
 .connectNumOfNetworks = hypervConnectNumOfNetworks, /* 7.1.0 */
 .connectListAllNetworks = hypervConnectListAllNetworks, /* 7.1.0 */
+.networkLookupByUUID = hypervNetworkLookupByUUID, /* 7.1.0 */
+.networkLookupByName = hypervNetworkLookupByName, /* 7.1.0 */
 };
-- 
2.30.0

[PATCH 04/12] hyperv: add support for creating network adapters

[Matt Coleman]

Co-authored-by: Sri Ramanujam 
Signed-off-by: Matt Coleman 
---
 src/hyperv/hyperv_driver.c| 138 ++
 src/hyperv/hyperv_wmi_classes.h   |   2 +
 src/hyperv/hyperv_wmi_generator.input |  32 ++
 3 files changed, 172 insertions(+)

diff --git a/src/hyperv/hyperv_driver.c b/src/hyperv/hyperv_driver.c
index ca75de98a9..8c34f5addd 100644
--- a/src/hyperv/hyperv_driver.c
+++ b/src/hyperv/hyperv_driver.c
@@ -978,6 +978,132 @@ hypervDomainAttachSerial(virDomainPtr domain, 
virDomainChrDefPtr serial)
 }
 
 
+static int
+hypervDomainAttachSyntheticEthernetAdapter(virDomainPtr domain,
+   virDomainNetDefPtr net,
+   char *hostname)
+{
+hypervPrivate *priv = domain->conn->privateData;
+g_autofree char *portResourceType = NULL;
+unsigned char vsiGuid[VIR_UUID_BUFLEN];
+char guidString[VIR_UUID_STRING_BUFLEN];
+g_autofree char *virtualSystemIdentifiers = NULL;
+char macString[VIR_MAC_STRING_BUFLEN];
+g_autofree char *macAddressNoColons = NULL;
+g_autoptr(GHashTable) portResource = NULL;
+g_auto(WsXmlDocH) sepsdResponse = NULL;
+g_auto(virBuffer) query = VIR_BUFFER_INITIALIZER;
+g_autoptr(Msvm_VirtualEthernetSwitch) vSwitch = NULL;
+g_autofree char *enabledState = NULL;
+g_autofree char *switch__PATH = NULL;
+g_autofree char *sepsd__PATH = NULL;
+g_autofree char *sepsdInstanceID = NULL;
+g_autofree char *sepsdInstanceEscaped = NULL;
+g_autofree char *connectionResourceType = NULL;
+g_autoptr(GHashTable) connectionResource = NULL;
+
+/*
+ * Step 1: Create the Msvm_SyntheticEthernetPortSettingData object
+ * that holds half the settings for the new adapter we are creating
+ */
+portResourceType = g_strdup_printf("%d", 
MSVM_RASD_RESOURCETYPE_ETHERNET_ADAPTER);
+
+virUUIDGenerate(vsiGuid);
+virUUIDFormat(vsiGuid, guidString);
+virtualSystemIdentifiers = g_strdup_printf("{%s}", guidString);
+
+virMacAddrFormat(>mac, macString);
+macAddressNoColons = virStringReplace(macString, ":", "");
+
+/* prepare embedded param */
+portResource = 
hypervCreateEmbeddedParam(Msvm_SyntheticEthernetPortSettingData_WmiInfo);
+if (!portResource)
+return -1;
+
+if (hypervSetEmbeddedProperty(portResource, "ResourceType", 
portResourceType) < 0)
+return -1;
+
+if (hypervSetEmbeddedProperty(portResource, "ResourceSubType",
+  "Microsoft:Hyper-V:Synthetic Ethernet Port") 
< 0)
+return -1;
+
+if (hypervSetEmbeddedProperty(portResource,
+  "VirtualSystemIdentifiers", 
virtualSystemIdentifiers) < 0)
+return -1;
+
+if (hypervSetEmbeddedProperty(portResource, "Address", macAddressNoColons) 
< 0)
+return -1;
+
+if (hypervSetEmbeddedProperty(portResource, "StaticMacAddress", "true") < 
0)
+return -1;
+
+if (hypervMsvmVSMSAddResourceSettings(domain, ,
+  
Msvm_SyntheticEthernetPortSettingData_WmiInfo,
+  ) < 0)
+return -1;
+
+/*
+ * Step 2: Get the Msvm_VirtualEthernetSwitch object
+ */
+virBufferAsprintf(,
+  MSVM_VIRTUALETHERNETSWITCH_WQL_SELECT "WHERE Name='%s'",
+  net->data.bridge.brname);
+
+if (hypervGetWmiClass(Msvm_VirtualEthernetSwitch, ) < 0)
+return -1;
+
+if (!vSwitch)
+return -1;
+
+/*
+ * Step 3: Create the Msvm_EthernetPortAllocationSettingData object that
+ * holds the other half of the network configuration
+ */
+enabledState = g_strdup_printf("%d", 
MSVM_ETHERNETPORTALLOCATIONSETTINGDATA_ENABLEDSTATE_ENABLED);
+
+/* build the two __PATH variables */
+switch__PATH = g_strdup_printf("%s\\Root\\Virtualization\\V2:"
+   
"Msvm_VirtualEthernetSwitch.CreationClassName=\"Msvm_VirtualEthernetSwitch\","
+   "Name=\"%s\"",
+   hostname, vSwitch->data->Name);
+
+/* Get the sepsd instance ID out of the XML response */
+sepsdInstanceID = hypervGetInstanceIDFromXMLResponse(sepsdResponse);
+sepsdInstanceEscaped = virStringReplace(sepsdInstanceID, "\\", "");
+sepsd__PATH = g_strdup_printf("%s\\root\\virtualization\\v2:"
+  
"Msvm_SyntheticEthernetPortSettingData.InstanceID=\"%s\"",
+  hostname, sepsdInstanceEscaped);
+
+connectionResourceType = g_strdup_printf("%d", 
MSVM_RASD_RESOURCETYPE_ETHERNET_CONNECTION);
+
+connectionResource = 
hypervCreateEmbeddedParam(Msvm_EthernetPortAllocationSettingData_WmiInfo);
+if (!connectionResource)
+return -1;
+
+if (hypervSetEmbeddedProperty(connectionResource, "EnabledState", 
enabledState) < 0)
+return -1;
+
+if 

[PATCH 05/12] hyperv: implement connectListAllNetworks and connectNumOfNetworks

[Matt Coleman]

Co-authored-by: Dawid Zamirski 
Signed-off-by: Matt Coleman 
---
 po/POTFILES.in |   1 +
 src/hyperv/hyperv_driver.c |   2 +
 src/hyperv/hyperv_network_driver.c | 127 +
 src/hyperv/hyperv_network_driver.h |  26 ++
 src/hyperv/meson.build |   1 +
 5 files changed, 157 insertions(+)
 create mode 100644 src/hyperv/hyperv_network_driver.c
 create mode 100644 src/hyperv/hyperv_network_driver.h

diff --git a/po/POTFILES.in b/po/POTFILES.in
index a2c46dd239..fc5850b59d 100644
--- a/po/POTFILES.in
+++ b/po/POTFILES.in
@@ -78,6 +78,7 @@
 @SRCDIR@src/esx/esx_vi_methods.c
 @SRCDIR@src/esx/esx_vi_types.c
 @SRCDIR@src/hyperv/hyperv_driver.c
+@SRCDIR@src/hyperv/hyperv_network_driver.c
 @SRCDIR@src/hyperv/hyperv_util.c
 @SRCDIR@src/hyperv/hyperv_wmi.c
 @SRCDIR@src/hypervisor/domain_cgroup.c
diff --git a/src/hyperv/hyperv_driver.c b/src/hyperv/hyperv_driver.c
index 8c34f5addd..22cfe6657d 100644
--- a/src/hyperv/hyperv_driver.c
+++ b/src/hyperv/hyperv_driver.c
@@ -31,6 +31,7 @@
 #include "viruuid.h"
 #include "virutil.h"
 #include "hyperv_driver.h"
+#include "hyperv_network_driver.h"
 #include "hyperv_private.h"
 #include "hyperv_util.h"
 #include "hyperv_wmi.h"
@@ -3560,6 +3561,7 @@ static virConnectDriver hypervConnectDriver = {
 .remoteOnly = true,
 .uriSchemes = (const char *[]){ "hyperv", NULL },
 .hypervisorDriver = ,
+.networkDriver = ,
 };
 
 int
diff --git a/src/hyperv/hyperv_network_driver.c 
b/src/hyperv/hyperv_network_driver.c
new file mode 100644
index 00..3931e548f5
--- /dev/null
+++ b/src/hyperv/hyperv_network_driver.c
@@ -0,0 +1,127 @@
+/*
+ * hyperv_network_driver.c: network driver functions for Microsoft Hyper-V 
hosts
+ *
+ * Copyright (C) 2020 Datto Inc
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library.  If not, see
+ * .
+ *
+ */
+
+#include 
+
+#include "datatypes.h"
+#include "viralloc.h"
+#include "network_conf.h"
+#include "hyperv_network_driver.h"
+#include "hyperv_wmi.h"
+
+#define VIR_FROM_THIS VIR_FROM_HYPERV
+
+/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
+ * Utility functions
+ */
+
+static virNetworkPtr
+hypervMsvmVirtualSwitchToNetwork(virConnectPtr conn, 
Msvm_VirtualEthernetSwitch *virtualSwitch)
+{
+unsigned char uuid[VIR_UUID_BUFLEN];
+
+if (virUUIDParse(virtualSwitch->data->Name, uuid) < 0) {
+virReportError(VIR_ERR_INTERNAL_ERROR,
+   _("Could not parse UUID from string '%s'"),
+   virtualSwitch->data->Name);
+return NULL;
+}
+
+return virGetNetwork(conn, virtualSwitch->data->ElementName, uuid);
+}
+
+
+/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
+ * Exported API functions
+ */
+
+#define MATCH(FLAG) (flags & (FLAG))
+static int
+hypervConnectListAllNetworks(virConnectPtr conn,
+ virNetworkPtr **nets,
+ unsigned int flags)
+{
+int ret = -1;
+hypervPrivate *priv = conn->privateData;
+size_t count = 0;
+size_t i;
+g_auto(virBuffer) query = { 
g_string_new(MSVM_VIRTUALETHERNETSWITCH_WQL_SELECT
+ "WHERE HealthState = 5"), 0 };
+g_autoptr(Msvm_VirtualEthernetSwitch) switches = NULL;
+Msvm_VirtualEthernetSwitch *entry = NULL;
+
+virCheckFlags(VIR_CONNECT_LIST_NETWORKS_FILTERS_ALL, -1);
+
+/*
+ * Hyper-V networks are always active, persistent, and
+ * autostarted, so return zero elements in case we are asked
+ * for networks different than that.
+ */
+if (MATCH(VIR_CONNECT_LIST_NETWORKS_FILTERS_ACTIVE) &&
+!(MATCH(VIR_CONNECT_LIST_NETWORKS_ACTIVE)))
+return 0;
+if (MATCH(VIR_CONNECT_LIST_NETWORKS_FILTERS_PERSISTENT) &&
+!(MATCH(VIR_CONNECT_LIST_NETWORKS_PERSISTENT)))
+return 0;
+if (MATCH(VIR_CONNECT_LIST_NETWORKS_FILTERS_AUTOSTART) &&
+!(MATCH(VIR_CONNECT_LIST_NETWORKS_AUTOSTART)))
+return 0;
+
+if (hypervGetWmiClass(Msvm_VirtualEthernetSwitch, ) < 0)
+goto cleanup;
+
+for (entry = switches; entry; entry = entry->next) {
+if (nets) {
+virNetworkPtr net = hypervMsvmVirtualSwitchToNetwork(conn, entry);
+if (!net)
+goto cleanup;
+if (VIR_APPEND_ELEMENT(*nets, count, net) 

[PATCH 02/12] hyperv: add support for creating serial devices

[Matt Coleman]

Co-authored-by: Sri Ramanujam 
Signed-off-by: Matt Coleman 
---
 src/hyperv/hyperv_driver.c | 92 ++
 1 file changed, 92 insertions(+)

diff --git a/src/hyperv/hyperv_driver.c b/src/hyperv/hyperv_driver.c
index 9bbbcfc88c..9394794c8f 100644
--- a/src/hyperv/hyperv_driver.c
+++ b/src/hyperv/hyperv_driver.c
@@ -899,6 +899,85 @@ hypervDomainAttachStorage(virDomainPtr domain, 
virDomainDefPtr def, const char *
 }
 
 
+static int
+hypervDomainAttachSerial(virDomainPtr domain, virDomainChrDefPtr serial)
+{
+char uuid_string[VIR_UUID_STRING_BUFLEN];
+hypervPrivate *priv = domain->conn->privateData;
+g_autofree char *com_string = g_strdup_printf("COM %d", 
serial->target.port);
+g_autoptr(Msvm_VirtualSystemSettingData) vssd = NULL;
+g_autoptr(Msvm_ResourceAllocationSettingData) rasd = NULL;
+g_autoptr(Msvm_SerialPortSettingData) spsd = NULL;
+hypervWmiClassInfoPtr classInfo = NULL;
+Msvm_ResourceAllocationSettingData *current = NULL;
+Msvm_ResourceAllocationSettingData *entry = NULL;
+g_autoptr(GHashTable) serialResource = NULL;
+const char *connectionValue = NULL;
+g_autofree const char *resourceType = NULL;
+
+virUUIDFormat(domain->uuid, uuid_string);
+
+if (hypervGetMsvmVirtualSystemSettingDataFromUUID(priv, uuid_string, 
) < 0)
+return -1;
+
+if (g_str_has_prefix(priv->version, "6.")) {
+if (hypervGetResourceAllocationSD(priv, vssd->data->InstanceID, ) 
< 0)
+return -1;
+
+current = rasd;
+classInfo = Msvm_ResourceAllocationSettingData_WmiInfo;
+} else {
+if (hypervGetSerialPortSD(priv, vssd->data->InstanceID, ) < 0)
+return -1;
+
+current = (Msvm_ResourceAllocationSettingData *)spsd;
+classInfo = Msvm_SerialPortSettingData_WmiInfo;
+}
+
+while (current) {
+if (current->data->ResourceType == MSVM_RASD_RESOURCETYPE_SERIAL_PORT 
&&
+STREQ(current->data->ElementName, com_string)) {
+/* found our com port */
+entry = current;
+break;
+}
+current = current->next;
+}
+
+if (!entry)
+return -1;
+
+if (STRNEQ(serial->source->data.file.path, "-1"))
+connectionValue = serial->source->data.file.path;
+else
+connectionValue = "";
+
+resourceType = g_strdup_printf("%d", entry->data->ResourceType);
+
+serialResource = hypervCreateEmbeddedParam(classInfo);
+if (!serialResource)
+return -1;
+
+if (hypervSetEmbeddedProperty(serialResource, "Connection", 
connectionValue) < 0)
+return -1;
+
+if (hypervSetEmbeddedProperty(serialResource, "InstanceID", 
entry->data->InstanceID) < 0)
+return -1;
+
+if (hypervSetEmbeddedProperty(serialResource, "ResourceType", 
resourceType) < 0)
+return -1;
+
+if (hypervSetEmbeddedProperty(serialResource, "ResourceSubType",
+  entry->data->ResourceSubType) < 0)
+return -1;
+
+if (hypervMsvmVSMSModifyResourceSettings(priv, , classInfo) 
< 0)
+return -1;
+
+return 0;
+}
+
+
 /*
  * Functions for deserializing device entries
  */
@@ -2425,6 +2504,7 @@ hypervDomainDefineXML(virConnectPtr conn, const char *xml)
 virDomainPtr domain = NULL;
 g_autoptr(hypervInvokeParamsList) params = NULL;
 g_autoptr(GHashTable) defineSystemParam = NULL;
+size_t i = 0;
 
 /* parse xml */
 def = virDomainDefParseString(xml, priv->xmlopt, NULL,
@@ -2484,6 +2564,14 @@ hypervDomainDefineXML(virConnectPtr conn, const char 
*xml)
 if (hypervDomainAttachStorage(domain, def, hostname) < 0)
 goto error;
 
+/* Attach serials */
+for (i = 0; i < def->nserials; i++) {
+if (hypervDomainAttachSerial(domain, def->serials[i]) < 0) {
+virReportError(VIR_ERR_INTERNAL_ERROR, _("Could not attach serial 
port %lu"), i);
+goto error;
+}
+}
+
 return domain;
 
  error:
@@ -2583,6 +2671,10 @@ hypervDomainAttachDeviceFlags(virDomainPtr domain, const 
char *xml, unsigned int
 if (hypervDomainAttachStorageVolume(domain, dev->data.disk, 
controller, hostname) < 0)
 return -1;
 break;
+case VIR_DOMAIN_DEVICE_CHR:
+if (hypervDomainAttachSerial(domain, dev->data.chr) < 0)
+return -1;
+break;
 default:
 /* unsupported device type */
 virReportError(VIR_ERR_INTERNAL_ERROR,
-- 
2.30.0

[PATCH 01/12] hyperv: XML parsing of serial ports

[Matt Coleman]

Co-authored-by: Sri Ramanujam 
Signed-off-by: Matt Coleman 
---
 src/hyperv/hyperv_driver.c| 68 +++
 src/hyperv/hyperv_wmi.c   | 10 
 src/hyperv/hyperv_wmi.h   |  4 ++
 src/hyperv/hyperv_wmi_classes.h   |  1 +
 src/hyperv/hyperv_wmi_generator.input |  5 ++
 5 files changed, 88 insertions(+)

diff --git a/src/hyperv/hyperv_driver.c b/src/hyperv/hyperv_driver.c
index bdc084790a..9bbbcfc88c 100644
--- a/src/hyperv/hyperv_driver.c
+++ b/src/hyperv/hyperv_driver.c
@@ -1253,6 +1253,61 @@ hypervDomainDefParseStorage(hypervPrivate *priv,
 }
 
 
+static int
+hypervDomainDefParseSerial(virDomainDefPtr def, 
Msvm_ResourceAllocationSettingData *rasd)
+{
+int port_num = 0;
+char **conn = NULL;
+const char *srcPath = NULL;
+Msvm_ResourceAllocationSettingData *entry = rasd;
+virDomainChrDefPtr serial = NULL;
+
+while (entry) {
+if (entry->data->ResourceType == MSVM_RASD_RESOURCETYPE_SERIAL_PORT) {
+/* clear some vars */
+serial = NULL;
+port_num = 0;
+conn = NULL;
+srcPath = NULL;
+
+/* get port number */
+port_num = entry->data->ElementName[4] - '0';
+if (port_num < 1) {
+entry = entry->next;
+continue;
+}
+
+serial = virDomainChrDefNew(NULL);
+
+serial->deviceType = VIR_DOMAIN_CHR_DEVICE_TYPE_SERIAL;
+serial->source->type = VIR_DOMAIN_CHR_TYPE_PIPE;
+serial->target.port = port_num;
+
+/* set up source */
+if (entry->data->Connection.count < 1) {
+srcPath = "-1";
+} else {
+conn = entry->data->Connection.data;
+if (!*conn)
+srcPath = "-1";
+else
+srcPath = *conn;
+}
+
+serial->source->data.file.path = g_strdup(srcPath);
+
+if (VIR_APPEND_ELEMENT(def->serials, def->nserials, serial) < 0) {
+virDomainChrDefFree(serial);
+return -1;
+}
+}
+
+entry = entry->next;
+}
+
+return 0;
+}
+
 
 /*
  * Driver functions
@@ -2113,6 +2168,8 @@ hypervDomainGetXMLDesc(virDomainPtr domain, unsigned int 
flags)
 g_autoptr(Msvm_MemorySettingData) memorySettingData = NULL;
 g_autoptr(Msvm_ResourceAllocationSettingData) rasd = NULL;
 g_autoptr(Msvm_StorageAllocationSettingData) sasd = NULL;
+g_autoptr(Msvm_SerialPortSettingData) spsd = NULL;
+Msvm_ResourceAllocationSettingData *serialDevices = NULL;
 
 virCheckFlags(VIR_DOMAIN_XML_COMMON_FLAGS, NULL);
 
@@ -2151,6 +2208,9 @@ hypervDomainGetXMLDesc(virDomainPtr domain, unsigned int 
flags)
 return NULL;
 }
 
+if (hypervGetSerialPortSD(priv, 
virtualSystemSettingData->data->InstanceID, ) < 0)
+return NULL;
+
 /* Fill struct */
 def->virtType = VIR_DOMAIN_VIRT_HYPERV;
 
@@ -2215,6 +2275,14 @@ hypervDomainGetXMLDesc(virDomainPtr domain, unsigned int 
flags)
 if (hypervDomainDefParseStorage(priv, def, rasd, sasd) < 0)
 return NULL;
 
+if (g_str_has_prefix(priv->version, "6."))
+serialDevices = rasd;
+else
+serialDevices = (Msvm_ResourceAllocationSettingData *)spsd;
+
+if (hypervDomainDefParseSerial(def, serialDevices) < 0)
+return NULL;
+
 /* XXX xmlopts must be non-NULL */
 return virDomainDefFormat(def, NULL, 
virDomainDefFormatConvertXMLFlags(flags));
 }
diff --git a/src/hyperv/hyperv_wmi.c b/src/hyperv/hyperv_wmi.c
index a28bb0e815..f5f0797605 100644
--- a/src/hyperv/hyperv_wmi.c
+++ b/src/hyperv/hyperv_wmi.c
@@ -1488,6 +1488,16 @@ hypervGetStorageAllocationSD(hypervPrivate *priv,
 }
 
 
+int
+hypervGetSerialPortSD(hypervPrivate *priv,
+  const char *id,
+  Msvm_SerialPortSettingData **data)
+{
+hypervGetSettingData(Msvm_SerialPortSettingData, id, data);
+return 0;
+}
+
+
 /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
  * Msvm_VirtualSystemManagementService
  */
diff --git a/src/hyperv/hyperv_wmi.h b/src/hyperv/hyperv_wmi.h
index f09948895e..2b0d0e4a3f 100644
--- a/src/hyperv/hyperv_wmi.h
+++ b/src/hyperv/hyperv_wmi.h
@@ -254,6 +254,10 @@ int hypervGetStorageAllocationSD(hypervPrivate *priv,
  const char *id,
  Msvm_StorageAllocationSettingData **data);
 
+int hypervGetSerialPortSD(hypervPrivate *priv,
+  const char *id,
+  Msvm_SerialPortSettingData **data);
+
 /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
  * Msvm_VirtualSystemManagementService
  */
diff --git a/src/hyperv/hyperv_wmi_classes.h b/src/hyperv/hyperv_wmi_classes.h
index 2c03ca3745..2f813bedb3 100644
--- a/src/hyperv/hyperv_wmi_classes.h
+++ b/src/hyperv/hyperv_wmi_classes.h

[PATCH 03/12] hyperv: XML parsing of Ethernet adapters

[Matt Coleman]

Co-authored-by: Sri Ramanujam 
Signed-off-by: Matt Coleman 
---
 src/hyperv/hyperv_driver.c| 113 ++
 src/hyperv/hyperv_wmi.c   |  20 +
 src/hyperv/hyperv_wmi.h   |   8 ++
 src/hyperv/hyperv_wmi_classes.h   |  12 +++
 src/hyperv/hyperv_wmi_generator.input | 109 +
 5 files changed, 262 insertions(+)

diff --git a/src/hyperv/hyperv_driver.c b/src/hyperv/hyperv_driver.c
index 9394794c8f..ca75de98a9 100644
--- a/src/hyperv/hyperv_driver.c
+++ b/src/hyperv/hyperv_driver.c
@@ -1388,6 +1388,107 @@ hypervDomainDefParseSerial(virDomainDefPtr def, 
Msvm_ResourceAllocationSettingDa
 }
 
 
+static int
+hypervDomainDefParseEthernetAdapter(virDomainDefPtr def,
+Msvm_EthernetPortAllocationSettingData 
*net,
+hypervPrivate *priv)
+{
+virDomainNetDefPtr ndef = NULL;
+g_autoptr(Msvm_SyntheticEthernetPortSettingData) sepsd = NULL;
+g_autoptr(Msvm_VirtualEthernetSwitch) vSwitch = NULL;
+char **switchConnection = NULL;
+g_autofree char *switchConnectionEscaped = NULL;
+char *sepsdPATH = NULL;
+g_autofree char *sepsdEscaped = NULL;
+g_auto(virBuffer) query = VIR_BUFFER_INITIALIZER;
+
+VIR_DEBUG("Parsing ethernet adapter '%s'", net->data->InstanceID);
+
+ndef = g_new0(virDomainNetDef, 1);
+
+ndef->type = VIR_DOMAIN_NET_TYPE_BRIDGE;
+
+/*
+ * If there's no switch port connection or the EnabledState is disabled,
+ * then the adapter isn't hooked up to anything and we don't have to
+ * do anything more.
+ */
+switchConnection = net->data->HostResource.data;
+if (net->data->HostResource.count < 1 || !*switchConnection ||
+net->data->EnabledState == 
MSVM_ETHERNETPORTALLOCATIONSETTINGDATA_ENABLEDSTATE_DISABLED) {
+VIR_DEBUG("Adapter not connected to switch");
+return 0;
+}
+
+/*
+ * Now we retrieve the associated Msvm_SyntheticEthernetPortSettingData and
+ * Msvm_VirtualEthernetSwitch objects and use them to build the XML 
definition.
+ */
+
+/* begin by getting the Msvm_SyntheticEthernetPortSettingData object */
+sepsdPATH = net->data->Parent;
+sepsdEscaped = virStringReplace(sepsdPATH, "\\", "");
+virBufferAsprintf(,
+  MSVM_SYNTHETICETHERNETPORTSETTINGDATA_WQL_SELECT "WHERE 
__PATH = '%s'",
+  sepsdEscaped);
+
+if (hypervGetWmiClass(Msvm_SyntheticEthernetPortSettingData, ) < 0)
+return -1;
+
+if (!sepsd) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Could not retrieve NIC 
settings"));
+return -1;
+}
+
+/* set mac address */
+if (virMacAddrParseHex(sepsd->data->Address, >mac) < 0)
+return -1;
+
+/* now we get the Msvm_VirtualEthernetSwitch */
+virBufferFreeAndReset();
+switchConnectionEscaped = virStringReplace(*switchConnection, "\\", 
"");
+virBufferAsprintf(,
+  MSVM_VIRTUALETHERNETSWITCH_WQL_SELECT "WHERE __PATH = 
'%s'",
+  switchConnectionEscaped);
+
+if (hypervGetWmiClass(Msvm_VirtualEthernetSwitch, ) < 0)
+return -1;
+
+if (!vSwitch) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Could not retrieve 
virtual switch"));
+return -1;
+}
+
+/* get bridge name */
+ndef->data.bridge.brname = g_strdup(vSwitch->data->Name);
+
+if (VIR_APPEND_ELEMENT(def->nets, def->nnets, ndef) < 0) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Could not append 
definition to domain"));
+return -1;
+}
+
+return 0;
+}
+
+static int
+hypervDomainDefParseEthernet(virDomainPtr domain,
+ virDomainDefPtr def,
+ Msvm_EthernetPortAllocationSettingData *nets)
+{
+Msvm_EthernetPortAllocationSettingData *entry = nets;
+hypervPrivate *priv = domain->conn->privateData;
+
+while (entry) {
+if (hypervDomainDefParseEthernetAdapter(def, entry, priv) < 0)
+return -1;
+
+entry = entry->next;
+}
+
+return 0;
+}
+
+
 /*
  * Driver functions
  */
@@ -2249,6 +2350,7 @@ hypervDomainGetXMLDesc(virDomainPtr domain, unsigned int 
flags)
 g_autoptr(Msvm_StorageAllocationSettingData) sasd = NULL;
 g_autoptr(Msvm_SerialPortSettingData) spsd = NULL;
 Msvm_ResourceAllocationSettingData *serialDevices = NULL;
+g_autoptr(Msvm_EthernetPortAllocationSettingData) nets = NULL;
 
 virCheckFlags(VIR_DOMAIN_XML_COMMON_FLAGS, NULL);
 
@@ -2290,6 +2392,10 @@ hypervDomainGetXMLDesc(virDomainPtr domain, unsigned int 
flags)
 if (hypervGetSerialPortSD(priv, 
virtualSystemSettingData->data->InstanceID, ) < 0)
 return NULL;
 
+if (hypervGetEthernetPortAllocationSD(priv,
+  
virtualSystemSettingData->data->InstanceID, ) < 0)
+return NULL;
+
 /* Fill struct */
 def->virtType = 

[PATCH 00/12] Hyper-V serial ports, NICs, and screeshots

[Matt Coleman]

This patchset makes the following changes to the Hyper-V driver:
* enable XML parsing and creation of serial ports and NICs
* implement several networking APIs
* implement screenshots

Matt Coleman (12):
  hyperv: XML parsing of serial ports
  hyperv: add support for creating serial devices
  hyperv: XML parsing of Ethernet adapters
  hyperv: add support for creating network adapters
  hyperv: implement connectListAllNetworks and connectNumOfNetworks
  hyperv: implement networkLookupByName and networkLookupByUUID
  hyperv: implement connectNumOfDefinedNetworks and
connectListDefinedNetworks
  hyperv: implement networkGetAutostart, networkIsActive, and
networkIsPersistent
  hyperv: implement networkGetXMLDesc
  hyperv: implement domainScreenshot
  hyperv: provide a more detailed error message for WSMan faults
  news: implement new Hyper-V APIs

 NEWS.rst  |  12 +
 po/POTFILES.in|   1 +
 src/hyperv/hyperv_driver.c| 620 ++
 src/hyperv/hyperv_network_driver.c| 242 ++
 src/hyperv/hyperv_network_driver.h|  26 ++
 src/hyperv/hyperv_wmi.c   |  45 +-
 src/hyperv/hyperv_wmi.h   |  12 +
 src/hyperv/hyperv_wmi_classes.h   |  36 ++
 src/hyperv/hyperv_wmi_generator.input | 309 +
 src/hyperv/meson.build|   1 +
 10 files changed, 1301 insertions(+), 3 deletions(-)
 create mode 100644 src/hyperv/hyperv_network_driver.c
 create mode 100644 src/hyperv/hyperv_network_driver.h

-- 
2.30.0

Re: [PATCH 00/10] Introduce virtio-mem model

[Daniel Henrique Barboza]

On 1/22/21 6:19 PM, David Hildenbrand wrote:



Out of curiosity: are you aware of anyone working in enabling virtio-mem
for pseries/ppc64? I'm wondering if there's some kind of architecture
limitation in Power or if it's just a lack of interest.


I remember there is interest, however:

- arm64 and x86-64 is used more frequently in applicable (cloud?) setups, so it 
has high prio
- s390x doesn‘t have any proper memory hot(un)plug, and as I have a strong 
s399x background, it‘s rather easy for me to implement
- ppc64 at least supports hot(un)plug of DIMMs

There is nothing fundamental speaking against ppc64 support AFAIR.


That's good to hear.


A block size of 16MB should be possible. I‘m planning on looking into it, 
however, there are a lot of other things on my todo list for virtio-mem.


I'm not familiar with the 'block size' concept of the virtio-mem device that 
would
allow for 16MB increments. My knowledge of the pseries kernel/QEMU is that the
guest visible memory must always be 256MiB aligned due to PAPR mechanics that
forces a memory block to be at least this size. Albeit I believe that there is
no constraints of the memory this device is providing being counted as
non-hotplugable, then in this case the alignment shouldn't be needed.

But I digress. Thanks for the insights. I'll ping some people inside IBM and
see if we have a more immediate use case for virtio-mem in Power. Perhaps
we can do some sort of collaboration with your work.



Thanks,


DHB








The QEMU code has an advanced block-size auto-detection code - e.g., querying 
from the kernel but limiting it to sane values (e.g., 512 MB on some arm64 
configurations). Maybe we can borrow some of that or even sense the block size 
via QEMU? Borrowing might be easier. :)


I guess it's a good candidate for a fancy QMP API.



One can at least query the block-size via „qom-get“, but that requires to spin 
up an QEMU instance with a virtio-mem device.




On x86-64 we are good to go with a 2MB default.



- in patch 03 it is mentioned that:

"If it wants to give more memory to the guest it changes 'requested-size' to
a bigger value, and if it wants to shrink guest memory it changes the
'requested-size' to a smaller value. Note, value of zero means that guest
should release all memory offered by the device."

Does size zero implicates the virtio-mem device unplug? Will the device still
exist in the guest even with zeroed memory, acting as a sort of 'deflated
virtio-balloon'?

Yes, the device will still exist, to be grown again later. Hotunplugging the 
device itself is not supported (yet, and also not in the near future).



Assuming that virtio-mem has low overhead in the guest when it's 'deflated',
I don't see any urgency into implementing hotunplug for this device TBH.


There are still things to be optimized in QEMU regarding virtual memory 
consumption, but that‘s more general work to be tackled within the next months. 
After that, not too much speaks against just letting the device stick around to 
provide more nemory later on demand.

Thanks!

Re: [PATCH] spec: Increase meson test timeout 10x

[Jim Fehlig]

On 1/22/21 2:46 AM, Andrea Bolognani wrote:

On Thu, 2021-01-21 at 16:55 -0500, Cole Robinson wrote:

  %check
-VIR_TEST_DEBUG=1 %meson_test --no-suite syntax-check
+# Building on slow archs, like emulated s390x in Fedora copr, requires
+# raising the test timeout
+VIR_TEST_DEBUG=1 %meson_test --no-suite syntax-check --timeout-multiplier 10


Debian has been doing the same thing for a while, even though in that
case it's to cope with slow native hardware rather than emulation:

   
https://salsa.debian.org/libvirt-team/libvirt/-/blob/debian/6.9.0-3/debian/rules#L170-179


FYI, I also use the timeout multiplier in the SUSE libvirt package to avoid test 
failures in OBS, particularly for non-x86 architectures. A value of 5 has worked 
fine thus far.


Regards,
Jim

Re: [PATCH 00/55] Hyper-V: code cleanup & prep for future changes

[Matt Coleman]

> On Jan 22, 2021, at 12:12 PM, Matt Coleman  wrote:
> 
>> On Jan 22, 2021, at 12:07 PM, Matt Coleman  wrote:
>> 
>>> On Jan 22, 2021, at 11:30 AM, Laine Stump  wrote:
>>> 
>>> On 1/22/21 11:05 AM, Laine Stump wrote:
 1) There are several cleanup functions in external libraries that in the 
 past were only called after checking that the pointer was != NULL. 
 g_autoptr cleanups need to handle being called with NULL as a NOP, and I'm 
 concerned that these functions may not behave properly in that case. Can 
 you either verify that it's safe to call them with NULL, or provide a 
 wrapper function that checks for NULL and use that as the cleanup?
>>> 
>>> I asked about item (2) on IRC just now, and danpb produced a short example 
>>> program that proves it is okay to use values from auto-freed objects as the 
>>> return value of a function. So there is only question (1) left. Let me know 
>>> and I'll either push or wait for modified patches accordingly.
>> 
>> For WsXmlDocH, GLib's documentation says that 
>> G_DEFINE_AUTOPTR_CLEANUP_FUNC() is NULL-safe:
>>> The function will not be called if the variable to be cleaned up contains 
>>> NULL.
>> 
>> https://developer.gnome.org/glib/stable/glib-Miscellaneous-Macros.html#G-DEFINE-AUTOPTR-CLEANUP-FUNC:CAPS
>> 
>> For client_opt_t and filter_t, the third parameter to 
>> G_DEFINE_AUTO_CLEANUP_FREE_FUNC() was used to prevent the types' free 
>> functions from being called if they're NULL.
> 
> Oh, woops. Caffeine hasn't kicked in. I got that all backwards.
> 
> There is an issue with client_opt_t, which doesn't have a NULL check:
> https://github.com/Openwsman/openwsman/blob/bcf670dba4d81ec1669cfd1dd7e2b57f0957e534/src/lib/wsman-client.c#L361
> 
> filter_t does have a NULL check:
> https://github.com/Openwsman/openwsman/blob/bcf670dba4d81ec1669cfd1dd7e2b57f0957e534/src/lib/wsman-filter.c#L250
> 
> -- 
> Matt

Evidently, I'm not the sharpest knife in the drawer today: this is 
NULL-safe as it is.

For WsXmlDocH, G_DEFINE_AUTO_CLEANUP_FREE_FUNC's third parameter ensures that 
the cleanup function won't be called if the variable is NULL:
https://developer.gnome.org/glib/stable/glib-Miscellaneous-Macros.html#G-DEFINE-AUTO-CLEANUP-FREE-FUNC:CAPS

On top of that, its cleanup function also includes a NULL check:
https://github.com/Openwsman/openwsman/blob/bcf670dba4d81ec1669cfd1dd7e2b57f0957e534/src/lib/wsman-xml.c#L663

For client_opt_t and filter_t, G_DEFINE_AUTOPTR_CLEANUP_FUNC expands to code 
that contains a NULL check:
https://developer.gnome.org/glib/stable/glib-Miscellaneous-Macros.html#G-DEFINE-AUTOPTR-CLEANUP-FUNC:CAPS

-- 
Matt

Re: [PATCH 00/10] Introduce virtio-mem model

[David Hildenbrand]

> Out of curiosity: are you aware of anyone working in enabling virtio-mem
> for pseries/ppc64? I'm wondering if there's some kind of architecture
> limitation in Power or if it's just a lack of interest.

I remember there is interest, however:

- arm64 and x86-64 is used more frequently in applicable (cloud?) setups, so it 
has high prio
- s390x doesn‘t have any proper memory hot(un)plug, and as I have a strong 
s399x background, it‘s rather easy for me to implement
- ppc64 at least supports hot(un)plug of DIMMs

There is nothing fundamental speaking against ppc64 support AFAIR. A block size 
of 16MB should be possible. I‘m planning on looking into it, however, there are 
a lot of other things on my todo list for virtio-mem.


> 
> 
>> The QEMU code has an advanced block-size auto-detection code - e.g., 
>> querying from the kernel but limiting it to sane values (e.g., 512 MB on 
>> some arm64 configurations). Maybe we can borrow some of that or even sense 
>> the block size via QEMU? Borrowing might be easier. :)
> 
> I guess it's a good candidate for a fancy QMP API.
> 

One can at least query the block-size via „qom-get“, but that requires to spin 
up an QEMU instance with a virtio-mem device.

> 
>> On x86-64 we are good to go with a 2MB default.
>>> 
>>> 
>>> - in patch 03 it is mentioned that:
>>> 
>>> "If it wants to give more memory to the guest it changes 'requested-size' to
>>> a bigger value, and if it wants to shrink guest memory it changes the
>>> 'requested-size' to a smaller value. Note, value of zero means that guest
>>> should release all memory offered by the device."
>>> 
>>> Does size zero implicates the virtio-mem device unplug? Will the device 
>>> still
>>> exist in the guest even with zeroed memory, acting as a sort of 'deflated
>>> virtio-balloon'?
>> Yes, the device will still exist, to be grown again later. Hotunplugging the 
>> device itself is not supported (yet, and also not in the near future).
> 
> 
> Assuming that virtio-mem has low overhead in the guest when it's 'deflated',
> I don't see any urgency into implementing hotunplug for this device TBH.

There are still things to be optimized in QEMU regarding virtual memory 
consumption, but that‘s more general work to be tackled within the next months. 
After that, not too much speaks against just letting the device stick around to 
provide more nemory later on demand.

Thanks!

Re: [PATCH 00/55] Hyper-V: code cleanup & prep for future changes

[Matt Coleman]

> On Jan 22, 2021, at 12:07 PM, Matt Coleman  wrote:
> 
>> On Jan 22, 2021, at 11:30 AM, Laine Stump  wrote:
>> 
>> On 1/22/21 11:05 AM, Laine Stump wrote:
>>> 1) There are several cleanup functions in external libraries that in the 
>>> past were only called after checking that the pointer was != NULL. 
>>> g_autoptr cleanups need to handle being called with NULL as a NOP, and I'm 
>>> concerned that these functions may not behave properly in that case. Can 
>>> you either verify that it's safe to call them with NULL, or provide a 
>>> wrapper function that checks for NULL and use that as the cleanup?
>> 
>> I asked about item (2) on IRC just now, and danpb produced a short example 
>> program that proves it is okay to use values from auto-freed objects as the 
>> return value of a function. So there is only question (1) left. Let me know 
>> and I'll either push or wait for modified patches accordingly.
> 
> For WsXmlDocH, GLib's documentation says that 
> G_DEFINE_AUTOPTR_CLEANUP_FUNC() is NULL-safe:
>> The function will not be called if the variable to be cleaned up contains 
>> NULL.
> 
> https://developer.gnome.org/glib/stable/glib-Miscellaneous-Macros.html#G-DEFINE-AUTOPTR-CLEANUP-FUNC:CAPS
> 
> For client_opt_t and filter_t, the third parameter to 
> G_DEFINE_AUTO_CLEANUP_FREE_FUNC() was used to prevent the types' free 
> functions from being called if they're NULL.

Oh, woops. Caffeine hasn't kicked in. I got that all backwards.

There is an issue with client_opt_t, which doesn't have a NULL check:
https://github.com/Openwsman/openwsman/blob/bcf670dba4d81ec1669cfd1dd7e2b57f0957e534/src/lib/wsman-client.c#L361

filter_t does have a NULL check:
https://github.com/Openwsman/openwsman/blob/bcf670dba4d81ec1669cfd1dd7e2b57f0957e534/src/lib/wsman-filter.c#L250

-- 
Matt

Re: [PATCH 00/55] Hyper-V: code cleanup & prep for future changes

[Matt Coleman]

> On Jan 22, 2021, at 11:30 AM, Laine Stump  wrote:
> 
> On 1/22/21 11:05 AM, Laine Stump wrote:
>> 1) There are several cleanup functions in external libraries that in the 
>> past were only called after checking that the pointer was != NULL. g_autoptr 
>> cleanups need to handle being called with NULL as a NOP, and I'm concerned 
>> that these functions may not behave properly in that case. Can you either 
>> verify that it's safe to call them with NULL, or provide a wrapper function 
>> that checks for NULL and use that as the cleanup?
> 
> I asked about item (2) on IRC just now, and danpb produced a short example 
> program that proves it is okay to use values from auto-freed objects as the 
> return value of a function. So there is only question (1) left. Let me know 
> and I'll either push or wait for modified patches accordingly.

For WsXmlDocH, GLib's documentation says that 
G_DEFINE_AUTOPTR_CLEANUP_FUNC() is NULL-safe:
> The function will not be called if the variable to be cleaned up contains 
> NULL.

https://developer.gnome.org/glib/stable/glib-Miscellaneous-Macros.html#G-DEFINE-AUTOPTR-CLEANUP-FUNC:CAPS

For client_opt_t and filter_t, the third parameter to 
G_DEFINE_AUTO_CLEANUP_FREE_FUNC() was used to prevent the types' free 
functions from being called if they're NULL.

-- 
Matt

Re: [PATCH 00/10] Introduce virtio-mem model

[Daniel Henrique Barboza]

On 1/22/21 4:54 PM, David Hildenbrand wrote:



Am 22.01.2021 um 19:53 schrieb Daniel Henrique Barboza :




On 1/22/21 9:50 AM, Michal Privoznik wrote:
Technically, this is another version of:
https://www.redhat.com/archives/libvir-list/2020-December/msg00199.html
But since virtio-pmem part is pushed now, I've reworked virtio-mem a bit
and sending it as a new series.
For curious ones, David summarized behaviour well when implementing
virtio-mem support in kernel:
https://lwn.net/Articles/755423/
For less curious ones:
   # virsh update-memory $dom --requested-size 4G
adds additional 4GiB of RAM to guest;
   # virsh update-memory $dom --requested-size 0
removes those 4GiB added earlier.
Patches are also available on my GitLab:
https://gitlab.com/MichalPrivoznik/libvirt/-/tree/virtio_mem_v3


Code LGTM:

Reviewed-by: Daniel Henrique Barboza 



Hi,

Let me answer your questions.


Thanks for the reply!






A few questions about the overall design:

- it is mentioned that 'requested-size' should respect the granularity
of the block unit, but later on the 'actual' attribute is added to track
the size that the device was expanded/shrunk. What happens if we forfeit
the granularity check of the memory increments? Will QEMU error out because
we're requesting an invalid value or it will silently size the device to a
plausible size?


QEMU will error out, stating that the request-size has to be properly aligned 
to the block-size.


'requested-size' granularity check stays then :)






- Reading the lwn article I understood that David implemented this support
for s390x as well. If that's the case, then I believe you should double
check later on what's the THP size that Z uses to be sure that it's the
same 2MiB value you're considering in patch 03.


In the near future we might see arm64 and s390x support. The latter might 
probably take a bit longer. Both are not supported yet in QEMU/kernel.


Out of curiosity: are you aware of anyone working in enabling virtio-mem
for pseries/ppc64? I'm wondering if there's some kind of architecture
limitation in Power or if it's just a lack of interest.




The QEMU code has an advanced block-size auto-detection code - e.g., querying 
from the kernel but limiting it to sane values (e.g., 512 MB on some arm64 
configurations). Maybe we can borrow some of that or even sense the block size 
via QEMU? Borrowing might be easier. :)


I guess it's a good candidate for a fancy QMP API.




On x86-64 we are good to go with a 2MB default.




- in patch 03 it is mentioned that:

"If it wants to give more memory to the guest it changes 'requested-size' to
a bigger value, and if it wants to shrink guest memory it changes the
'requested-size' to a smaller value. Note, value of zero means that guest
should release all memory offered by the device."

Does size zero implicates the virtio-mem device unplug? Will the device still
exist in the guest even with zeroed memory, acting as a sort of 'deflated
virtio-balloon'?


Yes, the device will still exist, to be grown again later. Hotunplugging the 
device itself is not supported (yet, and also not in the near future).



Assuming that virtio-mem has low overhead in the guest when it's 'deflated',
I don't see any urgency into implementing hotunplug for this device TBH.



Thanks,


DHB




Thanks!

Re: [PATCH 00/12] Hyper-V serial ports, NICs, and screeshots

[Neal Gompa]

On Fri, Jan 22, 2021 at 3:19 PM Matt Coleman  wrote:
>
> This patchset makes the following changes to the Hyper-V driver:
> * enable XML parsing and creation of serial ports and NICs
> * implement several networking APIs
> * implement screenshots
>
> Matt Coleman (12):
>   hyperv: XML parsing of serial ports
>   hyperv: add support for creating serial devices
>   hyperv: XML parsing of Ethernet adapters
>   hyperv: add support for creating network adapters
>   hyperv: implement connectListAllNetworks and connectNumOfNetworks
>   hyperv: implement networkLookupByName and networkLookupByUUID
>   hyperv: implement connectNumOfDefinedNetworks and
> connectListDefinedNetworks
>   hyperv: implement networkGetAutostart, networkIsActive, and
> networkIsPersistent
>   hyperv: implement networkGetXMLDesc
>   hyperv: implement domainScreenshot
>   hyperv: provide a more detailed error message for WSMan faults
>   news: implement new Hyper-V APIs
>
>  NEWS.rst  |  12 +
>  po/POTFILES.in|   1 +
>  src/hyperv/hyperv_driver.c| 620 ++
>  src/hyperv/hyperv_network_driver.c| 242 ++
>  src/hyperv/hyperv_network_driver.h|  26 ++
>  src/hyperv/hyperv_wmi.c   |  45 +-
>  src/hyperv/hyperv_wmi.h   |  12 +
>  src/hyperv/hyperv_wmi_classes.h   |  36 ++
>  src/hyperv/hyperv_wmi_generator.input | 309 +
>  src/hyperv/meson.build|   1 +
>  10 files changed, 1301 insertions(+), 3 deletions(-)
>  create mode 100644 src/hyperv/hyperv_network_driver.c
>  create mode 100644 src/hyperv/hyperv_network_driver.h
>
> --
> 2.30.0
>
>

The patch set builds and works for me locally, and I don't spot any
obvious issues in the code...

Reviewed-by: Neal Gompa 


-- 
真実はいつも一つ！/ Always, there's only one truth!

Re: [PATCH 00/10] Introduce virtio-mem model

[David Hildenbrand]

> Am 22.01.2021 um 19:53 schrieb Daniel Henrique Barboza 
> :
> 
> 
> 
>> On 1/22/21 9:50 AM, Michal Privoznik wrote:
>> Technically, this is another version of:
>> https://www.redhat.com/archives/libvir-list/2020-December/msg00199.html
>> But since virtio-pmem part is pushed now, I've reworked virtio-mem a bit
>> and sending it as a new series.
>> For curious ones, David summarized behaviour well when implementing
>> virtio-mem support in kernel:
>> https://lwn.net/Articles/755423/
>> For less curious ones:
>>   # virsh update-memory $dom --requested-size 4G
>> adds additional 4GiB of RAM to guest;
>>   # virsh update-memory $dom --requested-size 0
>> removes those 4GiB added earlier.
>> Patches are also available on my GitLab:
>> https://gitlab.com/MichalPrivoznik/libvirt/-/tree/virtio_mem_v3
> 
> Code LGTM:
> 
> Reviewed-by: Daniel Henrique Barboza 
> 

Hi,

Let me answer your questions.


> 
> A few questions about the overall design:
> 
> - it is mentioned that 'requested-size' should respect the granularity
> of the block unit, but later on the 'actual' attribute is added to track
> the size that the device was expanded/shrunk. What happens if we forfeit
> the granularity check of the memory increments? Will QEMU error out because
> we're requesting an invalid value or it will silently size the device to a
> plausible size?

QEMU will error out, stating that the request-size has to be properly aligned 
to the block-size.

> 
> 
> - Reading the lwn article I understood that David implemented this support
> for s390x as well. If that's the case, then I believe you should double
> check later on what's the THP size that Z uses to be sure that it's the
> same 2MiB value you're considering in patch 03.

In the near future we might see arm64 and s390x support. The latter might 
probably take a bit longer. Both are not supported yet in QEMU/kernel.

The QEMU code has an advanced block-size auto-detection code - e.g., querying 
from the kernel but limiting it to sane values (e.g., 512 MB on some arm64 
configurations). Maybe we can borrow some of that or even sense the block size 
via QEMU? Borrowing might be easier. :)

On x86-64 we are good to go with a 2MB default.

> 
> 
> - in patch 03 it is mentioned that:
> 
> "If it wants to give more memory to the guest it changes 'requested-size' to
> a bigger value, and if it wants to shrink guest memory it changes the
> 'requested-size' to a smaller value. Note, value of zero means that guest
> should release all memory offered by the device."
> 
> Does size zero implicates the virtio-mem device unplug? Will the device still
> exist in the guest even with zeroed memory, acting as a sort of 'deflated
> virtio-balloon'?

Yes, the device will still exist, to be grown again later. Hotunplugging the 
device itself is not supported (yet, and also not in the near future).

Thanks!

Re: [PATCH] spec: Increase meson test timeout 10x

[Neal Gompa]

On Thu, Jan 21, 2021 at 4:59 PM Cole Robinson  wrote:
>
> Tests time out when building in slow environments, like emulated
> s390x in Fedora copr. Bump up the test timeout
>
> Signed-off-by: Cole Robinson 
> ---
>  libvirt.spec.in | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/libvirt.spec.in b/libvirt.spec.in
> index 0a8b0ebad4..1731aa8bd9 100644
> --- a/libvirt.spec.in
> +++ b/libvirt.spec.in
> @@ -1288,7 +1288,9 @@ mv 
> $RPM_BUILD_ROOT%{_datadir}/systemtap/tapset/libvirt_qemu_probes.stp \
>  %endif
>
>  %check
> -VIR_TEST_DEBUG=1 %meson_test --no-suite syntax-check
> +# Building on slow archs, like emulated s390x in Fedora copr, requires
> +# raising the test timeout
> +VIR_TEST_DEBUG=1 %meson_test --no-suite syntax-check --timeout-multiplier 10
>
>  %post libs
>  %if 0%{?rhel} == 7
> --
> 2.29.2
>

I have to do this for slow hardware too, so I'm fine with it.

Reviewed-by: Neal Gompa 


-- 
真実はいつも一つ！/ Always, there's only one truth!

Re: [PATCH 00/10] Introduce virtio-mem model

[Daniel Henrique Barboza]

On 1/22/21 9:50 AM, Michal Privoznik wrote:

Technically, this is another version of:

https://www.redhat.com/archives/libvir-list/2020-December/msg00199.html

But since virtio-pmem part is pushed now, I've reworked virtio-mem a bit
and sending it as a new series.

For curious ones, David summarized behaviour well when implementing
virtio-mem support in kernel:

https://lwn.net/Articles/755423/

For less curious ones:

   # virsh update-memory $dom --requested-size 4G

adds additional 4GiB of RAM to guest;

   # virsh update-memory $dom --requested-size 0

removes those 4GiB added earlier.

Patches are also available on my GitLab:

https://gitlab.com/MichalPrivoznik/libvirt/-/tree/virtio_mem_v3


Code LGTM:

Reviewed-by: Daniel Henrique Barboza 


A few questions about the overall design:

- it is mentioned that 'requested-size' should respect the granularity
of the block unit, but later on the 'actual' attribute is added to track
the size that the device was expanded/shrunk. What happens if we forfeit
the granularity check of the memory increments? Will QEMU error out because
we're requesting an invalid value or it will silently size the device to a
plausible size?


- Reading the lwn article I understood that David implemented this support
for s390x as well. If that's the case, then I believe you should double
check later on what's the THP size that Z uses to be sure that it's the
same 2MiB value you're considering in patch 03.


- in patch 03 it is mentioned that:

"If it wants to give more memory to the guest it changes 'requested-size' to
a bigger value, and if it wants to shrink guest memory it changes the
'requested-size' to a smaller value. Note, value of zero means that guest
should release all memory offered by the device."

Does size zero implicates the virtio-mem device unplug? Will the device still
exist in the guest even with zeroed memory, acting as a sort of 'deflated
virtio-balloon'?



Thanks,


DHB






Michal Prívozník (10):
   virhostmem: Introduce virHostMemGetTHPSize()
   qemu_capabilities: Introduce QEMU_CAPS_DEVICE_VIRTIO_MEM_PCI
   conf: Introduce virtio-mem  model
   qemu: Build command line for virtio-mem
   qemu: Wire up  live update
   qemu: Wire up MEMORY_DEVICE_SIZE_CHANGE event
   qemu: Refresh the actual size of virtio-mem on monitor reconnect
   qemu: Recalculate balloon on MEMORY_DEVICE_SIZE_CHANGE event and
 reconnect
   virsh: Introduce update-memory command
   news: document recent virtio memory addition

  NEWS.rst  |   7 +
  docs/formatdomain.rst |  42 +++-
  docs/manpages/virsh.rst   |  31 +++
  docs/schemas/domaincommon.rng |  16 ++
  src/conf/domain_conf.c| 100 -
  src/conf/domain_conf.h|  13 ++
  src/conf/domain_validate.c|  39 
  src/libvirt_private.syms  |   3 +
  src/qemu/qemu_alias.c |  10 +-
  src/qemu/qemu_capabilities.c  |   2 +
  src/qemu/qemu_capabilities.h  |   1 +
  src/qemu/qemu_command.c   |  13 +-
  src/qemu/qemu_domain.c|  50 -
  src/qemu/qemu_domain.h|   1 +
  src/qemu/qemu_domain_address.c|  37 ++-
  src/qemu/qemu_driver.c| 211 +-
  src/qemu/qemu_hotplug.c   |  18 ++
  src/qemu/qemu_hotplug.h   |   5 +
  src/qemu/qemu_monitor.c   |  37 +++
  src/qemu/qemu_monitor.h   |  27 +++
  src/qemu/qemu_monitor_json.c  |  94 ++--
  src/qemu/qemu_monitor_json.h  |   5 +
  src/qemu/qemu_process.c   | 101 -
  src/qemu/qemu_validate.c  |   8 +
  src/security/security_apparmor.c  |   1 +
  src/security/security_dac.c   |   2 +
  src/security/security_selinux.c   |   2 +
  src/util/virhostmem.c |  63 ++
  src/util/virhostmem.h |   3 +
  tests/domaincapsmock.c|   9 +
  .../caps_5.1.0.x86_64.xml |   1 +
  .../caps_5.2.0.x86_64.xml |   1 +
  ...mory-hotplug-virtio-mem.x86_64-latest.args |  49 
  .../memory-hotplug-virtio-mem.xml |  66 ++
  tests/qemuxml2argvtest.c  |   1 +
  ...emory-hotplug-virtio-mem.x86_64-latest.xml |   1 +
  tests/qemuxml2xmltest.c   |   1 +
  tools/virsh-domain.c  | 154 +
  38 files changed, 1165 insertions(+), 60 deletions(-)
  create mode 100644 
tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args
  create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.xml
  create mode 12 

Re: [PATCH 09/10] virsh: Introduce update-memory command

[Daniel Henrique Barboza]

On 1/22/21 9:50 AM, Michal Privoznik wrote:

New 'update-memory' command is introduced which aims on making it
user friendly to change  device. So far I just need to
change  so I'm introducing --requested-size only; but
the idea is that this is extensible for other cases too. For
instance, want to change ? Nnew --my-element
argument can be easily introduced.


I think you meant:

"... want to change ? A new --my-element argument ..."



Reviewed-by: Daniel Henrique Barboza 



Signed-off-by: Michal Privoznik 
---
  docs/manpages/virsh.rst |  31 
  tools/virsh-domain.c| 154 
  2 files changed, 185 insertions(+)

diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst
index e3afa48f7b..32639e34ff 100644
--- a/docs/manpages/virsh.rst
+++ b/docs/manpages/virsh.rst
@@ -4891,6 +4891,37 @@ results as some fields may be autogenerated and thus 
match devices other than
  expected.
  
  
+update-memory

+-
+
+**Syntax:**
+
+::
+
+   update-memory domain [--print-xml] [--alias alias]
+ [[--live] [--config] | [--current]]
+ [--requested-size size]
+
+Update values for a  device. Not to be confused with overall
+domain memory which is tuned via ``setmem`` and ``setmaxmem``.
+This command finds  device inside given *domain*, changes
+requested values and passes updated device XML to daemon. If *--print-xml* is
+specified then the device is not changed, but the updated device XML is printed
+to stdout.  If there are more than one  devices in *domain* use
+*--alias* to select the desired one.
+
+If *--live* is specified, affect a running domain.
+If *--config* is specified, affect the next startup of a persistent guest.
+If *--current* is specified, it is equivalent to either *--live* or
+*--config*, depending on the current state of the guest.
+Both *--live* and *--config* flags may be given, but *--current* is
+exclusive. Not specifying any flag is the same as specifying *--current*.
+
+If *--requested-size* is specified then  under memory target is
+changed to requested *size* (as scaled integer, see ``NOTES`` above). It
+defaults to kibibytes if no suffix is provided.
+
+
  change-media
  
  
diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c

index 9746117bdb..0b32e6f408 100644
--- a/tools/virsh-domain.c
+++ b/tools/virsh-domain.c
@@ -9128,6 +9128,154 @@ cmdSetmaxmem(vshControl *ctl, const vshCmd *cmd)
  return ret;
  }
  
+

+/*
+ * "update-memory" command
+ */
+static const vshCmdInfo info_update_memory[] = {
+{.name = "help",
+ .data = N_("update memory device of a domain")
+},
+{.name = "desc",
+ .data = N_("Update values of a memory device of a domain")
+},
+{.name = NULL}
+};
+
+static const vshCmdOptDef opts_update_memory[] = {
+VIRSH_COMMON_OPT_DOMAIN_FULL(0),
+VIRSH_COMMON_OPT_DOMAIN_CONFIG,
+VIRSH_COMMON_OPT_DOMAIN_LIVE,
+VIRSH_COMMON_OPT_DOMAIN_CURRENT,
+{.name = "print-xml",
+ .type = VSH_OT_BOOL,
+ .help = N_("print updated memory device XML instead of executing the 
change")
+},
+{.name = "alias",
+ .type = VSH_OT_STRING,
+ .completer = virshDomainDeviceAliasCompleter,
+ .help = N_("memory device alias"),
+},
+{.name = "requested-size",
+ .type = VSH_OT_INT,
+ .help = N_("new value of  size, as scaled integer (default 
KiB)")
+},
+{.name = NULL}
+};
+
+static int
+virshGetUpdatedMemoryXML(char **updatedMemoryXML,
+ vshControl *ctl,
+ const vshCmd *cmd,
+ virDomainPtr dom,
+ unsigned int flags)
+{
+const char *alias = NULL;
+g_autoptr(xmlDoc) doc = NULL;
+g_autoptr(xmlXPathContext) ctxt = NULL;
+g_autofree char *xpath = NULL;
+int nmems;
+g_autofree xmlNodePtr *mems = NULL;
+g_autoptr(xmlBuffer) xmlbuf = NULL;
+unsigned int domainXMLFlags = 0;
+
+if (flags & VIR_DOMAIN_AFFECT_CONFIG)
+domainXMLFlags |= VIR_DOMAIN_XML_INACTIVE;
+
+if (virshDomainGetXMLFromDom(ctl, dom, domainXMLFlags, , ) < 0)
+return -1;
+
+if (vshCommandOptStringReq(ctl, cmd, "alias", ) < 0)
+return -1;
+
+if (alias) {
+xpath = g_strdup_printf("/domain/devices/memory[./alias/@name='%s']", 
alias);
+} else {
+xpath = g_strdup("/domain/devices/memory");
+}
+
+nmems = virXPathNodeSet(xpath, ctxt, );
+if (nmems < 0) {
+vshSaveLibvirtError();
+return -1;
+} else if (nmems == 0) {
+vshError(ctl, _("no memory device found"));
+return -1;
+} else if (nmems > 1) {
+vshError(ctl, _("multiple memory devices found, use --alias to select 
one"));
+return -1;
+}
+
+ctxt->node = mems[0];
+
+if (vshCommandOptBool(cmd, "requested-size")) {
+xmlNodePtr requestedSizeNode;
+g_autofree char *kibibytesStr = NULL;
+unsigned long long bytes = 0;
+

Re: [PATCH 05/10] qemu: Wire up live update

[Daniel Henrique Barboza]

On 1/22/21 9:50 AM, Michal Privoznik wrote:

As advertised in one of previous commits, we want' to be able to


Extra " ' " after want.

Reviewed-by: Daniel Henrique Barboza 



change 'requested-size' attribute of virtio-mem on the fly. This
commit does exactly that. Changing anything else is checked for
and forbidden.

Once guest has changed the allocation, QEMU emits an event which
we will use to track the allocation. In the next commit.

Signed-off-by: Michal Privoznik 
---
  src/conf/domain_conf.c   |  23 +
  src/conf/domain_conf.h   |   3 +
  src/libvirt_private.syms |   1 +
  src/qemu/qemu_driver.c   | 175 ++-
  src/qemu/qemu_hotplug.c  |  18 
  src/qemu/qemu_hotplug.h  |   5 +
  src/qemu/qemu_monitor.c  |  13 +++
  src/qemu/qemu_monitor.h  |   4 +
  src/qemu/qemu_monitor_json.c |  15 +++
  src/qemu/qemu_monitor_json.h |   5 +
  10 files changed, 261 insertions(+), 1 deletion(-)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index f8c5a40b24..b6fe5e4436 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -17297,6 +17297,29 @@ virDomainMemoryFindInactiveByDef(virDomainDefPtr def,
  }
  
  
+ssize_t

+virDomainMemoryFindByDeviceInfo(virDomainDefPtr def,
+virDomainDeviceInfoPtr info)
+{
+size_t i;
+
+for (i = 0; i < def->nmems; i++) {
+virDomainMemoryDefPtr tmp = def->mems[i];
+
+if (!virDomainDeviceInfoAddressIsEqual(>info, info))
+continue;
+
+/* alias, if present */
+if (STRNEQ_NULLABLE(tmp->info.alias, info->alias))
+continue;
+
+return i;
+}
+
+return -1;
+}
+
+
  /**
   * virDomainMemoryInsert:
   *
diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h
index 5d89ecfe9d..ef52328a6f 100644
--- a/src/conf/domain_conf.h
+++ b/src/conf/domain_conf.h
@@ -3609,6 +3609,9 @@ int virDomainMemoryFindByDef(virDomainDefPtr def, 
virDomainMemoryDefPtr mem)
  int virDomainMemoryFindInactiveByDef(virDomainDefPtr def,
   virDomainMemoryDefPtr mem)
  ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT;
+ssize_t virDomainMemoryFindByDeviceInfo(virDomainDefPtr dev,
+virDomainDeviceInfoPtr info)
+ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT;
  
  int virDomainShmemDefInsert(virDomainDefPtr def, virDomainShmemDefPtr shmem)

  ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT;
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index 962d82680e..2e7f92bcfe 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -494,6 +494,7 @@ virDomainMemballoonModelTypeFromString;
  virDomainMemballoonModelTypeToString;
  virDomainMemoryDefFree;
  virDomainMemoryFindByDef;
+virDomainMemoryFindByDeviceInfo;
  virDomainMemoryFindInactiveByDef;
  virDomainMemoryInsert;
  virDomainMemoryModelTypeToString;
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index ed966cf7e3..fadf0240fc 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -7087,6 +7087,168 @@ qemuDomainChangeDiskLive(virDomainObjPtr vm,
  return 0;
  }
  
+

+static bool
+qemuDomainChangeMemoryLiveValidateChange(const virDomainMemoryDef *oldDef,
+ const virDomainMemoryDef *newDef)
+{
+/* The only thing that is allowed to change is 'requestedsize' for virtio
+ * model. */
+if (oldDef->model != newDef->model) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("cannot modify memory model from '%s' to '%s'"),
+   virDomainMemoryModelTypeToString(oldDef->model),
+   virDomainMemoryModelTypeToString(newDef->model));
+return false;
+}
+
+if (oldDef->access != newDef->access) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("cannot modify memory access from '%s' to '%s'"),
+   virDomainMemoryAccessTypeToString(oldDef->access),
+   virDomainMemoryAccessTypeToString(newDef->access));
+return false;
+}
+
+if (oldDef->discard != newDef->discard) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("cannot modify memory discard from '%s' to '%s'"),
+   virTristateBoolTypeToString(oldDef->discard),
+   virTristateBoolTypeToString(newDef->discard));
+return false;
+}
+
+if (!virBitmapEqual(oldDef->sourceNodes,
+newDef->sourceNodes)) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
+   _("cannot modify memory source nodes"));
+return false;
+}
+
+if (oldDef->pagesize != newDef->pagesize) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("cannot modify memory pagesize 

Re: [PATCH 03/10] conf: Introduce virtio-mem model

[Daniel Henrique Barboza]

On 1/22/21 9:50 AM, Michal Privoznik wrote:

The virtio-mem is paravirtualized mechanism of adding/removing
memory to/from a VM. A virtio-mem-pci device is split into blocks
of equal size which are then exposed (all or only a requested
portion of them) to the guest kernel to use as regular memory.
Therefore, the device has two important attributes:

   1) block-size, which defines the size of a block
   2) requested-size, which defines how much memory (in bytes)
  is the device requested to expose to the guest.

The 'block-size' is configured on command line and immutable
throughout device's lifetime. The 'requested-size' can be set on
the command line too, but also is adjustable via monitor. In
fact, that is how management software places its requests to
change the memory allocation. If it wants to give more memory to
the guest it changes 'requested-size' to a bigger value, and if it
wants to shrink guest memory it changes the 'requested-size' to a
smaller value. Note, value of zero means that guest should
release all memory offered by the device. Of course, guest has to
cooperate. Therefore, there is a third attribute 'size' which is
read only and reflects how much memory the guest still has. This
can be different to 'requested-size', obviously. Because of name
clash, I've named it 'actualsize' and it is dealt with in future
commits (it is a runtime information anyway).

In the backend, memory for virtio-mem is backed by usual objects:
memory-backend-{ram,file,memfd} and their size puts the cap on
the amount of memory that a virtio-mem device can offer to a
guest. But we are already able to express this info using 
under .

Therefore, we need only two more elements to cover 'block-size'
and 'requested-size' attributes. This is the XML I've came up
with:

   
 
   1-3
   2048
 
 
   2097152
   0
   2048
   1048576
 
 
   

I hope by now it is obvious that:

   1) 'requested-size' must be an integer multiple of
  'block-size', and
   2) virtio-mem-pci device goes onto PCI bus and thus needs PCI
  address.

Then there is a limitation that the minimal 'block-size' is
transparent huge page size (I'll leave this without explanation).

Signed-off-by: Michal Privoznik 
---
  docs/formatdomain.rst | 35 --
  docs/schemas/domaincommon.rng | 11 
  src/conf/domain_conf.c| 53 ++-
  src/conf/domain_conf.h|  3 +
  src/conf/domain_validate.c| 39 +++
  src/qemu/qemu_alias.c |  1 +
  src/qemu/qemu_command.c   |  1 +
  src/qemu/qemu_domain.c| 10 +++
  src/qemu/qemu_domain_address.c| 37 ---
  src/qemu/qemu_validate.c  |  8 +++
  src/security/security_apparmor.c  |  1 +
  src/security/security_dac.c   |  2 +
  src/security/security_selinux.c   |  2 +
  tests/domaincapsmock.c|  9 +++
  .../memory-hotplug-virtio-mem.xml | 66 +++
  ...emory-hotplug-virtio-mem.x86_64-latest.xml |  1 +
  tests/qemuxml2xmltest.c   |  1 +
  17 files changed, 264 insertions(+), 16 deletions(-)
  create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.xml
  create mode 12 
tests/qemuxml2xmloutdata/memory-hotplug-virtio-mem.x86_64-latest.xml

diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst
index af540391db..2938758ec2 100644
--- a/docs/formatdomain.rst
+++ b/docs/formatdomain.rst
@@ -7267,6 +7267,18 @@ Example: usage of the memory devices
   524288
 
   
+ 
+   
+ 1-3
+ 2048
+   
+   
+ 2097152
+ 0
+ 2048
+ 1048576
+   
+ 
 
 ...
  
@@ -7274,7 +7286,8 @@ Example: usage of the memory devices

 Provide ``dimm`` to add a virtual DIMM module to the guest. :since:`Since
 1.2.14` Provide ``nvdimm`` model that adds a Non-Volatile DIMM module.
 :since:`Since 3.2.0` Provide ``virtio-pmem`` model to add a paravirtualized
-   persistent memory device. :since:`Since 7.1.0`
+   persistent memory device. :since:`Since 7.1.0` Provide ``virtio-mem`` model
+   to add paravirtualized memory device. :since: `Since 7.1.0`
  
  ``access``

 An optional attribute ``access`` ( :since:`since 3.2.0` ) that provides
@@ -7297,10 +7310,11 @@ Example: usage of the memory devices
 allowed only for ``model='nvdimm'`` for pSeries guests. :since:`Since 
6.2.0`
  
  ``source``

-   For model ``dimm`` this element is optional and allows to fine tune the
-   source of the memory used for the given memory device. If the element is not
-   provided defaults configured via ``numatune`` are used. If ``dimm`` is
-   provided, then the following optional elements can be provided as well:
+   For model ``dimm`` and model 

Re: [libvirt PATCH] docs: link to PCI docs from the kbase page

[Andrea Bolognani]

On Fri, 2021-01-22 at 17:08 +, Daniel P. Berrangé wrote:
> +`PCI hotplug <../pci-hotplug.html>`__
> +   Effectively usage of PCI hotplug

s/Effectively/Effective/

With that fixed

  Reviewed-by: Andrea Bolognani 

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [libvirt PATCH] docs: use a relative link to the kbase page

[Andrea Bolognani]

On Fri, 2021-01-22 at 17:07 +, Daniel P. Berrangé wrote:
> Signed-off-by: Daniel P. Berrangé 
> ---
>  docs/index.html.in | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Andrea Bolognani 

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [PATCH] virsh: Simplify @flags handing in cmdSetmem() and cmdSetmaxmem()

[Daniel Henrique Barboza]

Michal,


I see that this patch was sent standalone in the ML, and the cover-letter
of the series doesn't mention it. Is this an ooopsie? Should I review this
patch together with the rest of the series?


Thanks,


DHB


On 1/22/21 9:50 AM, Michal Privoznik wrote:

What code tries to achieve is that if no flags were provided to
either 'setmem' or 'setmaxmem' commands then the old (no flags)
API is called to be able to communicate with older daemons.
Well, the code can be simplified a bit.

Note that with this change the old no flag version of APIs is
used more often. Previously if --current argument was given it
resulted in *Flags() version to be called even though it is not
necessary - VIR_DOMAIN_AFFECT_CURRENT is implied.

Therefore, this change in fact allows virsh to talk with broader
set of daemons.

Signed-off-by: Michal Privoznik 
---
  tools/virsh-domain.c | 14 --
  1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c
index 2bb136333f..9746117bdb 100644
--- a/tools/virsh-domain.c
+++ b/tools/virsh-domain.c
@@ -9018,9 +9018,6 @@ cmdSetmem(vshControl *ctl, const vshCmd *cmd)
  flags |= VIR_DOMAIN_AFFECT_CONFIG;
  if (live)
  flags |= VIR_DOMAIN_AFFECT_LIVE;
-/* none of the options were specified */
-if (!current && !live && !config)
-flags = -1;
  
  if (!(dom = virshCommandOptDomain(ctl, cmd, NULL)))

  return false;
@@ -9037,7 +9034,7 @@ cmdSetmem(vshControl *ctl, const vshCmd *cmd)
  }
  kibibytes = VIR_DIV_UP(bytes, 1024);
  
-if (flags == -1) {

+if (flags == 0) {
  if (virDomainSetMemory(dom, kibibytes) != 0)
  ret = false;
  } else {
@@ -9090,7 +9087,7 @@ cmdSetmaxmem(vshControl *ctl, const vshCmd *cmd)
  bool config = vshCommandOptBool(cmd, "config");
  bool live = vshCommandOptBool(cmd, "live");
  bool current = vshCommandOptBool(cmd, "current");
-unsigned int flags = VIR_DOMAIN_AFFECT_CURRENT | VIR_DOMAIN_MEM_MAXIMUM;
+unsigned int flags = VIR_DOMAIN_AFFECT_CURRENT;
  
  VSH_EXCLUSIVE_OPTIONS_VAR(current, live);

  VSH_EXCLUSIVE_OPTIONS_VAR(current, config);
@@ -9099,9 +9096,6 @@ cmdSetmaxmem(vshControl *ctl, const vshCmd *cmd)
  flags |= VIR_DOMAIN_AFFECT_CONFIG;
  if (live)
  flags |= VIR_DOMAIN_AFFECT_LIVE;
-/* none of the options were specified */
-if (!current && !live && !config)
-flags = -1;
  
  if (!(dom = virshCommandOptDomain(ctl, cmd, NULL)))

  return false;
@@ -9118,13 +9112,13 @@ cmdSetmaxmem(vshControl *ctl, const vshCmd *cmd)
  }
  kibibytes = VIR_DIV_UP(bytes, 1024);
  
-if (flags == -1) {

+if (flags == 0) {
  if (virDomainSetMaxMemory(dom, kibibytes) != 0) {
  vshError(ctl, "%s", _("Unable to change MaxMemorySize"));
  ret = false;
  }
  } else {
-if (virDomainSetMemoryFlags(dom, kibibytes, flags) < 0) {
+if (virDomainSetMemoryFlags(dom, kibibytes, flags | 
VIR_DOMAIN_MEM_MAXIMUM) < 0) {
  vshError(ctl, "%s", _("Unable to change MaxMemorySize"));
  ret = false;
  }

Re: [PATCH] virNetDevOpenvswitchGetVhostuserIfname: Remove a single '\n' from ifname

[Daniel Henrique Barboza]

On 1/22/21 7:26 AM, Yalei Li wrote:

Ovs-vsctl returns a newline result.
Signed-off-by: Yalei Li 
---


Reviewed-by: Daniel Henrique Barboza 


  src/util/virnetdevopenvswitch.c | 1 +
  1 file changed, 1 insertion(+)

diff --git a/src/util/virnetdevopenvswitch.c b/src/util/virnetdevopenvswitch.c
index f9b3369b2a..bd840bd3b7 100644
--- a/src/util/virnetdevopenvswitch.c
+++ b/src/util/virnetdevopenvswitch.c
@@ -575,6 +575,7 @@ virNetDevOpenvswitchGetVhostuserIfname(const char *path,
  return 0;
  }
  
+virStringTrimOptionalNewline(*ifname);

  if (virNetDevOpenvswitchMaybeUnescapeReply(*ifname) < 0) {
  VIR_FREE(*ifname);
  return -1;

Re: [libvirt PATCH] qemu: fix release of virDomainObjPtr in SSH key APIs

[Daniel Henrique Barboza]

On 1/22/21 2:08 PM, Daniel P. Berrangé wrote:

The qemuDomainObjFromDomain() API must be paired with
the virDomainObjEndAPI API. The qemuDomainAuthorizedSSHKeysGet
method simply did 'return -1' leaking a reference in two paths.

The qemuDomainAuthorizedSSHKeysSet method marked the object
as an autoptr while also have some code paths that will call
virDomainObjEndAPI, resulting in attempted double free.

Signed-off-by: Daniel P. Berrangé 
---


Reviewed-by: Daniel Henrique Barboza 


  src/qemu/qemu_driver.c | 12 +++-
  1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 027617deef..05e021cce4 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -20254,10 +20254,10 @@ qemuDomainAuthorizedSSHKeysGet(virDomainPtr dom,
  return -1;
  
  if (virDomainAuthorizedSshKeysGetEnsureACL(dom->conn, vm->def) < 0)

-return -1;
+goto cleanup;
  
  if (qemuDomainObjBeginAgentJob(driver, vm, QEMU_AGENT_JOB_QUERY) < 0)

-return -1;
+goto cleanup;
  
  if (!qemuDomainAgentAvailable(vm, true))

  goto endagentjob;
@@ -20268,6 +20268,7 @@ qemuDomainAuthorizedSSHKeysGet(virDomainPtr dom,
  
   endagentjob:

  qemuDomainObjEndAgentJob(vm);
+ cleanup:
  virDomainObjEndAPI();
  return rv;
  }
@@ -20281,7 +20282,7 @@ qemuDomainAuthorizedSSHKeysSet(virDomainPtr dom,
 unsigned int flags)
  {
  virQEMUDriverPtr driver = dom->conn->privateData;
-g_autoptr(virDomainObj) vm = NULL;
+virDomainObjPtr vm = NULL;
  qemuAgentPtr agent;
  const bool append = flags & VIR_DOMAIN_AUTHORIZED_SSH_KEYS_SET_APPEND;
  const bool remove = flags & VIR_DOMAIN_AUTHORIZED_SSH_KEYS_SET_REMOVE;
@@ -20294,10 +20295,10 @@ qemuDomainAuthorizedSSHKeysSet(virDomainPtr dom,
  return -1;
  
  if (virDomainAuthorizedSshKeysSetEnsureACL(dom->conn, vm->def) < 0)

-return -1;
+goto cleanup;
  
  if (qemuDomainObjBeginAgentJob(driver, vm, QEMU_AGENT_JOB_QUERY) < 0)

-return -1;
+goto cleanup;
  
  if (!qemuDomainAgentAvailable(vm, true))

  goto endagentjob;
@@ -20311,6 +20312,7 @@ qemuDomainAuthorizedSSHKeysSet(virDomainPtr dom,
  
   endagentjob:

  qemuDomainObjEndAgentJob(vm);
+ cleanup:
  virDomainObjEndAPI();
  return rv;
  }

[libvirt PATCH 16/16] tools: report tainting for 'dominfo' command

[Daniel P . Berrangé]

$ virsh dominfo demo
Id: 2
Name:   demo
UUID:   eadf8ef0-bf14-4c5f-9708-4a19bacf9e81
OS Type:hvm
State:  running
CPU(s): 2
CPU time:   15.8s
Max memory: 1536000 KiB
Used memory:1536000 KiB
Persistent: yes
Autostart:  disable
Managed save:   no
Security model: selinux
Security DOI:   0
Security label: unconfined_u:unconfined_r:svirt_t:s0:c443,c956 (permissive)
Tainting:   custom-monitor
deprecated-config
Deprecations:   machine type 'pc-1.3'
CPU model 'Icelake-Client'

Signed-off-by: Daniel P. Berrangé 
---
 tools/virsh-domain-monitor.c | 12 
 1 file changed, 12 insertions(+)

diff --git a/tools/virsh-domain-monitor.c b/tools/virsh-domain-monitor.c
index 6b39b60339..ca85851810 100644
--- a/tools/virsh-domain-monitor.c
+++ b/tools/virsh-domain-monitor.c
@@ -1292,6 +1292,7 @@ cmdDominfo(vshControl *ctl, const vshCmd *cmd)
 int has_managed_save = 0;
 virshControlPtr priv = ctl->privData;
 char **deprecations = NULL;
+char **tainting = NULL;
 
 if (!(dom = virshCommandOptDomain(ctl, cmd, NULL)))
 return false;
@@ -1393,6 +1394,17 @@ cmdDominfo(vshControl *ctl, const vshCmd *cmd)
 }
 }
 
+if (virDomainGetTainting(dom, , 0) > 0) {
+size_t i;
+for (i = 0; tainting[i] != NULL; i++) {
+if (i == 0) {
+vshPrint(ctl, "%-15s %s\n", _("Tainting:"), tainting[i]);
+} else {
+vshPrint(ctl, "%-15s %s\n", "", tainting[i]);
+}
+}
+}
+
 if (virDomainGetDeprecations(dom, , 0) > 0) {
 size_t i;
 for (i = 0; deprecations[i] != NULL; i++) {
-- 
2.29.2

[libvirt PATCH 15/16] qemu: implement virDomainGetTainting API

[Daniel P . Berrangé]

Signed-off-by: Daniel P. Berrangé 
---
 src/qemu/qemu_driver.c | 35 +++
 1 file changed, 35 insertions(+)

diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 43cdb53f22..48d2dafb2f 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -20346,6 +20346,40 @@ qemuDomainGetDeprecations(virDomainPtr dom,
 }
 
 
+static int
+qemuDomainGetTainting(virDomainPtr dom,
+  char ***codes,
+  unsigned int flags)
+{
+virDomainObjPtr vm = NULL;
+int rv = -1;
+size_t i, n;
+size_t ntaint;
+
+virCheckFlags(0, -1);
+
+if (!(vm = qemuDomainObjFromDomain(dom)))
+return -1;
+
+if (virDomainGetTaintingEnsureACL(dom->conn, vm->def) < 0)
+goto cleanup;
+
+ntaint = __builtin_popcount(vm->taint);
+*codes = g_new0(char *, ntaint + 1);
+for (i = 0, n = 0; i < VIR_DOMAIN_TAINT_LAST && n < ntaint; i++) {
+if (vm->taint & (1 << i)) {
+(*codes)[n++] = g_strdup(virDomainTaintTypeToString(i));
+}
+}
+(*codes)[n] = NULL;
+rv = ntaint;
+
+ cleanup:
+virDomainObjEndAPI();
+return rv;
+}
+
+
 static virHypervisorDriver qemuHypervisorDriver = {
 .name = QEMU_DRIVER_NAME,
 .connectURIProbe = qemuConnectURIProbe,
@@ -20588,6 +20622,7 @@ static virHypervisorDriver qemuHypervisorDriver = {
 .domainAuthorizedSSHKeysGet = qemuDomainAuthorizedSSHKeysGet, /* 6.10.0 */
 .domainAuthorizedSSHKeysSet = qemuDomainAuthorizedSSHKeysSet, /* 6.10.0 */
 .domainGetDeprecations = qemuDomainGetDeprecations, /* 7.1.0 */
+.domainGetTainting = qemuDomainGetTainting, /* 7.1.0 */
 };
 
 
-- 
2.29.2

[libvirt PATCH 14/16] remote: add RPC support for the virDomainGetTainting API

[Daniel P . Berrangé]

Signed-off-by: Daniel P. Berrangé 
---
 src/remote/remote_daemon_dispatch.c | 46 +
 src/remote/remote_driver.c  | 44 +++
 src/remote/remote_protocol.x| 20 -
 3 files changed, 109 insertions(+), 1 deletion(-)

diff --git a/src/remote/remote_daemon_dispatch.c 
b/src/remote/remote_daemon_dispatch.c
index aa6a088222..f2249dfa0f 100644
--- a/src/remote/remote_daemon_dispatch.c
+++ b/src/remote/remote_daemon_dispatch.c
@@ -7508,3 +7508,49 @@ remoteDispatchDomainGetDeprecations(virNetServerPtr 
server G_GNUC_UNUSED,
 
 return rv;
 }
+
+
+static int
+remoteDispatchDomainGetTainting(virNetServerPtr server G_GNUC_UNUSED,
+virNetServerClientPtr client,
+virNetMessagePtr msg G_GNUC_UNUSED,
+virNetMessageErrorPtr rerr,
+remote_domain_get_tainting_args *args,
+remote_domain_get_tainting_ret *ret)
+{
+int rv = -1;
+virConnectPtr conn = remoteGetHypervisorConn(client);
+int ncodes = 0;
+char **codes = NULL;
+virDomainPtr dom = NULL;
+
+if (!conn)
+goto cleanup;
+
+if (!(dom = get_nonnull_domain(conn, args->dom)))
+goto cleanup;
+
+if ((ncodes = virDomainGetTainting(dom, , args->flags)) < 0)
+goto cleanup;
+
+if (ncodes > REMOTE_DOMAIN_TAINTING_MAX) {
+virReportError(VIR_ERR_INTERNAL_ERROR,
+   _("Number of codes %d, which exceeds max limit: %d"),
+   ncodes, REMOTE_DOMAIN_TAINTING_MAX);
+goto cleanup;
+}
+
+ret->codes.codes_val = g_steal_pointer();
+ret->codes.codes_len = ncodes;
+
+rv = ncodes;
+
+ cleanup:
+if (rv < 0)
+virNetMessageSaveError(rerr);
+if (ncodes > 0)
+virStringListFreeCount(codes, ncodes);
+virObjectUnref(dom);
+
+return rv;
+}
diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c
index 009df01a54..c3cdb751f6 100644
--- a/src/remote/remote_driver.c
+++ b/src/remote/remote_driver.c
@@ -8141,6 +8141,49 @@ remoteDomainGetDeprecations(virDomainPtr domain,
 return rv;
 }
 
+static int
+remoteDomainGetTainting(virDomainPtr domain,
+char ***codes,
+unsigned int flags)
+{
+int rv = -1;
+size_t i;
+struct private_data *priv = domain->conn->privateData;
+remote_domain_get_tainting_args args;
+remote_domain_get_tainting_ret ret;
+
+remoteDriverLock(priv);
+
+make_nonnull_domain(, domain);
+args.flags = flags;
+memset(, 0, sizeof(ret));
+
+if (call(domain->conn, priv, 0, REMOTE_PROC_DOMAIN_GET_TAINTING,
+ (xdrproc_t) xdr_remote_domain_get_tainting_args, (char *),
+ (xdrproc_t) xdr_remote_domain_get_tainting_ret, (char *)) == 
-1) {
+goto cleanup;
+}
+
+if (ret.codes.codes_len > REMOTE_DOMAIN_TAINTING_MAX) {
+virReportError(VIR_ERR_RPC, "%s",
+   _("remoteDomainGetTainting: "
+ "returned number of codes exceeds limit"));
+goto cleanup;
+}
+
+*codes = g_new0(char *, ret.codes.codes_len + 1);
+for (i = 0; i < ret.codes.codes_len; i++)
+(*codes)[i] = g_strdup(ret.codes.codes_val[i]);
+
+rv = ret.codes.codes_len;
+
+ cleanup:
+remoteDriverUnlock(priv);
+xdr_free((xdrproc_t)xdr_remote_domain_get_tainting_ret,
+ (char *) );
+return rv;
+}
+
 /* get_nonnull_domain and get_nonnull_network turn an on-wire
  * (name, uuid) pair into virDomainPtr or virNetworkPtr object.
  * These can return NULL if underlying memory allocations fail,
@@ -8575,6 +8618,7 @@ static virHypervisorDriver hypervisor_driver = {
 .domainAuthorizedSSHKeysGet = remoteDomainAuthorizedSSHKeysGet, /* 6.10.0 
*/
 .domainAuthorizedSSHKeysSet = remoteDomainAuthorizedSSHKeysSet, /* 6.10.0 
*/
 .domainGetDeprecations = remoteDomainGetDeprecations, /* 7.1.0 */
+.domainGetTainting = remoteDomainGetTainting, /* 7.1.0 */
 };
 
 static virNetworkDriver network_driver = {
diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x
index 4ddfb09631..bcb15b251f 100644
--- a/src/remote/remote_protocol.x
+++ b/src/remote/remote_protocol.x
@@ -286,6 +286,9 @@ const REMOTE_DOMAIN_AUTHORIZED_SSH_KEYS_MAX = 2048;
 /* Upper limit on number of deprecation messages */
 const REMOTE_DOMAIN_DEPRECATIONS_MAX = 2048;
 
+/* Upper limit on number of tainting codes */
+const REMOTE_DOMAIN_TAINTING_MAX = 2048;
+
 
 /* UUID.  VIR_UUID_BUFLEN definition comes from libvirt.h */
 typedef opaque remote_uuid[VIR_UUID_BUFLEN];
@@ -3811,6 +3814,15 @@ struct remote_domain_get_deprecations_ret {
 remote_nonnull_string msgs;
 };
 
+struct remote_domain_get_tainting_args {
+remote_nonnull_domain dom;
+unsigned int flags;
+};
+
+struct remote_domain_get_tainting_ret {
+remote_nonnull_string codes;
+};
+
 

[libvirt PATCH 13/16] src: define virDomainGetTainting API

[Daniel P . Berrangé]

This API allows fetching a list of tainting codes against the domain.

Internally tainting is defined as a bitmask with enum fields. Most,
but not all, of the fields are somewhat QEMU specific in reality
though. The idea of exposing the enum in the public API thus feels
a little questionable. This API takes the approach of exposing the
the string format of the enum codes and declaring the string codes
are (potentially) hypervisor specific.

IOW, callers of the API can display the codes but should treat them
as a opaque strings.

Signed-off-by: Daniel P. Berrangé 
---
 include/libvirt/libvirt-domain.h |  3 ++
 src/driver-hypervisor.h  |  5 
 src/libvirt-domain.c | 48 
 src/libvirt_public.syms  |  1 +
 4 files changed, 57 insertions(+)

diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h
index cef16e6361..d04140bf60 100644
--- a/include/libvirt/libvirt-domain.h
+++ b/include/libvirt/libvirt-domain.h
@@ -5122,5 +5122,8 @@ int virDomainAuthorizedSSHKeysSet(virDomainPtr domain,
 int virDomainGetDeprecations(virDomainPtr domain,
  char ***msgs,
  unsigned int flags);
+int virDomainGetTainting(virDomainPtr domain,
+ char ***codes,
+ unsigned int flags);
 
 #endif /* LIBVIRT_DOMAIN_H */
diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h
index 2f804dba1f..96111cd10a 100644
--- a/src/driver-hypervisor.h
+++ b/src/driver-hypervisor.h
@@ -1404,6 +1404,10 @@ typedef int
 (*virDrvDomainGetDeprecations)(virDomainPtr domain,
char ***msgs,
unsigned int flags);
+typedef int
+(*virDrvDomainGetTainting)(virDomainPtr domain,
+   char ***codes,
+   unsigned int flags);
 
 typedef struct _virHypervisorDriver virHypervisorDriver;
 typedef virHypervisorDriver *virHypervisorDriverPtr;
@@ -1671,4 +1675,5 @@ struct _virHypervisorDriver {
 virDrvDomainAuthorizedSSHKeysGet domainAuthorizedSSHKeysGet;
 virDrvDomainAuthorizedSSHKeysSet domainAuthorizedSSHKeysSet;
 virDrvDomainGetDeprecations domainGetDeprecations;
+virDrvDomainGetTainting domainGetTainting;
 };
diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c
index 9e3b118483..c053f35d3e 100644
--- a/src/libvirt-domain.c
+++ b/src/libvirt-domain.c
@@ -13149,3 +13149,51 @@ virDomainGetDeprecations(virDomainPtr domain,
 virDispatchError(conn);
 return -1;
 }
+
+
+/**
+ * virDomainGetTainting:
+ * @domain: a domain object
+ * @codes: pointer to a variable to store tainting codes
+ * @flags: extra flags; not used yet, so callers should always pass 0
+ *
+ * Fetch a list of all tainting codes for the VM and
+ * store them into @msgs array which is allocated upon
+ * successful return and is NULL terminated. The caller is
+ * responsible for freeing @msgs when no longer needed.
+ *
+ * Note that some hypervisors may only report tainting
+ * codes while the VM is in a running state. The list
+ * tainting codes is also hypervisor specific.
+ *
+ * Returns: number of messages stored in @msgs,
+ *  -1 otherwise.
+ */
+int
+virDomainGetTainting(virDomainPtr domain,
+ char ***codes,
+ unsigned int flags)
+{
+virConnectPtr conn;
+
+VIR_DOMAIN_DEBUG(domain, "codes=%p, flags=0x%x", codes, flags);
+
+virResetLastError();
+
+virCheckDomainReturn(domain, -1);
+conn = domain->conn;
+virCheckNonNullArgGoto(codes, error);
+
+if (conn->driver->domainGetTainting) {
+int ret;
+ret = conn->driver->domainGetTainting(domain, codes, flags);
+if (ret < 0)
+goto error;
+return ret;
+}
+
+virReportUnsupportedError();
+ error:
+virDispatchError(conn);
+return -1;
+}
diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms
index 345c5685bf..246fbaf1dc 100644
--- a/src/libvirt_public.syms
+++ b/src/libvirt_public.syms
@@ -882,6 +882,7 @@ LIBVIRT_6.10.0 {
 LIBVIRT_7.1.0 {
 global:
 virDomainGetDeprecations;
+virDomainGetTainting;
 } LIBVIRT_6.10.0;
 
 #  define new API here using predicted next version number 
-- 
2.29.2

[libvirt PATCH 12/16] tools: report deprecations for 'dominfo' command

[Daniel P . Berrangé]

$ virsh dominfo demo
Id: 2
Name:   demo
UUID:   eadf8ef0-bf14-4c5f-9708-4a19bacf9e81
OS Type:hvm
State:  running
CPU(s): 2
CPU time:   15.8s
Max memory: 1536000 KiB
Used memory:1536000 KiB
Persistent: yes
Autostart:  disable
Managed save:   no
Security model: selinux
Security DOI:   0
Security label: unconfined_u:unconfined_r:svirt_t:s0:c443,c956 (permissive)
Deprecations:   machine type 'pc-1.3'
CPU model 'Icelake-Client'

Signed-off-by: Daniel P. Berrangé 
---
 tools/virsh-domain-monitor.c | 13 +
 1 file changed, 13 insertions(+)

diff --git a/tools/virsh-domain-monitor.c b/tools/virsh-domain-monitor.c
index 5d0a03afa9..6b39b60339 100644
--- a/tools/virsh-domain-monitor.c
+++ b/tools/virsh-domain-monitor.c
@@ -1291,6 +1291,7 @@ cmdDominfo(vshControl *ctl, const vshCmd *cmd)
 char *str, uuid[VIR_UUID_STRING_BUFLEN];
 int has_managed_save = 0;
 virshControlPtr priv = ctl->privData;
+char **deprecations = NULL;
 
 if (!(dom = virshCommandOptDomain(ctl, cmd, NULL)))
 return false;
@@ -1391,6 +1392,18 @@ cmdDominfo(vshControl *ctl, const vshCmd *cmd)
 VIR_FREE(seclabel);
 }
 }
+
+if (virDomainGetDeprecations(dom, , 0) > 0) {
+size_t i;
+for (i = 0; deprecations[i] != NULL; i++) {
+if (i == 0) {
+vshPrint(ctl, "%-15s %s\n", _("Deprecations:"), 
deprecations[i]);
+} else {
+vshPrint(ctl, "%-15s %s\n", "", deprecations[i]);
+}
+}
+}
+
 virshDomainFree(dom);
 return ret;
 }
-- 
2.29.2

[libvirt PATCH 11/16] qemu: implement virDomainGetDeprecations API

[Daniel P . Berrangé]

Signed-off-by: Daniel P. Berrangé 
---
 src/qemu/qemu_driver.c | 31 +++
 1 file changed, 31 insertions(+)

diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 027617deef..43cdb53f22 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -20316,6 +20316,36 @@ qemuDomainAuthorizedSSHKeysSet(virDomainPtr dom,
 }
 
 
+static int
+qemuDomainGetDeprecations(virDomainPtr dom,
+  char ***msgs,
+  unsigned int flags)
+{
+virDomainObjPtr vm = NULL;
+int rv = -1;
+size_t i;
+
+virCheckFlags(0, -1);
+
+if (!(vm = qemuDomainObjFromDomain(dom)))
+return -1;
+
+if (virDomainGetDeprecationsEnsureACL(dom->conn, vm->def) < 0)
+goto cleanup;
+
+*msgs = g_new0(char *, vm->ndeprecations + 1);
+for (i = 0; i < vm->ndeprecations; i++) {
+(*msgs)[i] = g_strdup(vm->deprecations[i]);
+}
+(*msgs)[vm->ndeprecations] = NULL;
+rv = vm->ndeprecations;
+
+ cleanup:
+virDomainObjEndAPI();
+return rv;
+}
+
+
 static virHypervisorDriver qemuHypervisorDriver = {
 .name = QEMU_DRIVER_NAME,
 .connectURIProbe = qemuConnectURIProbe,
@@ -20557,6 +20587,7 @@ static virHypervisorDriver qemuHypervisorDriver = {
 .domainBackupGetXMLDesc = qemuDomainBackupGetXMLDesc, /* 6.0.0 */
 .domainAuthorizedSSHKeysGet = qemuDomainAuthorizedSSHKeysGet, /* 6.10.0 */
 .domainAuthorizedSSHKeysSet = qemuDomainAuthorizedSSHKeysSet, /* 6.10.0 */
+.domainGetDeprecations = qemuDomainGetDeprecations, /* 7.1.0 */
 };
 
 
-- 
2.29.2

[libvirt PATCH 10/16] remote: add RPC support for the virDomainGetDeprecations API

[Daniel P . Berrangé]

Signed-off-by: Daniel P. Berrangé 
---
 src/remote/remote_daemon_dispatch.c | 45 +
 src/remote/remote_driver.c  | 44 
 src/remote/remote_protocol.x| 21 +-
 3 files changed, 109 insertions(+), 1 deletion(-)

diff --git a/src/remote/remote_daemon_dispatch.c 
b/src/remote/remote_daemon_dispatch.c
index 46683aa4a7..aa6a088222 100644
--- a/src/remote/remote_daemon_dispatch.c
+++ b/src/remote/remote_daemon_dispatch.c
@@ -7463,3 +7463,48 @@ remoteDispatchDomainAuthorizedSshKeysSet(virNetServerPtr 
server G_GNUC_UNUSED,
 
 return rv;
 }
+
+static int
+remoteDispatchDomainGetDeprecations(virNetServerPtr server G_GNUC_UNUSED,
+virNetServerClientPtr client,
+virNetMessagePtr msg G_GNUC_UNUSED,
+virNetMessageErrorPtr rerr,
+remote_domain_get_deprecations_args *args,
+remote_domain_get_deprecations_ret *ret)
+{
+int rv = -1;
+virConnectPtr conn = remoteGetHypervisorConn(client);
+int nmsgs = 0;
+char **msgs = NULL;
+virDomainPtr dom = NULL;
+
+if (!conn)
+goto cleanup;
+
+if (!(dom = get_nonnull_domain(conn, args->dom)))
+goto cleanup;
+
+if ((nmsgs = virDomainGetDeprecations(dom, , args->flags)) < 0)
+goto cleanup;
+
+if (nmsgs > REMOTE_DOMAIN_DEPRECATIONS_MAX) {
+virReportError(VIR_ERR_INTERNAL_ERROR,
+   _("Number of msgs %d, which exceeds max limit: %d"),
+   nmsgs, REMOTE_DOMAIN_DEPRECATIONS_MAX);
+goto cleanup;
+}
+
+ret->msgs.msgs_val = g_steal_pointer();
+ret->msgs.msgs_len = nmsgs;
+
+rv = nmsgs;
+
+ cleanup:
+if (rv < 0)
+virNetMessageSaveError(rerr);
+if (nmsgs > 0)
+virStringListFreeCount(msgs, nmsgs);
+virObjectUnref(dom);
+
+return rv;
+}
diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c
index 1b784e61c7..009df01a54 100644
--- a/src/remote/remote_driver.c
+++ b/src/remote/remote_driver.c
@@ -8098,6 +8098,49 @@ remoteDomainAuthorizedSSHKeysSet(virDomainPtr domain,
 }
 
 
+static int
+remoteDomainGetDeprecations(virDomainPtr domain,
+char ***msgs,
+unsigned int flags)
+{
+int rv = -1;
+size_t i;
+struct private_data *priv = domain->conn->privateData;
+remote_domain_get_deprecations_args args;
+remote_domain_get_deprecations_ret ret;
+
+remoteDriverLock(priv);
+
+make_nonnull_domain(, domain);
+args.flags = flags;
+memset(, 0, sizeof(ret));
+
+if (call(domain->conn, priv, 0, REMOTE_PROC_DOMAIN_GET_DEPRECATIONS,
+ (xdrproc_t) xdr_remote_domain_get_deprecations_args, (char 
*),
+ (xdrproc_t) xdr_remote_domain_get_deprecations_ret, (char *)) 
== -1) {
+goto cleanup;
+}
+
+if (ret.msgs.msgs_len > REMOTE_DOMAIN_DEPRECATIONS_MAX) {
+virReportError(VIR_ERR_RPC, "%s",
+   _("remoteDomainGetDeprecations: "
+ "returned number of msgs exceeds limit"));
+goto cleanup;
+}
+
+*msgs = g_new0(char *, ret.msgs.msgs_len + 1);
+for (i = 0; i < ret.msgs.msgs_len; i++)
+(*msgs)[i] = g_strdup(ret.msgs.msgs_val[i]);
+
+rv = ret.msgs.msgs_len;
+
+ cleanup:
+remoteDriverUnlock(priv);
+xdr_free((xdrproc_t)xdr_remote_domain_get_deprecations_ret,
+ (char *) );
+return rv;
+}
+
 /* get_nonnull_domain and get_nonnull_network turn an on-wire
  * (name, uuid) pair into virDomainPtr or virNetworkPtr object.
  * These can return NULL if underlying memory allocations fail,
@@ -8531,6 +8574,7 @@ static virHypervisorDriver hypervisor_driver = {
 .domainBackupGetXMLDesc = remoteDomainBackupGetXMLDesc, /* 6.0.0 */
 .domainAuthorizedSSHKeysGet = remoteDomainAuthorizedSSHKeysGet, /* 6.10.0 
*/
 .domainAuthorizedSSHKeysSet = remoteDomainAuthorizedSSHKeysSet, /* 6.10.0 
*/
+.domainGetDeprecations = remoteDomainGetDeprecations, /* 7.1.0 */
 };
 
 static virNetworkDriver network_driver = {
diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x
index 2df38cef77..4ddfb09631 100644
--- a/src/remote/remote_protocol.x
+++ b/src/remote/remote_protocol.x
@@ -283,6 +283,9 @@ const REMOTE_NETWORK_PORT_PARAMETERS_MAX = 16;
 /* Upper limit on number of SSH keys */
 const REMOTE_DOMAIN_AUTHORIZED_SSH_KEYS_MAX = 2048;
 
+/* Upper limit on number of deprecation messages */
+const REMOTE_DOMAIN_DEPRECATIONS_MAX = 2048;
+
 
 /* UUID.  VIR_UUID_BUFLEN definition comes from libvirt.h */
 typedef opaque remote_uuid[VIR_UUID_BUFLEN];
@@ -3799,6 +3802,16 @@ struct remote_domain_authorized_ssh_keys_set_args {
 unsigned int flags;
 };
 
+struct remote_domain_get_deprecations_args {
+remote_nonnull_domain dom;
+unsigned int flags;
+};
+
+struct 

[libvirt PATCH 09/16] src: define virDomainGetDeprecations API

[Daniel P . Berrangé]

This API allows fetching a list of deprecation messages against the domain.

Signed-off-by: Daniel P. Berrangé 
---
 include/libvirt/libvirt-domain.h |  4 +++
 src/driver-hypervisor.h  |  6 
 src/libvirt-domain.c | 47 
 src/libvirt_public.syms  |  5 
 4 files changed, 62 insertions(+)

diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h
index de2456812c..cef16e6361 100644
--- a/include/libvirt/libvirt-domain.h
+++ b/include/libvirt/libvirt-domain.h
@@ -5119,4 +5119,8 @@ int virDomainAuthorizedSSHKeysSet(virDomainPtr domain,
   unsigned int nkeys,
   unsigned int flags);
 
+int virDomainGetDeprecations(virDomainPtr domain,
+ char ***msgs,
+ unsigned int flags);
+
 #endif /* LIBVIRT_DOMAIN_H */
diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h
index 9e8fe89921..2f804dba1f 100644
--- a/src/driver-hypervisor.h
+++ b/src/driver-hypervisor.h
@@ -1400,6 +1400,11 @@ typedef int
 unsigned int nkeys,
 unsigned int flags);
 
+typedef int
+(*virDrvDomainGetDeprecations)(virDomainPtr domain,
+   char ***msgs,
+   unsigned int flags);
+
 typedef struct _virHypervisorDriver virHypervisorDriver;
 typedef virHypervisorDriver *virHypervisorDriverPtr;
 
@@ -1665,4 +1670,5 @@ struct _virHypervisorDriver {
 virDrvDomainBackupGetXMLDesc domainBackupGetXMLDesc;
 virDrvDomainAuthorizedSSHKeysGet domainAuthorizedSSHKeysGet;
 virDrvDomainAuthorizedSSHKeysSet domainAuthorizedSSHKeysSet;
+virDrvDomainGetDeprecations domainGetDeprecations;
 };
diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c
index c9f8ffdb56..9e3b118483 100644
--- a/src/libvirt-domain.c
+++ b/src/libvirt-domain.c
@@ -13102,3 +13102,50 @@ virDomainAuthorizedSSHKeysSet(virDomainPtr domain,
 virDispatchError(conn);
 return -1;
 }
+
+
+/**
+ * virDomainGetDeprecations:
+ * @domain: a domain object
+ * @msgs: pointer to a variable to store deprecation messages
+ * @flags: extra flags; not used yet, so callers should always pass 0
+ *
+ * Fetch a list of all deprecation messages for the VM and
+ * store them into @msgs array which is allocated upon
+ * successful return and is NULL terminated. The caller is
+ * responsible for freeing @msgs when no longer needed.
+ *
+ * Note that some hypervisors may only report deprecation
+ * messages while the VM is in a running state.
+ *
+ * Returns: number of messages stored in @msgs,
+ *  -1 otherwise.
+ */
+int
+virDomainGetDeprecations(virDomainPtr domain,
+ char ***msgs,
+ unsigned int flags)
+{
+virConnectPtr conn;
+
+VIR_DOMAIN_DEBUG(domain, "msgs=%p, flags=0x%x", msgs, flags);
+
+virResetLastError();
+
+virCheckDomainReturn(domain, -1);
+conn = domain->conn;
+virCheckNonNullArgGoto(msgs, error);
+
+if (conn->driver->domainGetDeprecations) {
+int ret;
+ret = conn->driver->domainGetDeprecations(domain, msgs, flags);
+if (ret < 0)
+goto error;
+return ret;
+}
+
+virReportUnsupportedError();
+ error:
+virDispatchError(conn);
+return -1;
+}
diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms
index cf31f937d5..345c5685bf 100644
--- a/src/libvirt_public.syms
+++ b/src/libvirt_public.syms
@@ -879,4 +879,9 @@ LIBVIRT_6.10.0 {
 virDomainAuthorizedSSHKeysSet;
 } LIBVIRT_6.0.0;
 
+LIBVIRT_7.1.0 {
+global:
+virDomainGetDeprecations;
+} LIBVIRT_6.10.0;
+
 #  define new API here using predicted next version number 
-- 
2.29.2

[libvirt PATCH 08/16] qemu: record deprecation messages against the domain

[Daniel P . Berrangé]

These messages are only valid while the domain is running.

Signed-off-by: Daniel P. Berrangé 
---
 src/qemu/qemu_domain.c  | 5 +
 src/qemu/qemu_process.c | 5 +
 2 files changed, 10 insertions(+)

diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index f00e8d9384..64dc7b5eff 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -6205,6 +6205,11 @@ void qemuDomainObjTaintMsg(virQEMUDriverPtr driver,
 va_end(args);
 }
 
+if (taint == VIR_DOMAIN_TAINT_DEPRECATED_CONFIG &&
+extramsg) {
+virDomainObjDeprecation(obj, extramsg);
+}
+
 VIR_WARN("Domain id=%d name='%s' uuid=%s is tainted: %s%s%s%s",
  obj->def->id,
  obj->def->name,
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index 202d867289..4a34e5ab9f 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -7856,6 +7856,11 @@ void qemuProcessStop(virQEMUDriverPtr driver,
 }
 }
 
+for (i = 0; i < vm->ndeprecations; i++)
+g_free(vm->deprecations[i]);
+g_free(vm->deprecations);
+vm->ndeprecations = 0;
+vm->deprecations = NULL;
 vm->taint = 0;
 vm->pid = -1;
 virDomainObjSetState(vm, VIR_DOMAIN_SHUTOFF, reason);
-- 
2.29.2

[libvirt PATCH 07/16] conf: record deprecation messages against the domain

[Daniel P . Berrangé]

These messages will be stored in the live status XML.

Signed-off-by: Daniel P. Berrangé 
---
 src/conf/domain_conf.c   | 28 +---
 src/conf/domain_conf.h   |  4 
 src/libvirt_private.syms |  1 +
 3 files changed, 30 insertions(+), 3 deletions(-)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 5346c40a81..86c4639bda 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -1774,6 +1774,15 @@ bool virDomainObjTaint(virDomainObjPtr obj,
 }
 
 
+void virDomainObjDeprecation(virDomainObjPtr obj,
+ const char *msg)
+{
+obj->deprecations = g_renew(char *, obj->deprecations,
+obj->ndeprecations + 1);
+obj->deprecations[obj->ndeprecations++] = g_strdup(msg);
+}
+
+
 static void
 virDomainGraphicsAuthDefClear(virDomainGraphicsAuthDefPtr def)
 {
@@ -21205,7 +21214,8 @@ virDomainObjParseXML(xmlDocPtr xml,
 int reason = 0;
 void *parseOpaque = NULL;
 g_autofree char *tmp = NULL;
-g_autofree xmlNodePtr *nodes = NULL;
+g_autofree xmlNodePtr *taintNodes = NULL;
+g_autofree xmlNodePtr *depNodes = NULL;
 
 if (!(obj = virDomainObjNew(xmlopt)))
 return NULL;
@@ -21252,10 +21262,10 @@ virDomainObjParseXML(xmlDocPtr xml,
 }
 obj->pid = (pid_t)val;
 
-if ((n = virXPathNodeSet("./taint", ctxt, )) < 0)
+if ((n = virXPathNodeSet("./taint", ctxt, )) < 0)
 goto error;
 for (i = 0; i < n; i++) {
-char *str = virXMLPropString(nodes[i], "flag");
+char *str = virXMLPropString(taintNodes[i], "flag");
 if (str) {
 int flag = virDomainTaintTypeFromString(str);
 if (flag < 0) {
@@ -21269,6 +21279,13 @@ virDomainObjParseXML(xmlDocPtr xml,
 }
 }
 
+if ((n = virXPathNodeSet("./deprecation", ctxt, )) < 0)
+goto error;
+for (i = 0; i < n; i++) {
+g_autofree char *str = virXMLNodeContentString(depNodes[i]);
+virDomainObjDeprecation(obj, str);
+}
+
 if (xmlopt->privateData.parse &&
 xmlopt->privateData.parse(ctxt, obj, >config) < 0)
 goto error;
@@ -29073,6 +29090,11 @@ virDomainObjFormat(virDomainObjPtr obj,
   virDomainTaintTypeToString(i));
 }
 
+for (i = 0; i < obj->ndeprecations; i++) {
+virBufferEscapeString(, "%s\n",
+  obj->deprecations[i]);
+}
+
 if (xmlopt->privateData.format &&
 xmlopt->privateData.format(, obj) < 0)
 return NULL;
diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h
index 55abbec0fe..9ae1bd3eeb 100644
--- a/src/conf/domain_conf.h
+++ b/src/conf/domain_conf.h
@@ -2799,6 +2799,8 @@ struct _virDomainObj {
 void (*privateDataFreeFunc)(void *);
 
 int taint;
+size_t ndeprecations;
+char **deprecations;
 
 unsigned long long original_memlock; /* Original RLIMIT_MEMLOCK, zero if no
   * restore will be required later */
@@ -3056,6 +3058,8 @@ void virDomainObjEndAPI(virDomainObjPtr *vm);
 
 bool virDomainObjTaint(virDomainObjPtr obj,
virDomainTaintFlags taint);
+void virDomainObjDeprecation(virDomainObjPtr obj,
+ const char *msg);
 
 void virDomainObjBroadcast(virDomainObjPtr vm);
 int virDomainObjWait(virDomainObjPtr vm);
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index c325040b60..7a73943ba0 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -547,6 +547,7 @@ virDomainObjAssignDef;
 virDomainObjBroadcast;
 virDomainObjCheckActive;
 virDomainObjCopyPersistentDef;
+virDomainObjDeprecation;
 virDomainObjEndAPI;
 virDomainObjFormat;
 virDomainObjGetDefs;
-- 
2.29.2

[libvirt PATCH 06/16] qemu: taint the VM if it is using a deprecated machine type

[Daniel P . Berrangé]

Signed-off-by: Daniel P. Berrangé 
---
 src/qemu/qemu_capabilities.c | 16 
 src/qemu/qemu_capabilities.h |  3 +++
 src/qemu/qemu_domain.c   | 20 
 3 files changed, 39 insertions(+)

diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c
index f6d7a222c4..b6d4c09e7e 100644
--- a/src/qemu/qemu_capabilities.c
+++ b/src/qemu/qemu_capabilities.c
@@ -2454,6 +2454,22 @@ virQEMUCapsIsCPUDeprecated(virQEMUCapsPtr qemuCaps,
 }
 
 
+bool
+virQEMUCapsIsMachineDeprecated(virQEMUCapsPtr qemuCaps,
+   virDomainVirtType type,
+   const char *machine)
+{
+virQEMUCapsAccelPtr accel = virQEMUCapsGetAccel(qemuCaps, type);
+size_t i;
+
+for (i = 0; i < accel->nmachineTypes; i++) {
+if (STREQ_NULLABLE(accel->machineTypes[i].name, machine))
+return accel->machineTypes[i].deprecated;
+}
+return false;
+}
+
+
 bool
 virQEMUCapsGetMachineNumaMemSupported(virQEMUCapsPtr qemuCaps,
   virDomainVirtType virtType,
diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h
index a6270edb0e..19e2cd1fff 100644
--- a/src/qemu/qemu_capabilities.h
+++ b/src/qemu/qemu_capabilities.h
@@ -689,6 +689,9 @@ const char *virQEMUCapsGetMachineDefaultCPU(virQEMUCapsPtr 
qemuCaps,
 bool virQEMUCapsIsCPUDeprecated(virQEMUCapsPtr qemuCaps,
 virDomainVirtType type,
 const char *model);
+bool virQEMUCapsIsMachineDeprecated(virQEMUCapsPtr qemuCaps,
+virDomainVirtType type,
+const char *machine);
 bool virQEMUCapsGetMachineNumaMemSupported(virQEMUCapsPtr qemuCaps,
virDomainVirtType virtType,
const char *name);
diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index ed5fd6aa0d..f00e8d9384 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -6243,6 +6243,24 @@ void qemuDomainObjTaintMsg(virQEMUDriverPtr driver,
 virErrorRestore(_err);
 }
 
+static void
+qemuDomainObjCheckMachineTaint(virQEMUDriverPtr driver,
+   virDomainObjPtr obj,
+   qemuDomainLogContextPtr logCtxt)
+{
+qemuDomainObjPrivatePtr priv = obj->privateData;
+virQEMUCapsPtr qemuCaps = priv->qemuCaps;
+
+if (virQEMUCapsIsMachineDeprecated(qemuCaps,
+   obj->def->virtType,
+   obj->def->os.machine)) {
+qemuDomainObjTaintMsg(driver, obj, VIR_DOMAIN_TAINT_DEPRECATED_CONFIG, 
logCtxt,
+  _("machine type '%s'"),
+  obj->def->os.machine);
+}
+}
+
+
 static void
 qemuDomainObjCheckCPUTaint(virQEMUDriverPtr driver,
virDomainObjPtr obj,
@@ -6307,6 +6325,8 @@ void qemuDomainObjCheckTaint(virQEMUDriverPtr driver,
VIR_DOMAIN_TAINT_CUSTOM_HYPERVISOR_FEATURE, 
logCtxt);
 }
 
+qemuDomainObjCheckMachineTaint(driver, obj, logCtxt);
+
 if (obj->def->cpu)
 qemuDomainObjCheckCPUTaint(driver, obj, logCtxt, incomingMigration);
 
-- 
2.29.2

[libvirt PATCH 05/16] qemu: taint the VM if it is using a deprecated CPU model

[Daniel P . Berrangé]

Signed-off-by: Daniel P. Berrangé 
---
 src/qemu/qemu_capabilities.c | 17 +
 src/qemu/qemu_capabilities.h |  3 +++
 src/qemu/qemu_domain.c   | 37 
 3 files changed, 53 insertions(+), 4 deletions(-)

diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c
index e3d1de0779..f6d7a222c4 100644
--- a/src/qemu/qemu_capabilities.c
+++ b/src/qemu/qemu_capabilities.c
@@ -2437,6 +2437,23 @@ virQEMUCapsGetMachineDefaultCPU(virQEMUCapsPtr qemuCaps,
 }
 
 
+bool
+virQEMUCapsIsCPUDeprecated(virQEMUCapsPtr qemuCaps,
+   virDomainVirtType type,
+   const char *model)
+{
+virQEMUCapsAccelPtr accel = virQEMUCapsGetAccel(qemuCaps, type);
+qemuMonitorCPUDefsPtr defs = accel->cpuModels;
+size_t i;
+
+for (i = 0; i < defs->ncpus; i++) {
+if (STREQ_NULLABLE(defs->cpus[i].name, model))
+return defs->cpus[i].deprecated;
+}
+return false;
+}
+
+
 bool
 virQEMUCapsGetMachineNumaMemSupported(virQEMUCapsPtr qemuCaps,
   virDomainVirtType virtType,
diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h
index a14a78f959..a6270edb0e 100644
--- a/src/qemu/qemu_capabilities.h
+++ b/src/qemu/qemu_capabilities.h
@@ -686,6 +686,9 @@ bool virQEMUCapsGetMachineHotplugCpus(virQEMUCapsPtr 
qemuCaps,
 const char *virQEMUCapsGetMachineDefaultCPU(virQEMUCapsPtr qemuCaps,
 const char *name,
 virDomainVirtType type);
+bool virQEMUCapsIsCPUDeprecated(virQEMUCapsPtr qemuCaps,
+virDomainVirtType type,
+const char *model);
 bool virQEMUCapsGetMachineNumaMemSupported(virQEMUCapsPtr qemuCaps,
virDomainVirtType virtType,
const char *name);
diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index e63bea8b32..ed5fd6aa0d 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -6243,6 +6243,37 @@ void qemuDomainObjTaintMsg(virQEMUDriverPtr driver,
 virErrorRestore(_err);
 }
 
+static void
+qemuDomainObjCheckCPUTaint(virQEMUDriverPtr driver,
+   virDomainObjPtr obj,
+   qemuDomainLogContextPtr logCtxt,
+   bool incomingMigration)
+{
+qemuDomainObjPrivatePtr priv = obj->privateData;
+virQEMUCapsPtr qemuCaps = priv->qemuCaps;
+
+switch (obj->def->cpu->mode) {
+case VIR_CPU_MODE_HOST_PASSTHROUGH:
+if (incomingMigration)
+qemuDomainObjTaint(driver, obj, VIR_DOMAIN_TAINT_HOST_CPU, 
logCtxt);
+break;
+case VIR_CPU_MODE_CUSTOM:
+if (obj->def->cpu->model &&
+virQEMUCapsIsCPUDeprecated(qemuCaps,
+   obj->def->virtType,
+   obj->def->cpu->model)) {
+qemuDomainObjTaintMsg(driver, obj, 
VIR_DOMAIN_TAINT_DEPRECATED_CONFIG, logCtxt,
+  _("CPU model '%s'"),
+  obj->def->cpu->model);
+}
+break;
+case VIR_CPU_MODE_HOST_MODEL:
+case VIR_CPU_MODE_LAST:
+default:
+break;
+}
+}
+
 
 void qemuDomainObjCheckTaint(virQEMUDriverPtr driver,
  virDomainObjPtr obj,
@@ -6276,10 +6307,8 @@ void qemuDomainObjCheckTaint(virQEMUDriverPtr driver,
VIR_DOMAIN_TAINT_CUSTOM_HYPERVISOR_FEATURE, 
logCtxt);
 }
 
-if (obj->def->cpu &&
-obj->def->cpu->mode == VIR_CPU_MODE_HOST_PASSTHROUGH &&
-incomingMigration)
-qemuDomainObjTaint(driver, obj, VIR_DOMAIN_TAINT_HOST_CPU, logCtxt);
+if (obj->def->cpu)
+qemuDomainObjCheckCPUTaint(driver, obj, logCtxt, incomingMigration);
 
 for (i = 0; i < obj->def->ndisks; i++)
 qemuDomainObjCheckDiskTaint(driver, obj, obj->def->disks[i], logCtxt);
-- 
2.29.2

[libvirt PATCH 04/16] qemu: add ability to associate a string message with taint warning

[Daniel P . Berrangé]

Signed-off-by: Daniel P. Berrangé 
---
 src/qemu/qemu_domain.c | 49 ++
 src/qemu/qemu_domain.h |  7 ++
 2 files changed, 47 insertions(+), 9 deletions(-)

diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index dd79cfd9d9..e63bea8b32 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -6166,22 +6166,51 @@ void qemuDomainObjTaint(virQEMUDriverPtr driver,
 virDomainObjPtr obj,
 virDomainTaintFlags taint,
 qemuDomainLogContextPtr logCtxt)
+{
+qemuDomainObjTaintMsg(driver, obj, taint, logCtxt, NULL);
+}
+
+void qemuDomainObjTaintMsg(virQEMUDriverPtr driver,
+   virDomainObjPtr obj,
+   virDomainTaintFlags taint,
+   qemuDomainLogContextPtr logCtxt,
+   const char *fmt, ...)
 {
 virErrorPtr orig_err = NULL;
 g_autofree char *timestamp = NULL;
 char uuidstr[VIR_UUID_STRING_BUFLEN];
 int rc;
-
-if (!virDomainObjTaint(obj, taint))
-return;
+g_autofree char *extra = NULL;
+const char *extraprefix = "";
+const char *extramsg = "";
+const char *extrasuffix = "";
+va_list args;
+
+if (virDomainObjTaint(obj, taint)) {
+/* If an extra message was given we must always
+ * emit the taint warning, otherwise it is a
+ * one-time only warning per VM
+ */
+if (!fmt)
+return;
+}
 
 virUUIDFormat(obj->def->uuid, uuidstr);
 
-VIR_WARN("Domain id=%d name='%s' uuid=%s is tainted: %s",
+if (fmt) {
+va_start(args, fmt);
+extraprefix = " (";
+extramsg = extra = g_strdup_vprintf(fmt, args);
+extrasuffix = ")";
+va_end(args);
+}
+
+VIR_WARN("Domain id=%d name='%s' uuid=%s is tainted: %s%s%s%s",
  obj->def->id,
  obj->def->name,
  uuidstr,
- virDomainTaintTypeToString(taint));
+ virDomainTaintTypeToString(taint),
+ extraprefix, extramsg, extrasuffix);
 
 /* We don't care about errors logging taint info, so
  * preserve original error, and clear any error that
@@ -6193,16 +6222,18 @@ void qemuDomainObjTaint(virQEMUDriverPtr driver,
 
 if (logCtxt) {
 rc = qemuDomainLogContextWrite(logCtxt,
-   "%s: Domain id=%d is tainted: %s\n",
+   "%s: Domain id=%d is tainted: 
%s%s%s%s\n",
timestamp,
obj->def->id,
-   virDomainTaintTypeToString(taint));
+   virDomainTaintTypeToString(taint),
+   extraprefix, extramsg, extrasuffix);
 } else {
 rc = qemuDomainLogAppendMessage(driver, obj,
-"%s: Domain id=%d is tainted: %s\n",
+"%s: Domain id=%d is tainted: 
%s%s%s%s\n",
 timestamp,
 obj->def->id,
-virDomainTaintTypeToString(taint));
+virDomainTaintTypeToString(taint),
+extraprefix, extramsg, extrasuffix);
 }
 
 if (rc < 0)
diff --git a/src/qemu/qemu_domain.h b/src/qemu/qemu_domain.h
index 154339ef8f..7453881a31 100644
--- a/src/qemu/qemu_domain.h
+++ b/src/qemu/qemu_domain.h
@@ -565,6 +565,13 @@ void qemuDomainObjTaint(virQEMUDriverPtr driver,
 virDomainTaintFlags taint,
 qemuDomainLogContextPtr logCtxt);
 
+void qemuDomainObjTaintMsg(virQEMUDriverPtr driver,
+   virDomainObjPtr obj,
+   virDomainTaintFlags taint,
+   qemuDomainLogContextPtr logCtxt,
+   const char *msg,
+   ...) G_GNUC_PRINTF(5, 6);
+
 void qemuDomainObjCheckTaint(virQEMUDriverPtr driver,
  virDomainObjPtr obj,
  qemuDomainLogContextPtr logCtxt,
-- 
2.29.2

[libvirt PATCH 03/16] conf: introduce new taint flag for deprecated configuration

[Daniel P . Berrangé]

Hypervisors are capable of reporting that some features are deprecated.
This should be used to mark a domain as tainted.

Signed-off-by: Daniel P. Berrangé 
---
 src/conf/domain_conf.c | 1 +
 src/conf/domain_conf.h | 1 +
 2 files changed, 2 insertions(+)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index fcf332fe44..5346c40a81 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -87,6 +87,7 @@ VIR_ENUM_IMPL(virDomainTaint,
   "custom-dtb",
   "custom-ga-command",
   "custom-hypervisor-feature",
+  "deprecated-config",
 );
 
 VIR_ENUM_IMPL(virDomainVirt,
diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h
index d514f6360f..55abbec0fe 100644
--- a/src/conf/domain_conf.h
+++ b/src/conf/domain_conf.h
@@ -2757,6 +2757,7 @@ typedef enum {
 VIR_DOMAIN_TAINT_CUSTOM_DTB,   /* Custom device tree blob was 
specified */
 VIR_DOMAIN_TAINT_CUSTOM_GA_COMMAND, /* Custom guest agent command */
 VIR_DOMAIN_TAINT_CUSTOM_HYPERVISOR_FEATURE, /* custom hypervisor feature 
control */
+VIR_DOMAIN_TAINT_DEPRECATED_CONFIG,  /* Configuration that is marked 
deprecated */
 
 VIR_DOMAIN_TAINT_LAST
 } virDomainTaintFlags;
-- 
2.29.2

[libvirt PATCH 02/16] qemu: report whether a machine type is deprecated in capabilities

[Daniel P . Berrangé]

QEMU has the ability to mark machine types as deprecated. This should be
exposed to management applications in the capabilities.

Signed-off-by: Daniel P. Berrangé 
---
 docs/schemas/capability.rng |  8 
 src/conf/capabilities.c |  2 ++
 src/conf/capabilities.h |  1 +
 src/qemu/qemu_capabilities.c| 17 ++---
 src/qemu/qemu_capspriv.h|  3 ++-
 src/qemu/qemu_monitor.h |  1 +
 src/qemu/qemu_monitor_json.c|  4 
 .../qemucapabilitiesdata/caps_4.1.0.x86_64.xml  | 16 
 tests/qemucapabilitiesdata/caps_4.2.0.ppc64.xml |  4 ++--
 .../qemucapabilitiesdata/caps_4.2.0.x86_64.xml  | 16 
 .../qemucapabilitiesdata/caps_5.0.0.x86_64.xml  | 16 
 .../qemucapabilitiesdata/caps_5.1.0.x86_64.xml  | 16 
 .../qemucapabilitiesdata/caps_5.2.0.x86_64.xml  | 16 
 tests/testutilsqemu.c   |  8 +---
 14 files changed, 79 insertions(+), 49 deletions(-)

diff --git a/docs/schemas/capability.rng b/docs/schemas/capability.rng
index 91a046eb48..c4cafc47ee 100644
--- a/docs/schemas/capability.rng
+++ b/docs/schemas/capability.rng
@@ -407,6 +407,14 @@
   
 
   
+  
+
+  
+yes
+no
+  
+
+  
   
 
   
diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
index 425f34113a..1ea059ea6f 100644
--- a/src/conf/capabilities.c
+++ b/src/conf/capabilities.c
@@ -1244,6 +1244,8 @@ virCapabilitiesFormatGuestXML(virCapsGuestPtr *guests,
 virBufferAsprintf(buf, " canonical='%s'", machine->canonical);
 if (machine->maxCpus > 0)
 virBufferAsprintf(buf, " maxCpus='%d'", machine->maxCpus);
+if (machine->deprecated)
+virBufferAddLit(buf, " deprecated='yes'");
 virBufferAsprintf(buf, ">%s\n", machine->name);
 }
 
diff --git a/src/conf/capabilities.h b/src/conf/capabilities.h
index e2581fac8b..5fd59efc05 100644
--- a/src/conf/capabilities.h
+++ b/src/conf/capabilities.h
@@ -56,6 +56,7 @@ struct _virCapsGuestMachine {
 char *name;
 char *canonical;
 unsigned int maxCpus;
+bool deprecated;
 };
 
 struct _virCapsGuestDomainInfo {
diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c
index 61467eb6c2..e3d1de0779 100644
--- a/src/qemu/qemu_capabilities.c
+++ b/src/qemu/qemu_capabilities.c
@@ -624,6 +624,7 @@ struct _virQEMUCapsMachineType {
 char *defaultCPU;
 bool numaMemSupported;
 char *defaultRAMid;
+bool deprecated;
 };
 
 typedef struct _virQEMUCapsHostCPUData virQEMUCapsHostCPUData;
@@ -943,6 +944,7 @@ virQEMUCapsGetMachineTypesCaps(virQEMUCapsPtr qemuCaps,
 mach->name = g_strdup(accel->machineTypes[i].name);
 }
 mach->maxCpus = accel->machineTypes[i].maxCpus;
+mach->deprecated = accel->machineTypes[i].deprecated;
 }
 
 /* Make sure all canonical machine types also have their own entry so that
@@ -976,6 +978,7 @@ virQEMUCapsGetMachineTypesCaps(virQEMUCapsPtr qemuCaps,
 }
 mach->name = g_strdup(machine->canonical);
 mach->maxCpus = machine->maxCpus;
+mach->deprecated = machine->deprecated;
 i++;
 }
 i++;
@@ -1874,6 +1877,7 @@ virQEMUCapsAccelCopyMachineTypes(virQEMUCapsAccelPtr dst,
 dst->machineTypes[i].qemuDefault = src->machineTypes[i].qemuDefault;
 dst->machineTypes[i].numaMemSupported = 
src->machineTypes[i].numaMemSupported;
 dst->machineTypes[i].defaultRAMid = 
g_strdup(src->machineTypes[i].defaultRAMid);
+dst->machineTypes[i].deprecated = src->machineTypes[i].deprecated;
 }
 }
 
@@ -2708,7 +2712,8 @@ virQEMUCapsAddMachine(virQEMUCapsPtr qemuCaps,
   bool hotplugCpus,
   bool isDefault,
   bool numaMemSupported,
-  const char *defaultRAMid)
+  const char *defaultRAMid,
+  bool deprecated)
 {
 virQEMUCapsAccelPtr accel = virQEMUCapsGetAccel(qemuCaps, virtType);
 virQEMUCapsMachineTypePtr mach;
@@ -2731,6 +2736,7 @@ virQEMUCapsAddMachine(virQEMUCapsPtr qemuCaps,
 mach->numaMemSupported = numaMemSupported;
 
 mach->defaultRAMid = g_strdup(defaultRAMid);
+mach->deprecated = deprecated;
 }
 
 /**
@@ -2778,7 +2784,8 @@ virQEMUCapsProbeQMPMachineTypes(virQEMUCapsPtr qemuCaps,
   machines[i]->hotplugCpus,
   machines[i]->isDefault,
   machines[i]->numaMemSupported,
-  machines[i]->defaultRAMid);
+  machines[i]->defaultRAMid,
+  machines[i]->deprecated);
 
 if (preferredMachine &&
  

[libvirt PATCH 01/16] qemu: report whether a CPU model is deprecated in dom capabilities

[Daniel P . Berrangé]

QEMU has the ability to mark CPUs as deprecated. This should be exposed
to management applications in the domain capabilities.

This attribute is only set when the model is actually deprecated.

Signed-off-by: Daniel P. Berrangé 
---
 docs/formatdomaincaps.html.in| 10 ++
 docs/schemas/domaincaps.rng  |  8 
 src/conf/domain_capabilities.c   | 14 ++
 src/conf/domain_capabilities.h   |  4 +++-
 src/qemu/qemu_capabilities.c | 10 +-
 src/qemu/qemu_monitor.c  |  1 +
 src/qemu/qemu_monitor.h  |  1 +
 src/qemu/qemu_monitor_json.c |  4 
 tests/cputest.c  |  4 ++--
 tests/domaincapsdata/qemu_5.2.0-q35.x86_64.xml   |  4 ++--
 tests/domaincapsdata/qemu_5.2.0-tcg.x86_64.xml   |  4 ++--
 tests/domaincapsdata/qemu_5.2.0.x86_64.xml   |  4 ++--
 tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml | 16 
 13 files changed, 58 insertions(+), 26 deletions(-)

diff --git a/docs/formatdomaincaps.html.in b/docs/formatdomaincaps.html.in
index 65ab5574d3..8df88f5355 100644
--- a/docs/formatdomaincaps.html.in
+++ b/docs/formatdomaincaps.html.in
@@ -214,9 +214,9 @@
   feature policy='require' name='vmx'/
 /mode
 mode name='custom' supported='yes'
-  model usable='no'Broadwell/model
-  model usable='yes'Broadwell-noTSX/model
-  model usable='no'Haswell/model
+  model usable='no' deprecated='no'Broadwell/model
+  model usable='yes' deprecated='no'Broadwell-noTSX/model
+  model usable='no' deprecated='yes'Haswell/model
   ...
 /mode
   /cpu
@@ -262,7 +262,9 @@
 cannot be used without disabling some features that the CPU of such
 model is expected to have. A special value unknown
 indicates libvirt does not have enough information to provide the
-usability data.
+usability data. The deprecated attribute reflects
+the hypervisor's policy on usage of this model
+(since 7.1.0).
   
 
 
diff --git a/docs/schemas/domaincaps.rng b/docs/schemas/domaincaps.rng
index 0dbffb28ac..a57ef715c3 100644
--- a/docs/schemas/domaincaps.rng
+++ b/docs/schemas/domaincaps.rng
@@ -138,6 +138,14 @@
   unknown
 
   
+  
+
+  
+yes
+no
+  
+
+  
   
 
   
diff --git a/src/conf/domain_capabilities.c b/src/conf/domain_capabilities.c
index 8130311590..cdb1b31af6 100644
--- a/src/conf/domain_capabilities.c
+++ b/src/conf/domain_capabilities.c
@@ -175,7 +175,8 @@ virDomainCapsCPUModelsCopy(virDomainCapsCPUModelsPtr old)
 if (virDomainCapsCPUModelsAdd(cpuModels,
   old->models[i].name,
   old->models[i].usable,
-  old->models[i].blockers) < 0)
+  old->models[i].blockers,
+  old->models[i].deprecated) < 0)
 goto error;
 }
 
@@ -191,7 +192,8 @@ int
 virDomainCapsCPUModelsAdd(virDomainCapsCPUModelsPtr cpuModels,
   const char *name,
   virDomainCapsCPUUsable usable,
-  char **blockers)
+  char **blockers,
+  bool deprecated)
 {
 g_autofree char * nameCopy = NULL;
 virDomainCapsCPUModelPtr cpu;
@@ -208,6 +210,7 @@ virDomainCapsCPUModelsAdd(virDomainCapsCPUModelsPtr 
cpuModels,
 cpu->usable = usable;
 cpu->name = g_steal_pointer();
 cpu->blockers = g_strdupv(blockers);
+cpu->deprecated = deprecated;
 
 return 0;
 }
@@ -388,8 +391,11 @@ virDomainCapsCPUCustomFormat(virBufferPtr buf,
 
 for (i = 0; i < custom->nmodels; i++) {
 virDomainCapsCPUModelPtr model = custom->models + i;
-virBufferAsprintf(buf, "%s\n",
-  virDomainCapsCPUUsableTypeToString(model->usable),
+virBufferAsprintf(buf, "usable));
+if (model->deprecated)
+virBufferAddLit(buf, " deprecated='yes'");
+virBufferAsprintf(buf, ">%s\n",
   model->name);
 }
 
diff --git a/src/conf/domain_capabilities.h b/src/conf/domain_capabilities.h
index b22d40abb2..f454780185 100644
--- a/src/conf/domain_capabilities.h
+++ b/src/conf/domain_capabilities.h
@@ -146,6 +146,7 @@ struct _virDomainCapsCPUModel {
 char *name;
 virDomainCapsCPUUsable usable;
 char **blockers; /* NULL-terminated list of usability blockers */
+bool deprecated;
 };
 
 typedef struct _virDomainCapsCPUModels virDomainCapsCPUModels;
@@ -228,7 +229,8 @@ virDomainCapsCPUModelsPtr 
virDomainCapsCPUModelsCopy(virDomainCapsCPUModelsPtr o
 int virDomainCapsCPUModelsAdd(virDomainCapsCPUModelsPtr 

[libvirt PATCH 00/16] expose tainting and deprecations in public API

[Daniel P . Berrangé]

Libvirt has a notion of "tainting" which we use to mark a guest which
has some undesirable configuration or behaviour from libvirt's POV.
This ends up in the libvirtd logs and in the per-VM log file, but is
not exposed to management applications directly.

QMP has the ability to report whether a CPU or machine type is
deprecated.

QEMU itself prints warnings to stderr which end up in the per VM log:

2021-01-22T12:22:53.566239Z qemu-system-x86_64: Machine type 'pc-1.3' is depr=
ecated: use a newer machine type instead
2021-01-22T12:22:53.566613Z qemu-system-x86_64: warning: CPU model Icelake-Cl=
ient-x86_64-cpu is deprecated -- use Icelake-Server instead

We can use the deprecation info from QMP to add tainting to the
domain too. This will appear in the pre-VM log file again:

2021-01-22 12:22:53.492+: Domain id=3D2 is tainted: deprecated-configurat=
ion (machine type 'pc-1.3')
2021-01-22 12:22:53.492+: Domain id=3D2 is tainted: deprecated-configurat=
ion (CPU model 'Icelake-Client')

and more usefully in the libvirtd log

2021-01-22 13:18:09.619+: 3299849: warning :
qemuDomainObjTaintMsg:6208 :
Domain id=3D3 name=3D'demo' uuid=3Deadf8ef0-bf14-4c5f-9708-4a19bacf9e81
is tainted: deprecated-configuration (machine type 'pc-1.3')

2021-01-22 13:18:09.619+: 3299849: warning :
qemuDomainObjTaintMsg:6208 :
Domain id=3D3 name=3D'demo' uuid=3Deadf8ef0-bf14-4c5f-9708-4a19bacf9e81
is tainted: deprecated-configuration (CPU model 'Icelake-Client')

This series goes further and also exposes the deprecation info in the
capabilities (machine types) or domain capabilities (CPU) XML. This
lets mgmt apps avoid using the feature upfront if desired.

Finally both deprecation messages and tainting flags are exposed in
new public APIs, and wired into virsh

$ virsh dominfo demo
Id: 3
Name:   demo
UUID:   eadf8ef0-bf14-4c5f-9708-4a19bacf9e81
OS Type:hvm
State:  running
CPU(s): 2
CPU time:   1.3s
Max memory: 1536000 KiB
Used memory:1536000 KiB
Persistent: yes
Autostart:  disable
Managed save:   no
Security model: selinux
Security DOI:   0
Security label: unconfined_u:unconfined_r:svirt_t:s0:c578,c807 (permissive)
Tainting:   custom-monitor
deprecated-config
Deprecations:   CPU model 'Icelake-Client'
machine type 'pc-1.3'

The deprecations API is simple, just returning a list of free form
opaque strings, which are eeffectively warning messages.

I'm not entirely convinced by tainting API though. I didn't especially
want to expose the virDomainTaintFlags  enum in the public API since it
feels like the enum flags are (almost) all QEMU driver specific. I thus
took the approach of having an API return opaque strings which are
declared to be hypervisor specific.

I'm worried though that mgmt apps will none the less simply match on
the strings to detect things, at which point we might as well just use
an enum after all.

So perhaps it should just be turned into

  virDomainGetTainting(virDomainPtr obj,
   int **codes,
   unsigned int flags);

  enum virDomainTaintCodes {
 
  }

Daniel P. Berrang=C3=A9 (16):
  qemu: report whether a CPU model is deprecated in dom capabilities
  qemu: report whether a machine type is deprecated in capabilities
  conf: introduce new taint flag for deprecated configuration
  qemu: add ability to associate a string message with taint warning
  qemu: taint the VM if it is using a deprecated CPU model
  qemu: taint the VM if it is using a deprecated machine type
  conf: record deprecation messages against the domain
  qemu: record deprecation messages against the domain
  src: define virDomainGetDeprecations API
  remote: add RPC support for the virDomainGetDeprecations API
  qemu: implement virDomainGetDeprecations API
  tools: report deprecations for 'dominfo' command
  src: define virDomainGetTainting API
  remote: add RPC support for the virDomainGetTainting API
  qemu: implement virDomainGetTainting API
  tools: report tainting for 'dominfo' command

 docs/formatdomaincaps.html.in |  10 +-
 docs/schemas/capability.rng   |   8 ++
 docs/schemas/domaincaps.rng   |   8 ++
 include/libvirt/libvirt-domain.h  |   7 ++
 src/conf/capabilities.c   |   2 +
 src/conf/capabilities.h   |   1 +
 src/conf/domain_capabilities.c|  14 ++-
 src/conf/domain_capabilities.h|   4 +-
 src/conf/domain_conf.c|  29 -
 src/conf/domain_conf.h|   5 +
 src/driver-hypervisor.h   |  11 ++
 src/libvirt-domain.c  |  95 +++
 src/libvirt_private.syms  |   1 +
 src/libvirt_public.syms   |   6 +
 src/qemu/qemu_capabilities.c  |  60 +-
 src/qemu/qemu_capabilities.h   

Re: [PATCH RESEND 06/20] virhostdev.c: virHostdevGetPCIHostDevice() now reports missing device

[Daniel Henrique Barboza]

Sorry for the delay in answering. I usually get the replies to my
patches in my inbox because I usually get my email CC'ed in the
reply.

On 1/20/21 10:33 PM, Laine Stump wrote:

On 1/18/21 2:53 PM, Daniel Henrique Barboza wrote:

Gitlab issue #72 [1] reports that removing SR-IOVs VFs before
removing the devices from the running domains can have strange
consequences.



Resolution: "Don't do that!!" :-)


Seriously, though, there are any number of things that someone with root access 
to the host could do that would completely confuse libvirt and break 
everything. For example, assigning a PF to a guest after having assigned one or 
more of that PF's VFs to guests.


But I guess there's no harm in trying to mitigate the damage. As long as it 
doesn't complicate the code to the extent that it becomes more confusing, 
difficult to maintain, and liable to other regressions.


This was the idea with this series. I'm trying my best to not interfere
with working code/logic by adding these mitigations.






  QEMU might be able to hotunplug the device inside the
guest, but Libvirt will not be aware of that,



If qemu knows enough to unplug the device from the guest, then I'm presuming 
that it must also be sending a DEVICE_DELETED event up to libvirt. Of course 
libvirt isn't expecting this, and so probably throws it away, right?


 long story warning 

We have 2 scenarios I'm trying to deal with in this work: a SR-IOV disappearing
from the host, while a running domain was using one of its functions, in
a condition where:

1) the VF was coldplugged in the domain (i.e. it was in the domain XML
before starting the domain)
2) the VF was hotplugged in the domain


In scenario (2), QEMU will throw a DEVICE_DEL event and we'll capture it
accordingly via processDeviceDeleteEvent ->qemuDomainRemoveDevice ->
qemuDomainRemoveHostDevice. The problem is,by the time we reach
qemuDomainRemoveHostDevice, the VF is already gone from the host  and we didn't
foresee this as a possibility.

All of this is pinned on how we interpret virPCIDeviceNew(). My initial idea
for this work was to change virPCIDeviceNew() to allow the object to be created,
regardless of whether the device actually exists in the host, and then we
would have a 'present' flag to determine if the device is present or not.
I got terrified by the idea of changing the semantics of virPCIDeviceNew(),
which will return -1 if the device isn't present, for all Libvirt devices that
uses the object, just to handle a niche case. I decided then to do the
'device is present' checks you'll see around in this series.


As far as scenario (1) goes, I don't fully understand why it happens TBH. For
QEMU there is no difference - QEMU will throw a DEVICE_DEL event for a VF that
disappears from the host, even if the VF was coldplugged. When launching the VM
with Libvirt, QEMU internals aren't aware of kernel intent to remove the device.

Looking in the vfio driver in the kernel I learned that this intent is 
expressed by
an eventfd call of vfio_pci_request. My somewhat educated guess is that Libvirt
is wrapping up the QEMU process in a way that QEMU never sees this eventfd. The
result, as you can see in the bug, is that unless the admin kills the terminal 
where
"echo 0 > num_vfs" was issue, 'echo' will lock for as long as the domain using 
the VF
keeps running.

And when the domain is shut down the VF is removed from the process, and the 
host,
and then we have a similar problem in our shutdown path, via qemuProcessStop.







  and then the guest is
now inconsistent with the domain definition.

There's also the possibility of the VFs removal not succeeding
while the domain is running but then, as soon as the domain
is shutdown, all the VFs are removed. Libvirt can't handle
the removal of the PCI devices while trying to reattach the
hostdevs, and the Libvirt daemon can be left in an inconsistent
state (see [2]).

This patch starts to address the issue related in Gitlab #72, most
notably the issue described in [2]. When shutting down a domain
with SR-IOV hostdevs that got missing, virHostdevReAttachPCIDevices()
is failing the whole process and failing to reattach all the
PCI devices, including the ones that aren't related to the VFs that
went missing. Let's make it more resilient with host changes by
changing virHostdevGetPCIHostDevice() to return an exclusive error
code '-2' for this case. virHostdevGetPCIHostDeviceList() can then
tell when virHostdevGetPCIHostDevice() failed to find the PCI
device of a hostdev and continue to make the list of PCI devices.

virHostdevReAttachPCIDevices() will now be able to proceed reattaching
all other valid PCI devices, at least. The 'ghost hostdevs' will be
handled later on.

[1] https://gitlab.com/libvirt/libvirt/-/issues/72
[2] https://gitlab.com/libvirt/libvirt/-/issues/72#note_459032148

Signed-off-by: Daniel Henrique Barboza 
---
  src/hypervisor/virhostdev.c | 8 ++--
  1 file changed, 6 insertions(+), 2 deletions(-)

diff 

[libvirt PATCH] qemu: fix release of virDomainObjPtr in SSH key APIs

[Daniel P . Berrangé]

The qemuDomainObjFromDomain() API must be paired with
the virDomainObjEndAPI API. The qemuDomainAuthorizedSSHKeysGet
method simply did 'return -1' leaking a reference in two paths.

The qemuDomainAuthorizedSSHKeysSet method marked the object
as an autoptr while also have some code paths that will call
virDomainObjEndAPI, resulting in attempted double free.

Signed-off-by: Daniel P. Berrangé 
---
 src/qemu/qemu_driver.c | 12 +++-
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 027617deef..05e021cce4 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -20254,10 +20254,10 @@ qemuDomainAuthorizedSSHKeysGet(virDomainPtr dom,
 return -1;
 
 if (virDomainAuthorizedSshKeysGetEnsureACL(dom->conn, vm->def) < 0)
-return -1;
+goto cleanup;
 
 if (qemuDomainObjBeginAgentJob(driver, vm, QEMU_AGENT_JOB_QUERY) < 0)
-return -1;
+goto cleanup;
 
 if (!qemuDomainAgentAvailable(vm, true))
 goto endagentjob;
@@ -20268,6 +20268,7 @@ qemuDomainAuthorizedSSHKeysGet(virDomainPtr dom,
 
  endagentjob:
 qemuDomainObjEndAgentJob(vm);
+ cleanup:
 virDomainObjEndAPI();
 return rv;
 }
@@ -20281,7 +20282,7 @@ qemuDomainAuthorizedSSHKeysSet(virDomainPtr dom,
unsigned int flags)
 {
 virQEMUDriverPtr driver = dom->conn->privateData;
-g_autoptr(virDomainObj) vm = NULL;
+virDomainObjPtr vm = NULL;
 qemuAgentPtr agent;
 const bool append = flags & VIR_DOMAIN_AUTHORIZED_SSH_KEYS_SET_APPEND;
 const bool remove = flags & VIR_DOMAIN_AUTHORIZED_SSH_KEYS_SET_REMOVE;
@@ -20294,10 +20295,10 @@ qemuDomainAuthorizedSSHKeysSet(virDomainPtr dom,
 return -1;
 
 if (virDomainAuthorizedSshKeysSetEnsureACL(dom->conn, vm->def) < 0)
-return -1;
+goto cleanup;
 
 if (qemuDomainObjBeginAgentJob(driver, vm, QEMU_AGENT_JOB_QUERY) < 0)
-return -1;
+goto cleanup;
 
 if (!qemuDomainAgentAvailable(vm, true))
 goto endagentjob;
@@ -20311,6 +20312,7 @@ qemuDomainAuthorizedSSHKeysSet(virDomainPtr dom,
 
  endagentjob:
 qemuDomainObjEndAgentJob(vm);
+ cleanup:
 virDomainObjEndAPI();
 return rv;
 }
-- 
2.29.2

[libvirt PATCH] docs: link to PCI docs from the kbase page

[Daniel P . Berrangé]

While the PCI docs are linked from formatdomain.html, finding those
links is not straightforward. It is good for users to highlight them in
the kbase pages.  The PCI docs are intentionally not moved to the kbase/
sub-directory in order to avoid breaking hyperlinks.

Signed-off-by: Daniel P. Berrangé 
---
 docs/kbase/index.rst | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/docs/kbase/index.rst b/docs/kbase/index.rst
index 1494be1e77..83b9bb7200 100644
--- a/docs/kbase/index.rst
+++ b/docs/kbase/index.rst
@@ -40,6 +40,12 @@ Usage
 `KVM real time `__
Run real time workloads in guests on a KVM hypervisor
 
+`PCI hotplug <../pci-hotplug.html>`__
+   Effectively usage of PCI hotplug
+
+`PCI topology <../pci-addresses.html>`__
+   Addressing schemes for PCI devices
+
 Internals / Debugging
 -
 
-- 
2.29.2

[libvirt PATCH] docs: use a relative link to the kbase page

[Daniel P . Berrangé]

Signed-off-by: Daniel P. Berrangé 
---
 docs/index.html.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/index.html.in b/docs/index.html.in
index f44e055174..bf164edb58 100644
--- a/docs/index.html.in
+++ b/docs/index.html.in
@@ -64,7 +64,7 @@
 https://wiki.libvirt.org;>Wiki
 Read further community contributed content
 
-https://libvirt.org/kbase/index.html;>Knowledge 
base
+Knowledge base
 Learn more about libvirt through knowledge base
   
 
-- 
2.29.2

Re: [PATCH 3/3] rpm: disable netcf for the interface driver on RHEL/Fedora/CentOS

[Laine Stump]

On 1/22/21 4:49 AM, Daniel P. Berrangé wrote:

On Fri, Jan 22, 2021 at 03:01:57AM -0500, Laine Stump wrote:

Since libvirt.spec explicitly adds -Dnetcf=enabled to the meson
commandline, just setting the default in the meson.build file won't
have any effect for rpm builds. This patch changes the meson
commandline in the spec file to -Dnetcf=disabled and removes the
associated BuildRequires: and Requires:

(I debated whether or not to do that conditionally only for (fedora >
33 || rhel > 8) but haven't done that for now (still may if it's
considered important; for Fedora probably not important (since we
never rebase after release), for RHEL we might want to preserve
current behavior for RHEL8 even through a rebase though)

Yeah, our normal policy is that we don't disable stuff that was
already present. So changing this should be conditional on
newer distros only.



Okay, I'll make it more elaborate (with its own variable ala 
firewalld_zones) and repost.



Speaking of this - I just noticed that Fedora will be branching for F34 
prior to the next libvirt release, and I was hoping I could get this 
into the next Fedora (wish I'd been paying closer attention to the 
important dates :-( - I really need those entered in the same calendar 
that reminds me of my doctor appointments and meetings).

Re: [PATCH 2/3] build: make netcf=disabled the default

[Laine Stump]

On 1/22/21 4:51 AM, Daniel P. Berrangé wrote:

On Fri, Jan 22, 2021 at 03:01:56AM -0500, Laine Stump wrote:

Even after the previous patch, in order to build without netcf you
would need to add "-Dnetcf=disabled" to the meson commandline (or
uninstall netcf-devel). This patch makes -Dnetcf=disabled the
default. (Without this change, a lot of people would just blindly
continue building with netcf enabled.)

Signed-off-by: Laine Stump 
---
  meson.build | 7 ---
  1 file changed, 4 insertions(+), 3 deletions(-)

I don't much like this idea. I consider it a bug if I have
installed the -devel package for a pre-requisite and it isn't
then detected.


If a distro no longer wants to support netcf why not just
retire the netcf package from that distro version(s), that
way users won't have it installed in the first place ?



Yeah, that's what patch 3 does. I added this patch in later as a way of 
preventing developers (and people who routinely build their own packages 
from upstream without using the "rpm" target) from naively continuing to 
use a setup that is deprecated (and then wondering why it behaves 
differently when the build locally).



I'm okay leaving out this patch and just recommending that everyone do 
their own builds, though.

Re: [PATCH] qemu: Avoid crash in qemuStateShutdownPrepare() and qemuStateShutdownWait()

[Nikolay Shirokovskiy]

On Fri, Jan 22, 2021 at 2:24 PM Michal Privoznik 
wrote:

> On 1/22/21 12:09 PM, Nikolay Shirokovskiy wrote:
> > On Fri, Jan 22, 2021 at 12:45 PM Michal Privoznik 
> > wrote:
> >
> >> If libvirtd is sent SIGTERM while it is still initializing, it
> >> may crash. The following scenario was observed (using 'stress' to
> >> slow down CPU so much that the window where the problem exists is
> >> bigger):
> >>
> >> 1) The main thread is already executing virNetDaemonRun() and is
> >> in virEventRunDefaultImpl().
> >> 2) The thread that's supposed to run daemonRunStateInit() is
> >> spawned already, but daemonRunStateInit() is in its very early
> >> stage (in the stack trace I see it's executing
> >> virIdentityGetSystem()).
> >>
> >> If SIGTERM (or any other signal that we don't override handler
> >> for) arrives at this point, the main thread jumps out from
> >> virEventRunDefaultImpl() and enters virStateShutdownPrepare()
> >> (via shutdownPrepareCb which was set earlier). This iterates
> >> through stateShutdownPrepare() callbacks of state drivers and
> >> reaching qemuStateShutdownPrepare() eventually only to
> >> dereference qemu_driver. But since thread 2) has not been
> >> scheduled/not proceeded yet, qemu_driver was not allocated yet.
> >>
> >> Solution is simple - just check if qemu_driver is not NULL. But
> >> doing so only in qemuStateShutdownPrepare() would push the
> >> problem down to virStateShutdownWait(), well
> >> qemuStateShutdownWait(). Therefore, duplicate the trick there
> >> too.
> >>
> >
> > I guess this is a partial solution. Initialization may be in a state when
> > qemu_driver is initialized but qemu_driver->workerPool is still NULL
> > for example.
>
> Yes.
>
> >
> > Maybe we'd better delay shutdown until initialization is finished?
>
> I'm not exactly sure how to achieve that. Do you have a hint? Also, part
> of qemu driver state init is autostarting domains (which may take ages).
>

I'm thinking of adding a new variable 'initialized' to virnetdaemon. It can
be
set by call from daemonRunStateInit after initialization is finished.
And we should call shutdownPrepare/shutdownWait only when both 'quit' and
'initialized' are true. It will require another pipe pair probably to wake
up
event loop thread when 'initialized' it set to true.

As initialization can take a lot of time (autostart as you mentioned) we
can arm finishTimer right when quit is set without waiting for
initialization
to finish. This way we exit ungracefully just as in other cases when
shutdown
finishing takes too much time. Libvirtd has a goal to handle crashes at
any time so this exit should be fine.

Nikolay

Re: [PATCH 00/55] Hyper-V: code cleanup & prep for future changes

[Laine Stump]

On 1/22/21 11:05 AM, Laine Stump wrote:
(Thought I sent this 7 hours ago before I went to sleep, but when I 
sat down this morning I saw it was still sitting there as a draft :-/)


On 1/21/21 1:50 PM, Matt Coleman wrote:

This series of patches simplifies the code in several ways and makes a
few changes required by the next round of patches that I'll submit.

Simplifications:

* add a macro to cut down on repetitive SettingData code
* enable GLib auto-cleanup for hypervObject and several OpenWSMAN types

Changes:

* store the version in hypervPrivate, which will be used to handle
   breaking changes in the Hyper-V API: despite 2012R2 and 2016+ all
   using Hyper-V's "V2" API, backwards-incompatible changes were made in
   2016
* add inheritance to the WMI generator to simplify handling of the
   backwards-incompatible changes introduced in Hyper-V 2016



I've gone through all of these, and just have two questions that 
affect multiple patches each (I've replied to the associated patches):



1) There are several cleanup functions in external libraries that in 
the past were only called after checking that the pointer was != NULL. 
g_autoptr cleanups need to handle being called with NULL as a NOP, and 
I'm concerned that these functions may not behave properly in that 
case. Can you either verify that it's safe to call them with NULL, or 
provide a wrapper function that checks for NULL and use that as the 
cleanup?



2) There are several places where you're returning a value that is 
retrieved from an object that is being auto-freed, and I don't know 
enough about the details of the teardown code that's generated by the 
compiler to be certain that the return value would be retrieved from 
the object *before* it's freed rather than *after*. If someone knows 
the answer to this, then that's great, otherwise someone should 
compile a test program and list out the assembly code using gdb to see 
what the order is.



I asked about item (2) on IRC just now, and danpb produced a short 
example program that proves it is okay to use values from auto-freed 
objects as the return value of a function. So there is only question (1) 
left. Let me know and I'll either push or wait for modified patches 
accordingly.






Once those two questions are resolved (possibly requiring no change to 
your patches), then



 Reviewed-by: Laine Stump 




Matt Coleman (55):
   hyperv: add a macro for retrieving setting data
   hyperv: store the Hyper-V version when connecting
   hyperv: add inheritance to the WMI generator
   hyperv: store hypervPrivate in hypervObject
   hyperv: enable use of g_autoptr for hypervObject
   hyperv: enable use of g_autoptr for the rest of the CIM/WMI classes
   hyperv: enable automatic cleanup for OpenWSMAN types
   hyperv: use g_autoptr for Win32_OperatingSystem in hypervConnectOpen
   hyperv: use g_autoptr for Win32_ComputerSystem in
 hypervConnectGetHostname
   hyperv: use g_autoptr for Msvm_ProcessorSettingData in
 hypervConnectGetMaxVcpus
   hyperv: use g_autoptr for WMI classes in hypervNodeGetInfo
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervConnectNumOfDomains
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervConnectListDomains
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervDomainLookupByID
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervDomainLookupByUUID
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervDomainLookupByName
   hyperv: use g_autoptr for Msvm_ComputerSystem in hypervDomainResume
   hyperv: use g_autoptr for WMI classes in hypervDomainShutdownFlags
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervDomainDestroyFlags
   hyperv: use g_autoptr for WMI classes in hypervDomainGetMaxMemory
   hyperv: use g_autoptr for WMI classes in 
hypervDomainSetMemoryProperty

   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervRequestStateChange
   hyperv: use g_autoptr for Win32_ComputerSystemProduct in
 hypervLookupHostSystemBiosUuid
   hyperv: use g_autoptr for Msvm_ResourceAllocationSettingData in
 hypervDomainAttachPhysicalDisk
   hyperv: use g_autoptr for WMI classes in hypervDomainAttachStorage
   hyperv: use g_autoptr for Msvm_DiskDrive in
 hypervDomainDefParsePhysicalDisk
   hyperv: use g_autoptr for WMI classes in hypervDomainGetInfo
   hyperv: use g_autoptr for Msvm_ComputerSystem in hypervDomainGetState
   hyperv: use g_autoptr for WMI classes in hypervDomainSetVcpusFlags
   hyperv: use g_autoptr for WMI classes in hypervDomainGetVcpusFlags
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervConnectListDefinedDomains
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervConnectNumOfDefinedDomains
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervDomainCreateWithFlags
   hyperv: use g_autoptr for Msvm_VirtualSystemSettingData in
 hypervDomainGetAutostart
   hyperv: use g_autoptr for Msvm_VirtualSystemSettingData in
 

Re: [PATCH 00/55] Hyper-V: code cleanup & prep for future changes

[Daniel P . Berrangé]

On Fri, Jan 22, 2021 at 11:05:22AM -0500, Laine Stump wrote:
> (Thought I sent this 7 hours ago before I went to sleep, but when I sat down
> this morning I saw it was still sitting there as a draft :-/)
> 
> On 1/21/21 1:50 PM, Matt Coleman wrote:
> > This series of patches simplifies the code in several ways and makes a
> > few changes required by the next round of patches that I'll submit.
> > 
> > Simplifications:
> > 
> > * add a macro to cut down on repetitive SettingData code
> > * enable GLib auto-cleanup for hypervObject and several OpenWSMAN types
> > 
> > Changes:
> > 
> > * store the version in hypervPrivate, which will be used to handle
> >breaking changes in the Hyper-V API: despite 2012R2 and 2016+ all
> >using Hyper-V's "V2" API, backwards-incompatible changes were made in
> >2016
> > * add inheritance to the WMI generator to simplify handling of the
> >backwards-incompatible changes introduced in Hyper-V 2016
> 
> 
> I've gone through all of these, and just have two questions that affect
> multiple patches each (I've replied to the associated patches):
> 
> 
> 1) There are several cleanup functions in external libraries that in the
> past were only called after checking that the pointer was != NULL. g_autoptr
> cleanups need to handle being called with NULL as a NOP, and I'm concerned
> that these functions may not behave properly in that case. Can you either
> verify that it's safe to call them with NULL, or provide a wrapper function
> that checks for NULL and use that as the cleanup?

The G_DEFINE_AUTOPTR_*  macros alrady define wrappers that include
a NULL check I believe.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [PATCH 00/55] Hyper-V: code cleanup & prep for future changes

[Laine Stump]

(Thought I sent this 7 hours ago before I went to sleep, but when I sat 
down this morning I saw it was still sitting there as a draft :-/)


On 1/21/21 1:50 PM, Matt Coleman wrote:

This series of patches simplifies the code in several ways and makes a
few changes required by the next round of patches that I'll submit.

Simplifications:

* add a macro to cut down on repetitive SettingData code
* enable GLib auto-cleanup for hypervObject and several OpenWSMAN types

Changes:

* store the version in hypervPrivate, which will be used to handle
   breaking changes in the Hyper-V API: despite 2012R2 and 2016+ all
   using Hyper-V's "V2" API, backwards-incompatible changes were made in
   2016
* add inheritance to the WMI generator to simplify handling of the
   backwards-incompatible changes introduced in Hyper-V 2016



I've gone through all of these, and just have two questions that affect 
multiple patches each (I've replied to the associated patches):



1) There are several cleanup functions in external libraries that in the 
past were only called after checking that the pointer was != NULL. 
g_autoptr cleanups need to handle being called with NULL as a NOP, and 
I'm concerned that these functions may not behave properly in that case. 
Can you either verify that it's safe to call them with NULL, or provide 
a wrapper function that checks for NULL and use that as the cleanup?



2) There are several places where you're returning a value that is 
retrieved from an object that is being auto-freed, and I don't know 
enough about the details of the teardown code that's generated by the 
compiler to be certain that the return value would be retrieved from the 
object *before* it's freed rather than *after*. If someone knows the 
answer to this, then that's great, otherwise someone should compile a 
test program and list out the assembly code using gdb to see what the 
order is.



Once those two questions are resolved (possibly requiring no change to 
your patches), then



 Reviewed-by: Laine Stump 




Matt Coleman (55):
   hyperv: add a macro for retrieving setting data
   hyperv: store the Hyper-V version when connecting
   hyperv: add inheritance to the WMI generator
   hyperv: store hypervPrivate in hypervObject
   hyperv: enable use of g_autoptr for hypervObject
   hyperv: enable use of g_autoptr for the rest of the CIM/WMI classes
   hyperv: enable automatic cleanup for OpenWSMAN types
   hyperv: use g_autoptr for Win32_OperatingSystem in hypervConnectOpen
   hyperv: use g_autoptr for Win32_ComputerSystem in
 hypervConnectGetHostname
   hyperv: use g_autoptr for Msvm_ProcessorSettingData in
 hypervConnectGetMaxVcpus
   hyperv: use g_autoptr for WMI classes in hypervNodeGetInfo
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervConnectNumOfDomains
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervConnectListDomains
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervDomainLookupByID
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervDomainLookupByUUID
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervDomainLookupByName
   hyperv: use g_autoptr for Msvm_ComputerSystem in hypervDomainResume
   hyperv: use g_autoptr for WMI classes in hypervDomainShutdownFlags
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervDomainDestroyFlags
   hyperv: use g_autoptr for WMI classes in hypervDomainGetMaxMemory
   hyperv: use g_autoptr for WMI classes in hypervDomainSetMemoryProperty
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervRequestStateChange
   hyperv: use g_autoptr for Win32_ComputerSystemProduct in
 hypervLookupHostSystemBiosUuid
   hyperv: use g_autoptr for Msvm_ResourceAllocationSettingData in
 hypervDomainAttachPhysicalDisk
   hyperv: use g_autoptr for WMI classes in hypervDomainAttachStorage
   hyperv: use g_autoptr for Msvm_DiskDrive in
 hypervDomainDefParsePhysicalDisk
   hyperv: use g_autoptr for WMI classes in hypervDomainGetInfo
   hyperv: use g_autoptr for Msvm_ComputerSystem in hypervDomainGetState
   hyperv: use g_autoptr for WMI classes in hypervDomainSetVcpusFlags
   hyperv: use g_autoptr for WMI classes in hypervDomainGetVcpusFlags
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervConnectListDefinedDomains
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervConnectNumOfDefinedDomains
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervDomainCreateWithFlags
   hyperv: use g_autoptr for Msvm_VirtualSystemSettingData in
 hypervDomainGetAutostart
   hyperv: use g_autoptr for Msvm_VirtualSystemSettingData in
 hypervDomainSetAutostart
   hyperv: use g_autoptr for WMI classes in
 hypervDomainGetSchedulerParametersFlags
   hyperv: use g_autoptr for Msvm_ComputerSystem in hypervDomainIsActive
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 hypervDomainManagedSave
   hyperv: use g_autoptr for Msvm_ComputerSystem in
 

Re: [PATCH] qemu: Avoid crash in qemuStateShutdownPrepare() and qemuStateShutdownWait()

[Nikolay Shirokovskiy]

On Fri, Jan 22, 2021 at 12:45 PM Michal Privoznik 
wrote:

> If libvirtd is sent SIGTERM while it is still initializing, it
> may crash. The following scenario was observed (using 'stress' to
> slow down CPU so much that the window where the problem exists is
> bigger):
>
> 1) The main thread is already executing virNetDaemonRun() and is
>in virEventRunDefaultImpl().
> 2) The thread that's supposed to run daemonRunStateInit() is
>spawned already, but daemonRunStateInit() is in its very early
>stage (in the stack trace I see it's executing
>virIdentityGetSystem()).
>
> If SIGTERM (or any other signal that we don't override handler
> for) arrives at this point, the main thread jumps out from
> virEventRunDefaultImpl() and enters virStateShutdownPrepare()
> (via shutdownPrepareCb which was set earlier). This iterates
> through stateShutdownPrepare() callbacks of state drivers and
> reaching qemuStateShutdownPrepare() eventually only to
> dereference qemu_driver. But since thread 2) has not been
> scheduled/not proceeded yet, qemu_driver was not allocated yet.
>
> Solution is simple - just check if qemu_driver is not NULL. But
> doing so only in qemuStateShutdownPrepare() would push the
> problem down to virStateShutdownWait(), well
> qemuStateShutdownWait(). Therefore, duplicate the trick there
> too.
>

I guess this is a partial solution. Initialization may be in a state when
qemu_driver is initialized but qemu_driver->workerPool is still NULL
for example.

Maybe we'd better delay shutdown until initialization is finished?

Nikolay



>
> Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1895359#c14
> Signed-off-by
> :
> Michal Privoznik 
> ---
>  src/qemu/qemu_driver.c | 6 ++
>  1 file changed, 6 insertions(+)
>
> diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
> index 027617deef..ca4f366323 100644
> --- a/src/qemu/qemu_driver.c
> +++ b/src/qemu/qemu_driver.c
> @@ -1072,6 +1072,9 @@ qemuStateStop(void)
>  static int
>  qemuStateShutdownPrepare(void)
>  {
> +if (!qemu_driver)
> +return 0;
> +
>  virThreadPoolStop(qemu_driver->workerPool);
>  return 0;
>  }
> @@ -1091,6 +1094,9 @@ qemuDomainObjStopWorkerIter(virDomainObjPtr vm,
>  static int
>  qemuStateShutdownWait(void)
>  {
> +if (!qemu_driver)
> +return 0;
> +
>  virDomainObjListForEach(qemu_driver->domains, false,
>  qemuDomainObjStopWorkerIter, NULL);
>  virThreadPoolDrain(qemu_driver->workerPool);
> --
> 2.26.2
>
>

Re: [PATCH 00/10] Introduce virtio-mem model

[David Hildenbrand]

On 22.01.21 13:50, Michal Privoznik wrote:
> Technically, this is another version of:
> 
> https://www.redhat.com/archives/libvir-list/2020-December/msg00199.html
> 
> But since virtio-pmem part is pushed now, I've reworked virtio-mem a bit
> and sending it as a new series.
> 
> For curious ones, David summarized behaviour well when implementing
> virtio-mem support in kernel:
> 
> https://lwn.net/Articles/755423/

... and for the really curious ones (because some details in that patch
are a little outdated), there is plenty more at:

https://virtio-mem.gitlab.io/

Thanks for all your effort Michal!

-- 
Thanks,

David / dhildenb

[PATCH 07/10] qemu: Refresh the actual size of virtio-mem on monitor reconnect

[Michal Privoznik]

If the QEMU driver restarts it loses the track of the actual size
of virtio-mem (because it's runtime type of information and thus
not stored in XML) and therefore, we have to refresh it when
reconnecting to the domain monitor.

Signed-off-by: Michal Privoznik 
---
 src/qemu/qemu_domain.c   | 37 
 src/qemu/qemu_monitor.h  |  3 ++
 src/qemu/qemu_monitor_json.c | 55 +---
 src/qemu/qemu_process.c  |  3 ++
 4 files changed, 70 insertions(+), 28 deletions(-)

diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index 783756b191..5c40c02180 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -7992,9 +7992,21 @@ qemuDomainUpdateMemoryDeviceInfo(virQEMUDriverPtr driver,
 return -1;
 }
 
-/* if qemu doesn't support the info request, just carry on */
-if (rc == -2)
+/* If qemu doesn't support the info request, just carry on, unless we
+ * really need it. */
+if (rc == -2) {
+for (i = 0; i < vm->def->nmems; i++) {
+virDomainMemoryDefPtr mem = vm->def->mems[i];
+
+if (mem->model == VIR_DOMAIN_MEMORY_MODEL_VIRTIO_MEM) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("qemu did not return info on vitio-mem 
device"));
+return -1;
+}
+}
+
 return 0;
+}
 
 if (rc < 0)
 return -1;
@@ -8009,9 +8021,24 @@ qemuDomainUpdateMemoryDeviceInfo(virQEMUDriverPtr driver,
 if (!(dimm = virHashLookup(meminfo, mem->info.alias)))
 continue;
 
-mem->info.type = VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DIMM;
-mem->info.addr.dimm.slot = dimm->slot;
-mem->info.addr.dimm.base = dimm->address;
+switch (mem->model) {
+case VIR_DOMAIN_MEMORY_MODEL_VIRTIO_MEM:
+mem->actualsize = VIR_DIV_UP(dimm->size, 1024);
+break;
+
+case VIR_DOMAIN_MEMORY_MODEL_DIMM:
+case VIR_DOMAIN_MEMORY_MODEL_NVDIMM:
+mem->info.type = VIR_DOMAIN_DEVICE_ADDRESS_TYPE_DIMM;
+mem->info.addr.dimm.slot = dimm->slot;
+mem->info.addr.dimm.base = dimm->address;
+break;
+
+case VIR_DOMAIN_MEMORY_MODEL_VIRTIO_PMEM:
+case VIR_DOMAIN_MEMORY_MODEL_NONE:
+case VIR_DOMAIN_MEMORY_MODEL_LAST:
+/* nada */
+break;
+}
 }
 
 virHashFree(meminfo);
diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h
index 00428c14d2..9668e287a1 100644
--- a/src/qemu/qemu_monitor.h
+++ b/src/qemu/qemu_monitor.h
@@ -1391,10 +1391,13 @@ typedef struct _qemuMonitorMemoryDeviceInfo 
qemuMonitorMemoryDeviceInfo;
 typedef qemuMonitorMemoryDeviceInfo *qemuMonitorMemoryDeviceInfoPtr;
 
 struct _qemuMonitorMemoryDeviceInfo {
+/* For pc-dimm */
 unsigned long long address;
 unsigned int slot;
 bool hotplugged;
 bool hotpluggable;
+/* For virtio-mem */
+unsigned long long size;
 };
 
 int qemuMonitorGetMemoryDeviceInfo(qemuMonitorPtr mon,
diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c
index b1dc527e8b..1922f84a5e 100644
--- a/src/qemu/qemu_monitor_json.c
+++ b/src/qemu/qemu_monitor_json.c
@@ -8228,7 +8228,6 @@ qemuMonitorJSONGetMemoryDeviceInfo(qemuMonitorPtr mon,
 virJSONValuePtr cmd;
 virJSONValuePtr reply = NULL;
 virJSONValuePtr data = NULL;
-qemuMonitorMemoryDeviceInfoPtr meminfo = NULL;
 size_t i;
 
 if (!(cmd = qemuMonitorJSONMakeCommand("query-memory-devices", NULL)))
@@ -8249,6 +8248,9 @@ qemuMonitorJSONGetMemoryDeviceInfo(qemuMonitorPtr mon,
 
 for (i = 0; i < virJSONValueArraySize(data); i++) {
 virJSONValuePtr elem = virJSONValueArrayGet(data, i);
+g_autofree qemuMonitorMemoryDeviceInfoPtr meminfo = NULL;
+virJSONValuePtr dimminfo;
+const char *devalias;
 const char *type;
 
 if (!(type = virJSONValueObjectGetString(elem, "type"))) {
@@ -8258,26 +8260,23 @@ qemuMonitorJSONGetMemoryDeviceInfo(qemuMonitorPtr mon,
 goto cleanup;
 }
 
+if (!(dimminfo = virJSONValueObjectGetObject(elem, "data"))) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("query-memory-devices reply data doesn't "
+ "contain enum data"));
+goto cleanup;
+}
+
+if (!(devalias = virJSONValueObjectGetString(dimminfo, "id"))) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("dimm memory info data is missing 'id'"));
+goto cleanup;
+}
+
+meminfo = g_new0(qemuMonitorMemoryDeviceInfo, 1);
+
 /* dimm memory devices */
 if (STREQ(type, "dimm")) {
-virJSONValuePtr dimminfo;
-const char *devalias;
-
-if (!(dimminfo = virJSONValueObjectGetObject(elem, "data"))) {
-virReportError(VIR_ERR_INTERNAL_ERROR, 

[PATCH 06/10] qemu: Wire up MEMORY_DEVICE_SIZE_CHANGE event

[Michal Privoznik]

As advertised in previous commit, this event is delivered to us
when virtio-mem module changes the allocation inside the guest.
It comes with one attribute - size - which holds the new size of
the virtio-mem (well, allocated size), in bytes.
Mind you, this is not necessarily the same number as 'requested
size'. It almost certainly will be when sizing the memory up, but
it might not be when sizing the memory down - the guest kernel
might be unable to free some blocks.

This actual size is reported in the domain XML as an output
element only.

Signed-off-by: Michal Privoznik 
---
 docs/formatdomain.rst |  7 ++
 docs/schemas/domaincommon.rng |  5 +
 src/conf/domain_conf.c| 24 ++--
 src/conf/domain_conf.h|  7 ++
 src/libvirt_private.syms  |  1 +
 src/qemu/qemu_domain.c|  3 +++
 src/qemu/qemu_domain.h|  1 +
 src/qemu/qemu_driver.c| 33 
 src/qemu/qemu_monitor.c   | 24 
 src/qemu/qemu_monitor.h   | 20 +
 src/qemu/qemu_monitor_json.c  | 24 
 src/qemu/qemu_process.c   | 41 +++
 12 files changed, 188 insertions(+), 2 deletions(-)

diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst
index 2938758ec2..3088da1243 100644
--- a/docs/formatdomain.rst
+++ b/docs/formatdomain.rst
@@ -7277,6 +7277,7 @@ Example: usage of the memory devices
  0
  2048
  1048576
+ 524288

  

@@ -7391,6 +7392,12 @@ Example: usage of the memory devices
  The total size of blocks exposed to the guest. Must respect ``block``
  granularity.
 
+   ``actual``
+ Active XML for ``virtio-mem`` model may contain ``actual`` element that
+ reflects the actual size of the corresponding virtio memory device. The
+ element is formatted into live XML and never parsed, i.e. it is
+ output-only element.
+
 :anchor:``
 
 IOMMU devices
diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng
index 5bc120073e..6d5c983bc6 100644
--- a/docs/schemas/domaincommon.rng
+++ b/docs/schemas/domaincommon.rng
@@ -6115,6 +6115,11 @@
 
   
 
+
+  
+
+  
+
 
   
 
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index b6fe5e4436..74c897b53e 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -17320,6 +17320,21 @@ virDomainMemoryFindByDeviceInfo(virDomainDefPtr def,
 }
 
 
+ssize_t
+virDomainMemoryFindByDeviceAlias(virDomainDefPtr def,
+ const char *alias)
+{
+size_t i;
+
+for (i = 0; i < def->nmems; i++) {
+if (STREQ_NULLABLE(def->mems[i]->info.alias, alias))
+return i;
+}
+
+return -1;
+}
+
+
 /**
  * virDomainMemoryInsert:
  *
@@ -26611,7 +26626,8 @@ virDomainMemorySourceDefFormat(virBufferPtr buf,
 
 static void
 virDomainMemoryTargetDefFormat(virBufferPtr buf,
-   virDomainMemoryDefPtr def)
+   virDomainMemoryDefPtr def,
+   unsigned int flags)
 {
 g_auto(virBuffer) childBuf = VIR_BUFFER_INIT_CHILD(buf);
 
@@ -26633,6 +26649,10 @@ virDomainMemoryTargetDefFormat(virBufferPtr buf,
 
 virBufferAsprintf(, "%llu\n",
   def->requestedsize);
+if (!(flags & VIR_DOMAIN_DEF_FORMAT_INACTIVE)) {
+virBufferAsprintf(, "%llu\n",
+  def->actualsize);
+}
 }
 
 virXMLFormatElement(buf, "target", NULL, );
@@ -26665,7 +26685,7 @@ virDomainMemoryDefFormat(virBufferPtr buf,
 if (virDomainMemorySourceDefFormat(buf, def) < 0)
 return -1;
 
-virDomainMemoryTargetDefFormat(buf, def);
+virDomainMemoryTargetDefFormat(buf, def, flags);
 
 virDomainDeviceInfoFormat(buf, >info, flags);
 
diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h
index ef52328a6f..5f4a455963 100644
--- a/src/conf/domain_conf.h
+++ b/src/conf/domain_conf.h
@@ -2331,6 +2331,9 @@ struct _virDomainMemoryDef {
 unsigned long long labelsize; /* kibibytes; valid only for NVDIMM */
 unsigned long long blocksize; /* kibibytes; valid only for VIRTIO_MEM */
 unsigned long long requestedsize; /* kibibytes; valid only for VIRTIO_MEM 
*/
+unsigned long long actualsize; /* kibibytes, valid for VIRTIO_MEM and
+  active domain only, only to report never
+  parse */
 bool readonly; /* valid only for NVDIMM */
 
 /* required for QEMU NVDIMM ppc64 support */
@@ -3613,6 +3616,10 @@ ssize_t virDomainMemoryFindByDeviceInfo(virDomainDefPtr 
dev,
 virDomainDeviceInfoPtr info)
 ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT;
 
+ssize_t virDomainMemoryFindByDeviceAlias(virDomainDefPtr def,
+   

[PATCH 10/10] news: document recent virtio memory addition

[Michal Privoznik]

Signed-off-by: Michal Privoznik 
---
 NEWS.rst | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/NEWS.rst b/NEWS.rst
index 7a2d6649b4..33f316c8d4 100644
--- a/NEWS.rst
+++ b/NEWS.rst
@@ -18,6 +18,13 @@ v7.1.0 (unreleased)
 The virtio-pmem is a virtio variant of NVDIMM and just like NVDIMM
 virtio-pmem also allows accessing host pages bypassing guest page cache.
 
+  * Introduce virtio-mem  model
+
+New virtio-mem model is introduced for  device which is a
+paravirtualized mechanism of adding/removing memory to/from a VM. Use
+``virDomainUpdateDeviceFlags()`` API to adjust amount of memory or ``virsh
+update-memory`` for convenience.
+
 * **Improvements**
 
 * **Bug fixes**
-- 
2.26.2

[PATCH 09/10] virsh: Introduce update-memory command

[Michal Privoznik]

New 'update-memory' command is introduced which aims on making it
user friendly to change  device. So far I just need to
change  so I'm introducing --requested-size only; but
the idea is that this is extensible for other cases too. For
instance, want to change ? Nnew --my-element
argument can be easily introduced.

Signed-off-by: Michal Privoznik 
---
 docs/manpages/virsh.rst |  31 
 tools/virsh-domain.c| 154 
 2 files changed, 185 insertions(+)

diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst
index e3afa48f7b..32639e34ff 100644
--- a/docs/manpages/virsh.rst
+++ b/docs/manpages/virsh.rst
@@ -4891,6 +4891,37 @@ results as some fields may be autogenerated and thus 
match devices other than
 expected.
 
 
+update-memory
+-
+
+**Syntax:**
+
+::
+
+   update-memory domain [--print-xml] [--alias alias]
+ [[--live] [--config] | [--current]]
+ [--requested-size size]
+
+Update values for a  device. Not to be confused with overall
+domain memory which is tuned via ``setmem`` and ``setmaxmem``.
+This command finds  device inside given *domain*, changes
+requested values and passes updated device XML to daemon. If *--print-xml* is
+specified then the device is not changed, but the updated device XML is printed
+to stdout.  If there are more than one  devices in *domain* use
+*--alias* to select the desired one.
+
+If *--live* is specified, affect a running domain.
+If *--config* is specified, affect the next startup of a persistent guest.
+If *--current* is specified, it is equivalent to either *--live* or
+*--config*, depending on the current state of the guest.
+Both *--live* and *--config* flags may be given, but *--current* is
+exclusive. Not specifying any flag is the same as specifying *--current*.
+
+If *--requested-size* is specified then  under memory target is
+changed to requested *size* (as scaled integer, see ``NOTES`` above). It
+defaults to kibibytes if no suffix is provided.
+
+
 change-media
 
 
diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c
index 9746117bdb..0b32e6f408 100644
--- a/tools/virsh-domain.c
+++ b/tools/virsh-domain.c
@@ -9128,6 +9128,154 @@ cmdSetmaxmem(vshControl *ctl, const vshCmd *cmd)
 return ret;
 }
 
+
+/*
+ * "update-memory" command
+ */
+static const vshCmdInfo info_update_memory[] = {
+{.name = "help",
+ .data = N_("update memory device of a domain")
+},
+{.name = "desc",
+ .data = N_("Update values of a memory device of a domain")
+},
+{.name = NULL}
+};
+
+static const vshCmdOptDef opts_update_memory[] = {
+VIRSH_COMMON_OPT_DOMAIN_FULL(0),
+VIRSH_COMMON_OPT_DOMAIN_CONFIG,
+VIRSH_COMMON_OPT_DOMAIN_LIVE,
+VIRSH_COMMON_OPT_DOMAIN_CURRENT,
+{.name = "print-xml",
+ .type = VSH_OT_BOOL,
+ .help = N_("print updated memory device XML instead of executing the 
change")
+},
+{.name = "alias",
+ .type = VSH_OT_STRING,
+ .completer = virshDomainDeviceAliasCompleter,
+ .help = N_("memory device alias"),
+},
+{.name = "requested-size",
+ .type = VSH_OT_INT,
+ .help = N_("new value of  size, as scaled integer (default 
KiB)")
+},
+{.name = NULL}
+};
+
+static int
+virshGetUpdatedMemoryXML(char **updatedMemoryXML,
+ vshControl *ctl,
+ const vshCmd *cmd,
+ virDomainPtr dom,
+ unsigned int flags)
+{
+const char *alias = NULL;
+g_autoptr(xmlDoc) doc = NULL;
+g_autoptr(xmlXPathContext) ctxt = NULL;
+g_autofree char *xpath = NULL;
+int nmems;
+g_autofree xmlNodePtr *mems = NULL;
+g_autoptr(xmlBuffer) xmlbuf = NULL;
+unsigned int domainXMLFlags = 0;
+
+if (flags & VIR_DOMAIN_AFFECT_CONFIG)
+domainXMLFlags |= VIR_DOMAIN_XML_INACTIVE;
+
+if (virshDomainGetXMLFromDom(ctl, dom, domainXMLFlags, , ) < 0)
+return -1;
+
+if (vshCommandOptStringReq(ctl, cmd, "alias", ) < 0)
+return -1;
+
+if (alias) {
+xpath = g_strdup_printf("/domain/devices/memory[./alias/@name='%s']", 
alias);
+} else {
+xpath = g_strdup("/domain/devices/memory");
+}
+
+nmems = virXPathNodeSet(xpath, ctxt, );
+if (nmems < 0) {
+vshSaveLibvirtError();
+return -1;
+} else if (nmems == 0) {
+vshError(ctl, _("no memory device found"));
+return -1;
+} else if (nmems > 1) {
+vshError(ctl, _("multiple memory devices found, use --alias to select 
one"));
+return -1;
+}
+
+ctxt->node = mems[0];
+
+if (vshCommandOptBool(cmd, "requested-size")) {
+xmlNodePtr requestedSizeNode;
+g_autofree char *kibibytesStr = NULL;
+unsigned long long bytes = 0;
+unsigned long kibibytes = 0;
+
+if (vshCommandOptScaledInt(ctl, cmd, "requested-size", , 1024, 
ULLONG_MAX) < 0)
+return -1;
+kibibytes = VIR_DIV_UP(bytes, 

[PATCH 08/10] qemu: Recalculate balloon on MEMORY_DEVICE_SIZE_CHANGE event and reconnect

[Michal Privoznik]

Just like we are recalculating the amount of guest memory on
BALLOON_CHANGE and on reconnect to the monitor, we should include
the actual size of virtio-mem too.

Signed-off-by: Michal Privoznik 
---
 src/qemu/qemu_driver.c  |  3 +++
 src/qemu/qemu_process.c | 57 +
 2 files changed, 50 insertions(+), 10 deletions(-)

diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index d64eb4d399..2fd4429ba8 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -4298,6 +4298,9 @@ processMemoryDeviceSizeChange(virQEMUDriverPtr driver,
 mem = vm->def->mems[idx];
 mem->actualsize = VIR_DIV_UP(info->size, 1024);
 
+/* fix the balloon size */
+ignore_value(qemuProcessRefreshBalloonState(driver, vm, 
QEMU_ASYNC_JOB_NONE));
+
  endjob:
 qemuDomainObjEndJob(driver, vm);
 }
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index 8d41f947af..01d261d538 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -1250,10 +1250,31 @@ qemuProcessHandleBalloonChange(qemuMonitorPtr mon 
G_GNUC_UNUSED,
 virQEMUDriverPtr driver = opaque;
 virObjectEventPtr event = NULL;
 g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver);
+size_t i;
 
 virObjectLock(vm);
 event = virDomainEventBalloonChangeNewFromObj(vm, actual);
 
+VIR_DEBUG("New balloon size before fixup: %lld", actual);
+
+for (i = 0; i < vm->def->nmems; i++) {
+virDomainMemoryDefPtr mem = vm->def->mems[i];
+
+switch (mem->model) {
+case VIR_DOMAIN_MEMORY_MODEL_VIRTIO_MEM:
+actual += mem->actualsize;
+break;
+
+case VIR_DOMAIN_MEMORY_MODEL_NONE:
+case VIR_DOMAIN_MEMORY_MODEL_DIMM:
+case VIR_DOMAIN_MEMORY_MODEL_NVDIMM:
+case VIR_DOMAIN_MEMORY_MODEL_VIRTIO_PMEM:
+case VIR_DOMAIN_MEMORY_MODEL_LAST:
+/* nada */
+break;
+}
+}
+
 VIR_DEBUG("Updating balloon from %lld to %lld kb",
   vm->def->mem.cur_balloon, actual);
 vm->def->mem.cur_balloon = actual;
@@ -2451,21 +2472,37 @@ qemuProcessRefreshBalloonState(virQEMUDriverPtr driver,
int asyncJob)
 {
 unsigned long long balloon;
+size_t i;
 int rc;
 
-/* if no ballooning is available, the current size equals to the current
- * full memory size */
-if (!virDomainDefHasMemballoon(vm->def)) {
-vm->def->mem.cur_balloon = virDomainDefGetMemoryTotal(vm->def);
-return 0;
+if (virDomainDefHasMemballoon(vm->def)) {
+if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) < 0)
+return -1;
+
+rc = qemuMonitorGetBalloonInfo(qemuDomainGetMonitor(vm), );
+if (qemuDomainObjExitMonitor(driver, vm) < 0 || rc < 0)
+return -1;
+} else {
+balloon = virDomainDefGetMemoryTotal(vm->def);
 }
 
-if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) < 0)
-return -1;
+for (i = 0; i < vm->def->nmems; i++) {
+virDomainMemoryDefPtr mem = vm->def->mems[i];
+
+switch (mem->model) {
+case VIR_DOMAIN_MEMORY_MODEL_VIRTIO_MEM:
+balloon += mem->actualsize;
+break;
 
-rc = qemuMonitorGetBalloonInfo(qemuDomainGetMonitor(vm), );
-if (qemuDomainObjExitMonitor(driver, vm) < 0 || rc < 0)
-return -1;
+case VIR_DOMAIN_MEMORY_MODEL_NONE:
+case VIR_DOMAIN_MEMORY_MODEL_DIMM:
+case VIR_DOMAIN_MEMORY_MODEL_NVDIMM:
+case VIR_DOMAIN_MEMORY_MODEL_VIRTIO_PMEM:
+case VIR_DOMAIN_MEMORY_MODEL_LAST:
+/* nada */
+break;
+}
+}
 
 vm->def->mem.cur_balloon = balloon;
 
-- 
2.26.2

[PATCH 05/10] qemu: Wire up live update

[Michal Privoznik]

As advertised in one of previous commits, we want' to be able to
change 'requested-size' attribute of virtio-mem on the fly. This
commit does exactly that. Changing anything else is checked for
and forbidden.

Once guest has changed the allocation, QEMU emits an event which
we will use to track the allocation. In the next commit.

Signed-off-by: Michal Privoznik 
---
 src/conf/domain_conf.c   |  23 +
 src/conf/domain_conf.h   |   3 +
 src/libvirt_private.syms |   1 +
 src/qemu/qemu_driver.c   | 175 ++-
 src/qemu/qemu_hotplug.c  |  18 
 src/qemu/qemu_hotplug.h  |   5 +
 src/qemu/qemu_monitor.c  |  13 +++
 src/qemu/qemu_monitor.h  |   4 +
 src/qemu/qemu_monitor_json.c |  15 +++
 src/qemu/qemu_monitor_json.h |   5 +
 10 files changed, 261 insertions(+), 1 deletion(-)

diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index f8c5a40b24..b6fe5e4436 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -17297,6 +17297,29 @@ virDomainMemoryFindInactiveByDef(virDomainDefPtr def,
 }
 
 
+ssize_t
+virDomainMemoryFindByDeviceInfo(virDomainDefPtr def,
+virDomainDeviceInfoPtr info)
+{
+size_t i;
+
+for (i = 0; i < def->nmems; i++) {
+virDomainMemoryDefPtr tmp = def->mems[i];
+
+if (!virDomainDeviceInfoAddressIsEqual(>info, info))
+continue;
+
+/* alias, if present */
+if (STRNEQ_NULLABLE(tmp->info.alias, info->alias))
+continue;
+
+return i;
+}
+
+return -1;
+}
+
+
 /**
  * virDomainMemoryInsert:
  *
diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h
index 5d89ecfe9d..ef52328a6f 100644
--- a/src/conf/domain_conf.h
+++ b/src/conf/domain_conf.h
@@ -3609,6 +3609,9 @@ int virDomainMemoryFindByDef(virDomainDefPtr def, 
virDomainMemoryDefPtr mem)
 int virDomainMemoryFindInactiveByDef(virDomainDefPtr def,
  virDomainMemoryDefPtr mem)
 ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT;
+ssize_t virDomainMemoryFindByDeviceInfo(virDomainDefPtr dev,
+virDomainDeviceInfoPtr info)
+ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT;
 
 int virDomainShmemDefInsert(virDomainDefPtr def, virDomainShmemDefPtr shmem)
 ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) G_GNUC_WARN_UNUSED_RESULT;
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index 962d82680e..2e7f92bcfe 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -494,6 +494,7 @@ virDomainMemballoonModelTypeFromString;
 virDomainMemballoonModelTypeToString;
 virDomainMemoryDefFree;
 virDomainMemoryFindByDef;
+virDomainMemoryFindByDeviceInfo;
 virDomainMemoryFindInactiveByDef;
 virDomainMemoryInsert;
 virDomainMemoryModelTypeToString;
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index ed966cf7e3..fadf0240fc 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -7087,6 +7087,168 @@ qemuDomainChangeDiskLive(virDomainObjPtr vm,
 return 0;
 }
 
+
+static bool
+qemuDomainChangeMemoryLiveValidateChange(const virDomainMemoryDef *oldDef,
+ const virDomainMemoryDef *newDef)
+{
+/* The only thing that is allowed to change is 'requestedsize' for virtio
+ * model. */
+if (oldDef->model != newDef->model) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("cannot modify memory model from '%s' to '%s'"),
+   virDomainMemoryModelTypeToString(oldDef->model),
+   virDomainMemoryModelTypeToString(newDef->model));
+return false;
+}
+
+if (oldDef->access != newDef->access) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("cannot modify memory access from '%s' to '%s'"),
+   virDomainMemoryAccessTypeToString(oldDef->access),
+   virDomainMemoryAccessTypeToString(newDef->access));
+return false;
+}
+
+if (oldDef->discard != newDef->discard) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("cannot modify memory discard from '%s' to '%s'"),
+   virTristateBoolTypeToString(oldDef->discard),
+   virTristateBoolTypeToString(newDef->discard));
+return false;
+}
+
+if (!virBitmapEqual(oldDef->sourceNodes,
+newDef->sourceNodes)) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
+   _("cannot modify memory source nodes"));
+return false;
+}
+
+if (oldDef->pagesize != newDef->pagesize) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("cannot modify memory pagesize from '%llu' to 
'%llu'"),
+   oldDef->pagesize,
+   newDef->pagesize);
+return false;
+}
+
+if 

[PATCH 04/10] qemu: Build command line for virtio-mem

[Michal Privoznik]

Nothing special is happening here. All important changes were
done when for 'virtio-pmem' (adjusting the code to put virtio
memory on PCI bus, generating alias using
qemuDomainDeviceAliasIndex(). The only bit that might look
suspicious is no prealloc for virtio-mem. But if you think about
it, the whole purpose of this device is to change amount of
memory exposed to guest on the fly. There is no point in locking
the whole backend in memory.

Signed-off-by: Michal Privoznik 
---
 src/qemu/qemu_alias.c |  9 +++-
 src/qemu/qemu_command.c   | 12 -
 ...mory-hotplug-virtio-mem.x86_64-latest.args | 49 +++
 tests/qemuxml2argvtest.c  |  1 +
 4 files changed, 68 insertions(+), 3 deletions(-)
 create mode 100644 
tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args

diff --git a/src/qemu/qemu_alias.c b/src/qemu/qemu_alias.c
index 2a4e2393c8..e539bf9c8e 100644
--- a/src/qemu/qemu_alias.c
+++ b/src/qemu/qemu_alias.c
@@ -475,8 +475,11 @@ qemuDeviceMemoryGetAliasID(virDomainDefPtr def,
 size_t i;
 int maxidx = 0;
 
-/* virtio-pmem goes onto PCI bus and thus DIMM address is not valid */
-if (!oldAlias && mem->model != VIR_DOMAIN_MEMORY_MODEL_VIRTIO_PMEM)
+/* virtio-pmem and virtio-mem go onto PCI bus and thus DIMM address is not
+ * valid */
+if (!oldAlias &&
+mem->model != VIR_DOMAIN_MEMORY_MODEL_VIRTIO_PMEM &&
+mem->model != VIR_DOMAIN_MEMORY_MODEL_VIRTIO_MEM)
 return mem->info.addr.dimm.slot;
 
 for (i = 0; i < def->nmems; i++) {
@@ -523,6 +526,8 @@ qemuAssignDeviceMemoryAlias(virDomainDefPtr def,
 prefix = "virtiopmem";
 break;
 case VIR_DOMAIN_MEMORY_MODEL_VIRTIO_MEM:
+prefix = "virtiomem";
+break;
 case VIR_DOMAIN_MEMORY_MODEL_NONE:
 case VIR_DOMAIN_MEMORY_MODEL_LAST:
 default:
diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index 0a4a8f2646..4e50dbc0fd 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -3094,7 +3094,9 @@ qemuBuildMemoryBackendProps(virJSONValuePtr *backendProps,
 } else if (useHugepage) {
 if (qemuGetDomainHupageMemPath(priv->driver, def, pagesize, 
) < 0)
 return -1;
-prealloc = true;
+/* For virtio-mem backed by hugepages we don't need prealloc. */
+if (mem->model != VIR_DOMAIN_MEMORY_MODEL_VIRTIO_MEM)
+prealloc = true;
 } else {
 /* We can have both pagesize and mem source. If that's the case,
  * prefer hugepages as those are more specific. */
@@ -3304,6 +3306,9 @@ qemuBuildMemoryDeviceStr(const virDomainDef *def,
 break;
 
 case VIR_DOMAIN_MEMORY_MODEL_VIRTIO_MEM:
+device = "virtio-mem-pci";
+break;
+
 case VIR_DOMAIN_MEMORY_MODEL_NONE:
 case VIR_DOMAIN_MEMORY_MODEL_LAST:
 default:
@@ -3320,6 +3325,11 @@ qemuBuildMemoryDeviceStr(const virDomainDef *def,
 if (mem->labelsize)
 virBufferAsprintf(, "label-size=%llu,", mem->labelsize * 1024);
 
+if (mem->blocksize) {
+virBufferAsprintf(, "block-size=%llu,", mem->blocksize * 1024);
+virBufferAsprintf(, "requested-size=%llu,", mem->requestedsize * 
1024);
+}
+
 if (mem->uuid) {
 char uuidstr[VIR_UUID_STRING_BUFLEN];
 
diff --git 
a/tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args 
b/tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args
new file mode 100644
index 00..403e05865f
--- /dev/null
+++ b/tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args
@@ -0,0 +1,49 @@
+LC_ALL=C \
+PATH=/bin \
+HOME=/tmp/lib/domain--1-QEMUGuest1 \
+USER=test \
+LOGNAME=test \
+XDG_DATA_HOME=/tmp/lib/domain--1-QEMUGuest1/.local/share \
+XDG_CACHE_HOME=/tmp/lib/domain--1-QEMUGuest1/.cache \
+XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
+QEMU_AUDIO_DRV=none \
+/usr/bin/qemu-system-i386 \
+-name guest=QEMUGuest1,debug-threads=on \
+-S \
+-object secret,id=masterKey0,format=raw,\
+file=/tmp/lib/domain--1-QEMUGuest1/master-key.aes \
+-machine pc,accel=kvm,usb=off,dump-guest-core=off \
+-cpu qemu64 \
+-m size=2095104k,slots=16,maxmem=1099511627776k \
+-overcommit mem-lock=off \
+-smp 2,sockets=2,dies=1,cores=1,threads=1 \
+-object memory-backend-ram,id=ram-node0,size=2145386496 \
+-numa node,nodeid=0,cpus=0-1,memdev=ram-node0 \
+-object memory-backend-ram,id=memvirtiomem0,size=1073741824 \
+-device virtio-mem-pci,node=0,block-size=2097152,requested-size=536870912,\
+memdev=memvirtiomem0,id=virtiomem0,bus=pci.0,addr=0x2 \
+-object memory-backend-file,id=memvirtiomem1,\
+mem-path=/dev/hugepages2M/libvirt/qemu/-1-QEMUGuest1,size=2147483648,\
+host-nodes=1-3,policy=bind \
+-device virtio-mem-pci,node=0,block-size=2097152,requested-size=1073741824,\
+memdev=memvirtiomem1,id=virtiomem1,bus=pci.0,addr=0x4 \
+-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \
+-display none \

[PATCH 01/10] virhostmem: Introduce virHostMemGetTHPSize()

[Michal Privoznik]

New virHostMemGetTHPSize() is introduced which allows caller to
obtain THP PMD (Page Middle Directory) size, which is equal to
the minimal size that THP can use, taken from kernel doc
(Documentation/admin-guide/mm/transhuge.rst):

  Some userspace (such as a test program, or an optimized memory allocation
  library) may want to know the size (in bytes) of a transparent hugepage::

cat /sys/kernel/mm/transparent_hugepage/hpage_pmd_size

Since this size depends on the host architecture and the kernel
it won't change whilst libvirtd is running. Therefore, we can use
virOnce() and cache the value. Of course, we can be running under
kernel that has THP disabled or has no notion of THP at all. In
that case a negative value is returned to signal error.

Signed-off-by: Michal Privoznik 
---
 src/libvirt_private.syms |  1 +
 src/util/virhostmem.c| 63 
 src/util/virhostmem.h|  3 ++
 3 files changed, 67 insertions(+)

diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms
index fbaf16704b..962d82680e 100644
--- a/src/libvirt_private.syms
+++ b/src/libvirt_private.syms
@@ -2365,6 +2365,7 @@ virHostMemGetFreePages;
 virHostMemGetInfo;
 virHostMemGetParameters;
 virHostMemGetStats;
+virHostMemGetTHPSize;
 virHostMemSetParameters;
 
 
diff --git a/src/util/virhostmem.c b/src/util/virhostmem.c
index ae42978ed2..ef7b97806f 100644
--- a/src/util/virhostmem.c
+++ b/src/util/virhostmem.c
@@ -45,11 +45,14 @@
 #include "virstring.h"
 #include "virnuma.h"
 #include "virlog.h"
+#include "virthread.h"
 
 #define VIR_FROM_THIS VIR_FROM_NONE
 
 VIR_LOG_INIT("util.hostmem");
 
+static unsigned long long virHostTHPPMDSize;
+static virOnceControl virHostMemGetTHPSizeOnce = VIR_ONCE_CONTROL_INITIALIZER;
 
 #ifdef __FreeBSD__
 # define BSD_MEMORY_STATS_ALL 4
@@ -920,3 +923,63 @@ virHostMemAllocPages(unsigned int npages,
 
 return ncounts;
 }
+
+#if defined(__linux__)
+# define HPAGE_PMD_SIZE_PATH 
"/sys/kernel/mm/transparent_hugepage/hpage_pmd_size"
+static int
+virHostMemGetTHPSizeSysfs(unsigned long long *size)
+{
+g_autofree char *buf = NULL;
+
+/* 1KiB limit is more than enough. */
+if (virFileReadAll(HPAGE_PMD_SIZE_PATH, 1024, ) < 0)
+return -1;
+
+virStringTrimOptionalNewline(buf);
+if (virStrToLong_ull(buf, NULL, 10, size) < 0) {
+virReportError(VIR_ERR_INTERNAL_ERROR,
+   _("unable to parse THP PMD size: %s"), buf);
+return -1;
+}
+
+/* Size is now in bytes. Convert to KiB. */
+*size >>= 10;
+return 0;
+}
+#endif /* defined(__linux__) */
+
+
+static void
+virHostMemGetTHPSizeOnceInit(void)
+{
+#if defined(__linux__)
+virHostMemGetTHPSizeSysfs();
+#else /* !defined(__linux__) */
+VIR_WARN("Getting THP size not ported yet");
+#endif /* !defined(__linux__) */
+}
+
+
+/**
+ * virHostMemGetTHPSize:
+ * @size: returned size of THP in kibibytes
+ *
+ * Obtain Transparent Huge Page size in kibibytes. The size
+ * depends on host architecture and kernel. Because of virOnce(),
+ * do not rely on errno in case of failure.
+ *
+ * Returns: 0 on success,
+ * -1 on failure.
+ */
+int
+virHostMemGetTHPSize(unsigned long long *size)
+{
+if (virOnce(, virHostMemGetTHPSizeOnceInit) < 0)
+return -1;
+
+if (virHostTHPPMDSize == 0)
+return -1;
+
+*size = virHostTHPPMDSize;
+return 0;
+}
diff --git a/src/util/virhostmem.h b/src/util/virhostmem.h
index 1369829807..bf15c40698 100644
--- a/src/util/virhostmem.h
+++ b/src/util/virhostmem.h
@@ -53,3 +53,6 @@ int virHostMemAllocPages(unsigned int npages,
  int startCell,
  unsigned int cellCount,
  bool add);
+
+int virHostMemGetTHPSize(unsigned long long *size)
+G_GNUC_NO_INLINE;
-- 
2.26.2

[PATCH 03/10] conf: Introduce virtio-mem model

[Michal Privoznik]

The virtio-mem is paravirtualized mechanism of adding/removing
memory to/from a VM. A virtio-mem-pci device is split into blocks
of equal size which are then exposed (all or only a requested
portion of them) to the guest kernel to use as regular memory.
Therefore, the device has two important attributes:

  1) block-size, which defines the size of a block
  2) requested-size, which defines how much memory (in bytes)
 is the device requested to expose to the guest.

The 'block-size' is configured on command line and immutable
throughout device's lifetime. The 'requested-size' can be set on
the command line too, but also is adjustable via monitor. In
fact, that is how management software places its requests to
change the memory allocation. If it wants to give more memory to
the guest it changes 'requested-size' to a bigger value, and if it
wants to shrink guest memory it changes the 'requested-size' to a
smaller value. Note, value of zero means that guest should
release all memory offered by the device. Of course, guest has to
cooperate. Therefore, there is a third attribute 'size' which is
read only and reflects how much memory the guest still has. This
can be different to 'requested-size', obviously. Because of name
clash, I've named it 'actualsize' and it is dealt with in future
commits (it is a runtime information anyway).

In the backend, memory for virtio-mem is backed by usual objects:
memory-backend-{ram,file,memfd} and their size puts the cap on
the amount of memory that a virtio-mem device can offer to a
guest. But we are already able to express this info using 
under .

Therefore, we need only two more elements to cover 'block-size'
and 'requested-size' attributes. This is the XML I've came up
with:

  

  1-3
  2048


  2097152
  0
  2048
  1048576


  

I hope by now it is obvious that:

  1) 'requested-size' must be an integer multiple of
 'block-size', and
  2) virtio-mem-pci device goes onto PCI bus and thus needs PCI
 address.

Then there is a limitation that the minimal 'block-size' is
transparent huge page size (I'll leave this without explanation).

Signed-off-by: Michal Privoznik 
---
 docs/formatdomain.rst | 35 --
 docs/schemas/domaincommon.rng | 11 
 src/conf/domain_conf.c| 53 ++-
 src/conf/domain_conf.h|  3 +
 src/conf/domain_validate.c| 39 +++
 src/qemu/qemu_alias.c |  1 +
 src/qemu/qemu_command.c   |  1 +
 src/qemu/qemu_domain.c| 10 +++
 src/qemu/qemu_domain_address.c| 37 ---
 src/qemu/qemu_validate.c  |  8 +++
 src/security/security_apparmor.c  |  1 +
 src/security/security_dac.c   |  2 +
 src/security/security_selinux.c   |  2 +
 tests/domaincapsmock.c|  9 +++
 .../memory-hotplug-virtio-mem.xml | 66 +++
 ...emory-hotplug-virtio-mem.x86_64-latest.xml |  1 +
 tests/qemuxml2xmltest.c   |  1 +
 17 files changed, 264 insertions(+), 16 deletions(-)
 create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.xml
 create mode 12 
tests/qemuxml2xmloutdata/memory-hotplug-virtio-mem.x86_64-latest.xml

diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst
index af540391db..2938758ec2 100644
--- a/docs/formatdomain.rst
+++ b/docs/formatdomain.rst
@@ -7267,6 +7267,18 @@ Example: usage of the memory devices
  524288

  
+ 
+   
+ 1-3
+ 2048
+   
+   
+ 2097152
+ 0
+ 2048
+ 1048576
+   
+ 

...
 
@@ -7274,7 +7286,8 @@ Example: usage of the memory devices
Provide ``dimm`` to add a virtual DIMM module to the guest. :since:`Since
1.2.14` Provide ``nvdimm`` model that adds a Non-Volatile DIMM module.
:since:`Since 3.2.0` Provide ``virtio-pmem`` model to add a paravirtualized
-   persistent memory device. :since:`Since 7.1.0`
+   persistent memory device. :since:`Since 7.1.0` Provide ``virtio-mem`` model
+   to add paravirtualized memory device. :since: `Since 7.1.0`
 
 ``access``
An optional attribute ``access`` ( :since:`since 3.2.0` ) that provides
@@ -7297,10 +7310,11 @@ Example: usage of the memory devices
allowed only for ``model='nvdimm'`` for pSeries guests. :since:`Since 6.2.0`
 
 ``source``
-   For model ``dimm`` this element is optional and allows to fine tune the
-   source of the memory used for the given memory device. If the element is not
-   provided defaults configured via ``numatune`` are used. If ``dimm`` is
-   provided, then the following optional elements can be provided as well:
+   For model ``dimm`` and model ``virtio-mem`` this element is optional and
+   allows to fine tune the source of the memory used for the given memory
+ 

[PATCH 02/10] qemu_capabilities: Introduce QEMU_CAPS_DEVICE_VIRTIO_MEM_PCI

[Michal Privoznik]

This commit introduces a new capability that reflects virtio-mem-pci
device support in QEMU:

  QEMU_CAPS_DEVICE_VIRTIO_MEM_PCI, /* -device virtio-mem-pci */

The virtio-mem-pci device was introduced in QEMU 5.1.

Signed-off-by: Michal Privoznik 
---
 src/qemu/qemu_capabilities.c | 2 ++
 src/qemu/qemu_capabilities.h | 1 +
 tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml | 1 +
 tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml | 1 +
 4 files changed, 5 insertions(+)

diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c
index d656732c3e..0be7a24c9a 100644
--- a/src/qemu/qemu_capabilities.c
+++ b/src/qemu/qemu_capabilities.c
@@ -610,6 +610,7 @@ VIR_ENUM_IMPL(virQEMUCaps,
   "dc390",
   "am53c974",
   "virtio-pmem-pci",
+  "virtio-mem-pci",
 );
 
 
@@ -1327,6 +1328,7 @@ struct virQEMUCapsStringFlags virQEMUCapsObjectTypes[] = {
 { "dc390", QEMU_CAPS_SCSI_DC390 },
 { "am53c974", QEMU_CAPS_SCSI_AM53C974 },
 { "virtio-pmem-pci", QEMU_CAPS_DEVICE_VIRTIO_PMEM_PCI },
+{ "virtio-mem-pci", QEMU_CAPS_DEVICE_VIRTIO_MEM_PCI },
 };
 
 
diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h
index a14a78f959..76c7465c0e 100644
--- a/src/qemu/qemu_capabilities.h
+++ b/src/qemu/qemu_capabilities.h
@@ -590,6 +590,7 @@ typedef enum { /* virQEMUCapsFlags grouping marker for 
syntax-check */
 QEMU_CAPS_SCSI_DC390, /* -device dc-390 */
 QEMU_CAPS_SCSI_AM53C974, /* -device am53c974 */
 QEMU_CAPS_DEVICE_VIRTIO_PMEM_PCI, /* -device virtio-pmem-pci */
+QEMU_CAPS_DEVICE_VIRTIO_MEM_PCI, /* -device virtio-mem-pci */
 
 QEMU_CAPS_LAST /* this must always be the last item */
 } virQEMUCapsFlags;
diff --git a/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml 
b/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml
index 07466093c9..a54806fb0c 100644
--- a/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml
+++ b/tests/qemucapabilitiesdata/caps_5.1.0.x86_64.xml
@@ -250,6 +250,7 @@
   
   
   
+  
   5001000
   0
   43100242
diff --git a/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml 
b/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml
index dea2ff4b54..1bbc38297d 100644
--- a/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml
+++ b/tests/qemucapabilitiesdata/caps_5.2.0.x86_64.xml
@@ -251,6 +251,7 @@
   
   
   
+  
   5002000
   0
   43100243
-- 
2.26.2

[PATCH 00/10] Introduce virtio-mem model

[Michal Privoznik]

Technically, this is another version of:

https://www.redhat.com/archives/libvir-list/2020-December/msg00199.html

But since virtio-pmem part is pushed now, I've reworked virtio-mem a bit
and sending it as a new series.

For curious ones, David summarized behaviour well when implementing
virtio-mem support in kernel:

https://lwn.net/Articles/755423/

For less curious ones:

  # virsh update-memory $dom --requested-size 4G

adds additional 4GiB of RAM to guest;

  # virsh update-memory $dom --requested-size 0

removes those 4GiB added earlier.

Patches are also available on my GitLab:

https://gitlab.com/MichalPrivoznik/libvirt/-/tree/virtio_mem_v3


Michal Prívozník (10):
  virhostmem: Introduce virHostMemGetTHPSize()
  qemu_capabilities: Introduce QEMU_CAPS_DEVICE_VIRTIO_MEM_PCI
  conf: Introduce virtio-mem  model
  qemu: Build command line for virtio-mem
  qemu: Wire up  live update
  qemu: Wire up MEMORY_DEVICE_SIZE_CHANGE event
  qemu: Refresh the actual size of virtio-mem on monitor reconnect
  qemu: Recalculate balloon on MEMORY_DEVICE_SIZE_CHANGE event and
reconnect
  virsh: Introduce update-memory command
  news: document recent virtio memory addition

 NEWS.rst  |   7 +
 docs/formatdomain.rst |  42 +++-
 docs/manpages/virsh.rst   |  31 +++
 docs/schemas/domaincommon.rng |  16 ++
 src/conf/domain_conf.c| 100 -
 src/conf/domain_conf.h|  13 ++
 src/conf/domain_validate.c|  39 
 src/libvirt_private.syms  |   3 +
 src/qemu/qemu_alias.c |  10 +-
 src/qemu/qemu_capabilities.c  |   2 +
 src/qemu/qemu_capabilities.h  |   1 +
 src/qemu/qemu_command.c   |  13 +-
 src/qemu/qemu_domain.c|  50 -
 src/qemu/qemu_domain.h|   1 +
 src/qemu/qemu_domain_address.c|  37 ++-
 src/qemu/qemu_driver.c| 211 +-
 src/qemu/qemu_hotplug.c   |  18 ++
 src/qemu/qemu_hotplug.h   |   5 +
 src/qemu/qemu_monitor.c   |  37 +++
 src/qemu/qemu_monitor.h   |  27 +++
 src/qemu/qemu_monitor_json.c  |  94 ++--
 src/qemu/qemu_monitor_json.h  |   5 +
 src/qemu/qemu_process.c   | 101 -
 src/qemu/qemu_validate.c  |   8 +
 src/security/security_apparmor.c  |   1 +
 src/security/security_dac.c   |   2 +
 src/security/security_selinux.c   |   2 +
 src/util/virhostmem.c |  63 ++
 src/util/virhostmem.h |   3 +
 tests/domaincapsmock.c|   9 +
 .../caps_5.1.0.x86_64.xml |   1 +
 .../caps_5.2.0.x86_64.xml |   1 +
 ...mory-hotplug-virtio-mem.x86_64-latest.args |  49 
 .../memory-hotplug-virtio-mem.xml |  66 ++
 tests/qemuxml2argvtest.c  |   1 +
 ...emory-hotplug-virtio-mem.x86_64-latest.xml |   1 +
 tests/qemuxml2xmltest.c   |   1 +
 tools/virsh-domain.c  | 154 +
 38 files changed, 1165 insertions(+), 60 deletions(-)
 create mode 100644 
tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args
 create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.xml
 create mode 12 
tests/qemuxml2xmloutdata/memory-hotplug-virtio-mem.x86_64-latest.xml

-- 
2.26.2

[PATCH] virsh: Simplify @flags handing in cmdSetmem() and cmdSetmaxmem()

[Michal Privoznik]

What code tries to achieve is that if no flags were provided to
either 'setmem' or 'setmaxmem' commands then the old (no flags)
API is called to be able to communicate with older daemons.
Well, the code can be simplified a bit.

Note that with this change the old no flag version of APIs is
used more often. Previously if --current argument was given it
resulted in *Flags() version to be called even though it is not
necessary - VIR_DOMAIN_AFFECT_CURRENT is implied.

Therefore, this change in fact allows virsh to talk with broader
set of daemons.

Signed-off-by: Michal Privoznik 
---
 tools/virsh-domain.c | 14 --
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c
index 2bb136333f..9746117bdb 100644
--- a/tools/virsh-domain.c
+++ b/tools/virsh-domain.c
@@ -9018,9 +9018,6 @@ cmdSetmem(vshControl *ctl, const vshCmd *cmd)
 flags |= VIR_DOMAIN_AFFECT_CONFIG;
 if (live)
 flags |= VIR_DOMAIN_AFFECT_LIVE;
-/* none of the options were specified */
-if (!current && !live && !config)
-flags = -1;
 
 if (!(dom = virshCommandOptDomain(ctl, cmd, NULL)))
 return false;
@@ -9037,7 +9034,7 @@ cmdSetmem(vshControl *ctl, const vshCmd *cmd)
 }
 kibibytes = VIR_DIV_UP(bytes, 1024);
 
-if (flags == -1) {
+if (flags == 0) {
 if (virDomainSetMemory(dom, kibibytes) != 0)
 ret = false;
 } else {
@@ -9090,7 +9087,7 @@ cmdSetmaxmem(vshControl *ctl, const vshCmd *cmd)
 bool config = vshCommandOptBool(cmd, "config");
 bool live = vshCommandOptBool(cmd, "live");
 bool current = vshCommandOptBool(cmd, "current");
-unsigned int flags = VIR_DOMAIN_AFFECT_CURRENT | VIR_DOMAIN_MEM_MAXIMUM;
+unsigned int flags = VIR_DOMAIN_AFFECT_CURRENT;
 
 VSH_EXCLUSIVE_OPTIONS_VAR(current, live);
 VSH_EXCLUSIVE_OPTIONS_VAR(current, config);
@@ -9099,9 +9096,6 @@ cmdSetmaxmem(vshControl *ctl, const vshCmd *cmd)
 flags |= VIR_DOMAIN_AFFECT_CONFIG;
 if (live)
 flags |= VIR_DOMAIN_AFFECT_LIVE;
-/* none of the options were specified */
-if (!current && !live && !config)
-flags = -1;
 
 if (!(dom = virshCommandOptDomain(ctl, cmd, NULL)))
 return false;
@@ -9118,13 +9112,13 @@ cmdSetmaxmem(vshControl *ctl, const vshCmd *cmd)
 }
 kibibytes = VIR_DIV_UP(bytes, 1024);
 
-if (flags == -1) {
+if (flags == 0) {
 if (virDomainSetMaxMemory(dom, kibibytes) != 0) {
 vshError(ctl, "%s", _("Unable to change MaxMemorySize"));
 ret = false;
 }
 } else {
-if (virDomainSetMemoryFlags(dom, kibibytes, flags) < 0) {
+if (virDomainSetMemoryFlags(dom, kibibytes, flags | 
VIR_DOMAIN_MEM_MAXIMUM) < 0) {
 vshError(ctl, "%s", _("Unable to change MaxMemorySize"));
 ret = false;
 }
-- 
2.26.2

Re: [PATCH] qemu: Avoid crash in qemuStateShutdownPrepare() and qemuStateShutdownWait()

[Michal Privoznik]

On 1/22/21 12:09 PM, Nikolay Shirokovskiy wrote:

On Fri, Jan 22, 2021 at 12:45 PM Michal Privoznik 
wrote:


If libvirtd is sent SIGTERM while it is still initializing, it
may crash. The following scenario was observed (using 'stress' to
slow down CPU so much that the window where the problem exists is
bigger):

1) The main thread is already executing virNetDaemonRun() and is
in virEventRunDefaultImpl().
2) The thread that's supposed to run daemonRunStateInit() is
spawned already, but daemonRunStateInit() is in its very early
stage (in the stack trace I see it's executing
virIdentityGetSystem()).

If SIGTERM (or any other signal that we don't override handler
for) arrives at this point, the main thread jumps out from
virEventRunDefaultImpl() and enters virStateShutdownPrepare()
(via shutdownPrepareCb which was set earlier). This iterates
through stateShutdownPrepare() callbacks of state drivers and
reaching qemuStateShutdownPrepare() eventually only to
dereference qemu_driver. But since thread 2) has not been
scheduled/not proceeded yet, qemu_driver was not allocated yet.

Solution is simple - just check if qemu_driver is not NULL. But
doing so only in qemuStateShutdownPrepare() would push the
problem down to virStateShutdownWait(), well
qemuStateShutdownWait(). Therefore, duplicate the trick there
too.



I guess this is a partial solution. Initialization may be in a state when
qemu_driver is initialized but qemu_driver->workerPool is still NULL
for example.


Yes.



Maybe we'd better delay shutdown until initialization is finished?


I'm not exactly sure how to achieve that. Do you have a hint? Also, part 
of qemu driver state init is autostarting domains (which may take ages).


Michal

[PATCH] virNetDevOpenvswitchGetVhostuserIfname: Remove a single '\n' from ifname

[Yalei Li]

Ovs-vsctl returns a newline result.
Signed-off-by: Yalei Li 
---
 src/util/virnetdevopenvswitch.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/util/virnetdevopenvswitch.c b/src/util/virnetdevopenvswitch.c
index f9b3369b2a..bd840bd3b7 100644
--- a/src/util/virnetdevopenvswitch.c
+++ b/src/util/virnetdevopenvswitch.c
@@ -575,6 +575,7 @@ virNetDevOpenvswitchGetVhostuserIfname(const char *path,
 return 0;
 }
 
+virStringTrimOptionalNewline(*ifname);
 if (virNetDevOpenvswitchMaybeUnescapeReply(*ifname) < 0) {
 VIR_FREE(*ifname);
 return -1;
-- 
2.27.0

Re: [libvirt PATCH v2 06/13] util: extract storage file probe code into virtstoragefileprobe.c

[Pavel Hrdina]

On Fri, Jan 22, 2021 at 10:09:27AM +0100, Pavel Hrdina wrote:
> On Fri, Jan 22, 2021 at 09:23:00AM +0100, Peter Krempa wrote:
> > On Thu, Jan 21, 2021 at 20:34:20 +0100, Pavel Hrdina wrote:
> > > This code is not directly relevant to virStorageSource so move it to
> > > separate file.
> > > 
> > > Signed-off-by: Pavel Hrdina 
> > > ---
> > >  po/POTFILES.in|   1 +
> > >  src/libvirt_private.syms  |   6 +-
> > >  src/qemu/qemu_driver.c|   1 +
> > >  src/storage/storage_backend_gluster.c |   1 +
> > >  src/storage/storage_util.c|   1 +
> > >  src/util/meson.build  |   1 +
> > >  src/util/virstoragefile.c | 939 +
> > >  src/util/virstoragefile.h |  11 -
> > >  src/util/virstoragefileprobe.c| 967 ++
> > >  src/util/virstoragefileprobe.h|  44 ++
> > >  10 files changed, 1026 insertions(+), 946 deletions(-)
> > >  create mode 100644 src/util/virstoragefileprobe.c
> > >  create mode 100644 src/util/virstoragefileprobe.h
> > 
> > [...]
> > 
> > > diff --git a/src/util/virstoragefile.c b/src/util/virstoragefile.c
> > > index 13a86f34e5..98a3222d09 100644
> > > --- a/src/util/virstoragefile.c
> > > +++ b/src/util/virstoragefile.c
> > 
> > [...]
> > 
> > > @@ -792,289 +146,6 @@ virStorageIsRelative(const char *backing)
> > >  }
> > >  
> > >  
> > > -static int
> > > -virStorageFileProbeFormatFromBuf(const char *path,
> > > - char *buf,
> > > - size_t buflen)
> > 
> > I'd prefer if this is the function exported from the new module ...
> > 
> > > -{
> > > -int format = VIR_STORAGE_FILE_RAW;
> > > -size_t i;
> > > -int possibleFormat = VIR_STORAGE_FILE_RAW;
> > > -VIR_DEBUG("path=%s, buf=%p, buflen=%zu", path, buf, buflen);
> > > -
> > > -/* First check file magic */
> > > -for (i = 0; i < VIR_STORAGE_FILE_LAST; i++) {
> > > -if (virStorageFileMatchesMagic(fileTypeInfo[i].magicOffset,
> > > -   fileTypeInfo[i].magic,
> > > -   buf, buflen)) {
> > > -if 
> > > (!virStorageFileMatchesVersion(fileTypeInfo[i].versionOffset,
> > > -  
> > > fileTypeInfo[i].versionSize,
> > > -  
> > > fileTypeInfo[i].versionNumbers,
> > > -  fileTypeInfo[i].endian,
> > > -  buf, buflen)) {
> > > -possibleFormat = i;
> > > -continue;
> > > -}
> > > -format = i;
> > > -goto cleanup;
> > > -}
> > > -}
> > > -
> > > -if (possibleFormat != VIR_STORAGE_FILE_RAW)
> > > -VIR_WARN("File %s matches %s magic, but version is wrong. "
> > > - "Please report new version to libvir-list@redhat.com",
> > > - path, virStorageFileFormatTypeToString(possibleFormat));
> > > -
> > > - cleanup:
> > > -VIR_DEBUG("format=%d", format);
> > > -return format;
> > > -}
> > 
> > [...]
> > 
> > > -/**
> > > - * virStorageFileProbeFormat:
> > > - *
> > > - * Probe for the format of 'path', returning the detected
> > > - * disk format.
> > > - *
> > > - * Callers are advised never to trust the returned 'format'
> > > - * unless it is listed as VIR_STORAGE_FILE_RAW, since a
> > > - * malicious guest can turn a raw file into any other non-raw
> > > - * format at will.
> > > - *
> > > - * Best option: Don't use this function
> > > - */
> > > -int
> > > -virStorageFileProbeFormat(const char *path, uid_t uid, gid_t gid)
> > > -{
> > 
> > ... and not this.
> > 
> > > -struct stat sb;
> > > -ssize_t len = VIR_STORAGE_MAX_HEADER;
> > > -VIR_AUTOCLOSE fd = -1;
> > > -g_autofree char *header = NULL;
> > > -
> > > -if ((fd = virFileOpenAs(path, O_RDONLY, 0, uid, gid, 0)) < 0) {
> > 
> > Specifically the new module should not ever touch any real storage and
> > should just be a self-contained prober of metadata froma buffer.
> > 
> > > -virReportSystemError(-fd, _("Failed to open file '%s'"), path);
> > > -return -1;
> > > -}
> > > -
> > > -if (fstat(fd, ) < 0) {
> > > -virReportSystemError(errno, _("cannot stat file '%s'"), path);
> > > -return -1;
> > > -}
> > > -
> > > -/* No header to probe for directories */
> > > -if (S_ISDIR(sb.st_mode))
> > > -return VIR_STORAGE_FILE_DIR;
> > > -
> > > -if (lseek(fd, 0, SEEK_SET) == (off_t)-1) {
> > > -virReportSystemError(errno, _("cannot set to start of '%s'"), 
> > > path);
> > > -return -1;
> > > -}
> > > -
> > > -if ((len = virFileReadHeaderFD(fd, len, )) < 0) {
> > > -virReportSystemError(errno, _("cannot read header '%s'"), path);
> > > -return -1;
> > > -}
> > > -
> 

Re: [PATCH 2/3] build: make netcf=disabled the default

[Daniel P . Berrangé]

On Fri, Jan 22, 2021 at 03:01:56AM -0500, Laine Stump wrote:
> Even after the previous patch, in order to build without netcf you
> would need to add "-Dnetcf=disabled" to the meson commandline (or
> uninstall netcf-devel). This patch makes -Dnetcf=disabled the
> default. (Without this change, a lot of people would just blindly
> continue building with netcf enabled.)
> 
> Signed-off-by: Laine Stump 
> ---
>  meson.build | 7 ---
>  1 file changed, 4 insertions(+), 3 deletions(-)

I don't much like this idea. I consider it a bug if I have
installed the -devel package for a pre-requisite and it isn't
then detected.


If a distro no longer wants to support netcf why not just
retire the netcf package from that distro version(s), that
way users won't have it installed in the first place ?


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [PATCH 3/3] rpm: disable netcf for the interface driver on RHEL/Fedora/CentOS

[Daniel P . Berrangé]

On Fri, Jan 22, 2021 at 03:01:57AM -0500, Laine Stump wrote:
> Since libvirt.spec explicitly adds -Dnetcf=enabled to the meson
> commandline, just setting the default in the meson.build file won't
> have any effect for rpm builds. This patch changes the meson
> commandline in the spec file to -Dnetcf=disabled and removes the
> associated BuildRequires: and Requires:
> 
> (I debated whether or not to do that conditionally only for (fedora >
> 33 || rhel > 8) but haven't done that for now (still may if it's
> considered important; for Fedora probably not important (since we
> never rebase after release), for RHEL we might want to preserve
> current behavior for RHEL8 even through a rebase though)

Yeah, our normal policy is that we don't disable stuff that was
already present. So changing this should be conditional on
newer distros only.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [PATCH] spec: Increase meson test timeout 10x

[Andrea Bolognani]

On Thu, 2021-01-21 at 16:55 -0500, Cole Robinson wrote:
>  %check
> -VIR_TEST_DEBUG=1 %meson_test --no-suite syntax-check
> +# Building on slow archs, like emulated s390x in Fedora copr, requires
> +# raising the test timeout
> +VIR_TEST_DEBUG=1 %meson_test --no-suite syntax-check --timeout-multiplier 10

Debian has been doing the same thing for a while, even though in that
case it's to cope with slow native hardware rather than emulation:

  
https://salsa.debian.org/libvirt-team/libvirt/-/blob/debian/6.9.0-3/debian/rules#L170-179

So

  Reviewed-by: Andrea Bolognani 

from my side, but please leave the message on the list for a few days
to give other people the opportunity to voice any concerns they might
have.

-- 
Andrea Bolognani / Red Hat / Virtualization

[PATCH] qemu: Avoid crash in qemuStateShutdownPrepare() and qemuStateShutdownWait()

[Michal Privoznik]

If libvirtd is sent SIGTERM while it is still initializing, it
may crash. The following scenario was observed (using 'stress' to
slow down CPU so much that the window where the problem exists is
bigger):

1) The main thread is already executing virNetDaemonRun() and is
   in virEventRunDefaultImpl().
2) The thread that's supposed to run daemonRunStateInit() is
   spawned already, but daemonRunStateInit() is in its very early
   stage (in the stack trace I see it's executing
   virIdentityGetSystem()).

If SIGTERM (or any other signal that we don't override handler
for) arrives at this point, the main thread jumps out from
virEventRunDefaultImpl() and enters virStateShutdownPrepare()
(via shutdownPrepareCb which was set earlier). This iterates
through stateShutdownPrepare() callbacks of state drivers and
reaching qemuStateShutdownPrepare() eventually only to
dereference qemu_driver. But since thread 2) has not been
scheduled/not proceeded yet, qemu_driver was not allocated yet.

Solution is simple - just check if qemu_driver is not NULL. But
doing so only in qemuStateShutdownPrepare() would push the
problem down to virStateShutdownWait(), well
qemuStateShutdownWait(). Therefore, duplicate the trick there
too.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1895359#c14
Signed-off-by: Michal Privoznik 
---
 src/qemu/qemu_driver.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 027617deef..ca4f366323 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -1072,6 +1072,9 @@ qemuStateStop(void)
 static int
 qemuStateShutdownPrepare(void)
 {
+if (!qemu_driver)
+return 0;
+
 virThreadPoolStop(qemu_driver->workerPool);
 return 0;
 }
@@ -1091,6 +1094,9 @@ qemuDomainObjStopWorkerIter(virDomainObjPtr vm,
 static int
 qemuStateShutdownWait(void)
 {
+if (!qemu_driver)
+return 0;
+
 virDomainObjListForEach(qemu_driver->domains, false,
 qemuDomainObjStopWorkerIter, NULL);
 virThreadPoolDrain(qemu_driver->workerPool);
-- 
2.26.2

Re: [PATCH RESEND 04/20] virpci.c: simplify virPCIDeviceNew() signature

[Daniel Henrique Barboza]

On 1/20/21 10:17 PM, Laine Stump wrote:

On 1/18/21 2:53 PM, Daniel Henrique Barboza wrote:

The current virPCIDeviceNew() signature, receiving 4 uints in sequence
(domain, bus, slot, function), is not neat.

We already have a way to represent a PCI address in virPCIDeviceAddress
that is used in the code. Aside from the test files, most of
virPCIDeviceNew() callers have access to a virPCIDeviceAddress reference,
but then we need to retrieve the 4 required uints (addr.domain, addr.bus,
addr.slot, addr.function) to satisfy virPCIDeviceNew(). The result is
that we have extra verbosity/boilerplate to retrieve an information that
is already available in virPCIDeviceAddress.

A better way is presented by virNVMEDeviceNew(), where the caller just
supplies a virPCIDeviceAddress pointer and the function handles the
details internally.

This patch changes virPCIDeviceNew() to receive a virPCIDeviceAddress
pointer instead of 4 uints.

Signed-off-by: Daniel Henrique Barboza 
---
  src/hypervisor/virhostdev.c    |  3 +--
  src/libxl/libxl_driver.c   |  6 ++---
  src/node_device/node_device_udev.c | 11 +---
  src/qemu/qemu_domain_address.c |  5 +---
  src/qemu/qemu_driver.c |  6 ++---
  src/security/security_apparmor.c   |  3 +--
  src/security/security_dac.c    |  6 ++---
  src/security/security_selinux.c    |  6 ++---
  src/security/virt-aa-helper.c  |  6 +
  src/util/virnetdev.c   |  3 +--
  src/util/virnvme.c |  5 +---
  src/util/virpci.c  | 40 +-
  src/util/virpci.h  |  5 +---
  tests/virhostdevtest.c |  3 ++-
  tests/virpcitest.c | 35 +++---
  15 files changed, 64 insertions(+), 79 deletions(-)

diff --git a/src/hypervisor/virhostdev.c b/src/hypervisor/virhostdev.c
index be32a26164..bd35397f2c 100644
--- a/src/hypervisor/virhostdev.c
+++ b/src/hypervisor/virhostdev.c
@@ -235,8 +235,7 @@ virHostdevGetPCIHostDevice(const virDomainHostdevDef 
*hostdev,
  hostdev->source.subsys.type != VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI)
  return 0;
-    actual = virPCIDeviceNew(pcisrc->addr.domain, pcisrc->addr.bus,
- pcisrc->addr.slot, pcisrc->addr.function);
+    actual = virPCIDeviceNew(>addr);
  if (!actual)
  return -1;
diff --git a/src/libxl/libxl_driver.c b/src/libxl/libxl_driver.c
index 360d553a22..3eaf106006 100644
--- a/src/libxl/libxl_driver.c
+++ b/src/libxl/libxl_driver.c
@@ -5818,7 +5818,7 @@ libxlNodeDeviceDetachFlags(virNodeDevicePtr dev,
  if (virDomainDriverNodeDeviceGetPCIInfo(def, ) < 0)
  goto cleanup;
-    pci = virPCIDeviceNew(devAddr.domain, devAddr.bus, devAddr.slot, 
devAddr.function);
+    pci = virPCIDeviceNew();
  if (!pci)
  goto cleanup;
@@ -5889,7 +5889,7 @@ libxlNodeDeviceReAttach(virNodeDevicePtr dev)
  if (virDomainDriverNodeDeviceGetPCIInfo(def, ) < 0)
  goto cleanup;
-    pci = virPCIDeviceNew(devAddr.domain, devAddr.bus, devAddr.slot, 
devAddr.function);
+    pci = virPCIDeviceNew();
  if (!pci)
  goto cleanup;
@@ -5947,7 +5947,7 @@ libxlNodeDeviceReset(virNodeDevicePtr dev)
  if (virDomainDriverNodeDeviceGetPCIInfo(def, ) < 0)
  goto cleanup;
-    pci = virPCIDeviceNew(devAddr.domain, devAddr.bus, devAddr.slot, 
devAddr.function);
+    pci = virPCIDeviceNew();
  if (!pci)
  goto cleanup;
diff --git a/src/node_device/node_device_udev.c 
b/src/node_device/node_device_udev.c
index 55a2731681..fceb135aa5 100644
--- a/src/node_device/node_device_udev.c
+++ b/src/node_device/node_device_udev.c
@@ -367,6 +367,7 @@ udevProcessPCI(struct udev_device *device,
  virNodeDevCapPCIDevPtr pci_dev = >caps->data.pci_dev;
  virPCIEDeviceInfoPtr pci_express = NULL;
  virPCIDevicePtr pciDev = NULL;
+    virPCIDeviceAddress devAddr;
  int ret = -1;
  char *p;
  bool privileged;
@@ -416,10 +417,12 @@ udevProcessPCI(struct udev_device *device,
  if (virNodeDeviceGetPCIDynamicCaps(def->sysfs_path, pci_dev) < 0)
  goto cleanup;
-    if (!(pciDev = virPCIDeviceNew(pci_dev->domain,
-   pci_dev->bus,
-   pci_dev->slot,
-   pci_dev->function)))
+    devAddr.domain = pci_dev->domain;
+    devAddr.bus = pci_dev->bus;
+    devAddr.slot = pci_dev->slot;
+    devAddr.function = pci_dev->function;



Now see - there we are again! If virNodeDevCapPCIDev could just have a 
virPCIDeviceAddress, then this could have been a 4->1 conversion instead of 
4->4. Ah well, tomorrow is another day!




Convert nodedev_dev to use PCIDeviceAddr instead of 4 uints is not
that hard (just checked with a quick search). I'm not sure if this
would have an implication in the public API though.


Thanks,

DHB





+
+    if (!(pciDev = virPCIDeviceNew()))
  goto cleanup;
  /* We need to be root to read 

Re: [libvirt PATCH v2 09/13] util: move virStorageFileProbe code into storage_file

[Peter Krempa]

On Fri, Jan 22, 2021 at 10:15:52 +0100, Pavel Hrdina wrote:
> On Fri, Jan 22, 2021 at 09:57:03AM +0100, Peter Krempa wrote:
> > On Thu, Jan 21, 2021 at 20:34:23 +0100, Pavel Hrdina wrote:

[...]

> > >  @SRCDIR@src/storage_file/storage_file_backend.c
> > >  @SRCDIR@src/storage_file/storage_file_fs.c
> > >  @SRCDIR@src/storage_file/storage_file_gluster.c
> > > +@SRCDIR@src/storage_file/storage_file_probe.c
> > 
> > Hmm, I think we should add a prefix for the _fs and _gluster backend
> > drivers so that it doesn't get confused with the probing code.
> 
> Good point. How about storage_file_backend_fs ?

Sounds good to me.

Re: [libvirt PATCH v2 08/13] util: move virStorageFileBackend code into storage_file

[Peter Krempa]

On Fri, Jan 22, 2021 at 10:13:02 +0100, Pavel Hrdina wrote:
> On Fri, Jan 22, 2021 at 09:54:06AM +0100, Peter Krempa wrote:
> > On Thu, Jan 21, 2021 at 20:34:22 +0100, Pavel Hrdina wrote:
> > > It's used only by storage file code so it doesn't make sense to have
> > > it in util directory.
> > > 
> > > Signed-off-by: Pavel Hrdina 
> > > ---
> > >  po/POTFILES.in| 2 +-
> > >  src/libvirt_private.syms  | 8 
> > >  src/storage_file/meson.build  | 1 +
> > >  .../storage_file_backend.c}   | 4 ++--
> > >  .../storage_file_backend.h}   | 2 +-
> > >  src/storage_file/storage_file_fs.c| 2 +-
> > >  src/storage_file/storage_file_gluster.c   | 2 +-
> > >  src/storage_file/storage_source.c | 2 +-
> > >  src/util/meson.build  | 1 -
> > >  9 files changed, 12 insertions(+), 12 deletions(-)
> > >  rename src/{util/virstoragefilebackend.c => 
> > > storage_file/storage_file_backend.c} (97%)
> > >  rename src/{util/virstoragefilebackend.h => 
> > > storage_file/storage_file_backend.h} (97%)
> > 
> > Reviewed-by: Peter Krempa 
> > 
> > although I'm probably more inclined to keep the original name without
> > underscores and with vir prefix.
> 
> I personally don't have any preference. The only motivation was not to
> mix two different naming styles withing one directory.
> 
> We should eventually unify the file naming style across the whole
> project as we use underscores, dash and nothing all over the place.

Yes, it can be done separately.

Re: [PATCH] virsh: Fix XPATH in virshDomainDeviceAliasCompleter()

[Peter Krempa]

On Fri, Jan 22, 2021 at 09:39:01 +0100, Michal Privoznik wrote:
> The way this completer works is that it dumps XML of specified
> domain and then tries to look for @name attribute of 
> element. However, the XPATH it uses is not correct which results
> in no aliases returned by the completer.
> 
> Signed-off-by: Michal Privoznik 
> ---
>  tools/virsh-completer-domain.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/virsh-completer-domain.c b/tools/virsh-completer-domain.c
> index 4a3459f12a..e773af6552 100644
> --- a/tools/virsh-completer-domain.c
> +++ b/tools/virsh-completer-domain.c
> @@ -316,14 +316,14 @@ virshDomainDeviceAliasCompleter(vshControl *ctl,
>  if (virshDomainGetXML(ctl, cmd, domainXMLFlags, , ) < 0)
>  return NULL;
>  
> -naliases = virXPathNodeSet("./devices//alias/@name", ctxt, );
> +naliases = virXPathNodeSet("/domain/devices//alias[@name]", ctxt, 
> );

Another option would be //devices/ as start, but since we know it's a
domain XML full path is okay.

>  if (naliases < 0)
>  return NULL;
>  
>  tmp = g_new0(char *, naliases + 1);
>  
>  for (i = 0; i < naliases; i++) {
> -if (!(tmp[i] = virXMLNodeContentString(aliases[i])))
> +if (!(tmp[i] = virXMLPropString(aliases[i], "name")))

... and I actually prefer this.

>  return NULL;
>  }

Reviewed-by: Peter Krempa 

Re: [libvirt PATCH v2 12/13] virstoragefile: use virStorageFile prefix for all functions

[Pavel Hrdina]

On Fri, Jan 22, 2021 at 10:10:33AM +0100, Peter Krempa wrote:
> On Thu, Jan 21, 2021 at 20:34:26 +0100, Pavel Hrdina wrote:
> > Signed-off-by: Pavel Hrdina 
> > ---
> 
> I really hate that these functions are exported.
> 
> If this patch is necessary for the following, then commit it, otherwise
> drop it. I'll follow up with patches for moving the two functions
> appropriately and unexporting them.
> 
> Reviewed-by: Peter Krempa 
> 
> Otherwise just drop it.

I'll drop it then.


signature.asc
Description: PGP signature

Re: [libvirt PATCH v2 09/13] util: move virStorageFileProbe code into storage_file

[Pavel Hrdina]

On Fri, Jan 22, 2021 at 09:57:03AM +0100, Peter Krempa wrote:
> On Thu, Jan 21, 2021 at 20:34:23 +0100, Pavel Hrdina wrote:
> > Same as virStorageFileBackend, it doesn't belong into util directory.
> 
> Arguably, after you remove the file access, it kinda becomes just an
> utility function.
> 
> On the other hand, it's use is strictly limited to the storage source
> code, so it makes sense to stop linking it with the general util module.
> 
> > 
> > Signed-off-by: Pavel Hrdina 
> > ---
> >  po/POTFILES.in |  2 +-
> >  src/libvirt_private.syms   | 10 +-
> >  src/qemu/qemu_driver.c |  2 +-
> >  src/storage/storage_backend_gluster.c  |  2 +-
> >  src/storage/storage_util.c |  2 +-
> >  src/storage_file/meson.build   |  1 +
> >  .../storage_file_probe.c}  |  6 +++---
> >  .../storage_file_probe.h}  |  2 +-
> >  src/storage_file/storage_source.c  |  2 +-
> >  src/util/meson.build   |  1 -
> >  10 files changed, 15 insertions(+), 15 deletions(-)
> >  rename src/{util/virstoragefileprobe.c => 
> > storage_file/storage_file_probe.c} (99%)
> >  rename src/{util/virstoragefileprobe.h => 
> > storage_file/storage_file_probe.h} (95%)
> > 
> > diff --git a/po/POTFILES.in b/po/POTFILES.in
> > index 4da22ede8f..eeff3f9f2f 100644
> > --- a/po/POTFILES.in
> > +++ b/po/POTFILES.in
> > @@ -226,6 +226,7 @@
> >  @SRCDIR@src/storage_file/storage_file_backend.c
> >  @SRCDIR@src/storage_file/storage_file_fs.c
> >  @SRCDIR@src/storage_file/storage_file_gluster.c
> > +@SRCDIR@src/storage_file/storage_file_probe.c
> 
> Hmm, I think we should add a prefix for the _fs and _gluster backend
> drivers so that it doesn't get confused with the probing code.

Good point. How about storage_file_backend_fs ?

> Also same comment as previous patch regarding naming.
> 
> Reviewed-by: Peter Krempa 
> 


signature.asc
Description: PGP signature

Re: [libvirt PATCH v2 06/13] util: extract storage file probe code into virtstoragefileprobe.c

[Pavel Hrdina]

On Fri, Jan 22, 2021 at 09:23:00AM +0100, Peter Krempa wrote:
> On Thu, Jan 21, 2021 at 20:34:20 +0100, Pavel Hrdina wrote:
> > This code is not directly relevant to virStorageSource so move it to
> > separate file.
> > 
> > Signed-off-by: Pavel Hrdina 
> > ---
> >  po/POTFILES.in|   1 +
> >  src/libvirt_private.syms  |   6 +-
> >  src/qemu/qemu_driver.c|   1 +
> >  src/storage/storage_backend_gluster.c |   1 +
> >  src/storage/storage_util.c|   1 +
> >  src/util/meson.build  |   1 +
> >  src/util/virstoragefile.c | 939 +
> >  src/util/virstoragefile.h |  11 -
> >  src/util/virstoragefileprobe.c| 967 ++
> >  src/util/virstoragefileprobe.h|  44 ++
> >  10 files changed, 1026 insertions(+), 946 deletions(-)
> >  create mode 100644 src/util/virstoragefileprobe.c
> >  create mode 100644 src/util/virstoragefileprobe.h
> 
> [...]
> 
> > diff --git a/src/util/virstoragefile.c b/src/util/virstoragefile.c
> > index 13a86f34e5..98a3222d09 100644
> > --- a/src/util/virstoragefile.c
> > +++ b/src/util/virstoragefile.c
> 
> [...]
> 
> > @@ -792,289 +146,6 @@ virStorageIsRelative(const char *backing)
> >  }
> >  
> >  
> > -static int
> > -virStorageFileProbeFormatFromBuf(const char *path,
> > - char *buf,
> > - size_t buflen)
> 
> I'd prefer if this is the function exported from the new module ...
> 
> > -{
> > -int format = VIR_STORAGE_FILE_RAW;
> > -size_t i;
> > -int possibleFormat = VIR_STORAGE_FILE_RAW;
> > -VIR_DEBUG("path=%s, buf=%p, buflen=%zu", path, buf, buflen);
> > -
> > -/* First check file magic */
> > -for (i = 0; i < VIR_STORAGE_FILE_LAST; i++) {
> > -if (virStorageFileMatchesMagic(fileTypeInfo[i].magicOffset,
> > -   fileTypeInfo[i].magic,
> > -   buf, buflen)) {
> > -if 
> > (!virStorageFileMatchesVersion(fileTypeInfo[i].versionOffset,
> > -  fileTypeInfo[i].versionSize,
> > -  
> > fileTypeInfo[i].versionNumbers,
> > -  fileTypeInfo[i].endian,
> > -  buf, buflen)) {
> > -possibleFormat = i;
> > -continue;
> > -}
> > -format = i;
> > -goto cleanup;
> > -}
> > -}
> > -
> > -if (possibleFormat != VIR_STORAGE_FILE_RAW)
> > -VIR_WARN("File %s matches %s magic, but version is wrong. "
> > - "Please report new version to libvir-list@redhat.com",
> > - path, virStorageFileFormatTypeToString(possibleFormat));
> > -
> > - cleanup:
> > -VIR_DEBUG("format=%d", format);
> > -return format;
> > -}
> 
> [...]
> 
> > -/**
> > - * virStorageFileProbeFormat:
> > - *
> > - * Probe for the format of 'path', returning the detected
> > - * disk format.
> > - *
> > - * Callers are advised never to trust the returned 'format'
> > - * unless it is listed as VIR_STORAGE_FILE_RAW, since a
> > - * malicious guest can turn a raw file into any other non-raw
> > - * format at will.
> > - *
> > - * Best option: Don't use this function
> > - */
> > -int
> > -virStorageFileProbeFormat(const char *path, uid_t uid, gid_t gid)
> > -{
> 
> ... and not this.
> 
> > -struct stat sb;
> > -ssize_t len = VIR_STORAGE_MAX_HEADER;
> > -VIR_AUTOCLOSE fd = -1;
> > -g_autofree char *header = NULL;
> > -
> > -if ((fd = virFileOpenAs(path, O_RDONLY, 0, uid, gid, 0)) < 0) {
> 
> Specifically the new module should not ever touch any real storage and
> should just be a self-contained prober of metadata froma buffer.
> 
> > -virReportSystemError(-fd, _("Failed to open file '%s'"), path);
> > -return -1;
> > -}
> > -
> > -if (fstat(fd, ) < 0) {
> > -virReportSystemError(errno, _("cannot stat file '%s'"), path);
> > -return -1;
> > -}
> > -
> > -/* No header to probe for directories */
> > -if (S_ISDIR(sb.st_mode))
> > -return VIR_STORAGE_FILE_DIR;
> > -
> > -if (lseek(fd, 0, SEEK_SET) == (off_t)-1) {
> > -virReportSystemError(errno, _("cannot set to start of '%s'"), 
> > path);
> > -return -1;
> > -}
> > -
> > -if ((len = virFileReadHeaderFD(fd, len, )) < 0) {
> > -virReportSystemError(errno, _("cannot read header '%s'"), path);
> > -return -1;
> > -}
> > -
> > -return virStorageFileProbeFormatFromBuf(path, header, len);
> > -}
> 
> [...]
> 
> > diff --git a/src/util/virstoragefileprobe.h b/src/util/virstoragefileprobe.h
> > new file mode 100644
> > index 00..2b94a4ae51
> > --- /dev/null
> > +++ b/src/util/virstoragefileprobe.h
> > @@ -0,0 +1,44 @@
> > +/*
> > + * 

Re: [libvirt PATCH v2 08/13] util: move virStorageFileBackend code into storage_file

[Pavel Hrdina]

On Fri, Jan 22, 2021 at 09:54:06AM +0100, Peter Krempa wrote:
> On Thu, Jan 21, 2021 at 20:34:22 +0100, Pavel Hrdina wrote:
> > It's used only by storage file code so it doesn't make sense to have
> > it in util directory.
> > 
> > Signed-off-by: Pavel Hrdina 
> > ---
> >  po/POTFILES.in| 2 +-
> >  src/libvirt_private.syms  | 8 
> >  src/storage_file/meson.build  | 1 +
> >  .../storage_file_backend.c}   | 4 ++--
> >  .../storage_file_backend.h}   | 2 +-
> >  src/storage_file/storage_file_fs.c| 2 +-
> >  src/storage_file/storage_file_gluster.c   | 2 +-
> >  src/storage_file/storage_source.c | 2 +-
> >  src/util/meson.build  | 1 -
> >  9 files changed, 12 insertions(+), 12 deletions(-)
> >  rename src/{util/virstoragefilebackend.c => 
> > storage_file/storage_file_backend.c} (97%)
> >  rename src/{util/virstoragefilebackend.h => 
> > storage_file/storage_file_backend.h} (97%)
> 
> Reviewed-by: Peter Krempa 
> 
> although I'm probably more inclined to keep the original name without
> underscores and with vir prefix.

I personally don't have any preference. The only motivation was not to
mix two different naming styles withing one directory.

We should eventually unify the file naming style across the whole
project as we use underscores, dash and nothing all over the place.

Pavel


signature.asc
Description: PGP signature

Re: [libvirt PATCH v2 13/13] storage_source: use virStorageSource prefix for all functions

[Peter Krempa]

On Thu, Jan 21, 2021 at 20:34:27 +0100, Pavel Hrdina wrote:
> Signed-off-by: Pavel Hrdina 
> ---
>  src/libvirt_private.syms  |  42 ++---
>  src/qemu/qemu_backup.c|   8 +-
>  src/qemu/qemu_block.c |   6 +-
>  src/qemu/qemu_domain.c|  16 +-
>  src/qemu/qemu_driver.c|  66 +++
>  src/qemu/qemu_hotplug.c   |   4 +-
>  src/qemu/qemu_process.c   |   4 +-
>  src/qemu/qemu_snapshot.c  |  20 +--
>  src/security/virt-aa-helper.c |   2 +-
>  src/storage/storage_backend_gluster.c |   4 +-
>  src/storage/storage_util.c|   6 +-
>  src/storage_file/storage_source.c | 250 +-
>  src/storage_file/storage_source.h |  90 +-
>  tests/virstoragetest.c|  18 +-
>  14 files changed, 268 insertions(+), 268 deletions(-)

Reviewed-by: Peter Krempa 

Re: [libvirt PATCH v2 12/13] virstoragefile: use virStorageFile prefix for all functions

[Peter Krempa]

On Thu, Jan 21, 2021 at 20:34:26 +0100, Pavel Hrdina wrote:
> Signed-off-by: Pavel Hrdina 
> ---

I really hate that these functions are exported.

If this patch is necessary for the following, then commit it, otherwise
drop it. I'll follow up with patches for moving the two functions
appropriately and unexporting them.

Reviewed-by: Peter Krempa 

Otherwise just drop it.