[libvirt PATCH v6 09/36] qemu: query nbdkit module dir from binary

2023-07-20 Thread Jonathon Jongsma
` separately. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 77 -- 1 file changed, 60 insertions(+), 17 deletions(-) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 2f9c5f19ef..9e53c1893e 100644

[libvirt PATCH v6 29/36] schema: add configuration for host verification of ssh disks

2023-07-20 Thread Jonathon Jongsma
In order to make ssh disks usable, we need to be able to validate a remote host. To do this, add a xml element for ssh disks to allow the user to specify a location for a file that contains known host keys. Implementation to follow. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa

[libvirt PATCH v6 02/36] qemu: Add functions for determining nbdkit availability

2023-07-20 Thread Jonathon Jongsma
In future commits, we will optionally use nbdkit to serve some remote disk sources. This patch queries to see whether nbdkit is installed on the host and queries it for capabilities. The data will be used in later commits. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu

[libvirt PATCH v6 32/36] qemu: implement keyfile auth for ssh disks with nbdkit

2023-07-20 Thread Jonathon Jongsma
For ssh disks that are served by nbdkit, we can support logging in with an ssh key file. Pass the path to the configured key file and the username to the nbdkit process. Signed-off-by: Jonathon Jongsma --- src/conf/domain_conf.c| 32 ++ src/conf

[libvirt PATCH v6 07/36] qemu: use file cache for nbdkit caps

2023-07-20 Thread Jonathon Jongsma
Add the virFileCache implementation for nbdkit capabilities to the qemu driver. This allows us to determine whether nbdkit is installed and which plugins are supported. it also has persistent caching and the capabilities are re-queried whenever something changes. Signed-off-by: Jonathon Jongsma

[libvirt PATCH v6 24/36] qemu: add test for authenticating a https network disk

2023-07-20 Thread Jonathon Jongsma
Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- tests/qemunbdkitdata/disk-network-source-curl.args.disk1 | 4 +++- .../disk-network-source-curl.args.disk1.pipe.780 | 2 +- .../disk-network-source-curl.args.disk1.pipe.782 | 1 + tests/qemunbdkitdata

[libvirt PATCH v6 31/36] schema: add keyfile configuration for ssh disks

2023-07-20 Thread Jonathon Jongsma
: ... ... Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- docs/formatdomain.rst | 7 +++ src/conf/schemas/domaincommon.rng | 19 ++- 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index

[libvirt PATCH v6 20/36] qemu: pass sensitive data to nbdkit via pipe

2023-07-20 Thread Jonathon Jongsma
Rather than passing passwords and cookies (which could contain passwords) to nbdkit via commandline arguments, use the alternate format that nbdkit supports where we can specify a file descriptor which nbdkit will read to get the password or cookies. Signed-off-by: Jonathon Jongsma Reviewed

[libvirt PATCH v6 28/36] qemu: implement password auth for ssh disks with nbdkit

2023-07-20 Thread Jonathon Jongsma
For ssh disks that are served by nbdkit, lookup the password from the configured secret and securely pass it to the nbdkit process using fd passing. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_nbdkit.c| 87 ++- .../disk-network-ssh

[libvirt PATCH v6 30/36] qemu: implement knownHosts for ssh disks with nbdkit

2023-07-20 Thread Jonathon Jongsma
For ssh disks that are served by nbdkit, use the configured value for knownHosts and pass it to the nbdkit process. Signed-off-by: Jonathon Jongsma --- src/conf/domain_conf.c| 8 ++ src/conf/storage_source_conf.c| 1 + src/conf/storage_source_conf.h

[libvirt PATCH v6 13/36] qemu: Extract qemuDomainLogContext into a new file

2023-07-20 Thread Jonathon Jongsma
This will allow us to use it for nbdkit logging in upcoming commits. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- po/POTFILES| 1 + src/qemu/meson.build | 1 + src/qemu/qemu_domain.c | 247 ++ src/qemu/qemu_domain.h

[libvirt PATCH v6 04/36] util: Allow virFileCache data to be any GObject

2023-07-20 Thread Jonathon Jongsma
Since the libvirt documentation suggests to prefer GObject over virObject, and since virObject is a GObject, change virFileCache to allow GObjects as data. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/util/virfilecache.c | 14 -- src/util/virfilecache.h | 2

[libvirt PATCH v6 18/36] qemu: include nbdkit state in private xml

2023-07-20 Thread Jonathon Jongsma
Add xml to the private data for a disk source to represent the nbdkit process so that the state can be re-created if the libvirt daemon is restarted. Format: /path/to/nbdkit.pid /path/to/nbdkit.socket Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu

[libvirt PATCH v6 21/36] qemu: use nbdkit to serve network disks if available

2023-07-20 Thread Jonathon Jongsma
For virStorageSource objects that contain an nbdkitProcess, start that nbdkit process to serve that network drive and then pass the nbdkit socket to qemu rather than sending the network url to qemu directly. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_block.c

[libvirt PATCH v6 11/36] qemu: remove unused 'mode' param from qemuDomainLogContextNew()

2023-07-20 Thread Jonathon Jongsma
The only use of this function always passes QEMU_DOMAIN_LOG_CONTEXT_MODE_START. The other enum values are never used anywhere in the code. Remove them. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_domain.c | 26 +++--- src/qemu/qemu_domain.h

[libvirt PATCH v6 22/36] util: make virCommandSetSendBuffer testable

2023-07-20 Thread Jonathon Jongsma
Add a private function to peek at the list of send buffers in virCommand so that it is testable Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/libvirt_private.syms | 1 + src/util/vircommand.c | 17 + src/util/vircommand.h | 8 src/util

[libvirt PATCH v6 36/36] rpm: update spec file for for nbdkit support

2023-07-20 Thread Jonathon Jongsma
Require libnbd-devel when building the qemu driver, recommend nbdkit packages. Signed-off-by: Jonathon Jongsma --- libvirt.spec.in | 8 1 file changed, 8 insertions(+) diff --git a/libvirt.spec.in b/libvirt.spec.in index 1f77cd90b7..616087e849 100644 --- a/libvirt.spec.in +++ b

[libvirt PATCH v6 26/36] qemu: try to connect to nbdkit early to detect errors

2023-07-20 Thread Jonathon Jongsma
eturned error: 404 error: Failed to start domain 'nbdkit-test' error: internal error: Failed to connect to nbdkit for 'http://localhost:/nonexistent.iso]: error: problem doing HEAD request to fetch size of URL [http://localhost:/nonexistent.iso]: HTTP respon

[libvirt PATCH v6 05/36] qemu: implement basic virFileCache for nbdkit caps

2023-07-20 Thread Jonathon Jongsma
Preparatory step for caching nbdkit capabilities. This patch implements the newData and isValid virFileCacheHandlers callback functions. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 89 +- src/qemu/qemu_nbdkit.h

[libvirt PATCH v6 33/36] schema: add ssh-agent configuration for ssh disks

2023-07-20 Thread Jonathon Jongsma
Add the ability to specify a path to a ssh-agent socket in order to use the ssh-agent to authenticate to remote ssh disks. Example configuration: ... ... Signed-off-by: Jonathon Jongsma --- docs/formatdomain.rst | 13

[libvirt PATCH v6 35/36] run: add ability to set selinux context

2023-07-20 Thread Jonathon Jongsma
and must be enabled with the command line switch. Signed-off-by: Jonathon Jongsma --- run.in | 100 + 1 file changed, 80 insertions(+), 20 deletions(-) diff --git a/run.in b/run.in index 80a5d2dd43..5b89b3dcd5 100644 --- a/run.in +++ b

[libvirt PATCH v6 25/36] qemu: Monitor nbdkit process for exit

2023-07-20 Thread Jonathon Jongsma
was chosen somewhat arbitrarily. Signed-off-by: Jonathon Jongsma --- meson.build | 7 +++ src/qemu/qemu_nbdkit.c | 136 ++-- src/qemu/qemu_nbdkit.h | 4 +- src/qemu/qemu_process.c | 4 +- 4 files changed, 143 insertions(+), 8 deletions(-) diff

[libvirt PATCH v6 34/36] qemu: implement ssh-agent auth for ssh disks with nbdkit

2023-07-20 Thread Jonathon Jongsma
to access rather than unconfined_t. Signed-off-by: Jonathon Jongsma --- src/conf/domain_conf.c | 11 --- src/conf/storage_source_conf.c | 1 + src/conf/storage_source_conf.h | 1 + src/qemu/qemu_nbdkit.c

[libvirt PATCH v6 27/36] schema: add password configuration for ssh disk

2023-07-20 Thread Jonathon Jongsma
authentication. Implementation will follow. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- docs/formatdomain.rst | 27 ++- src/conf/schemas/domaincommon.rng | 23 ++- 2 files changed, 36 insertions

[libvirt PATCH v6 17/36] qemu: split qemuDomainSecretStorageSourcePrepare

2023-07-20 Thread Jonathon Jongsma
This prepares encryption secrets and authentication secrets. When we add nbdkit-backed network storage sources, we will not need to send authentication secrets to qemu, since they will be sent to nbdkit instead. So split this into two different functions. Signed-off-by: Jonathon Jongsma Reviewed

[libvirt PATCH v6 00/36] Use nbdkit for http/ftp/ssh network drives in libvirt

2023-07-20 Thread Jonathon Jongsma
smaller changes suggested by Peter in v5 Jonathon Jongsma (36): schema: allow 'ssh' as a protocol for network disks qemu: Add functions for determining nbdkit availability qemu: expand nbdkit capabilities util: Allow virFileCache data to be any GObject qemu: implement basic virFileCache

[libvirt PATCH v6 15/36] qemu: log error output from nbdkit

2023-07-20 Thread Jonathon Jongsma
log stderr and stdout from nbdkit into its own log so that nbdkit-related issues can be debugged more easily. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 16 +++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/src/qemu

[libvirt PATCH v6 19/36] util: secure erase virCommand send buffers

2023-07-20 Thread Jonathon Jongsma
All users of virCommandSetSendBuffer() are using it to send sensitive data to a child process. So, since these buffers contain sensitive information, clear it with virSecureErase(). Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/util/vircommand.c | 2 ++ 1 file changed, 2

[libvirt PATCH v6 16/36] tests: add ability to test various nbdkit capabilities

2023-07-20 Thread Jonathon Jongsma
Add new DO_TEST_CAPS_LATEST_NBDKIT macro to test xml2argv for various nbdkit capability scenarios. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 20 +--- tests/qemuxml2argvtest.c | 11 +++ tests/testutilsqemu.c| 26

[libvirt PATCH v6 10/36] qemu: add functions to start and stop nbdkit

2023-07-20 Thread Jonathon Jongsma
Add some helper functions to build a virCommand object and run the nbdkit process for a given virStorageSource. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 250 + src/qemu/qemu_nbdkit.h | 10 ++ 2 files changed

[libvirt PATCH v6 03/36] qemu: expand nbdkit capabilities

2023-07-20 Thread Jonathon Jongsma
. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 42 ++ 1 file changed, 42 insertions(+) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 9ff293698d..486df8c161 100644 --- a/src/qemu/qemu_nbdkit.c +++ b

[libvirt PATCH v6 08/36] qemu: Add qemuNbdkitProcess

2023-07-20 Thread Jonathon Jongsma
-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_conf.c | 22 src/qemu/qemu_conf.h | 2 ++ src/qemu/qemu_domain.c | 31 src/qemu/qemu_domain.h | 4 +++ src/qemu/qemu_nbdkit.c | 82 ++ src/qemu/qemu_nbdkit.h | 26

[libvirt PATCH v6 23/36] tests: add tests for nbdkit invocation

2023-07-20 Thread Jonathon Jongsma
We were testing the arguments that were being passed to qemu when a disk was being served by nbdkit, but the arguments used to start nbdkit itself were not testable. This adds a test to ensure that we're invoking nbdkit correctly for various disk source definitions. Signed-off-by: Jonathon

[libvirt PATCH v6 14/36] qemu: move qemuProcessReadLog() to qemuLogContext

2023-07-20 Thread Jonathon Jongsma
This code can be used by the nbdkit implementation for reading back filtered log data for error reporting. Move it to qemuLogContext so that it can be shared. Renamed to qemuLogContextReadFiltered(). Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_logcontext.c | 65

[libvirt PATCH v6 06/36] qemu: implement persistent file cache for nbdkit caps

2023-07-20 Thread Jonathon Jongsma
Implement the loadFile and saveFile virFileCacheHandlers callbacks so that nbdkit capabilities are cached perstistently across daemon restarts. The format and implementation is modeled on the qemu capabilities, but simplified slightly. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa

[libvirt PATCH v6 12/36] Generalize qemuDomainLogContextNew()

2023-07-20 Thread Jonathon Jongsma
Allow to specify a basename for the log file so that qemuDomainLogContextNew() can be used to create log contexts for secondary loggers. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_domain.c | 5 +++-- src/qemu/qemu_domain.h | 3 ++- src/qemu/qemu_process.c | 2

[libvirt PATCH v6 01/36] schema: allow 'ssh' as a protocol for network disks

2023-07-20 Thread Jonathon Jongsma
There was support in the code for parsing protocol='ssh' on network disk sources, but it was not present in the xml schema. Add this to the schema. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/conf/schemas/domaincommon.rng | 1 + tests/qemublocktest.c

[libvirt PATCH] nodedev: report mdev persistence properly

2023-07-18 Thread Jonathon Jongsma
using it to update transient mdevs. Refactor the function so that we can use it to update both persistent and transient mdevs. Signed-off-by: Jonathon Jongsma --- src/node_device/node_device_driver.c | 21 +++-- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/src

[libvirt PATCH] qemu: require memfd memory for virtio 'blob' support

2023-07-18 Thread Jonathon Jongsma
-07-18T02:33:57.083178Z qemu-kvm: -device {"driver":"virtio-vga","id":"video0","max_outputs":1,"blob":true,"bus":"pcie.0","addr":"0x1"}: cannot enable blob resources without udmabuf Signed-o

Re: [PATCH Libvirt 1/3] qemu_capabilities: Introduce virtio-blk DISCARD and WRITE_ZEROES capabilities

2023-07-17 Thread Jonathon Jongsma
I believe that qemu 4.2.0 is the oldest version of qemu that we still support, so I don't think that a new capability would actually be necessary for this. Jonathon On 7/16/23 8:41 AM, ~hyman wrote: From: Hyman Huang(黄勇) DISCARD and WRITE_ZEROES commands has been implemented in virtio-blk

[libvirt PATCH] docs: expand clangd instructions

2023-07-14 Thread Jonathon Jongsma
Add some additional information about running clangd for LSP when clang is not your normal compiler. Signed-off-by: Jonathon Jongsma --- docs/clangd.rst | 17 + 1 file changed, 17 insertions(+) diff --git a/docs/clangd.rst b/docs/clangd.rst index 93570ae178..25529948f6 100644

[libvirt PATCH 1/2] nodedev: refactor mdevctl thread functions

2023-07-06 Thread Jonathon Jongsma
a couple functions. The names weren't very descriptive of their behavior. For example, the old scheduleMdevctlHandler() function didn't actually schedule anything, it just started a thread. So rename it to free up the 'schedule' name for the above refactored function. Signed-off-by: Jonathon

[libvirt PATCH 0/2] nodedev: update transient mdev attributes

2023-07-06 Thread Jonathon Jongsma
move mdevctl updates to the mdevctl thread. These two patches apply on top of Boris's patch: "[PATCH v2] nodedev: transient mdev update on nodeDeviceCreateXML" Jonathon Jongsma (2): nodedev: refactor mdevctl thread functions nodedev: update mdevs from the mdevctl thread src/n

[libvirt PATCH 2/2] nodedev: update mdevs from the mdevctl thread

2023-07-06 Thread Jonathon Jongsma
a very large number of mdevs. Signed-off-by: Jonathon Jongsma --- src/node_device/node_device_udev.c | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/src/node_device/node_device_udev.c b/src/node_device/node_device_udev.c index 9ba550dbc1..da63d326a1 100644 --- a/src

Re: [PATCH v2] nodedev: transient mdev update on nodeDeviceCreateXML

2023-07-06 Thread Jonathon Jongsma
On 6/30/23 6:34 AM, Boris Fiuczynski wrote: Update the optional mdev attributes by running an mdevctl update on a new created nodedev object representing an mdev. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2143158 Signed-off-by: Boris Fiuczynski ---

Re: [libvirt] [PATCH RESEND 2/4] qemu: Validate notify VM exit feature is available only on x86

2023-07-05 Thread Jonathon Jongsma
On 7/3/23 1:30 AM, Lin Ma wrote: Signed-off-by: Lin Ma --- src/qemu/qemu_validate.c | 12 +++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_validate.c b/src/qemu/qemu_validate.c index a53729d349..6ec5af0028 100644 --- a/src/qemu/qemu_validate.c +++

Re: [libvirt] [PATCH RESEND 1/4] conf: Introduce notify VM exit feature

2023-07-05 Thread Jonathon Jongsma
On 7/3/23 1:30 AM, Lin Ma wrote: VMX(kernel v6.0) supports Notification VM exit feature under commit 2f4073e0. QEMU supports it as well since v7.2 under commit e2e69f6b. Add this feature into libvirt now. An example of Domain XML snippet to configure this feature:

Re: [libvirt PATCH v2 4/4] qemu: enable asynchronous teardown on s390x hosts

2023-06-28 Thread Jonathon Jongsma
On 6/27/23 10:51 AM, Boris Fiuczynski wrote: Enablement of asynchronous teardown on S390 and add tests for asynchronous teardown autogeneration support. I don't know all of the implications of enabling vs not enabling this feature. It sounds like it speeds up shutdown significantly in some

Re: [PATCH] nodedev: transient mdev update on nodeDeviceCreateXML

2023-06-28 Thread Jonathon Jongsma
On 6/28/23 3:40 AM, Boris Fiuczynski wrote: On 6/28/23 12:03 AM, Jonathon Jongsma wrote: On 6/23/23 5:43 AM, Boris Fiuczynski wrote: Update the optional mdev attributes on the new created nodedev object as they otherwise would not get set until the next mdevctl update cycle. Resolves: https

Re: [PATCH] nodedev: transient mdev update on nodeDeviceCreateXML

2023-06-27 Thread Jonathon Jongsma
On 6/23/23 5:43 AM, Boris Fiuczynski wrote: Update the optional mdev attributes on the new created nodedev object as they otherwise would not get set until the next mdevctl update cycle. Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2143158 Signed-off-by: Boris Fiuczynski ---

[libvirt PATCH v2] run: add ability to set selinux context

2023-06-21 Thread Jonathon Jongsma
script. Signed-off-by: Jonathon Jongsma --- I sent a version of this a couple months ago and Erik made a few comments and then I kind of forgot about it for a little while. Re-sending now with a couple minor changes changes in v2: - rebased to master - fixed to properly restore context

Re: [PATCH 0/3] Enable asynchronous teardown on s390x hosts

2023-06-21 Thread Jonathon Jongsma
On 6/13/23 10:42 AM, Boris Fiuczynski wrote: Update capabilities for QEMU 8.1 on s390x, add a new capability async-teardown and make use of it when running on s390x hosts to improve memory reclaiming. Is this really something that should be enabled unconditionally on all s390x guests, or

Re: [libvirt PATCH 1/5] conf: add ability to configure a vdpa block disk device

2023-06-08 Thread Jonathon Jongsma
On 6/8/23 8:44 AM, Peter Krempa wrote: On Tue, Jun 06, 2023 at 16:11:00 -0500, Jonathon Jongsma wrote: vDPA block devices can be configured as follows: Signed-off-by: Jonathon Jongsma --- docs/formatdomain.rst | 19 +-- src/ch/ch_monitor.c

Re: [PATCH] scripts: QMP schema query string helper script

2023-06-07 Thread Jonathon Jongsma
ser(description='A tool to generate QMP schema query strins and validator of schema coverage') typo: strins -> strings What do you mean by 'coverage' here? How is it validating schema coverage? As far as I can tell, it's just validating the schema replies file itself. I would cha

[libvirt PATCH 5/5] qemu: Implement support for vDPA block devices

2023-06-06 Thread Jonathon Jongsma
Requires recent qemu with support for the virtio-blk-vhost-vdpa device and the ability to pass a /dev/fdset/N path for the vdpa path (8.1.0) Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1900770 --- src/qemu/qemu_block.c | 20 -- src/qemu/qemu_domain.c

[libvirt PATCH 2/5] qemu: add virtio-blk-vhost-vdpa capability

2023-06-06 Thread Jonathon Jongsma
will be present in the qemu 8.1.0 release and was merged to qemu in commit 98b126f5. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + 2 files changed, 3 insertions(+) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu

[libvirt PATCH 4/5] qemu: consider vdpa block devices for memlock limits

2023-06-06 Thread Jonathon Jongsma
vDPA block devices will also need the same consideration for memlock limits as other vdpa devices, so consider these devices when calculating memlock limits. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_domain.c | 12 ++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff

[libvirt PATCH 1/5] conf: add ability to configure a vdpa block disk device

2023-06-06 Thread Jonathon Jongsma
vDPA block devices can be configured as follows: Signed-off-by: Jonathon Jongsma --- docs/formatdomain.rst | 19 +-- src/ch/ch_monitor.c | 1 + src/conf/domain_conf.c| 7 +++ src/conf/schemas/domaincommon.rng | 13

[libvirt PATCH 3/5] qemu: make vdpa connect function more generic

2023-06-06 Thread Jonathon Jongsma
qemuInterfaceVDPAConnect() was a helper function for connecting to the vdpa device file. But in order to support other vdpa devices besides network interfaces (e.g. vdpa block devices) make this function a bit more generic. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_command.c | 23

[libvirt PATCH 0/5] Add support for vDPA block devices

2023-06-06 Thread Jonathon Jongsma
see https://bugzilla.redhat.com/show_bug.cgi?id=1900770. Jonathon Jongsma (5): conf: add ability to configure a vdpa block disk device qemu: add virtio-blk-vhost-vdpa capability qemu: make vdpa connect function more generic qemu: consider vdpa block devices for memlock limits qemu

Re: [PATCH 03/18] virRaiseErrorLog: Don't skip error printing when enabling debug logging env variable

2023-05-25 Thread Jonathon Jongsma
On 5/25/23 10:13 AM, Peter Krempa wrote: When a user requests debug logging by setting the environment variable: LIBVIRT_DEBUG=1 we should log any errors regardless of the setting of e.g. 'LIBVIRT_LOG_OUTPUTS' as the code will log every 'debug' and 'info' level message to stderr but will skip

Re: [PATCH] nodedev: update transient mdevs

2023-05-23 Thread Jonathon Jongsma
Sorry for the delay Reviewed-by: Jonathon Jongsma On 5/22/23 5:56 AM, Boris Fiuczynski wrote: Polite ping. On 5/8/23 7:10 PM, Boris Fiuczynski wrote: Instead of updating defined mdevs only add another update for active devices as well to cover transient mdev devices as well. Resolves

Re: [libvirt PATCH] run: add ability to set selinux context

2023-04-26 Thread Jonathon Jongsma
On 4/25/23 11:54 AM, Erik Skultety wrote: On Tue, Apr 25, 2023 at 10:40:45AM -0500, Jonathon Jongsma wrote: On 4/25/23 9:43 AM, Jonathon Jongsma wrote: On 4/25/23 8:11 AM, Erik Skultety wrote: On Mon, Apr 24, 2023 at 03:50:48PM -0500, Jonathon Jongsma wrote: When running libvirt from

Re: [libvirt PATCH] run: add ability to set selinux context

2023-04-25 Thread Jonathon Jongsma
On 4/25/23 9:43 AM, Jonathon Jongsma wrote: On 4/25/23 8:11 AM, Erik Skultety wrote: On Mon, Apr 24, 2023 at 03:50:48PM -0500, Jonathon Jongsma wrote: When running libvirt from the build directory with the 'run' script, it will run as unconfined_t. This can result in unexpected behavior when

Re: [libvirt PATCH] run: add ability to set selinux context

2023-04-25 Thread Jonathon Jongsma
On 4/25/23 8:11 AM, Erik Skultety wrote: On Mon, Apr 24, 2023 at 03:50:48PM -0500, Jonathon Jongsma wrote: When running libvirt from the build directory with the 'run' script, it will run as unconfined_t. This can result in unexpected behavior when selinux is enforcing due to the fact

[libvirt PATCH] run: add ability to set selinux context

2023-04-24 Thread Jonathon Jongsma
and must be enabled with the command line switch. Signed-off-by: Jonathon Jongsma --- run.in | 56 ++-- 1 file changed, 50 insertions(+), 6 deletions(-) diff --git a/run.in b/run.in index c6d3411082..4aa458b791 100644 --- a/run.in +++ b/run.in

Re: [libvirt PATCH 0/3] tests: Don't use GIO_COMPILATION

2023-03-30 Thread Jonathon Jongsma
/meson.build | 11 ++- tests/virfirewalltest.c | 4 2 files changed, 6 insertions(+), 9 deletions(-) Reviewed-by: Jonathon Jongsma

Re: [libvirt PATCH v5 32/32] qemu: implement keyfile auth for ssh disk with nbdkit

2023-02-23 Thread Jonathon Jongsma
On 2/16/23 10:56 AM, Peter Krempa wrote: On Tue, Feb 14, 2023 at 11:08:19 -0600, Jonathon Jongsma wrote: For ssh disks that are served by nbdkit, we can support logging in with an ssh key file. Pass the path to the configured key file and the username to the nbdkit process. The key file may

Re: [libvirt PATCH v2] qemu: implement QEMU NBD source reconnect delay attribute

2023-02-21 Thread Jonathon Jongsma
On 2/17/23 10:50 AM, Christian Nautze wrote: Currently it's only possible to set this parameter during domain creation via QEMU commandline passthrough feature. With the new delay attribute it's also possible to set this parameter if you want to attach a new NBD disk

Re: [PATCH] qemu: fix reconnect of unix socket is wrong

2023-02-21 Thread Jonathon Jongsma
TATE_BOOL_NO) reconnect = 0; } Reviewed-by: Jonathon Jongsma

Re: [libvirt PATCH v5 29/32] schema: add configuration for host verification of ssh disks

2023-02-16 Thread Jonathon Jongsma
On 2/16/23 10:45 AM, Peter Krempa wrote: On Tue, Feb 14, 2023 at 11:08:16 -0600, Jonathon Jongsma wrote: In order to make ssh disks usable, we need to be able to validate a remote host. To do this, add a xml element for ssh disks to allow the user to specify a location for a file that contains

Re: [libvirt PATCH v5 27/32] schema: add password configuration for ssh disk

2023-02-16 Thread Jonathon Jongsma
On 2/16/23 10:43 AM, Peter Krempa wrote: On Tue, Feb 14, 2023 at 11:08:14 -0600, Jonathon Jongsma wrote: Right now, ssh network disks are not usable. There is some basic support in libvirt that is meant to support disk chains that have backing disks located at ssh urls, but there is no real way

Re: [libvirt PATCH v5 21/32] qemu: use nbdkit to serve network disks if available

2023-02-16 Thread Jonathon Jongsma
On 2/16/23 9:55 AM, Peter Krempa wrote: On Tue, Feb 14, 2023 at 11:08:08 -0600, Jonathon Jongsma wrote: For virStorageSource objects that contain an nbdkitProcess, start that nbdkit process to serve that network drive and then pass the nbdkit socket to qemu rather than sending the network url

Re: [PATCH v2 3/5] qemu_passt: Make passt report errors to stderr whenever possible

2023-02-16 Thread Jonathon Jongsma
On 2/16/23 7:32 AM, Michal Privoznik wrote: Passt has '--stderr' argument which makes it report error onto stderr rather to system log. Unfortunately, it's currently impossible to use both '--log-file' and '--stderr', so pass the latter only if the former isn't passed. Then, use the stderr to

Re: [libvirt PATCH v5 14/32] qemu: move qemuProcessReadLog() to qemuLogContext

2023-02-16 Thread Jonathon Jongsma
On 2/16/23 8:56 AM, Peter Krempa wrote: On Tue, Feb 14, 2023 at 11:08:01 -0600, Jonathon Jongsma wrote: This code can be used by the nbdkit implementation for reading back filtered log data for error reporting. Move it to qemuLogContext so that it can be shared. Renamed

[libvirt PATCH v5 32/32] qemu: implement keyfile auth for ssh disk with nbdkit

2023-02-14 Thread Jonathon Jongsma
adds this key to an ssh agent, they can configure the disk with the path to the unix socket for the ssh agent so libvirt can pass this socket path to nbdkit and we can make use of these keys. Signed-off-by: Jonathon Jongsma --- src/conf/domain_conf.c| 36

[libvirt PATCH v5 18/32] qemu: include nbdkit state in private xml

2023-02-14 Thread Jonathon Jongsma
Add xml to the private data for a disk source to represent the nbdkit process so that the state can be re-created if the libvirt daemon is restarted. Format: /path/to/nbdkit.pid /path/to/nbdkit.socket Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_domain.c

[libvirt PATCH v5 05/32] qemu: implement basic virFileCache for nbdkit caps

2023-02-14 Thread Jonathon Jongsma
Preparatory step for caching nbdkit capabilities. This patch implements the newData and isValid virFileCacheHandlers callback functions. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 89 +- src/qemu/qemu_nbdkit.h

[libvirt PATCH v5 13/32] qemu: Extract qemuDomainLogContext into a new file

2023-02-14 Thread Jonathon Jongsma
This will allow us to use it for nbdkit logging in upcoming commits. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- po/POTFILES| 1 + src/qemu/meson.build | 1 + src/qemu/qemu_domain.c | 247 ++ src/qemu/qemu_domain.h

[libvirt PATCH v5 14/32] qemu: move qemuProcessReadLog() to qemuLogContext

2023-02-14 Thread Jonathon Jongsma
This code can be used by the nbdkit implementation for reading back filtered log data for error reporting. Move it to qemuLogContext so that it can be shared. Renamed to qemuLogContextReadFiltered(). Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_logcontext.c | 65

[libvirt PATCH v5 29/32] schema: add configuration for host verification of ssh disks

2023-02-14 Thread Jonathon Jongsma
In order to make ssh disks usable, we need to be able to validate a remote host. To do this, add a xml element for ssh disks to allow the user to specify a location for a file that contains known host keys. Implementation to follow. Signed-off-by: Jonathon Jongsma --- docs/formatdomain.rst

[libvirt PATCH v5 20/32] qemu: pass sensitive data to nbdkit via pipe

2023-02-14 Thread Jonathon Jongsma
Rather than passing passwords and cookies (which could contain passwords) to nbdkit via commandline arguments, use the alternate format that nbdkit supports where we can specify a file descriptor which nbdkit will read to get the password or cookies. Signed-off-by: Jonathon Jongsma --- src/qemu

[libvirt PATCH v5 27/32] schema: add password configuration for ssh disk

2023-02-14 Thread Jonathon Jongsma
authentication. Implementation will follow. Signed-off-by: Jonathon Jongsma --- docs/formatdomain.rst | 27 ++- src/conf/schemas/domaincommon.rng | 23 ++- 2 files changed, 36 insertions(+), 14 deletions(-) diff --git

[libvirt PATCH v5 19/32] util: secure erase virCommand send buffers

2023-02-14 Thread Jonathon Jongsma
All users of virCommandSetSendBuffer() are using it to send sensitive data to a child process. So, since these buffers contain sensitive information, clear it with virSecureErase(). Signed-off-by: Jonathon Jongsma --- src/util/vircommand.c | 2 ++ 1 file changed, 2 insertions(+) diff --git

[libvirt PATCH v5 30/32] qemu: implement knownHosts for ssh disks with nbdkit

2023-02-14 Thread Jonathon Jongsma
For ssh disks that are served by nbdkit, use the configured value for knownHosts and pass it to the nbdkit process. Signed-off-by: Jonathon Jongsma --- src/conf/domain_conf.c| 8 src/conf/storage_source_conf.c| 1 + src

[libvirt PATCH v5 24/32] qemu: add test for authenticating a https network disk

2023-02-14 Thread Jonathon Jongsma
Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- tests/qemunbdkitdata/disk-network-source-curl.args.disk1 | 4 +++- .../disk-network-source-curl.args.disk1.pipe.780 | 2 +- .../disk-network-source-curl.args.disk1.pipe.782 | 1 + tests/qemunbdkitdata

[libvirt PATCH v5 31/32] schema: add keyfile configuration for ssh disks

2023-02-14 Thread Jonathon Jongsma
: ... ... Signed-off-by: Jonathon Jongsma --- docs/formatdomain.rst | 8 src/conf/schemas/domaincommon.rng | 22 +- 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst

[libvirt PATCH v5 28/32] qemu: implement password auth for ssh disks with nbdkit

2023-02-14 Thread Jonathon Jongsma
For ssh disks that are served by nbdkit, lookup the password from the configured secret and securely pass it to the nbdkit process using fd passing. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_nbdkit.c| 87 ++- .../disk-network-ssh

[libvirt PATCH v5 25/32] qemu: Monitor nbdkit process for exit

2023-02-14 Thread Jonathon Jongsma
was chosen somewhat arbitrarily. Signed-off-by: Jonathon Jongsma --- meson.build | 7 ++ src/qemu/qemu_nbdkit.c | 147 ++-- src/qemu/qemu_nbdkit.h | 4 +- src/qemu/qemu_process.c | 4 +- 4 files changed, 155 insertions(+), 7 deletions(-) diff

[libvirt PATCH v5 26/32] qemu: try to connect to nbdkit early to detect errors

2023-02-14 Thread Jonathon Jongsma
eturned error: 404 error: Failed to start domain 'nbdkit-test' error: internal error: Failed to connect to nbdkit for 'http://localhost:/nonexistent.iso]: error: problem doing HEAD request to fetch size of URL [http://localhost:/nonexistent.iso]: HTTP response code said error:

[libvirt PATCH v5 21/32] qemu: use nbdkit to serve network disks if available

2023-02-14 Thread Jonathon Jongsma
For virStorageSource objects that contain an nbdkitProcess, start that nbdkit process to serve that network drive and then pass the nbdkit socket to qemu rather than sending the network url to qemu directly. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_block.c | 162

[libvirt PATCH v5 11/32] qemu: remove unused 'mode' param from qemuDomainLogContextNew()

2023-02-14 Thread Jonathon Jongsma
The only use of this function always passes QEMU_DOMAIN_LOG_CONTEXT_MODE_START. The other enum values are never used anywhere in the code. Remove them. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_domain.c | 26 +++--- src/qemu/qemu_domain.h

[libvirt PATCH v5 23/32] tests: add tests for nbdkit invocation

2023-02-14 Thread Jonathon Jongsma
We were testing the arguments that were being passed to qemu when a disk was being served by nbdkit, but the arguments used to start nbdkit itself were not testable. This adds a test to ensure that we're invoking nbdkit correctly for various disk source definitions. Signed-off-by: Jonathon

[libvirt PATCH v5 12/32] Generalize qemuDomainLogContextNew()

2023-02-14 Thread Jonathon Jongsma
Allow to specify a basename for the log file so that qemuDomainLogContextNew() can be used to create log contexts for secondary loggers. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_domain.c | 5 +++-- src/qemu/qemu_domain.h | 3 ++- src/qemu/qemu_process.c | 2

[libvirt PATCH v5 16/32] tests: add ability to test various nbdkit capabilities

2023-02-14 Thread Jonathon Jongsma
Add new DO_TEST_CAPS_LATEST_NBDKIT macro to test xml2argv for various nbdkit capability scenarios. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 20 +--- tests/qemuxml2argvtest.c | 11 +++ tests/testutilsqemu.c| 27

[libvirt PATCH v5 10/32] qemu: add functions to start and stop nbdkit

2023-02-14 Thread Jonathon Jongsma
Add some helper functions to build a virCommand object and run the nbdkit process for a given virStorageSource. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 254 + src/qemu/qemu_nbdkit.h | 10 ++ 2 files changed

[libvirt PATCH v5 09/32] qemu: query nbdkit module dir from binary

2023-02-14 Thread Jonathon Jongsma
` separately. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu/qemu_nbdkit.c | 77 -- 1 file changed, 60 insertions(+), 17 deletions(-) diff --git a/src/qemu/qemu_nbdkit.c b/src/qemu/qemu_nbdkit.c index 7dcb61d9a6..840710862e 100644

[libvirt PATCH v5 22/32] util: make virCommandSetSendBuffer testable

2023-02-14 Thread Jonathon Jongsma
Add a private function to peek at the list of send buffers in virCommand so that it is testable Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/libvirt_private.syms | 1 + src/util/vircommand.c | 17 + src/util/vircommand.h | 8 src/util

[libvirt PATCH v5 02/32] qemu: Add functions for determining nbdkit availability

2023-02-14 Thread Jonathon Jongsma
In future commits, we will optionally use nbdkit to serve some remote disk sources. This patch queries to see whether nbdkit is installed on the host and queries it for capabilities. The data will be used in later commits. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/qemu

[libvirt PATCH v5 01/32] schema: allow 'ssh' as a protocol for network disks

2023-02-14 Thread Jonathon Jongsma
There was support in the code for parsing protocol='ssh' on network disk sources, but it was not present in the xml schema. Add this to the schema. Signed-off-by: Jonathon Jongsma Reviewed-by: Peter Krempa --- src/conf/schemas/domaincommon.rng | 1 + tests/qemublocktest.c

<    1   2   3   4   5   6   7   8   9   10   >