[Libguestfs] Benchmarks of asynch nbdublk (was: Re: [PATCH libnbd] ublk: Add new nbdublk program)

I have an interesting benchmark to share. I wanted to find out the answers to a couple of questions: (1) Is nbdublk stable, and (2) Is the performance terrible / good / excellent? I only wanted a rough impression for these, we can work on optimising it more later. Since I am developing nbdublk

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

FYI here is the asynch version: https://gitlab.com/rwmjones/libnbd/-/commits/nbdublk/ I kept it as two commits so you can see the differences but obviously it would be folded into one commit upstream. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read

Re: [Libguestfs] shebang usage in bash scripts

On Thu, Aug 25, 2022 at 03:32:15PM +0200, Laszlo Ersek wrote: > Hi, > > most shell scripts in the v2v projects start with a shebang like this: > > #!/bin/bash - > > I *think* I understand the intent of the single hyphen, but (a) it seems > unnecessary, (b) even if we insisted, using the

Re: [Libguestfs] virt-p2v VM testing bitrot?

On Thu, Aug 25, 2022 at 02:42:54PM +0200, Laszlo Ersek wrote: > I'd like to implement it in a separate script. > > Do I have to list that script under EXTRA_DIST? Usually yes, because we'd want it to go into the *.tar.gz file which is how some distros (eg. Fedora) consume virt-p2v. There is a

[Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

m B<-V> + +=item B<--version> + +Display the package name and version and exit. + +=back + +=head1 MODES + +Modes are used to select the NBD server. Possible modes are: + +=over 4 + +=item nbdublk DEVICE URI + +This mode uses an NBD URI (see L and +L<https://github.com/NetworkBlockDev

[Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

This patch adds simple support for a ublk-based NBD client. It is also available here: https://gitlab.com/rwmjones/libnbd/-/tree/nbdublk/ublk ublk is a way to write Linux block device drivers in userspace: https://lwn.net/Articles/903855/ For simplicity of implementation and because I don't

Re: [Libguestfs] virt-p2v VM testing bitrot?

On Wed, Aug 24, 2022 at 05:25:50PM +0200, Laszlo Ersek wrote: > On 08/24/22 16:13, Richard W.M. Jones wrote: > > On Wed, Aug 24, 2022 at 04:12:04PM +0200, Laszlo Ersek wrote: > >> I've started looking into RHBZ 1590721 for virt-p2v. > >> > >> For p2v develop

Re: [Libguestfs] virt-p2v VM testing bitrot?

On Wed, Aug 24, 2022 at 04:12:04PM +0200, Laszlo Ersek wrote: > I've started looking into RHBZ 1590721 for virt-p2v. > > For p2v development, quick local testing is helpful. "make > run-virt-p2v-directly" seems to be working great; however, the VM-based > test methods seem to have developed

Re: [Libguestfs] [PATCH nbdkit 2/2] file: Allow a file descriptor to be passed to the plugin

On Thu, Aug 18, 2022 at 10:28:17AM -0500, Eric Blake wrote: > On Wed, Aug 17, 2022 at 10:56:58PM +0100, Richard W.M. Jones wrote: > > On Wed, Aug 17, 2022 at 10:37:00PM +0100, Richard W.M. Jones wrote: > > > Is that actually possible? “fcntl (fd, F_GETFL) & O_W

Re: [Libguestfs] [PATCH nbdkit 1/2] file: Add an internal "mode"

On Thu, Aug 18, 2022 at 12:30:21PM +0200, Laszlo Ersek wrote: > On 08/18/22 11:53, Richard W.M. Jones wrote: > > > > This is upstream in 5 commits 5764fa6f4..dd28b0054: > > > > https://gitlab.com/nbdkit/nbdkit/-/commit/2620d95a4585204f0db0d55c0d41276d11970436 > >

Re: [Libguestfs] [PATCH nbdkit 1/2] file: Add an internal "mode"

This is upstream in 5 commits 5764fa6f4..dd28b0054: https://gitlab.com/nbdkit/nbdkit/-/commit/2620d95a4585204f0db0d55c0d41276d11970436 https://gitlab.com/nbdkit/nbdkit/-/commit/185e7d4010b353f36b5ca5d47467a770c530e58c

Re: [Libguestfs] [PATCH nbdkit 1/2] file: Add an internal "mode"

On Thu, Aug 18, 2022 at 10:27:11AM +0200, Laszlo Ersek wrote: > On 08/17/22 17:38, Richard W.M. Jones wrote: > > Previously we relied on the implicit assumption filename xor directory, > > representing two modes. Make this explicit with an internal mode > > variable.

Re: [Libguestfs] [PATCH nbdkit 2/2] file: Allow a file descriptor to be passed to the plugin

On Wed, Aug 17, 2022 at 10:37:00PM +0100, Richard W.M. Jones wrote: > Is that actually possible? “fcntl (fd, F_GETFL) & O_WRONLY” > should do it? So the answer is no as it's a kind of tri-state. I think this should work (untested)? r = fcntl (fd, F_GETFL); if (r == -1) ... r &a

Re: [Libguestfs] [PATCH nbdkit 2/2] file: Allow a file descriptor to be passed to the plugin

On Wed, Aug 17, 2022 at 03:57:24PM -0500, Eric Blake wrote: > On Wed, Aug 17, 2022 at 04:38:11PM +0100, Richard W.M. Jones wrote: > > This allows you to pass in a file descriptor to the plugin, eg: > > > > $ exec 6<>/var/tmp/fedora-36.img > > $ ./nbd

Re: [Libguestfs] [v2v PATCH] convert_linux: start the QEMU guest agent in a distro-specific way

On Wed, Aug 17, 2022 at 04:47:36PM +0200, Laszlo Ersek wrote: > The current command "service start" does not apply to > RHEL-6; the service name ("qemu-ga") differs from the package name > ("qemu-guest-agent") there. > > Overhaul the logic -- detach the command from the package name; cover the >

[Libguestfs] [PATCH nbdkit 1/2] file: Add an internal "mode"

Previously we relied on the implicit assumption filename xor directory, representing two modes. Make this explicit with an internal mode variable. This is just refactoring and should not change the functionality. However we're now more strict about duplicate file= or dir= parameters appearing on

[Libguestfs] [PATCH nbdkit 0/2] file: Add fd=

Patch 1 looks better if you look at it with "git show -w". This adds the ability to explicitly pass file descriptors to the file plugin, matching a recent feature of qemu. Actually this was already possible on some platforms using file=/dev/fd/ but it seems better to support it on all

[Libguestfs] [PATCH nbdkit 2/2] file: Allow a file descriptor to be passed to the plugin

This allows you to pass in a file descriptor to the plugin, eg: $ exec 6<>/var/tmp/fedora-36.img $ ./nbdkit file fd=6 --run 'nbdinfo $uri' Note this was previously possible on most platforms using /dev/fd/, but not all platforms that have file descriptors support this (although it is POSIX)

Re: [Libguestfs] [libnbd PATCH 4/4] states: Use RESYNC to handle more structured reply server bugs

On Fri, Aug 12, 2022 at 08:07:25AM -0500, Eric Blake wrote: > Plus a typo in the subject for 1/4. With everything fixed, the series > is now in as 185195d..0883029. Thanks - as mentioned before I'll give this a good "going over" with fuzzing next week. > Another leniency issue I am exploring is

Re: [Libguestfs] [libnbd PATCH 4/4] states: Use RESYNC to handle more structured reply server bugs

that our test cases use both simple & structured replies, or if not generate the missing test case. Series: Acked-by: Richard W.M. Jones Thanks, Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog:

Re: [Libguestfs] libnbd | Failed pipeline for master | 5bd16353

On Thu, Aug 11, 2022 at 07:42:17AM +, GitLab wrote: > Pipeline #610085241 triggered by ● Richard W.M. Jones >had 3 failed jobs > Failed jobs > ✖ builds x86_64-opensuse-leap-153 Weeks ago I worked with Jim Fehlig to update

[Libguestfs] Rust build failure on debian-11 (was: Fwd: nbdkit | Failed pipeline for master | 539ee349)

Hi Alan, https://gitlab.com/nbdkit/nbdkit/-/jobs/2835506354 cargo build --release Updating crates.io index Downloading crates ... Downloaded bitflags v1.3.2 Downloaded libc v0.2.128 error: failed to parse manifest at

Re: [Libguestfs] [PATCH nbdkit] ci: Expire artifacts after 1 week

On Tue, Aug 09, 2022 at 10:07:37AM +0100, Richard W.M. Jones wrote: > On Tue, Aug 09, 2022 at 09:23:57AM +0200, Martin Kletzander wrote: > > On Mon, Aug 08, 2022 at 09:51:32PM +0100, Richard W.M. Jones wrote: > > >nbdkit uses about 1.1G of storage for artifacts. I have per

Re: [Libguestfs] [PATCH nbdkit] ci: Expire artifacts after 1 week

On Tue, Aug 09, 2022 at 09:23:57AM +0200, Martin Kletzander wrote: > On Mon, Aug 08, 2022 at 09:51:32PM +0100, Richard W.M. Jones wrote: > >nbdkit uses about 1.1G of storage for artifacts. I have personally > >never looked at anything other than the artifacts of the latest fail

[Libguestfs] [PATCH nbdkit] ci: Expire artifacts after 1 week

nbdkit uses about 1.1G of storage for artifacts. I have personally never looked at anything other than the artifacts of the latest failed job, so I don't believe we need to keep these around for long. This commit proposes expiring (and thus deleting) artifacts after 1 week.

[Libguestfs] [PATCH nbdkit] ci: Expire artifacts after 1 week

--- .gitlab-ci.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 122bc0860..6e3cda33a 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -31,6 +31,7 @@ include: '/ci/gitlab.yml' - "**/test-suite.log" public: true when: always +

Re: [Libguestfs] [v2v PATCH] convert_linux: remove LVM2 "devices file"

On Fri, Aug 05, 2022 at 10:44:26AM +0200, Laszlo Ersek wrote: > A recent feature of LVM2 is the "devices file" > . It speeds up > LVM2 PV discovery on a normal system, but an old devices file in a > converted domain (with different hardware)

[Libguestfs] ANNOUNCE: nbdkit 1.32 and libnbd 1.14 released

nbdkit(1). AUTHORS Authors of nbdkit 1.32: Alan Somers Eric Blake Laszlo Ersek Martin Kletzander Nikolaus Rath Richard W.M. Jones COPYRIGHT Copyright (C) 2013-2022 Red Hat Inc. LICENSE Redistribution

Re: [Libguestfs] Anything for libnbd or nbdkit stable releases?

On Fri, Jul 08, 2022 at 06:20:43PM +0100, Richard W.M. Jones wrote: > > There's been a surprising amount of upstream change in these two > projects, so I think we are due new stable releases of both. The last > stable releases were on Feb 24th, about 4.5 months ago. Is there >

Re: [Libguestfs] [v2v PATCH] input-xen: cover RHEL9 OpenSSL crypto settings

e effect > +if the libvirt client library used by virt-v2v is at least version > +8.6.0. > + > =head2 Test libvirt connection to remote Xen host > > Use the L command to list the guests on the remote Xen host: > -- > 2.19.1.3.g30247aa5d201 Reviewed-by: Richard W.M. Jones

Re: [Libguestfs] [PATCH libnbd v2 0/3] Use GNUTLS_NO_SIGNAL if available

On Thu, Jul 28, 2022 at 12:16:15PM -0500, Eric Blake wrote: > On Thu, Jul 28, 2022 at 03:19:07PM +0100, Richard W.M. Jones wrote: > > V1 was here: > > > > https://listman.redhat.com/archives/libguestfs/2022-July/029545.html > > > > The original second patch

Re: [Libguestfs] [PATCH libnbd v2 1/3] lib/crypto: Use GNUTLS_NO_SIGNAL if available

On Thu, Jul 28, 2022 at 12:12:37PM -0500, Eric Blake wrote: > On Thu, Jul 28, 2022 at 03:19:08PM +0100, Richard W.M. Jones wrote: > > libnbd has long used MSG_NOSIGNAL to avoid receiving SIGPIPE if we > > accidentally write on a closed socket, which is a nice alternative to >

[Libguestfs] [PATCH libnbd v2 3/3] docs: Document signal handling

Document that libnbd doesn't set signal handlers, use of MSG_NOSIGNAL or SO_NOSIGPIPE, and optional registration of a global SIGPIPE handler. --- docs/libnbd.pod | 12 1 file changed, 12 insertions(+) diff --git a/docs/libnbd.pod b/docs/libnbd.pod index 7cb2a48473..dd880c3bff 100644

[Libguestfs] [PATCH libnbd v2 2/3] generator: Set SO_NOSIGPIPE on sockets

If the platform supports it, set the SO_NOSIGPIPE socket option on sockets that we create in nbd_connect_* functions (except nbd_connect_socket where the caller gets to choose). Link: https://www.doof.me.uk/2020/09/23/sigpipe-and-how-to-ignore-it/ --- generator/states-connect.c | 17

[Libguestfs] [PATCH libnbd v2 0/3] Use GNUTLS_NO_SIGNAL if available

V1 was here: https://listman.redhat.com/archives/libguestfs/2022-July/029545.html The original second patch is now upstream. As well as updating 1/3 with the feedback, I added two new patches. This has been compile tested on FreeBSD and doesn't seem to break anything, but testing if it

[Libguestfs] [PATCH libnbd v2 1/3] lib/crypto: Use GNUTLS_NO_SIGNAL if available

libnbd has long used MSG_NOSIGNAL to avoid receiving SIGPIPE if we accidentally write on a closed socket, which is a nice alternative to using a SIGPIPE signal handler. However with TLS connections, gnutls did not use this flag and so programs using libnbd + TLS would receive SIGPIPE in some

Re: [Libguestfs] [PATCH libnbd 1/2] lib/crypto: Use GNUTLS_NO_SIGNAL if available

On Thu, Jul 28, 2022 at 02:57:49PM +0200, Laszlo Ersek wrote: > On 07/27/22 18:30, Richard W.M. Jones wrote: > > libnbd has long used MSG_NOSIGNAL to avoid receiving SIGPIPE if we > > accidentally write on a closed socket, which is a nice alternative to > > using a SIGPIPE sig

Re: [Libguestfs] [PATCH libnbd 2/2] lib/crypto.c: Ignore TLS premature termination after write shutdown

On Thu, Jul 28, 2022 at 07:26:16AM -0500, Eric Blake wrote: > On Wed, Jul 27, 2022 at 05:30:59PM +0100, Richard W.M. Jones wrote: > > qemu-nbd doesn't call gnutls_bye to cleanly shut down the connection > > after we send NBD_CMD_DISC. When copying from a qemu-nbd server (or &g

Re: [Libguestfs] [PATCH libnbd 1/2] lib/crypto: Use GNUTLS_NO_SIGNAL if available

On Thu, Jul 28, 2022 at 07:22:23AM -0500, Eric Blake wrote: > On Wed, Jul 27, 2022 at 05:30:58PM +0100, Richard W.M. Jones wrote: > > libnbd has long used MSG_NOSIGNAL to avoid receiving SIGPIPE if we > > accidentally write on a closed socket, which is a nice alternative to >

[Libguestfs] [PATCH libnbd 2/2] lib/crypto.c: Ignore TLS premature termination after write shutdown

qemu-nbd doesn't call gnutls_bye to cleanly shut down the connection after we send NBD_CMD_DISC. When copying from a qemu-nbd server (or any operation which calls nbd_shutdown) you will see errors like this: $ nbdcopy nbds://foo?tls-certificates=/var/tmp/pki null:

[Libguestfs] [PATCH libnbd 1/2] lib/crypto: Use GNUTLS_NO_SIGNAL if available

libnbd has long used MSG_NOSIGNAL to avoid receiving SIGPIPE if we accidentally write on a closed socket, which is a nice alternative to using a SIGPIPE signal handler. However with TLS connections, gnutls did not use this flag and so programs using libnbd + TLS would receive SIGPIPE in some

Re: [Libguestfs] [nbdkit PATCH] tests: Better quoting for cleanup_fn

On Tue, Jul 26, 2022 at 06:56:04AM -0500, Eric Blake wrote: > On Tue, Jul 26, 2022 at 07:13:41AM +0200, Laszlo Ersek wrote: > > > > > > -for (( _i = 0; _i < ${#_cleanup_hook[@]}; ++_i )); do > > > -${_cleanup_hook[_i]} > > > +for (( _i = 0; _i < $_cleanup_hook_count; ++_i )); do >

Re: [Libguestfs] [nbdkit PATCH] tests: Better quoting for cleanup_fn

On Tue, Jul 26, 2022 at 12:22:11PM +0200, Laszlo Ersek wrote: > On 07/26/22 12:17, Laszlo Ersek wrote: > > On 07/26/22 10:11, Richard W.M. Jones wrote: > >> On Tue, Jul 26, 2022 at 07:13:41AM +0200, Laszlo Ersek wrote: > >>> On 07/25/22 16:02, Eric Blake wrote:

Re: [Libguestfs] [nbdkit PATCH] tests: Better quoting for cleanup_fn

On Tue, Jul 26, 2022 at 09:11:24AM +0100, Richard W.M. Jones wrote: > On Tue, Jul 26, 2022 at 07:13:41AM +0200, Laszlo Ersek wrote: > > On 07/25/22 16:02, Eric Blake wrote: > > > +for (( _i = 0; _i < $_cleanup_hook_count; ++_i )); do > > > +l

Re: [Libguestfs] [nbdkit PATCH] tests: Better quoting for cleanup_fn

On Tue, Jul 26, 2022 at 07:13:41AM +0200, Laszlo Ersek wrote: > On 07/25/22 16:02, Eric Blake wrote: > > +for (( _i = 0; _i < $_cleanup_hook_count; ++_i )); do > > +local -n _hook=_cleanup_hook$_i > > +"${_hook[@]}" > > BTW, "nameref" variables seem like a relatively new

Re: [Libguestfs] [nbdkit PATCH] tests: Better quoting for cleanup_fn

On Mon, Jul 25, 2022 at 10:01:21AM -0500, Eric Blake wrote: > On Mon, Jul 25, 2022 at 04:13:41PM +0200, Laszlo Ersek wrote: > > > > That is, our commands were munged by IFS splitting, because we stored > > > commands in a flat variable. Fix it by instead using an array > > > variable per

Re: [Libguestfs] [v2v PATCH] output/create_libvirt_xml: generate @check='none' CPU attribute

On Fri, Jul 22, 2022 at 12:53:00PM +0100, Daniel P. Berrangé wrote: > On Fri, Jul 22, 2022 at 01:49:21PM +0200, Laszlo Ersek wrote: > > On 07/22/22 11:50, Richard W.M. Jones wrote: > > > On Fri, Jul 22, 2022 at 10:42:48AM +0100, Daniel P. Berrangé wrote: > > >> On F

Re: [Libguestfs] [v2v PATCH] output/create_libvirt_xml: generate @check='none' CPU attribute

On Fri, Jul 22, 2022 at 10:42:48AM +0100, Daniel P. Berrangé wrote: > On Fri, Jul 22, 2022 at 10:34:44AM +0100, Richard W.M. Jones wrote: > > Sorry for the delayed response to this. I see you've posted an > > updated patch, so this is just a bit of FYI. > > > > I ori

Re: [Libguestfs] [v2v PATCH v2] output/create_libvirt_xml: relax VCPU feature checking for "qemu64"

On Fri, Jul 22, 2022 at 09:36:27AM +0200, Laszlo Ersek wrote: > When the source domain doesn't specify a VCPU model ("s_cpu_model" is > None), and the guest OS is assumed to work with the default VCPU model > ("gcaps_default_cpu" is true), we don't output any element. In that > case, libvirtd

Re: [Libguestfs] [v2v PATCH] output/create_libvirt_xml: generate @check='none' CPU attribute

Sorry for the delayed response to this. I see you've posted an updated patch, so this is just a bit of FYI. I originally added CPU modelling in commit 11505e4b84 (March 2017): https://github.com/libguestfs/virt-v2v/commit/11505e4b84ce8d7eda4e2a275fdcecc5f2a3288d What we were actually trying to

Re: [Libguestfs] [PATCH] Add simple parsing logic for hostname file.

On Wed, Jul 20, 2022 at 01:36:25PM -0400, dzr...@gmail.com wrote: > On Wed, 2022-07-20 at 12:45 +0100, Richard W.M. Jones wrote: > > How about this instead? > > > > It actually changes the behaviour slightly.  Currently a > > /etc/hostname > > which contains the s

Re: [Libguestfs] [p2v PATCH] virt-p2v-make-*.in: make @datadir@ and @libdir@ resolvable

fix@" > +exec_prefix="@exec_prefix@" > > # Parse the command line arguments. > shortopts=o:vV > diff --git a/virt-p2v-make-kiwi.in b/virt-p2v-make-kiwi.in > index 987de11b54aa..a648f6a3886c 100644 > --- a/virt-p2v-make-kiwi.in > +++ b/virt-p2v-make-kiwi.in > @@ -20,6

Re: [Libguestfs] [v2v PATCH] output/create_libvirt_xml: generate @check='none' CPU attribute

On Wed, Jul 20, 2022 at 01:09:13PM +0200, Laszlo Ersek wrote: > We currently don't generate any @check attribute for the /domain/cpu > element, which causes the following libvirtd behavior > : > > > Once the domain starts, libvirt will

Re: [Libguestfs] [PATCH] always 'max' for the appliance CPU model on all targes except ppc

On Wed, Jul 20, 2022 at 10:14:00AM +0200, Laszlo Ersek wrote: > On 05/27/22 10:21, Laszlo Ersek wrote: > > On 05/25/22 15:30, Daniel P. Berrangé wrote: > >> The current logic for selecting CPU model to use with the appliance > >> selects 'max' for all architectures except for ppc64le and aarch64.

Re: [Libguestfs] [PATCH] Add simple parsing logic for hostname file.

Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html >From 3b6897c8ba9c8d438e8599af93188e65cced6ec8 Mon Sep 17 00:00:00 2001 From: "Richard W.M. Jones" Date: Wed, 20 Jul 2022 12

Re: [Libguestfs] nbdkit test-checkwrite.sh failing occasionally on ppc64le

On Tue, Jul 19, 2022 at 04:41:02PM +0200, Laszlo Ersek wrote: > On 07/19/22 16:17, Richard W.M. Jones wrote: > > > Note I'm not claiming the assertion will not fail! Just that the > > assertion is correct, and if it fails we've got a bug. > > Fair enough, but how

Re: [Libguestfs] nbdkit test-checkwrite.sh failing occasionally on ppc64le

On Tue, Jul 19, 2022 at 03:21:05PM +0200, Laszlo Ersek wrote: > On 07/19/22 11:45, Richard W.M. Jones wrote: > > > > Fixed in: > > https://gitlab.com/nbdkit/nbdkit/-/commit/16a13fa9e1f3c4cbc0fae89e364f832e1a41fb40 > > https://gitlab

Re: [Libguestfs] nbdkit test-checkwrite.sh failing occasionally on ppc64le

Fixed in: https://gitlab.com/nbdkit/nbdkit/-/commit/16a13fa9e1f3c4cbc0fae89e364f832e1a41fb40 https://gitlab.com/nbdkit/nbdkit/-/commit/2c9a557f2ebe5a8647cc44dbdd7c157356783306 https://gitlab.com/nbdkit/nbdkit/-/commit/aaf5484462cd06505b7a1a8103a899d3551c451b -- Richard Jones, Virtualization

Re: [Libguestfs] nbdkit test-checkwrite.sh failing occasionally on ppc64le

No idea why this was only failing on ppc64le, since I've finally got a reproducer that works everywhere (see attached). $ ./bound.sh + rm -f /tmp/sock + sleep 1 + nbdkit --exit-with-parent -U /tmp/sock sh - --filter=checkwrite + nbdsh -u 'nbd+unix:///?socket=/tmp/sock' -c 'h.zero (655360, 262144,

Re: [Libguestfs] nbdkit test-checkwrite.sh failing occasionally on ppc64le

Arrrgh! https://koji.fedoraproject.org/koji/taskinfo?taskID=89618509 FAIL: test-checkwrite.sh Last week I reserved a ppc64le machine which had almost exactly the same spec as one of the Koji builders and ran both the full test suite and this single test thousands of times over a few days

Re: [Libguestfs] libnbd | Failed pipeline for master | d11ddb6b

Commit Author ● Richard W.M. Jones > > Pipeline #589493913 triggered by ● Richard W.M. Jones >had 19 failed jobs > Failed jobs > ✖ buildsx86_64-debian-11 > ✖ builds x86_64-alpine-314 > ✖

Re: [Libguestfs] [PATCH virt-v2v v3 2/2] -o rhv: Wait for the NBD server to exit to avoid a race with unmounting

On Fri, Jul 15, 2022 at 03:30:56PM +0200, Laszlo Ersek wrote: > On 07/15/22 14:11, Richard W.M. Jones wrote: > > Less churn version as described here: > > https://listman.redhat.com/archives/libguestfs/2022-July/029483.html > > > > > > Couple of confusing point

[Libguestfs] [PATCH virt-v2v v3 2/2] -o rhv: Wait for the NBD server to exit to avoid a race with unmounting

https://bugzilla.redhat.com/show_bug.cgi?id=1953286#c26 --- output/output_rhv.ml | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/output/output_rhv.ml b/output/output_rhv.ml index 15a2c14adf..45f831e39a 100644 --- a/output/output_rhv.ml +++ b/output/output_rhv.ml @@

[Libguestfs] [PATCH virt-v2v v3 1/2] output: Permit output modes to wait on the local NBD server

Output.output_to_local_file is used by several output modes that write to local files or devices. It launches an instance of qemu-nbd or nbdkit connected to the local file. Previously we unconditionally added an On_exit handler to kill the NBD server. This is usually safe because nbdcopy

[Libguestfs] [PATCH virt-v2v v3 2/2] -o rhv: Wait for the NBD server to exit to avoid a race with unmounting

Less churn version as described here: https://listman.redhat.com/archives/libguestfs/2022-July/029483.html ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs

Re: [Libguestfs] [PATCH virt-v2v 1/2] output: Permit output modes to wait on the local NBD server

On Fri, Jul 15, 2022 at 11:41:34AM +0100, Richard W.M. Jones wrote: > val output_to_local_file : ?changeuid:((unit -> unit) -> unit) -> > - ?compressed:bool -> > + ?compressed:bool

[Libguestfs] [PATCH virt-v2v 0/2] -o rhv: Wait for the NBD server to exit to avoid race with unmounting

This is an alternative approach to https://listman.redhat.com/archives/libguestfs/2022-July/029461.html This doesn't include a timeout. We expect that the NBD servers will exit when killed, if not it's a serious bug somewhere. Rich. ___ Libguestfs

[Libguestfs] [PATCH virt-v2v 1/2] output: Permit output modes to wait on the local NBD server

Output.output_to_local_file is used by several output modes that write to local files or devices. It launches an instance of qemu-nbd or nbdkit connected to the local file. Previously we unconditionally added an On_exit handler to kill the NBD server. This is usually safe because nbdcopy

[Libguestfs] [PATCH virt-v2v 2/2] -o rhv: Wait for the NBD server to exit to avoid a race with unmounting

https://bugzilla.redhat.com/show_bug.cgi?id=1953286#c26 --- output/output_rhv.ml | 17 +++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/output/output_rhv.ml b/output/output_rhv.ml index 5e3c031e38..f57ac9099e 100644 --- a/output/output_rhv.ml +++

Re: [Libguestfs] [PATCH common 4/4] mltools: Allow waiting for killed PIDs

On Fri, Jul 15, 2022 at 09:30:38AM +0200, Laszlo Ersek wrote: > On 07/14/22 14:36, Richard W.M. Jones wrote: > > Add a new, optional [?wait] parameter to On_exit.kill, allowing > > programs to wait for a number of seconds for the subprocess to exit. > > --- > >

Re: [Libguestfs] [PATCH common 3/4] mltools: Introduce priority for ordering actions in On_exit

On Fri, Jul 15, 2022 at 09:17:26AM +0200, Laszlo Ersek wrote: > I think this patch is good, but I'm surprised that ~prio:0 is for > placing stuff at the *end* of the list, where 1000 is the default. This > ordering differs from both the firstboot script priorities we did > recently (lower prio

Re: [Libguestfs] [PATCH common 3/4] mltools: Introduce priority for ordering actions in On_exit

On Fri, Jul 15, 2022 at 09:17:26AM +0200, Laszlo Ersek wrote: > On 07/14/22 14:36, Richard W.M. Jones wrote: > > Introduce a new, optional ?prio parameter which can be used to control > > the order that actions run on exit. By default actions have priority > > 1000. Higher

Re: [Libguestfs] [PATCH common 2/4] mltools: Reimplement On_exit to use a list of actions

On Fri, Jul 15, 2022 at 09:12:24AM +0200, Laszlo Ersek wrote: > > +let do_action action = > > + try > > +match action with > > +| Unlink file -> Unix.unlink file > > +| Rm_rf dir -> > > + let cmd = sprintf "rm -rf %s" (Filename.quote dir) in ... > > (2) Shouldn't we use two

Re: [Libguestfs] [PATCH common 1/4] mltools: Rename On_exit.rmdir to On_exit.rm_rf

On Fri, Jul 15, 2022 at 08:36:19AM +0200, Laszlo Ersek wrote: > On 07/14/22 14:36, Richard W.M. Jones wrote: > > Make it clearer what this function does and that it's potentially > > dangerous. The functionality itself is unchanged. > > --- > > mltools/on_exit.ml | 2

Re: [Libguestfs] [PATCH common 1/4] mltools: Rename On_exit.rmdir to On_exit.rm_rf

On Fri, Jul 15, 2022 at 08:36:19AM +0200, Laszlo Ersek wrote: > On 07/14/22 14:36, Richard W.M. Jones wrote: > > Make it clearer what this function does and that it's potentially > > dangerous. The functionality itself is unchanged. > > --- > > mltools/on_exit.ml | 2

Re: [Libguestfs] [guestfs-tools PATCH 2/2] sysprep: advise against cloning VMs with internal full disk encryption

ould examine > the L above and the guest afterwards. > -- For the whole series: Reviewed-by: Richard W.M. Jones Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets yo

[Libguestfs] [PATCH virt-v2v 2/3] -o rhv: Unmount the temporary NFS mountpoint as late as possible

To partially avoid a potential race against nbdkit or qemu-nbd releasing files on the mountpoint before they exit, unmount as late as we can. See also https://bugzilla.redhat.com/show_bug.cgi?id=1953286#c26 --- output/output_rhv.ml | 4 ++-- common | 2 +- 2 files changed, 3

[Libguestfs] [PATCH virt-v2v 3/3] output: Wait up to 5 seconds for nbdkit or qemu-nbd to exit

Before exiting virt-v2v, wait a few seconds for nbdkit or qemu-nbd subprocesses to exit. We expect they will exit almost instaneously so this won't have much effect. However in the -o rhv case it's helpful to wait briefly to ensure that the NFS mountpoint is released before we try to unmount it

[Libguestfs] [PATCH virt-v2v 0/3] -o rhv: Try to avoid race condition when exiting

Requires: https://listman.redhat.com/archives/libguestfs/2022-July/029453.html This patch series tries to avoid a race condition when using -o rhv mode when virt-v2v exits after a successful conversion. The potential race is described in this bug report:

[Libguestfs] [PATCH virt-v2v 1/3] common: Adapt to renamed function On_exit.rmdir -> On_exit.rm_rf

This function was renamed to make it clearer what it does (and that it's potentially dangerous). The functionality is unchanged. --- input/OVA.ml | 2 +- input/parse_domain_from_vmx.ml | 2 +- lib/nbdkit.ml | 2 +- lib/qemuNBD.ml | 2 +-

[Libguestfs] [PATCH common 4/4] mltools: Allow waiting for killed PIDs

Add a new, optional [?wait] parameter to On_exit.kill, allowing programs to wait for a number of seconds for the subprocess to exit. --- mltools/on_exit.ml | 30 +- mltools/on_exit.mli | 14 -- 2 files changed, 33 insertions(+), 11 deletions(-) diff --git

[Libguestfs] [PATCH common 2/4] mltools: Reimplement On_exit to use a list of actions

Previously we used separate lists of files, dirs, pids, etc. This makes it harder to introduce new features to reorder actions. Reimplement the module so we use a simple list of actions, where each action can have type File, Rm_rf, Kill, etc. Iterate through this list on exit to execute the

[Libguestfs] [PATCH common 1/4] mltools: Rename On_exit.rmdir to On_exit.rm_rf

Make it clearer what this function does and that it's potentially dangerous. The functionality itself is unchanged. --- mltools/on_exit.ml | 2 +- mltools/on_exit.mli | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/mltools/on_exit.ml b/mltools/on_exit.ml index

[Libguestfs] [PATCH common 0/4] mltools: Introduce priority for ordering actions

https://bugzilla.redhat.com/show_bug.cgi?id=1953286#c26 This set of changes enhances the common/mltools On_exit module to: - allow actions to be ordered using a priority field - allow waiting for killed subprocesses to exit (with some caveats) Related set of changes to virt-v2v will follow.

[Libguestfs] [PATCH common 3/4] mltools: Introduce priority for ordering actions in On_exit

Introduce a new, optional ?prio parameter which can be used to control the order that actions run on exit. By default actions have priority 1000. Higher numbered actions run first. Lower numbered actions run last. So to have an action run at the very end before exit you might use ~prio:0 Note

Re: [Libguestfs] nbdkit test-checkwrite.sh failing occasionally on ppc64le

On Wed, Jul 13, 2022 at 02:13:14PM +0100, Richard W.M. Jones wrote: > On Wed, Jul 13, 2022 at 03:10:37PM +0200, Laszlo Ersek wrote: > > On 07/13/22 13:27, Richard W.M. Jones wrote: > > > > > > It's not 100% reproducible, but it seems to happen reasonably often. > >

Re: [Libguestfs] A few Coverity errors in nbdkit

On Wed, Jul 13, 2022 at 04:43:38PM -0500, Eric Blake wrote: > On Tue, Jul 12, 2022 at 04:13:28PM -0500, Eric Blake wrote: > > On Tue, Jul 12, 2022 at 07:22:31PM +0100, Richard W.M. Jones wrote: > > > Hi Eric, > > > > > > None of this is important

Re: [Libguestfs] nbdkit test-checkwrite.sh failing occasionally on ppc64le

On Wed, Jul 13, 2022 at 03:10:37PM +0200, Laszlo Ersek wrote: > On 07/13/22 13:27, Richard W.M. Jones wrote: > > > > It's not 100% reproducible, but it seems to happen reasonably often. > > Unfortunately it only happens on ppc64le (so far). The error message > > lack

[Libguestfs] nbdkit test-checkwrite.sh failing occasionally on ppc64le

It's not 100% reproducible, but it seems to happen reasonably often. Unfortunately it only happens on ppc64le (so far). The error message lacks sufficient detail to tell what's going on. nbdkit: backend.c:624: backend_pread: Assertion `backend_valid_range (c, offset, count)' failed. nbdkit:

[Libguestfs] A few Coverity errors in nbdkit

Hi Eric, None of this is important, and may not even be bugs, but here are a few issues raised with the latest run of Coverity on nbdkit 1.30.7. ** *** filters/multi-conn It seems as if "group_vector" is really leaked from

Re: [Libguestfs] Error ret=-1 with EINTR in nbd_connect_systemd_socket_activation()

On Tue, Jul 12, 2022 at 05:33:41AM +, Motohiro Kawahito wrote: > Hi, I’d like to connect to an encrypted QCOW2 file by > nbd_connect_systemd_socket_activation(), but I got ret=-1 with EINTR. In our > program, signals are frequently occurred, so I think this is the background of > the problem.

Re: [Libguestfs] EVE-NG - VM - Compression

On Mon, Jul 11, 2022 at 06:53:02AM -0500, Jaime Rodriguez wrote: > Hello Team, > I am trying to compress a windows 10VM on Eve-ng with the 'virt-sparsify'.  > > Initial Command Run:  virt-sparsify  --compress virtioa.qcow2 > compressedvirtioa.qcow2 > > Added debugging and ran again with '-v -x'

Re: [Libguestfs] [v2v PATCH 2/2] input-xen: replace "enable LEGACY crypto" advice with targeted ssh options

pto algorithms required > +to interoperate with RHEL 5 sshd are disabled. To enable them, you may > +need to add the following C stanza to your F<~/.ssh/config>: > > - # update-crypto-policies --set LEGACY > + Host xen.example.com > + KexAlgorithms +diffie-hellman

[Libguestfs] Anything for libnbd or nbdkit stable releases?

There's been a surprising amount of upstream change in these two projects, so I think we are due new stable releases of both. The last stable releases were on Feb 24th, about 4.5 months ago. Is there anything you're waiting to get in? Rich. -- Richard Jones, Virtualization Group, Red Hat

Re: [Libguestfs] [PATCH libnbd v4 2/2] copy: Use preferred block size for copying

On Thu, Jul 07, 2022 at 02:35:41PM -0500, Eric Blake wrote: > > +++ b/copy/copy-file-to-qcow2-compressed.sh > > > + > > +# Create a compressed qcow2 file1. > > +# > > +# sparse-random files should compress easily because by default each > > +# block uses repeated bytes. > > +qemu-img create -f

Re: [Libguestfs] [nbdkit] ARRAY_SIZE [was: vddk: Demote another "phone home" error message to debug]

On Thu, Jul 07, 2022 at 03:10:19PM -0500, Eric Blake wrote: > There's also a question of whether the error message when the > constraint fails is going to be legible enough to give the developer a > hint as to what they did wrong when they pass in a non-array type; but > I'm less worried about

Re: [Libguestfs] [PATCH libnbd v4 1/2] copy: Store the preferred block size in the operations struct

On Thu, Jul 07, 2022 at 01:54:55PM -0500, Eric Blake wrote: > > + if (S_ISREG (stat.st_mode)) /* Regular file. */ > > +return file_create (filename, fd, > > +stat.st_size, (uint64_t) stat.st_blksize, false, > > d); > > Is the cast to uint64_t actually needed? No

Re: [Libguestfs] [nbdkit] ARRAY_SIZE [was: vddk: Demote another "phone home" error message to debug]

om/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW >From c62cffde2be0debcbb35590a1ffc993161fcdb28 Mon Sep 17 00:00:00

Re: [Libguestfs] [PATCH nbdkit] vddk: Demote another "phone home" error message to debug

On Thu, Jul 07, 2022 at 12:58:53PM +0200, Laszlo Ersek wrote: > I think it's time for us to introduce the ARRAY_SIZE macro. Yeah I thought about that while I was writing the patch :-) I'll do as you suggest, thanks. Rich. -- Richard Jones, Virtualization Group, Red Hat

[Libguestfs] [PATCH nbdkit] vddk: Demote another "phone home" error message to debug

Reported-by: Ming Xie Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2104720 --- plugins/vddk/vddk.c | 31 --- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/plugins/vddk/vddk.c b/plugins/vddk/vddk.c index dbd3fdbe..1ed9fc53 100644 ---

<    5   6   7   8   9   10   11   12   13   14   >