Re: [Libguestfs] [PATCH libnbd] generator: Pass LISTEN_FDNAMES=nbd with systemd socket activation

2023-02-07 Thread Richard W.M. Jones
We worried about getenv's safety (lack of) for quite a while when we were writing libguestfs, which uses a lot more environment variables in many more places. But we decided there was simply nothing we could do about it, and it was easier not to worry :-) Rich. -- Richard Jones,

[Libguestfs] [PATCH v2v] -o libvirt: Use cpu='host-model' for gcaps_default_cpu = false

2023-02-07 Thread Richard W.M. Jones
For RHEL >= 9 / x86-64 guests we cannot use the default qemu CPU (eg. "qemu64"), and so we have a mechanism for conversion to indicate to the output modes that a more capable CPU is required. We previously picked cpu='host-passthrough' (ie. the equivalent of qemu's -cpu host). However this is

Re: [Libguestfs] [PATCH v2v] convert: linux: Require host cpu for all RHEL-alike >= 9

2023-02-06 Thread Richard W.M. Jones
On Mon, Feb 06, 2023 at 12:37:30PM +, Daniel P. Berrangé wrote: > On Mon, Feb 06, 2023 at 12:19:40PM +0000, Richard W.M. Jones wrote: > > RHEL >= 9 and compatible distros like Rocky >= 9 will not boot using > > the default qemu CPU. You will see an error at boot: > &g

[Libguestfs] [PATCH v2v] convert: linux: Require host cpu for all RHEL-alike >= 9

2023-02-06 Thread Richard W.M. Jones
RHEL >= 9 and compatible distros like Rocky >= 9 will not boot using the default qemu CPU. You will see an error at boot: Fatal glibc error: CPU does not support x86-64-v2 Instead you need to use -cpu host. In commit f28757c6d1 ("convert_linux: set "gcaps_default_cpu = false" for x86_64

[Libguestfs] [PATCH v2v] convert: linux: Require host cpu for all RHEL-alike >= 9

2023-02-06 Thread Richard W.M. Jones
Already pushed as: https://github.com/libguestfs/virt-v2v/commit/9f12b95bbe7bc2850ce4ba134c46a3cc5fd1167d I'm posting to the list for awareness. Rich. ___ Libguestfs mailing list Libguestfs@redhat.com

Re: [Libguestfs] [PATCH nbdkit 0/6] curl: Use a curl handle pool

2023-02-05 Thread Richard W.M. Jones
On Sun, Feb 05, 2023 at 05:09:21PM +, Richard W.M. Jones wrote: > On Sun, Feb 05, 2023 at 04:35:41PM +0000, Richard W.M. Jones wrote: > > I'm still adding instrumentation to see if the theory above is right, > > plus I have no idea how to fix this. > > Turns out

Re: [Libguestfs] [PATCH nbdkit 0/6] curl: Use a curl handle pool

2023-02-05 Thread Richard W.M. Jones
On Sun, Feb 05, 2023 at 04:35:41PM +, Richard W.M. Jones wrote: > I'm still adding instrumentation to see if the theory above is right, > plus I have no idea how to fix this. Turns out I didn't need to add instrumentation. Simply forcing nbdcopy to use at most 1 request per connection

Re: [Libguestfs] [PATCH nbdkit 0/6] curl: Use a curl handle pool

2023-02-05 Thread Richard W.M. Jones
On Sat, Feb 04, 2023 at 12:34:52PM +, Richard W.M. Jones wrote: > Anyway, this all seems to work, but it actually reduces performance :-( > > In particular this simple test slows down quite substantially: > > time ./nbdkit -r -U - curl file:/var/tmp/fedora-36.img

Re: [Libguestfs] [PATCH libnbd] generator: Pass LISTEN_FDNAMES=nbd with systemd socket activation

2023-02-05 Thread Richard W.M. Jones
On Sun, Feb 05, 2023 at 09:10:29AM +0100, Laszlo Ersek wrote: > - ignore the whole thing? This one. > - update the documentation, add locking APIs, implemented with a mutex? I don't believe that would help since other libraries or the main program might be calling setenv. We cannot do anything

[Libguestfs] [PATCH nbdkit 5/6] curl: Use the parallel thread model

2023-02-04 Thread Richard W.M. Jones
After previous changes, it is now safe to use the parallel thread model in this plugin. The locking in pool.c protects a single curl handle from being used from multiple threads. An advantage of this is we can now combine the curl plugin with filters such as readahead and scan. ---

[Libguestfs] [PATCH nbdkit 4/6] curl: Introduce concept of getting/putting handles from a common pool

2023-02-04 Thread Richard W.M. Jones
Introduce the concept of a pool of curl handles from which the main code has to take a handle (get_handle()), and later return it to the pool (put_handle()). When a handle is taken from the pool it can be used exclusively by the main code until it is returned. The "pool" only has at most one

[Libguestfs] [PATCH nbdkit 6/6] curl: Complete implementation of the curl handle pool

2023-02-04 Thread Richard W.M. Jones
This commit implements the curl handle pool. By default it can grow to up to 4 curl handles, shared between all NBD connections. You can change this using the new connections=N parameter. --- plugins/curl/nbdkit-curl-plugin.pod | 7 +++ plugins/curl/curldefs.h | 2 +

[Libguestfs] [PATCH nbdkit 2/6] curl: Split out the libcurl handle from the plugin handle

2023-02-04 Thread Richard W.M. Jones
In a future commit we will create a pool of libcurl handles. In order to prepare for that, split out the libcurl handle and associated fields and buffers, from the plugin handle. The plugin handle (for the moment) contains a pointer to the libcurl handle so there is still a 1-1 relationship

[Libguestfs] [PATCH nbdkit 3/6] curl: Fix declarations of globals in "curldefs.h"

2023-02-04 Thread Richard W.M. Jones
Tidy up previous work on this plugin so that every global is declared in "curldefs.h". Move the HAVE_CURLINFO_CONTENT_LENGTH_DOWNLOAD_T macro to this header file too. --- plugins/curl/curldefs.h | 16 plugins/curl/curl.c | 11 +-- 2 files changed, 17 insertions(+),

[Libguestfs] [PATCH nbdkit 1/6] curl: Rename curl_handle -> handle

2023-02-04 Thread Richard W.M. Jones
Trivial refactoring to make future changes easier. --- plugins/curl/curldefs.h | 4 ++-- plugins/curl/curl.c | 18 +- plugins/curl/scripts.c | 14 +++--- 3 files changed, 18 insertions(+), 18 deletions(-) diff --git a/plugins/curl/curldefs.h

[Libguestfs] [PATCH nbdkit 0/6] curl: Use a curl handle pool

2023-02-04 Thread Richard W.M. Jones
This experimental series changes the way that the curl plugin deals with libcurl handles. It also changes the thread model of the plugin from SERIALIZE_REQUESTS to PARALLEL. Currently one NBD connection opens one libcurl handle. This also implies one TCP connection to the web server. If you

Re: [Libguestfs] Two multi-conn questions

2023-02-02 Thread Richard W.M. Jones
On Thu, Feb 02, 2023 at 12:54:16PM -0600, Eric Blake wrote: > On Thu, Feb 02, 2023 at 04:26:04PM +0000, Richard W.M. Jones wrote: > ... > > > > $ time nbdkit -r curl > > > > https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img > > >

Re: [Libguestfs] Two multi-conn questions

2023-02-02 Thread Richard W.M. Jones
On Thu, Feb 02, 2023 at 10:04:20AM -0600, Eric Blake wrote: > On Wed, Feb 01, 2023 at 12:40:45PM +0000, Richard W.M. Jones wrote: > > Hi Eric, > > > > A couple of semi-related multi-conn questions. > > > > (1) nbdkit-curl-plugin does not currently advertise multi

Re: [Libguestfs] [PATCH nbdkit] curl: Enable multi-conn for read-only connections

2023-02-02 Thread Richard W.M. Jones
On Thu, Feb 02, 2023 at 09:18:34AM -0600, Eric Blake wrote: > On Thu, Feb 02, 2023 at 03:06:30PM +0000, Richard W.M. Jones wrote: > > Multi-conn is enabled only when we know the connection is read-only: > > > > $ ./nbdkit -r curl file:/var/tmp/jammy-server-clo

[Libguestfs] [PATCH nbdkit] curl: Enable multi-conn for read-only connections

2023-02-02 Thread Richard W.M. Jones
Multi-conn is enabled only when we know the connection is read-only: $ ./nbdkit -r curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run ' nbdinfo $uri ' | grep can_multi_conn can_multi_conn: true $ ./nbdkit curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run ' nbdinfo $uri

Re: [Libguestfs] [p2v PATCH v2 10/11] Reenable the OpenStack output mode

2023-02-02 Thread Richard W.M. Jones
ad' drivers, since we do not > - * support passing all the options for them. > + * Ignore the 'rhv-upload' driver, since we do not support passing all the > + * options for it. > */ >if (STRNEQ (name, "vdsm") && > - STRNEQ (name, "ope

Re: [Libguestfs] [p2v PATCH v2 09/11] gui: expose "p2v.output.misc" (-oo)

2023-02-02 Thread Richard W.M. Jones
> +Note that the C text entry behaves differently from the You can use S> which will add a non-breaking space to prevent the output from being wrapped (not that it's likely to be line-wrapped in the current position, but it might be good practice). > +other text entries -- its forma

[Libguestfs] Two multi-conn questions

2023-02-01 Thread Richard W.M. Jones
Hi Eric, A couple of semi-related multi-conn questions. (1) nbdkit-curl-plugin does not currently advertise multi-conn. However forcing multi-conn makes it faster, eg: $ time nbdkit -r curl https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img timeout=2000 --run '

Re: [Libguestfs] Preliminary release notes for libguestfs 1.50, guestfs-tools 1.50

2023-01-31 Thread Richard W.M. Jones
On Tue, Jan 31, 2023 at 01:03:46PM -0600, Eric Blake wrote: > On Tue, Jan 31, 2023 at 06:36:04PM +0000, Richard W.M. Jones wrote: > > > > $SUBJECT here ... > > > > > > https://github.com/libguestfs/libguestfs/blob/master/docs/guestfs-release-notes-1.50.pod &g

[Libguestfs] Preliminary release notes for libguestfs 1.50, guestfs-tools 1.50

2023-01-31 Thread Richard W.M. Jones
$SUBJECT here ... https://github.com/libguestfs/libguestfs/blob/master/docs/guestfs-release-notes-1.50.pod https://github.com/rwmjones/guestfs-tools/blob/master/docs/guestfs-tools-release-notes-1.50.pod Any extra stuff that's pending for either project that isn't too invasive for 1.50,

Re: [Libguestfs] [libnbd PATCH v2 3/3] nbdsh: Improve --help and initial banner contents.

2023-01-31 Thread Richard W.M. Jones
On Tue, Jan 31, 2023 at 10:41:46AM -0600, Eric Blake wrote: > On Tue, Jan 31, 2023 at 11:52:27AM +0000, Richard W.M. Jones wrote: > > On Tue, Jan 31, 2023 at 01:33:25PM +0200, Nir Soffer wrote: > > > On Tue, Jan 31, 2023 at 12:34 AM Richard W.M. Jones > > > wrote: >

Re: [Libguestfs] [PATCH libnbd v2 3/4] generator: Add APIs to get/set the socket activation socket name

2023-01-31 Thread Richard W.M. Jones
On Tue, Jan 31, 2023 at 10:29:00AM -0600, Eric Blake wrote: > On Mon, Jan 30, 2023 at 10:55:20PM +0000, Richard W.M. Jones wrote: > > To allow us to name the socket passed down to the NBD server when > > calling nbd_connect_systemd_socket_activation(3), we need to add the > >

Re: [Libguestfs] [PATCH libnbd] generator: Pass LISTEN_FDNAMES=nbd with systemd socket activation

2023-01-31 Thread Richard W.M. Jones
On Tue, Jan 31, 2023 at 01:49:53PM +0100, Laszlo Ersek wrote: > On 1/28/23 13:47, Richard W.M. Jones wrote: > > systemd allows sockets passed through socket activation to be named > > with the protocol they require. We only ever pass one socket, name > > it. This

Re: [Libguestfs] [libnbd PATCH v2 3/3] nbdsh: Improve --help and initial banner contents.

2023-01-31 Thread Richard W.M. Jones
On Tue, Jan 31, 2023 at 01:33:25PM +0200, Nir Soffer wrote: > On Tue, Jan 31, 2023 at 12:34 AM Richard W.M. Jones wrote: > > > > On Fri, Nov 04, 2022 at 04:18:31PM -0500, Eric Blake wrote: > > > Document all options in --help output. If -n is not in use, then > >

[Libguestfs] [PATCH libnbd v2 4/4] generator/states-connect-socket-activation.c: Set LISTEN_FDNAMES

2023-01-30 Thread Richard W.M. Jones
When the user calls nbd_set_socket_activation_name before calling nbd_connect_system_socket_activation, pass the name down to the server through LISTEN_FDNAMES. This has no effect unless the new API has been called to set the socket name to a non-empty string. ---

[Libguestfs] [PATCH libnbd v2 0/4] Pass LISTEN_FDNAMES with systemd socket activation

2023-01-30 Thread Richard W.M. Jones
This is an alternative approach to https://listman.redhat.com/archives/libguestfs/2023-January/030535.html After discussing this with Dan Berrange we came to the conclusion that you really might want to set LISTEN_FDNAMES to arbitrary short strings (or not set it). Especially when talking to

[Libguestfs] [PATCH libnbd v2 3/4] generator: Add APIs to get/set the socket activation socket name

2023-01-30 Thread Richard W.M. Jones
To allow us to name the socket passed down to the NBD server when calling nbd_connect_systemd_socket_activation(3), we need to add the field to the handle and add access functions. --- generator/API.ml | 49 ++ lib/handle.c | 56

[Libguestfs] [PATCH libnbd v2 1/4] generator/states-connect-socket-activation.c: Refactor environment prep

2023-01-30 Thread Richard W.M. Jones
Some small refactorings which should not affect the code: - Use string_vector_reserve instead of checking each time we append. - Get rid of the hard-coded length, and use strncmp (..., s, strlen (s)). The compiler should compile this to the same code. ---

[Libguestfs] [PATCH libnbd v2 2/4] common/include: Copy ascii-ctype functions from nbdkit

2023-01-30 Thread Richard W.M. Jones
--- .gitignore| 1 + common/include/Makefile.am| 6 +++ common/include/ascii-ctype.h | 75 ++ common/include/test-ascii-ctype.c | 88 +++ 4 files changed, 170 insertions(+) diff --git a/.gitignore

Re: [Libguestfs] [libnbd PATCH v2 3/3] nbdsh: Improve --help and initial banner contents.

2023-01-30 Thread Richard W.M. Jones
On Fri, Nov 04, 2022 at 04:18:31PM -0500, Eric Blake wrote: > Document all options in --help output. If -n is not in use, then > enhance the banner to print the current state of h, and further tailor > the advice given on useful next steps to take to mention opt_go when > using --opt-mode. I had

Re: [Libguestfs] [p2v PATCH 11/11] make-kickstart: add URLs for RHEL-9

2023-01-30 Thread Richard W.M. Jones
I had a few reservations in a couple of comments, but basically the series is fine and therefore: Reviewed-by: Richard W.M. Jones Please push it whenever you want. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization

Re: [Libguestfs] [p2v PATCH 09/11] gui: expose "p2v.output.misc" (-oo)

2023-01-30 Thread Richard W.M. Jones
On Mon, Jan 30, 2023 at 03:22:26PM +0100, Laszlo Ersek wrote: > + str = gtk_entry_get_text (GTK_ENTRY (oo_entry)); > + guestfs_int_free_string_list (config->output.misc); > + config->output.misc = guestfs_int_split_string (',', str); My concern here is that someone is going to put "foo=bar,

Re: [Libguestfs] [p2v PATCH 02/11] Introduce "p2v.output.misc" for passing "-oo" options to virt-v2v

2023-01-30 Thread Richard W.M. Jones
On Mon, Jan 30, 2023 at 03:22:19PM +0100, Laszlo Ersek wrote: > + ConfigStringList->new(name => 'misc'), > ], >), > ]; > @@ -168,6 +169,7 @@ my @cmdline_aliases = ( >["p2v.output.connection", "p2v.oc"], >["p2v.output.format", "p2v.of"], >["p2v.output.storage",

Re: [Libguestfs] [nbdkit PATCH 1/2] retry: Add in retry support during .open

2023-01-28 Thread Richard W.M. Jones
On Fri, Jan 27, 2023 at 02:41:22PM -0600, Eric Blake wrote: > Now that a filter can open a backend as many times as it wants, > there's no longer a technical reason we can't retry .open. However, > adding retry logic here does mean we have to weaken an assert in the > server backend code, since

[Libguestfs] [PATCH libnbd] generator: Pass LISTEN_FDNAMES=nbd with systemd socket activation

2023-01-28 Thread Richard W.M. Jones
systemd allows sockets passed through socket activation to be named with the protocol they require. We only ever pass one socket, name it. This environment variable is currently ignored by qemu-nbd and nbdkit, but might be used by qemu-storage-daemon:

Re: [Libguestfs] Proposal for tools to inspect drivers and inject Windows virtio drivers

2023-01-27 Thread Richard W.M. Jones
On Thu, Jan 19, 2023 at 06:49:32PM +, Richard W.M. Jones wrote: > On Sat, Dec 10, 2022 at 09:45:11PM +0000, Richard W.M. Jones wrote: > > (a) Inspecting a virtual machine disk image to find out what virtual > > devices it needs (ie. what drivers are installed in the guest

[Libguestfs] [PATCH v2v v2] -o rhv-upload: Improve error message for invalid or missing -os parameter

2023-01-27 Thread Richard W.M. Jones
For -o rhv-upload, the -os parameter specifies the storage domain. Because the RHV API allows globs when searching for a domain, if you used a parameter like -os 'data*' then this would confuse the Python code, since it can glob to the name of a storage domain, but then later fail when we try to

[Libguestfs] [PATCH v2v] -o rhv-upload: Give a nicer error if the storage domain

2023-01-26 Thread Richard W.M. Jones
https://bugzilla.redhat.com/show_bug.cgi?id=1986386 My RHV instance is dead at the moment so I didn't do much more than check this compiles and passes the one test we have. Also I want to spend as little time as possible on RHV outputs for virt-v2v since the RHV product will be discontinued

[Libguestfs] [PATCH v2v] -o rhv-upload: Give a nicer error if the storage domain does not exist

2023-01-26 Thread Richard W.M. Jones
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1986386 Reported-by: Junqin Zhou --- output/rhv-upload-precheck.py | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/output/rhv-upload-precheck.py b/output/rhv-upload-precheck.py index 1dc1b8498a..35ea021032 100644 ---

[Libguestfs] FYI: libguestfs development version now depends on ocaml-augeas

2023-01-21 Thread Richard W.M. Jones
Hi Hillu, Stephane, This bug ... https://bugzilla.redhat.com/show_bug.cgi?id=2162845 ... possibly - I'm not sure if this person is using Debian, but it'll affect Debian too. Libguestfs used to bundle a copy of the OCaml bindings for Augeas: http://augeas.net/

Re: [Libguestfs] [p2v PATCH 4/4] make-disk: inject EPEL9 repo in RHEL9 disk image

2023-01-19 Thread Richard W.M. Jones
On Thu, Jan 19, 2023 at 01:14:48PM +0100, Laszlo Ersek wrote: > In RHEL9, "metacity" was replaced with "gnome-kiosk": > > https://bugzilla.redhat.com/show_bug.cgi?id=1951145#c4 > https://bugzilla.redhat.com/show_bug.cgi?id=1956984#c1 > > gnome-kiosk has particular requirements for the main

Re: [Libguestfs] [p2v PATCH 3/4] make-disk: replace here-docs with printfs for better indentation

2023-01-19 Thread Richard W.M. Jones
On Thu, Jan 19, 2023 at 01:14:47PM +0100, Laszlo Ersek wrote: > Here-documents break the indentation of the script; replace them with the > "printf" shell utility. (We could use the <<- redirection operator for > stripping leading TABs from the here-docs, but the script itself does not > use TABs

Re: [Libguestfs] [p2v PATCH 2/4] make-disk: restrict kernel update bug work-around

2023-01-19 Thread Richard W.M. Jones
On Thu, Jan 19, 2023 at 01:14:46PM +0100, Laszlo Ersek wrote: > Commit 2764abefaa14 ("make-disk: Don't upgrade the kernel to avoid kernel > command line bug", 2022-03-24) replaced the "--update" option of > virt-builder with an open-coded DNF update command, excluding the kernel > packages from

Re: [Libguestfs] [p2v PATCH 1/4] ssh: shield virt-v2v from bash RC files

2023-01-19 Thread Richard W.M. Jones
On Thu, Jan 19, 2023 at 01:14:45PM +0100, Laszlo Ersek wrote: > I don't have virt-v2v installed system-wide on my RHEL9 laptop, I rely on > the various "./run" scripts to put everything that virt-p2v's "make check" > requires on the PATH environment variable. > > However,

Re: [Libguestfs] Proposal for tools to inspect drivers and inject Windows virtio drivers

2023-01-19 Thread Richard W.M. Jones
On Sat, Dec 10, 2022 at 09:45:11PM +, Richard W.M. Jones wrote: > (a) Inspecting a virtual machine disk image to find out what virtual > devices it needs (ie. what drivers are installed in the guest). guestfs-tools 1.49.9 contains a new tool called 'virt-drivers' whi

Re: [Libguestfs] Proposal for tools to inspect drivers and inject Windows virtio drivers

2023-01-17 Thread Richard W.M. Jones
On Tue, Jan 17, 2023 at 12:36:58PM +, Richard W.M. Jones wrote: > On Sat, Dec 10, 2022 at 09:45:11PM +0000, Richard W.M. Jones wrote: > > For (b), you could specify the location of the Window disk image and > > the virtio-win path/ISO to have it attempt to install the drivers

Re: [Libguestfs] Proposal for tools to inspect drivers and inject Windows virtio drivers

2023-01-17 Thread Richard W.M. Jones
On Sat, Dec 10, 2022 at 09:45:11PM +, Richard W.M. Jones wrote: > For (b), you could specify the location of the Window disk image and > the virtio-win path/ISO to have it attempt to install the drivers in > the disk image: > > $ virt-customize -a windows.img \ >

Re: [Libguestfs] [PATCH nbdkit] New plugin: blkio

2023-01-17 Thread Richard W.M. Jones
On Mon, Jan 16, 2023 at 02:31:35PM -0600, Eric Blake wrote: > On Wed, Jan 04, 2023 at 06:14:35PM +0000, Richard W.M. Jones wrote: > > Using libblkio (https://libblkio.gitlab.io/) this implements a plugin > > for reading and writing various disk sources mainly used in > > vir

Re: [Libguestfs] [PATCH nbdkit] New plugin: blkio [incomplete]

2023-01-16 Thread Richard W.M. Jones
On Mon, Jan 16, 2023 at 02:35:41PM -0600, Eric Blake wrote: > On Wed, Jan 04, 2023 at 10:51:50PM +0000, Richard W.M. Jones wrote: > > Here's an updated version which supports read, write, flush, > > write-zeroes, discard and FUA: > > > > https://gitlab.com/rwmjones/nbd

Re: [Libguestfs] Cargo edition problem with Debian 11

2023-01-16 Thread Richard W.M. Jones
Thanks everyone. Just a note that nbdkit CI is passing right now. I disabled the rust plugin (only) for the troublesome distros. Also I fixed a bunch of other things that were broken. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming

Re: [Libguestfs] Cargo edition problem with Debian 11

2023-01-13 Thread Richard W.M. Jones
On Fri, Jan 13, 2023 at 01:06:09PM -0700, alan somers wrote: > Which files get overwritten by libvirt-ci ? I think it's everything except ci/manifest.yml, possibly. Eric and Martin (CC'd) know more about this. The libvirt-ci page is: https://libvirt.org/ci.html And the command I use to

Re: [Libguestfs] Cargo edition problem with Debian 11

2023-01-13 Thread Richard W.M. Jones
> https://github.com/libguestfs/nbdkit/pull/19 Actually I suspect that the changes made in PR19 will get overwritten since they touch files which are in git but are really controlled by libvirt-ci. I'm not sure how to make the changes in a way which are compatible with libvirt-ci, maybe Eric or

Re: [Libguestfs] Cargo edition problem with Debian 11

2023-01-13 Thread Richard W.M. Jones
On Fri, Jan 13, 2023 at 11:18:10AM -0700, alan somers wrote: > I approved PR 21. But I'd like to merge PR 19 too, to fix the Cirrus > CI that runs on the Github mirror. Sure thing! Please go ahead, as I'm going to wait for PR21 to see exactly what tests still fail. Rich. -- Richard Jones,

Re: [Libguestfs] Cargo edition problem with Debian 11

2023-01-13 Thread Richard W.M. Jones
On Fri, Jan 13, 2023 at 06:00:07PM +, Richard W.M. Jones wrote: > I'll push something to PR#19 in a few mins. Actually I don't think I can modify it, so I opened an alternate PR: https://gitlab.com/nbdkit/nbdkit/-/merge_requests/21 Rich. -- Richard Jones, Virtualization Group, Red

Re: [Libguestfs] Cargo edition problem with Debian 11

2023-01-13 Thread Richard W.M. Jones
on the platform that fails. I'll push something to PR#19 in a few mins. Rich. > On Fri, Jan 13, 2023 at 10:43 AM Richard W.M. Jones wrote: > > > > On Fri, Jan 13, 2023 at 09:17:29AM -0700, alan somers wrote: > > > Oh, I misread your initial mail. I saw &

Re: [Libguestfs] Cargo edition problem with Debian 11

2023-01-13 Thread Richard W.M. Jones
RUST: skip to the relevant section of ci/manifest.yml. However my main thing is could we actually fix it instead of covering up the problem in CI? Rich. > On Fri, Jan 13, 2023 at 8:57 AM Richard W.M. Jones wrote: > > > > On Fri, Jan 13, 2023 at 08:35:08AM -0700, alan some

Re: [Libguestfs] Cargo edition problem with Debian 11

2023-01-13 Thread Richard W.M. Jones
On Fri, Jan 13, 2023 at 08:35:08AM -0700, alan somers wrote: > The nbdkit crate. Why does Debian need a package for it? I'm not sure -- do they have one? I can't seem to find it. Rich. > On Fri, Jan 13, 2023 at 8:30 AM Richard W.M. Jones wrote: > > > > On Fri, Jan 13,

Re: [Libguestfs] Cargo edition problem with Debian 11

2023-01-13 Thread Richard W.M. Jones
ch. > On Fri, Jan 13, 2023 at 7:27 AM Richard W.M. Jones wrote: > > > > On Fri, Jan 13, 2023 at 07:17:14AM -0700, alan somers wrote: > > > I think we should downgrade predicates-tree to 1.0.5 or older. I can > > > submit a PR for that. What does "rustc --ve

Re: [Libguestfs] Cargo edition problem with Debian 11

2023-01-13 Thread Richard W.M. Jones
d do! - I'll fix that in a moment). But based on it being Debian 11 ("bullseye") it's likely to be: https://packages.debian.org/bullseye/rustc Package: rustc (1.48.0+dfsg1-2) Which is pretty ancient ... Rich. > On Fri, Jan 13, 2023 at 2:41 AM Richard W.M. Jones wrote: > > >

[Libguestfs] Cargo edition problem with Debian 11

2023-01-13 Thread Richard W.M. Jones
https://gitlab.com/nbdkit/nbdkit/-/jobs/3598390121 -- cargo build --release --example ramdisk Downloading crates ... Downloaded float-cmp v0.9.0 Downloaded downcast v0.11.0 Downloaded mockall_derive v0.11.3 Downloaded

Re: [Libguestfs] [nbdkit PATCH] data: don't ignore SIGPIPE

2023-01-13 Thread Richard W.M. Jones
On Fri, Jan 13, 2023 at 05:57:35AM +, Thomas Weißschuh wrote: > If all of the requested data has been read, the call to popen_close will > close the scripts output pipe and on the next write a SIGPIPE is > delivered. > When the scripts inherited signalhandler is ignoring the singal, it > won't

Re: [Libguestfs] [v2v PATCH 2/2] test-v2v-i-ova: spell out ntfs feature group dependency

2023-01-12 Thread Richard W.M. Jones
Looks good, series ACK, thanks! Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100

Re: [Libguestfs] [PATCH nbdkit] New plugin: blkio [incomplete]

2023-01-11 Thread Richard W.M. Jones
On Wed, Jan 11, 2023 at 10:52:01AM -0500, Stefan Hajnoczi wrote: > On Wed, Jan 04, 2023 at 06:14:34PM +0000, Richard W.M. Jones wrote: > > (1) There is no way to know which properties are readable, writable, > > and those which need to be set before or after blk

Re: [Libguestfs] [PATCH nbdkit] New plugin: blkio [incomplete]

2023-01-11 Thread Richard W.M. Jones
On Wed, Jan 11, 2023 at 11:34:50AM -0500, Stefan Hajnoczi wrote: > On Sat, Jan 07, 2023 at 07:44:37PM +0000, Richard W.M. Jones wrote: > > > > This is upstream in nbdkit now: > > https://gitlab.com/nbdkit/nbdkit/-/tree/master/plugins/blkio > > > > Another

Re: [Libguestfs] [PATCH nbdkit] New plugin: blkio [incomplete]

2023-01-09 Thread Richard W.M. Jones
On Sun, Jan 08, 2023 at 08:35:35PM +, Alberto Faria wrote: > With qemu-storage-daemon, you must set writable=on on the > vhost-user-blk export for the device to be writable. Ah, this was the thing I was missing, thanks. Rich. -- Richard Jones, Virtualization Group, Red Hat

Re: [Libguestfs] [PATCH nbdkit] New plugin: blkio [incomplete]

2023-01-07 Thread Richard W.M. Jones
This is upstream in nbdkit now: https://gitlab.com/nbdkit/nbdkit/-/tree/master/plugins/blkio Another question: (6) vhost-user + "read-only" property acts a bit strangely. After opening virtio-blk-vhost-user it throws an EROFS error if you try to "blkio_start" it. However if you set the

Re: [Libguestfs] [PATCH nbdkit v2] ssh: Improve the error message when all authentication methods fail

2023-01-06 Thread Richard W.M. Jones
On Fri, Jan 06, 2023 at 08:29:06AM +0100, Laszlo Ersek wrote: > On 1/5/23 17:17, Richard W.M. Jones wrote: > > The current error message: > > > > nbdkit: ssh[1]: error: all possible authentication methods failed > > > > is confusing and non-action

[Libguestfs] [PATCH nbdkit v2] ssh: Improve the error message when all authentication methods fail

2023-01-05 Thread Richard W.M. Jones
The current error message: nbdkit: ssh[1]: error: all possible authentication methods failed is confusing and non-actionable. It's hard even for experts to understand the relationship between the authentication methods offered by a server and what we require. Try to improve the error message

[Libguestfs] [PATCH nbdkit v2] ssh: Improve the error message when all authentication methods fail

2023-01-05 Thread Richard W.M. Jones
How about this much simpler patch? It's mostly functionally equivalent. I intentionally did not combine the if-statements together, in order to make it easier to understand. Also I separated out the removal of the bogus comment and pushed that, since it was unrelated. Rich.

Re: [Libguestfs] [PATCH nbdkit] ssh: Improve the error message when all authentication methods fail

2023-01-05 Thread Richard W.M. Jones
On Thu, Jan 05, 2023 at 04:49:16PM +0100, Laszlo Ersek wrote: > On 1/5/23 12:34, Richard W.M. Jones wrote: > > The current error message: > > > > nbdkit: ssh[1]: error: all possible authentication methods failed > > > > is confusing and non-action

[Libguestfs] [PATCH nbdkit] ssh: Improve the error message when all authentication methods fail

2023-01-05 Thread Richard W.M. Jones
The current error message: nbdkit: ssh[1]: error: all possible authentication methods failed is confusing and non-actionable. It's hard even for experts to understand the relationship between the authentication methods offered by a server and what we require. Try to improve the error message

Re: [Libguestfs] [PATCH] inspect: check presence of BIOS boot partition to handle BIOS+GPT setup

2023-01-05 Thread Richard W.M. Jones
Hi Andrey, Just looking through the patches posted vs what went upstream: https://github.com/libguestfs/virt-v2v/commit/73614313114a7a3f3dfdd857d5c0307b34d0599c https://github.com/libguestfs/virt-v2v/commit/d04b9a05b2a5350b55e1558a91e3477c35c16976

Re: [Libguestfs] [v2v PATCH] windows_virtio: favor "fwcfg" over "qemufwcfg"

2023-01-05 Thread Richard W.M. Jones
On Thu, Dec 22, 2022 at 08:20:56AM +0100, Laszlo Ersek wrote: > On 12/15/22 12:15, Laszlo Ersek wrote: > > Virtio-win may provide the "qemufwcfg" stub driver and/or the "fwcfg" > > actual driver. If both are provided, we must not install both, as they > > conflict with each other. Pick "fwcfg" in

Re: [Libguestfs] [v2v PATCH] rhv: Use osinfo to distinguish Windows >= 10 variants in "ovirt:id" too

2023-01-05 Thread Richard W.M. Jones
On Sun, Jan 01, 2023 at 03:09:26PM +0100, Laszlo Ersek wrote: > Reflect commit 38b35f3b7e5c ("rhv: Use osinfo to distinguish Windows >= 10 > variants", 2022-12-02) to the "ovirt:id" field in the OVF as well. Take > the values from "packaging/conf/osinfo-defaults.properties" in the > ovirt-engine

Re: [Libguestfs] [PATCH nbdkit] New plugin: blkio [incomplete]

2023-01-04 Thread Richard W.M. Jones
Here's an updated version which supports read, write, flush, write-zeroes, discard and FUA: https://gitlab.com/rwmjones/nbdkit/-/commits/2023-libblkio/ I was able to access a local file using the io_uring driver using guestfish. You can try it like this: $ ./nbdkit -fv blkio io_uring

Re: [Libguestfs] [PATCH nbdkit] New plugin: blkio [incomplete]

2023-01-04 Thread Richard W.M. Jones
On Wed, Jan 04, 2023 at 06:14:34PM +, Richard W.M. Jones wrote: > (3) It seems like some drivers require pre-allocated memory regions, > and since some do that means we might as well implement this. It > also seems like some drivers require file-backed pre-allocated > memory reg

Re: [Libguestfs] [PATCH nbdkit] New plugin: blkio [incomplete]

2023-01-04 Thread Richard W.M. Jones
(5) "The application is responsible for thread-safety. No thread synchronization is necessary when a queue is only used from a single thread. Proper synchronization is required when sharing a queue between multiple threads." Does this apply across multiple struct blkio handles? ie. Is there now,

Re: [Libguestfs] [PATCH nbdkit] New plugin: blkio [incomplete]

2023-01-04 Thread Richard W.M. Jones
On Wed, Jan 04, 2023 at 06:14:34PM +, Richard W.M. Jones wrote: > This is an incomplete outline implementation for a libblkio plugin for > nbdkit. At the moment it only supports reading the same ("capacity") s/same/size/ -- Richard Jones, Virtualization Gro

[Libguestfs] [PATCH nbdkit] New plugin: blkio

2023-01-04 Thread Richard W.M. Jones
he location of C<$plugindir>. + +=back + +=head1 VERSION + +C first appeared in nbdkit 1.34. + +=head1 SEE ALSO + +L, +L, +L, +L<https://libblkio.gitlab.io>, +L<https://libblkio.gitlab.io/libblkio/blkio.html>. + +=head1 AUTHORS + +Richard W.M. Jones + +=head1 COPYRIGHT + +Co

[Libguestfs] [PATCH nbdkit] New plugin: blkio [incomplete]

2023-01-04 Thread Richard W.M. Jones
This is an incomplete outline implementation for a libblkio plugin for nbdkit. At the moment it only supports reading the same ("capacity") of the device, and not even reading or writing. I have some questions about the libblkio API before I can complete the plugin (see below). The idea here is

Re: [Libguestfs] [V2V PATCH v4] parse_libvirt_xml: look for manual firmware in "/domain/os/loader/@type"

2022-12-20 Thread Richard W.M. Jones
BTW I'm on holiday, and I'm quite happy to go with whatever Laszlo agrees with. He is also able to push it upstream. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder

Re: [Libguestfs] [PATCH v2] parse_libvirt_xml: look for manual firmware in "/domain/os/loader"

2022-12-15 Thread Richard W.M. Jones
On Thu, Dec 15, 2022 at 08:11:15PM +0200, Andrey Drobyshev wrote: > According to [1], there're different ways to specify which firmware is > to be used by a libvirt-driven VM. Namely, there's an automatic > firmware selection, e.g.: > > ... > > ... > > and a manual one, e.g.: > >

Re: [Libguestfs] [PATCH] parse_libvirt_xml: look for manual firmware in "/domain/os/loader"

2022-12-15 Thread Richard W.M. Jones
On Thu, Dec 15, 2022 at 06:45:46PM +0200, Andrey Drobyshev wrote: > According to [1], there're different ways to specify which firmware is > to be used by a libvirt-driven VM. Namely, there's an automatic > firmware selection, e.g.: > > ... > > ... > > and a manual one, e.g.: > >

Re: [Libguestfs] LibGuest Issue

2022-12-13 Thread Richard W.M. Jones
Sorry your emails appeared to be blocked before. I have whitelisted both now. On Mon, Dec 12, 2022 at 06:46:28AM +, Helen Le wrote: > supermin: kernel: modpath /lib/modules/5.15.0-56-generic > cp: cannot open '/boot/vmlinuz-5.15.0-56-generic' for reading: Permission > denied > supermin: cp

Re: [Libguestfs] Proposal for tools to inspect drivers and inject Windows virtio drivers

2022-12-12 Thread Richard W.M. Jones
On Mon, Dec 12, 2022 at 04:05:20PM +0100, Laszlo Ersek wrote: > On 12/10/22 22:45, Richard W.M. Jones wrote: > > We've had requests from products that use libguestfs & virt-v2v to > > provide additional tooling for: > > > > (a) Inspecting a virtual machine disk

[Libguestfs] Proposal for tools to inspect drivers and inject Windows virtio drivers

2022-12-10 Thread Richard W.M. Jones
We've had requests from products that use libguestfs & virt-v2v to provide additional tooling for: (a) Inspecting a virtual machine disk image to find out what virtual devices it needs (ie. what drivers are installed in the guest). (b) Taking a Windows disk image and injecting virtio drivers

Re: [Libguestfs] [v2v PATCH] convert_linux: add Rocky Linux to supported distros

2022-12-09 Thread Richard W.M. Jones
Thanks, that is upstream in commit db831c167b. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows

Re: [Libguestfs] [PATCH] m4/guestfs-appliance.m4: add VIRTUOZZO to RedHat-based distros

2022-12-09 Thread Richard W.M. Jones
On Fri, Dec 09, 2022 at 09:15:14PM +0200, Andrey Drobyshev wrote: > On 12/9/22 20:59, Richard W.M. Jones wrote: > > On Fri, Dec 09, 2022 at 07:50:06PM +0100, Denis V. Lunev wrote: > >> On 12/9/22 19:36, Andrey Drobyshev wrote: > >>> This allows to detect VzLinux

Re: [Libguestfs] [PATCH] m4/guestfs-appliance.m4: add VIRTUOZZO to RedHat-based distros

2022-12-09 Thread Richard W.M. Jones
On Fri, Dec 09, 2022 at 07:50:06PM +0100, Denis V. Lunev wrote: > On 12/9/22 19:36, Andrey Drobyshev wrote: > >This allows to detect VzLinux guest. > > > >Signed-off-by: Pavel Butsykin > >Signed-off-by: Andrey Drobyshev > >--- > > m4/guestfs-appliance.m4 | 2 +- > > 1 file changed, 1

Re: [Libguestfs] [v2v PATCH 2/2] convert_windows: fix up the UEFI fallback boot loader if broken

2022-12-02 Thread Richard W.M. Jones
On Fri, Dec 02, 2022 at 02:25:53PM +0100, Laszlo Ersek wrote: > On 12/02/22 13:58, Richard W.M. Jones wrote: > > On Fri, Dec 02, 2022 at 01:44:09PM +0100, Laszlo Ersek wrote: > >> + info (f_"Fixing UEFI bootloader."); > >> + g#rm_rf bootd

Re: [Libguestfs] [v2v PATCH 2/2] convert_windows: fix up the UEFI fallback boot loader if broken

2022-12-02 Thread Richard W.M. Jones
gt; >List.iter ( > fun dev_path -> > g#mount dev_path esp_temp_path; > fix_win_uefi_bcd esp_temp_path; > +(match uefi_arch with > + | Some uefi_arch -> fix_win_uefi_fallback esp_temp_path uefi_arch > + | None ->

Re: [Libguestfs] [v2v PATCH 1/2] convert_linux.get_uefi_arch_suffix: move to Utils

2022-12-02 Thread Richard W.M. Jones
pect.i_arch with >| None -> cant_fix_uefi () >| Some suffix -> ( > diff --git a/lib/utils.ml b/lib/utils.ml > index fd45769415c7..26e7e259001b 100644 > --- a/lib/utils.ml > +++ b/lib/utils.ml > @@ -258,3 +258,8 @@ let get_disk_allocated ~dir ~disknr = >

Re: [Libguestfs] [PATCH v2v] convert: windows: Copy drivers for Windows 11, Windows 2019 & Windows 2022

2022-12-02 Thread Richard W.M. Jones
On Fri, Dec 02, 2022 at 10:16:53AM +0100, Laszlo Ersek wrote: > On 12/01/22 13:49, Richard W.M. Jones wrote: > > If the virtio-win ISO contains drivers for Windows 11, Windows 2019 or > > Windows 2022, and the guest matches these, then copy in the right > > drivers. For this

Re: [Libguestfs] [PATCH libguestfs 2/2] lib: Return correct osinfo field for Windows 11

2022-12-02 Thread Richard W.M. Jones
On Fri, Dec 02, 2022 at 10:11:32AM +0100, Laszlo Ersek wrote: > On 12/01/22 12:20, Richard W.M. Jones wrote: > > On Thu, Dec 01, 2022 at 10:34:09AM +, Richard W.M. Jones wrote: > >> For Windows Client, we can only distinguish between Windows 10 and > >> Wi

[Libguestfs] [PATCH v2v] convert: windows: Copy drivers for Windows 11, Windows 2019 & Windows 2022

2022-12-01 Thread Richard W.M. Jones
If the virtio-win ISO contains drivers for Windows 11, Windows 2019 or Windows 2022, and the guest matches these, then copy in the right drivers. For this to work you will need libguestfs >= 1.49.8 which allows osinfo to be used to detect Windows versions >= 10. Side note: virtio-win uses a mix

<    2   3   4   5   6   7   8   9   10   11   >