[Libguestfs] [PATCH nbdkit 00/10] Make --run imply -U -

2023-09-09 Thread Richard W.M. Jones
Should have done this a long time ago. I feel it is about time we change the default of nbdkit --run to imply -U -, rather than opening a public port. Patch series turned out to be a little bit more complicated than I anticipated, but it contains some nice clean ups. Last patch updating the

[Libguestfs] [PATCH nbdkit 03/10] server: Don't set port as a side effect

2023-09-09 Thread Richard W.M. Jones
This removes another long-range code dependency. --- server/sockets.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/server/sockets.c b/server/sockets.c index fe8b24409..d49b1755b 100644 --- a/server/sockets.c +++ b/server/sockets.c @@ -171,26 +171,26 @@

[Libguestfs] [PATCH nbdkit 06/10] server: Make --run imply -U -

2023-09-09 Thread Richard W.M. Jones
Almost always when you used nbdkit --run you should also use -U - (to use a private Unix domain socket). Otherwise nbdkit listened on TCP port 10809, which had two bad side effects: It permitted other processes to interfere with your --run command, and it reserved a public TCP port which would

[Libguestfs] [PATCH nbdkit 10/10] XXX docs: Remove references to -U - when it is implicit

2023-09-09 Thread Richard W.M. Jones
XXX NOTE XXX I would not apply this patch immediately, since online documentation will get updated as soon as I do that. Best to wait until after 1.36 is released at least. XXX END NOTE XXX --- docs/nbdkit-captive.pod | 6 +++---

[Libguestfs] [PATCH nbdkit 08/10] tests: Be punctilious about using requires_run in tests that use --run

2023-09-09 Thread Richard W.M. Jones
This requires that nbdkit is built with the --run feature, which (currently) is not true for Windows. (In some tests we separately checked for !Windows, but let's favour consistency.) --- plugins/rust/test-ramdisk.sh | 2 ++ tests/test-S3.sh | 1 +

[Libguestfs] [PATCH nbdkit 07/10] tests/test-parallel-*.sh: Remove redundant comment

2023-09-09 Thread Richard W.M. Jones
--- tests/test-parallel-file.sh | 1 - tests/test-parallel-nbd.sh | 1 - tests/test-parallel-sh.sh | 1 - 3 files changed, 3 deletions(-) diff --git a/tests/test-parallel-file.sh b/tests/test-parallel-file.sh index 3cbaa2d46..add60007a 100755 --- a/tests/test-parallel-file.sh +++

[Libguestfs] [PATCH nbdkit 05/10] server: Add the NBD URI to debug output

2023-09-09 Thread Richard W.M. Jones
Example after applying this patch: $ ./nbdkit -fv --port -e foo null 1M /home/rjones/d/nbdkit/server/nbdkit -f -v --port= -e foo -- /home/rjones/d/nbdkit/plugins/null/.libs/nbdkit-null-plugin.so 1M nbdkit: debug: nbdkit 1.35.12 nbdkit: debug: TLS disabled: could not load TLS

[Libguestfs] [PATCH nbdkit 02/10] server: Don't set export_name as a side effect of using --run

2023-09-09 Thread Richard W.M. Jones
Reduce long-range code dependencies by not setting export_name in run_command (--run functionality), especially as this function is not always called. If the -e option was not used at all then export_name will be NULL. --- server/captive.c | 12 +--- 1 file changed, 5 insertions(+), 7

Re: [Libguestfs] libnbd | Failed pipeline for master | 0b587f8a

2023-09-08 Thread Richard W.M. Jones
On Thu, Sep 07, 2023 at 12:52:07PM -0500, Eric Blake wrote: > On Thu, Sep 07, 2023 at 04:49:53PM +, GitLab wrote: > > > > > > Pipeline #996353812 has failed! > > > > Project: libnbd ( https://gitlab.com/nbdkit/libnbd ) > > Branch: master ( https://gitlab.com/nbdkit/libnbd/-/commits/master )

Re: [Libguestfs] [libnbd PATCH] info: Prefer NBD_OPT_INFO when possible

2023-09-07 Thread Richard W.M. Jones
On Thu, Sep 07, 2023 at 11:35:27AM -0500, Eric Blake wrote: > On Tue, Sep 05, 2023 at 03:23:58PM +0100, Richard W.M. Jones wrote: > > On Tue, Sep 05, 2023 at 08:47:34AM -0500, Eric Blake wrote: > > > diff --git a/info/info-list-uris.sh b/info/info-list-uris.sh > > > inde

Re: [Libguestfs] [libnbd PATCH] rust: Use mio::poll instead of requiring epoll

2023-09-07 Thread Richard W.M. Jones
On Thu, Sep 07, 2023 at 08:09:33AM -0500, Eric Blake wrote: > On Wed, Sep 06, 2023 at 05:02:06PM -0500, Eric Blake wrote: > > > Okay, I'll merge in that aspect of my original along with your other > > > improvements, and push something soon. Fingers crossed that we'll > > > finally get a green CI

Re: [Libguestfs] LIBGUESTFS supported version

2023-09-06 Thread Richard W.M. Jones
On Wed, Sep 06, 2023 at 09:04:17AM +, Teja Konapalli wrote: > Am trying to install libguestfs in my redhat 8.2 version default its > installing 1.38.4. Could you please help us with the installation of > version 1.50 libguestfs (through command line in RHEL 8.2) and > supported RHEL versions

Re: [Libguestfs] [libnbd PATCH] info: Prefer NBD_OPT_INFO when possible

2023-09-05 Thread Richard W.M. Jones
On Tue, Sep 05, 2023 at 08:47:34AM -0500, Eric Blake wrote: > --- a/info/info-list-uris.sh > +++ b/info/info-list-uris.sh > @@ -22,7 +22,14 @@ set -e > set -x > > requires nbdkit --version > -requires nbdkit file --version > +# Avoid tickling a double-free bug in nbdkit 1.35.11 > +requires

Re: [Libguestfs] [libnbd PATCH] info: Prefer NBD_OPT_INFO when possible

2023-09-05 Thread Richard W.M. Jones
On Tue, Sep 05, 2023 at 08:47:34AM -0500, Eric Blake wrote: > diff --git a/info/info-list-uris.sh b/info/info-list-uris.sh > index 0d6a16a0..d8ea9108 100755 > --- a/info/info-list-uris.sh > +++ b/info/info-list-uris.sh > @@ -22,7 +22,14 @@ set -e > set -x > > requires nbdkit --version >

Re: [Libguestfs] [PATCH libnbd 5/5] copy: Allow human sizes for --queue-size, --request-size, --sparse

2023-09-05 Thread Richard W.M. Jones
On Mon, Sep 04, 2023 at 10:13:45AM +0200, Laszlo Ersek wrote: > On 9/3/23 17:23, Richard W.M. Jones wrote: > > Allow these options to be specified using human sizes, for example > > this now works: > > > > nbdcopy --request-size=32M ... > > --- > >

Re: [Libguestfs] [PATCH nbdkit] server: Move size parsing code (nbdkit_parse_size) to common/include

2023-09-05 Thread Richard W.M. Jones
On Mon, Sep 04, 2023 at 09:52:53AM +0200, Laszlo Ersek wrote: > On 9/3/23 17:22, Richard W.M. Jones wrote: > > This is broadly simple code motion, intended so that we can reuse the > > same code in libnbd. > > --- > > common/include/Makefile.am | 6 ++ >

Re: [Libguestfs] [libnbd PATCH] rust: Cleanups in examples/concurrent-read-write.rs.

2023-09-05 Thread Richard W.M. Jones
On Mon, Sep 04, 2023 at 03:21:10PM +, Tage Johansson wrote: > This patch makes some small cleanups in > rust/examples/concurrent-read-write.rs. Specificly, it refrases one > comment, removes some outcommented code, and removes a completely > redundent if statement. It also replaces a hard

[Libguestfs] [PATCH libnbd 4/5] Revert "copy, info: Include common/utils/human-size.h"

2023-09-03 Thread Richard W.M. Jones
This reverts commit XXX FILL IN LATER XXX --- copy/main.c | 2 +- info/show.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/copy/main.c b/copy/main.c index d2f415dc47..6928a4acde 100644 --- a/copy/main.c +++ b/copy/main.c @@ -39,7 +39,7 @@ #include #include

[Libguestfs] [PATCH libnbd 5/5] copy: Allow human sizes for --queue-size, --request-size, --sparse

2023-09-03 Thread Richard W.M. Jones
Allow these options to be specified using human sizes, for example this now works: nbdcopy --request-size=32M ... --- copy/copy-sparse-allocated.sh| 2 +- copy/copy-sparse-no-extents.sh | 2 +- copy/copy-sparse-request-size.sh | 2 +- copy/main.c | 37

[Libguestfs] [PATCH libnbd 2/5] common/include: Import the human_size_parse (nbdkit_parse_size) function

2023-09-03 Thread Richard W.M. Jones
This function is copied from nbdkit commit XXX [fill in after nbdkit change is upstream] XXX --- .gitignore | 1 + common/include/Makefile.am | 6 ++ common/include/human-size.h | 137 +++ common/include/test-human-size.c | 133

[Libguestfs] [PATCH libnbd 3/5] common: Combine human-size.h headers into one

2023-09-03 Thread Richard W.M. Jones
Copy the human_size() function from common/utils/ into the new human-size.h header in common/include/. Remove human-size.c and combine the tests into one. --- common/include/human-size.h | 51 ++ common/include/test-human-size.c | 79 +---

[Libguestfs] [PATCH libnbd 1/5] copy, info: Include common/utils/human-size.h

2023-09-03 Thread Richard W.M. Jones
The next commit will add a new "human-size.h" header copied in from nbdkit. This breaks existing code that uses common/utils/human-size.h (which we will combine later). As a temporary hack to maintain bisection, make sure we are using the deprecated "human-size.h" header. This hack will be

[Libguestfs] [PATCH libnbd 0/5] copy: Allow human sizes for --queue-size, etc

2023-09-03 Thread Richard W.M. Jones
See companion patch: Subject: [PATCH nbdkit] server: Move size parsing code (nbdkit_parse_size) to common/include This is the second part of the patch. It adds the new human_size_parse function to libnbd and then uses it for parsing --queue-size, --request-size and --sparse. The main

[Libguestfs] [PATCH nbdkit] server: Move size parsing code (nbdkit_parse_size) to common/include

2023-09-03 Thread Richard W.M. Jones
This is broadly simple code motion, intended so that we can reuse the same code in libnbd. --- common/include/Makefile.am | 6 ++ common/include/human-size.h | 137 +++ common/include/test-human-size.c | 133 ++ server/public.c

[Libguestfs] [PATCH nbdkit] server: Move size parsing code (nbdkit_parse_size) to common/include

2023-09-03 Thread Richard W.M. Jones
This is the first part of a pair of patch series which aim to let us use nbdkit_parse_size (or rather, an equivalent common function) in nbdcopy, so we can write: nbdcopy --request-size=32M ... We can't do that now which was annoying me earlier in the week. This commit creates a new function

Re: [Libguestfs] [libnbd PATCH 1/2] maint: Favor 4-space indent in .rs files

2023-09-01 Thread Richard W.M. Jones
> +[*.rs] > +# Match rustfmt style > +indent_size = 4 > + > [{Makefile,Makefile.in,Makefile.am,*.mk}] > # Make requires tabs. > indent_style = tab Reviewed-by: Richard W.M. Jones -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read

Re: [Libguestfs] [libnbd PATCH 0/3] Simplify nbd_shutdown vs. opt mode

2023-09-01 Thread Richard W.M. Jones
ut instead). My feeling is this should be enabled by default, as that does the right thing by default. Whether or not we need to have a flag to disable it (ie the opposite sense to the proposed flag) is up to you. For the series: Reviewed-by: Richard W.M. Jones Rich. > Eric Blake (3): > tests:

Re: [Libguestfs] [libnbd PATCH 0/2] (Attempt to) fix some Rust CI failures

2023-09-01 Thread Richard W.M. Jones
Thanks - I believe we have equivalent fixes upstream now. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with plugins

Re: [Libguestfs] [nbdkit PATCH] sh: Allow pwrite to not consume all data

2023-08-31 Thread Richard W.M. Jones
I pushed this as commit d40eb85b2 including the changes suggested by Laszlo and Eric. Thanks! Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with

Re: [Libguestfs] [nbdkit PATCH] sh: Allow pwrite to not consume all data

2023-08-31 Thread Richard W.M. Jones
On Thu, Aug 31, 2023 at 12:05:17PM +0200, Laszlo Ersek wrote: > On 8/31/23 11:47, Richard W.M. Jones wrote: > > On Thu, Aug 31, 2023 at 11:12:59AM +0200, Laszlo Ersek wrote: > >> On 8/31/23 10:02, Richard W.M. Jones wrote: > >>> > >>> On Wed, Aug 30,

Re: [Libguestfs] [nbdkit PATCH] sh: Allow pwrite to not consume all data

2023-08-31 Thread Richard W.M. Jones
On Thu, Aug 31, 2023 at 11:18:27AM +0200, Laszlo Ersek wrote: > On 8/31/23 10:55, Richard W.M. Jones wrote: > > On Thu, Aug 31, 2023 at 10:40:53AM +0200, Laszlo Ersek wrote: > >> On 8/31/23 00:21, Eric Blake wrote: > >>> I hit another transient failure in libnbd CI wh

Re: [Libguestfs] [nbdkit PATCH] sh: Allow pwrite to not consume all data

2023-08-31 Thread Richard W.M. Jones
On Thu, Aug 31, 2023 at 11:12:59AM +0200, Laszlo Ersek wrote: > On 8/31/23 10:02, Richard W.M. Jones wrote: > > > > On Wed, Aug 30, 2023 at 05:21:19PM -0500, Eric Blake wrote: > >> I hit another transient failure in libnbd CI when a poorly-written > >> eval sc

Re: [Libguestfs] [nbdkit PATCH] sh: Allow pwrite to not consume all data

2023-08-31 Thread Richard W.M. Jones
On Thu, Aug 31, 2023 at 10:40:53AM +0200, Laszlo Ersek wrote: > On 8/31/23 00:21, Eric Blake wrote: > > I hit another transient failure in libnbd CI when a poorly-written > > eval script did not consume all of stdin during .pwrite. As behaving > > as a data sink can be a somewhat reasonable

[Libguestfs] [PATCH nbdkit] sh: In pwrite, allow scripts to ignore stdin

2023-08-31 Thread Richard W.M. Jones
See comment for explanation and https://listman.redhat.com/archives/libguestfs/2023-August/032468.html --- tests/Makefile.am| 2 + plugins/sh/call.c| 33 +++- tests/test-sh-pwrite-ignore-stdin.sh | 77 3 files changed,

Re: [Libguestfs] [nbdkit PATCH] sh: Allow pwrite to not consume all data

2023-08-31 Thread Richard W.M. Jones
On Wed, Aug 30, 2023 at 05:21:19PM -0500, Eric Blake wrote: > I hit another transient failure in libnbd CI when a poorly-written > eval script did not consume all of stdin during .pwrite. As behaving > as a data sink can be a somewhat reasonable feature of a > quickly-written sh or eval plugin,

Re: [Libguestfs] [libnbd PATCH v9 0/7] Rust Bindings for Libnbd

2023-08-29 Thread Richard W.M. Jones
On Tue, Aug 29, 2023 at 09:09:02AM +0100, Richard W.M. Jones wrote: > > Thanks, I pushed this series now. > > There are some CI failures, see: > https://gitlab.com/nbdkit/libnbd/-/pipelines/984910205 > > Can you suggest some fixes for these, or otherwise we can go throu

Re: [Libguestfs] [libnbd PATCH v9 0/7] Rust Bindings for Libnbd

2023-08-29 Thread Richard W.M. Jones
Thanks, I pushed this series now. There are some CI failures, see: https://gitlab.com/nbdkit/libnbd/-/pipelines/984910205 Can you suggest some fixes for these, or otherwise we can go through them on the call today. Rich. -- Richard Jones, Virtualization Group, Red Hat

Re: [Libguestfs] [nbdkit PATCH] cc: Allow configuration without absolute paths

2023-08-19 Thread Richard W.M. Jones
e settings used for nbdkit itself > > +AC_ARG_VAR([CC_PLUGIN_CC], > > + [Value to use for CC when building the cc plugin, default $CC]) > > I'm wondering if there is a better way to word this (it shows up in > './configure --help' output). Maybe: > > [Value to hard-co

Re: [Libguestfs] [libnbd PATCH v7 0/9] Rust Bindings for Libnbd

2023-08-17 Thread Richard W.M. Jones
On Thu, Aug 17, 2023 at 09:29:47AM -0500, Eric Blake wrote: > On Thu, Aug 17, 2023 at 04:13:36AM +, Tage Johansson wrote: > > It is possible to add a minimum version requirement in Cargo.toml, I guess I > > should do that to make the errors a bit more clear. > > It's okay if older platforms

Re: [Libguestfs] [libnbd PATCH] ci: Update lcitool to request cargo during CI

2023-08-16 Thread Richard W.M. Jones
It's interesting that awk is required. I've been attempting to keep nbdkit from requiring awk (but we do require sed). In config.log I see: $ grep -i awk config.log configure:5226: checking for gawk configure:5247: found /usr/bin/gawk configure:5258: result: gawk ac_cv_prog_AWK=gawk

Re: [Libguestfs] [libnbd PATCH v5 01/12] rust: create basic Rust bindings

2023-08-15 Thread Richard W.M. Jones
On Mon, Aug 14, 2023 at 07:55:15AM +, Tage Johansson wrote: > > On 8/11/2023 2:00 PM, Eric Blake wrote: > > On Thu, Aug 03, 2023 at 03:36:05PM +, Tage Johansson wrote: > > This commit creates basic Rust bindings in the rust directory. > The bindings are generated by

Re: [Libguestfs] [libnbd PATCH v7 9/9] rust: async: Add an example

2023-08-15 Thread Richard W.M. Jones
On Tue, Aug 15, 2023 at 12:19:15PM +, Tage Johansson wrote: > I guess that we still use nbdkit with a unix socket when testing the > examples in run-tests.sh, right? Yes. In fact, we must since there may be other tests running in parallel, or simply an NBD server already sitting on a port.

Re: [Libguestfs] [libnbd PATCH v7 9/9] rust: async: Add an example

2023-08-15 Thread Richard W.M. Jones
On Tue, Aug 15, 2023 at 11:25:56AM +, Tage Johansson wrote: > > On 8/15/2023 11:07 AM, Richard W.M. Jones wrote: > >So what do I think about the patch series as a whole ... (in > >particular, the patches I didn't add Reviewed-by tags to). > > > >It would be muc

Re: [Libguestfs] [libnbd PATCH v7 9/9] rust: async: Add an example

2023-08-15 Thread Richard W.M. Jones
So what do I think about the patch series as a whole ... (in particular, the patches I didn't add Reviewed-by tags to). It would be much nicer IMHO if we didn't have to define callback lifetimes in this way, since they were not intended to be classified into async_kind / cblifetime / cbcount,

Re: [Libguestfs] [libnbd PATCH v7 9/9] rust: async: Add an example

2023-08-15 Thread Richard W.M. Jones
+join_set > +.spawn(async move { nbd.pwrite(, offset, None).await > }); > +} > +stats.requests += 1; > +} > +join_set.join_next().await.unwrap().unwrap()?; > +} > + > +if task_idx == 0 {} > +

Re: [Libguestfs] [libnbd PATCH v7 8/9] rust: async: Add a couple of integration tests

2023-08-15 Thread Richard W.M. Jones
, > +//"-v", XXX: uncomment this Is there an issue with using -v here? It seems to be used by all the other tests. Reviewed-by: Richard W.M. Jones Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualizatio

Re: [Libguestfs] [libnbd PATCH v7 6/9] generator: Add `modifies_fd` flag to the [call] structure

2023-08-15 Thread Richard W.M. Jones
; > +modifies_fd = true; > shortdesc = "poll the handle once, with fd"; > longdesc = "\ > This is the same as L, but an additional > @@ -3497,6 +3527,7 @@ and invalidate the need to write more commands. >"aio_notify_read", { > default_call with

Re: [Libguestfs] [libnbd PATCH] golang: Bump minimum Go version to 1.17

2023-08-14 Thread Richard W.M. Jones
loses support, but is relatively old these days. All other > systems appear unaffected by this bump in requirements, at least if > they can be configured to use developer backports. > > Suggested-by: Nir Soffer > Signed-off-by: Eric Blake > --- Reviewed-by: Richard W.M. Jones I

Re: [Libguestfs] [libnbd PATCH] rust: Add cargo as an optional build dependency in README

2023-08-14 Thread Richard W.M. Jones
Thanks, I have pushed this as commit 41e921b554. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with plugins https://gitlab.com/nbdkit/nbdkit

Re: [Libguestfs] [libnbd PATCH v5 01/12] rust: create basic Rust bindings

2023-08-14 Thread Richard W.M. Jones
On Mon, Aug 14, 2023 at 08:15:53AM +, Tage Johansson wrote: > > On 8/11/2023 3:41 PM, Richard W.M. Jones wrote: > > On Fri, Aug 11, 2023 at 08:22:34AM -0500, Eric Blake wrote: > > On Thu, Aug 03, 2023 at 03:36:05PM +, Tage Johansson wrote: > >

Re: [Libguestfs] [libnbd PATCH v7 1/9] rust: Make it possible to run tests with Valgrind

2023-08-11 Thread Richard W.M. Jones
On Thu, Aug 10, 2023 at 11:24:28AM +, Tage Johansson wrote: > Make it possible to run Rust tests with Valgrind with > `make check-valgrind` in the rust directory. I think I forgot to mention it before. This is upstream in commit b656d1d987c. Rich. > rust/Makefile.am | 3 +++ >

Re: [Libguestfs] [libnbd PATCH v5 01/12] rust: create basic Rust bindings

2023-08-11 Thread Richard W.M. Jones
On Fri, Aug 11, 2023 at 08:22:34AM -0500, Eric Blake wrote: > On Thu, Aug 03, 2023 at 03:36:05PM +, Tage Johansson wrote: > > This commit creates basic Rust bindings in the rust directory. > > The bindings are generated by generator/Rust.ml and > > generator/RustSys.ml. > > --- > > > +++

Re: [Libguestfs] [libnbd PATCH] rust: Allow rust compilation without rustfmt

2023-08-10 Thread Richard W.M. Jones
!= "xno" -a "x$RUSTFMT" != "xno"]) > +AM_CONDITIONAL([HAVE_RUST],[test "x$CARGO" != "xno"]) Now we're allowing HAVE_RUST if cargo != "no", and no longer checking for rustfmt. That would fail when running the generator, except that in the genera

Re: [Libguestfs] [libnbd PATCH v7 0/9] Rust Bindings for Libnbd

2023-08-10 Thread Richard W.M. Jones
Valgrind now works(!) Not sure what was going on there before. I will push the valgrind patch shortly, thanks. There is some output from valgrind [attached] which you might want to look at, although I believe none of it points to any actual problem. To reproduce this use: $ cd rust $

Re: [Libguestfs] [libnbd PATCH v4 05/25] golang: Change logic of copy_uint32_array

2023-08-08 Thread Richard W.M. Jones
On Tue, Aug 08, 2023 at 02:36:12PM +0300, Nir Soffer wrote: > On Thu, Aug 3, 2023 at 4:57 AM Eric Blake wrote: > > func copy_uint32_array(entries *C.uint32_t, count C.size_t) []uint32 { > > +if (uint64(count) > 64*1024*1024) { > > +panic(\"violation of state machine guarantee\") > >

Re: [Libguestfs] [PATCH libnbd] copy/copy-file-to-qcow2-compressed.sh: Skip test for all zeroes disk

2023-08-08 Thread Richard W.M. Jones
On Tue, Aug 08, 2023 at 01:38:31PM +0200, Laszlo Ersek wrote: > On 8/8/23 00:28, Richard W.M. Jones wrote: > > Occasionally this test will choose a random seed which results in an > > all-zeroes disk. The test tries to convert this to a compressed qcow2 > > file, and fails

Re: [Libguestfs] [PATCH libnbd] copy/copy-file-to-qcow2-compressed.sh: Skip test for all zeroes disk

2023-08-08 Thread Richard W.M. Jones
After more testing today I pushed a slightly improved version of this patch as: https://gitlab.com/nbdkit/libnbd/-/commit/a333eec17a5ce4b2a4bbbd2f08bf5f4474382074 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization

[Libguestfs] [PATCH libnbd] copy/copy-file-to-qcow2-compressed.sh: Skip test for all zeroes disk

2023-08-07 Thread Richard W.M. Jones
Occasionally this test will choose a random seed which results in an all-zeroes disk. The test tries to convert this to a compressed qcow2 file, and fails because no compressed clusters are detected in the resulting file. This happens because qcow2 stores zero clusters with a special sparse

Re: [Libguestfs] [libnbd PATCH] golang: Optionally use gofmt on generated .go files

2023-08-07 Thread Richard W.M. Jones
On Mon, Aug 07, 2023 at 04:11:15PM -0500, Eric Blake wrote: > On Fri, Aug 04, 2023 at 09:29:48AM -0500, Eric Blake wrote: > > Combine my recent work on improving the generated Go output with > > Tage's work on using a canonical formatter for Rust. If gofmt is > > available during the build, then

Re: [Libguestfs] debug a corrupted VHDX file with libguestfs

2023-08-07 Thread Richard W.M. Jones
On Fri, Aug 04, 2023 at 10:37:50PM +, Lin, HongYi - Dell Team wrote: > I tried to recompile Ubuntu kernel and add some printk messages, they don’t > show up in dmesg when I run > > root@HenrikUbuntu2204:/mnt/sdj# guestmount -a > c1eef115-0006-ec888b8d-64888b8d-00155000-4a448456 -m

Re: [Libguestfs] [PATCH nbdkit v2 8/9] curl: Use curl multi interface

2023-08-07 Thread Richard W.M. Jones
On Mon, Aug 07, 2023 at 12:57:02PM +0200, Laszlo Ersek wrote: > On 8/4/23 20:04, Richard W.M. Jones wrote: > > On Fri, Aug 04, 2023 at 11:38:03AM -0500, Eric Blake wrote: > >> On Fri, Jul 28, 2023 at 06:17:52PM +0100, Richard W.M. Jones wrote: > >>> See the comm

Re: [Libguestfs] [PATCH nbdkit v2 8/9] curl: Use curl multi interface

2023-08-04 Thread Richard W.M. Jones
On Fri, Aug 04, 2023 at 11:38:03AM -0500, Eric Blake wrote: > On Fri, Jul 28, 2023 at 06:17:52PM +0100, Richard W.M. Jones wrote: > > See the comment at the top of plugins/curl/pool.c for general > > information about how this works. > > > > This makes a very large dif

Re: [Libguestfs] [libnbd PATCH v4 02/25] generator: Add Extent64 arg type for upcoming use

2023-08-04 Thread Richard W.M. Jones
On Fri, Aug 04, 2023 at 10:11:06AM -0500, Eric Blake wrote: > On Fri, Aug 04, 2023 at 10:09:14AM +0100, Richard W.M. Jones wrote: > > On Wed, Aug 02, 2023 at 08:50:22PM -0500, Eric Blake wrote: > > > The existing nbd_block_status() callback is permanently stuck with an >

Re: [Libguestfs] [libnbd PATCH v6 04/13] rust: Make it possible to run tests with Valgrind

2023-08-04 Thread Richard W.M. Jones
On Fri, Aug 04, 2023 at 11:34:07AM +, Tage Johansson wrote: > Make it possible to run Rust tests with Valgrind with > `make check-valgrind` in the rust directory. > --- > rust/Makefile.am | 3 +++ > rust/run-tests.sh.in | 6 +- > 2 files changed, 8 insertions(+), 1 deletion(-) > >

Re: [Libguestfs] [libnbd PATCH v6 05/13] rust: Add some examples

2023-08-04 Thread Richard W.M. Jones
I have pushed patches 1, 2, 3 and 5. Note I made some small changes, so you'll need to rebase your work on top. When using 'git pull --rebase' skip your version of any conflicting patches ('git rebase --skip') in preference to the upstream version (unless you wish to examine what I changed).

Re: [Libguestfs] [libnbd PATCH v4 01/25] block_status: Add some sanity checking of server lengths

2023-08-04 Thread Richard W.M. Jones
On Fri, Aug 04, 2023 at 12:14:32PM +0100, Richard W.M. Jones wrote: > On Fri, Aug 04, 2023 at 11:49:18AM +0100, Richard W.M. Jones wrote: > > On Wed, Aug 02, 2023 at 08:50:21PM -0500, Eric Blake wrote: > > > Previously, we had not been doing any validation of server extent >

Re: [Libguestfs] [libnbd PATCH v4 01/25] block_status: Add some sanity checking of server lengths

2023-08-04 Thread Richard W.M. Jones
On Fri, Aug 04, 2023 at 11:49:18AM +0100, Richard W.M. Jones wrote: > On Wed, Aug 02, 2023 at 08:50:21PM -0500, Eric Blake wrote: > > Previously, we had not been doing any validation of server extent > > responses, which means a client query at an offset near the end of the > &

Re: [Libguestfs] [libnbd PATCH v4 01/25] block_status: Add some sanity checking of server lengths

2023-08-04 Thread Richard W.M. Jones
> +/* Call the caller's extent function. Yes, our 32-bit public API > + * foolishly tracks the number of uint32_t instead of block > + * descriptors; see _block_desc_is_multiple_of_bs_entry above. > + */ > +if (stop) > + debug (h, "truncating server's

Re: [Libguestfs] [libnbd PATCH v4 18/25] examples: Update copy-libev to use 64-bit block status

2023-08-04 Thread Richard W.M. Jones
ngths. > > Signed-off-by: Eric Blake > Reviewed-by: Richard W.M. Jones > --- > > v4: no change > --- > examples/copy-libev.c | 21 ++--- > 1 file changed, 10 insertions(+), 11 deletions(-) > > diff --git a/examples/copy-libev.c b/examples/copy-lib

Re: [Libguestfs] [libnbd PATCH v4 17/25] info: Update nbdinfo --map to use 64-bit block status

2023-08-04 Thread Richard W.M. Jones
, that is > planned for the nbdkit v3 protocol. > > Signed-off-by: Eric Blake > Reviewed-by: Richard W.M. Jones > --- > > v4: no change > --- > info/map.c | 65 -- > 1 file changed, 34 insertions(+), 31 dele

Re: [Libguestfs] [libnbd PATCH v4 16/25] info: Expose extended-headers support through nbdinfo

2023-08-04 Thread Richard W.M. Jones
s_negotiated (nbd); > +eh_negotiated = nbd_get_extended_headers_negotiated (nbd); > > if (!json_output) { >if (protocol) { > @@ -318,8 +320,9 @@ main (int argc, char *argv[]) > fprintf (fp, "protocol: %s", protocol); > if (tls_negotiated &g

Re: [Libguestfs] [libnbd PATCH v4 14/25] dump: Update nbddump to use 64-bit block status

2023-08-04 Thread Richard W.M. Jones
e > Reviewed-by: Richard W.M. Jones > --- > > v4: no changes > --- > dump/dump.c | 27 ++- > 1 file changed, 14 insertions(+), 13 deletions(-) > > diff --git a/dump/dump.c b/dump/dump.c > index b4aebe84..71053277 100644 > --- a/dump/

Re: [Libguestfs] [libnbd PATCH v4 13/25] copy: Update nbdcopy to use 64-bit block status

2023-08-04 Thread Richard W.M. Jones
ngths. > > Signed-off-by: Eric Blake > Reviewed-by: Richard W.M. Jones > --- > > v4: no code changes > --- > copy/nbd-ops.c | 22 +++--- > 1 file changed, 11 insertions(+), 11 deletions(-) > > diff --git a/copy/nbd-ops.c b/copy/nbd-ops.c > inde

Re: [Libguestfs] [libnbd PATCH v4 11/25] api: Add tests for [aio_]nbd_block_status_64

2023-08-04 Thread Richard W.M. Jones
os.Getenv("abs_top_srcdir") > + script := srcdir + "/tests/meta-base-allocation.sh" > + > + h, err := Create() > + if err != nil { > + t.Fatalf("could not create handle: %s", err) > + } > + defer h.Close() > + >

Re: [Libguestfs] [libnbd PATCH v4 07/25] generator: Support Extent64 arg in OCaml code

2023-08-04 Thread Richard W.M. Jones
git a/ocaml/nbd-c.h b/ocaml/nbd-c.h > index e3abb912..adcdd15a 100644 > --- a/ocaml/nbd-c.h > +++ b/ocaml/nbd-c.h > @@ -62,6 +62,7 @@ extern void nbd_internal_ocaml_raise_closed (const char > *func) Noreturn; > > extern const char **nbd_internal_ocaml_string_list (value); > extern value n

Re: [Libguestfs] [libnbd PATCH v4 06/25] generator: Support Extent64 arg in Go code

2023-08-04 Thread Richard W.M. Jones
t; --- a/golang/handle.go > +++ b/golang/handle.go > @@ -58,6 +58,12 @@ func (h *Libnbd) String() string { > return "{}" > } > > +/* Used for block status callback. */ > +type LibnbdExtent struct { > + Length uint64 // length of the extent > + Flags uint64 //

Re: [Libguestfs] [libnbd PATCH v4 05/25] golang: Change logic of copy_uint32_array

2023-08-04 Thread Richard W.M. Jones
about golang, so glad to see it gone here. > -copy(ret, s) > +addr := uintptr(unsafe.Pointer(entries)) > +for i := 0; i < int(count); i++ { > +ptr := (*C.uint32_t)(unsafe.Pointer(addr)) > +ret[i] = uint32(*ptr) > +addr += unsafe.Sizeof(*ptr) > +}

Re: [Libguestfs] [libnbd PATCH v4 04/25] generator: Support Extent64 arg in Python code

2023-08-04 Thread Richard W.M. Jones
f "py_%s" n > | CBBytesIn (n, _) -> "O", sprintf "py_%s" n >| CBInt n -> "i", n >| CBInt64 n -> "L", n > @@ -254,7 +266,7 @@ let >pr " out:\n"; >List.iter ( > function > -

Re: [Libguestfs] [libnbd PATCH v4 03/25] generator: Support Extent64 arg in C code

2023-08-04 Thread Richard W.M. Jones
(** extent descriptor, struct nbd_extent in C *) With an improved comment / the comment change hunk here dropped (see my review of the prior commit): Reviewed-by: Richard W.M. Jones > | Fd of string (** file descriptor *) > | Flags of string * flags (** flags, uint32

Re: [Libguestfs] [libnbd PATCH v4 02/25] generator: Add Extent64 arg type for upcoming use

2023-08-04 Thread Richard W.M. Jones
e next patch): | Extent64 of string (** extent descriptor, struct nbd_extent in C, pair of 64 bit ints in other languages *) as that seems much clearer to me. With this change: Reviewed-by: Richard W.M. Jones Rich. > | Fd of stri

Re: [Libguestfs] [libnbd PATCH v4 19/25] ocaml: Add example for 64-bit extents

2023-08-04 Thread Richard W.M. Jones
-by: Eric Blake > Reviewed-by: Richard W.M. Jones > --- > > v4: no change > --- > ocaml/examples/Makefile.am | 1 + > ocaml/examples/extents64.ml | 42 + > 2 files changed, 43 insertions(+) > create mode 100644 ocaml/examples/ext

Re: [Libguestfs] [libnbd PATCH v4 15/25] info: Add --has alias for --can

2023-08-04 Thread Richard W.M. Jones
usage easier. > > Suggested-by: Richard W.M. Jones > Signed-off-by: Eric Blake Reviewed-by: Richard W.M. Jones No reason I can see to delay pushing this patch upstream right away. Rich. > --- > > v4: new patch > --- > info/nbdinfo.pod | 29 +++-- >

Re: [Libguestfs] [libnbd PATCH v5 01/12] rust: create basic Rust bindings

2023-08-04 Thread Richard W.M. Jones
I applied the first 4 patches to my local repo. 'make clean' now fails at: error: failed to load manifest for dependency `libnbd-sys` Caused by: failed to parse manifest at `/home/rjones/d/libnbd/rust/libnbd-sys/Cargo.toml` Caused by: no targets specified in the manifest either

Re: [Libguestfs] [libnbd PATCH v4 08/11] generator: Add `modifies_fd` flag to the [call] structure

2023-08-03 Thread Richard W.M. Jones
On Thu, Aug 03, 2023 at 03:30:36PM +, Tage Johansson wrote: > > On 8/2/2023 6:39 PM, Richard W.M. Jones wrote: > >On Wed, Aug 02, 2023 at 12:40:53PM +, Tage Johansson wrote: > >>Add a flag (`modifies_fd`) to the [call] structure in generator/API.ml > >>which

Re: [Libguestfs] [libnbd PATCH v4 01/11] rust: create basic Rust bindings

2023-08-03 Thread Richard W.M. Jones
On Thu, Aug 03, 2023 at 03:20:04PM +, Tage Johansson wrote: > On 8/2/2023 5:42 PM, Richard W.M. Jones wrote: > >Here we're using markdown, I guess, but ocamldoc comments prefer to > >use [ ... ] instead of ` ... ` (although we don't use ocamldoc really). > > > But

Re: [Libguestfs] [libnbd PATCH v4 11/11] rust: Add some examples

2023-08-02 Thread Richard W.M. Jones
On Wed, Aug 02, 2023 at 12:40:56PM +, Tage Johansson wrote: > This patch adds a few examples in rust/examples/. The examples are > compiled and run as part of the test suite. > --- > rust/Makefile.am | 3 + > rust/examples/concurrent-read-write.rs | 135

Re: [Libguestfs] [libnbd PATCH v4 09/11] rust: async: Use the `modifies_fd` flag to exclude calls

2023-08-02 Thread Richard W.M. Jones
On Wed, Aug 02, 2023 at 12:40:54PM +, Tage Johansson wrote: > All handle calls which has the `modifies_fd` flag set to [true] will be > excluded from `AsyncHandle` (the asynchronous handle in the rust > bindings). This is a better approach then listing all calls that should > be excluded in

Re: [Libguestfs] [libnbd PATCH v4 08/11] generator: Add `modifies_fd` flag to the [call] structure

2023-08-02 Thread Richard W.M. Jones
On Wed, Aug 02, 2023 at 12:40:53PM +, Tage Johansson wrote: > Add a flag (`modifies_fd`) to the [call] structure in generator/API.ml > which is set to [true] if the handle call may do something with the > file descriptor. That is, it is [true] for all calls which are or may > call

Re: [Libguestfs] [libnbd PATCH v4 06/11] rust: Make it possible to run tests with Valgrind

2023-08-02 Thread Richard W.M. Jones
ersion > > -$CARGO test -- --nocapture > +if [ -z "$VG" ]; then > +$CARGO test -- --nocapture > +else > +$CARGO test --config "target.'cfg(all())'.runner = \"$VG\"" -- > --nocapture > +fi > -- > 2.41.0 Reviewed-by: Richard W.M.

Re: [Libguestfs] [libnbd PATCH v4 05/11] rust: Add a couple of integration tests

2023-08-02 Thread Richard W.M. Jones
On Wed, Aug 02, 2023 at 12:40:50PM +, Tage Johansson wrote: > diff --git a/rust/run-tests.sh b/rust/run-tests.sh > index 7a0bc85..005000e 100755 > --- a/rust/run-tests.sh > +++ b/rust/run-tests.sh > @@ -1,6 +1,6 @@ > #!/bin/bash - > # nbd client library in userspace > -# Copyright Red Hat >

Re: [Libguestfs] [libnbd PATCH v4 04/11] rust: Use more specific closure traits

2023-08-02 Thread Richard W.M. Jones
On Wed, Aug 02, 2023 at 12:40:49PM +, Tage Johansson wrote: > For closures with `cb,count = CBOnce`, `FnOnce` will be used instead of I think there's an extra ',' here. Previous comments about use of markdown apply too. > `FnMut`. Moreover, closures in synchronous commands with >

Re: [Libguestfs] [PATCH nbdkit v2 9/9] curl: Redefine connections= parameter as number of HTTP connections

2023-08-01 Thread Richard W.M. Jones
On Mon, Jul 31, 2023 at 10:44:30PM +0100, Richard W.M. Jones wrote: > On Mon, Jul 31, 2023 at 04:36:16PM -0500, Eric Blake wrote: > > On Fri, Jul 28, 2023 at 06:17:53PM +0100, Richard W.M. Jones wrote: > > > Previously (nbdkit 1.34) this was the number of easy handles. However

Re: [Libguestfs] [PATCH nbdkit v2 9/9] curl: Redefine connections= parameter as number of HTTP connections

2023-07-31 Thread Richard W.M. Jones
On Mon, Jul 31, 2023 at 04:36:16PM -0500, Eric Blake wrote: > On Fri, Jul 28, 2023 at 06:17:53PM +0100, Richard W.M. Jones wrote: > > Previously (nbdkit 1.34) this was the number of easy handles. However > > it turns out that easy handles can open multiple HTTP connections,

Re: [Libguestfs] [libnbd PATCH 6/7] golang: Improve whitespace style in generated bindings

2023-07-31 Thread Richard W.M. Jones
On Fri, Jul 28, 2023 at 10:29:42AM +0200, Laszlo Ersek wrote: > On 7/27/23 21:22, Eric Blake wrote: > > On Thu, Jul 27, 2023 at 08:50:31PM +0200, Laszlo Ersek wrote: > >>> File "GoLang.ml", line 186, characters 11-71: > >>> 186 | pr ("\t" ^ "/* %s field is ignored unless %sSet == true. >

Re: [Libguestfs] [libnbd PATCH 6/7] golang: Improve whitespace style in generated bindings

2023-07-31 Thread Richard W.M. Jones
So the way I see this could work is: pr ~tab:8 "\n %s\n" "foo" would post-process the output to convert spaces to tabs (ie. output "\n\tfoo\n"). If this was necessary for a whole language you could do: let pr = pr ~tab:8 ;; at the top of the OCaml generator file for that language. I

Re: [Libguestfs] [libnbd PATCH 5/7] golang: Use 'gofmt' style recommendations on manual files

2023-07-31 Thread Richard W.M. Jones
up, or gofmt itself has changed what it regards as correct formatting? Anyway ... Reviewed-by: Richard W.M. Jones Rich. > --- > golang/configure/test.go | 14 > golang/examples/aio_copy/aio_copy.go | 7 ++-- > golang/examples/simple_copy/sim

Re: [Libguestfs] [libnbd PATCH 3/7] golang: Simplify RBool return

2023-07-31 Thread Richard W.M. Jones
il }\n" > + pr "return int (ret) != 0, nil\n" > | RStaticString -> >pr "/* ret is statically allocated, do not free it. */\n"; >pr "r := C.GoString (ret);\n"; Obvious improvement. Reviewed-by: Richard W.M. Jones

Re: [Libguestfs] [libnbd PATCH 2/7] golang: Export meta-context constants

2023-07-31 Thread Richard W.M. Jones
, this > patch needs to be rethought a bit, as it would impact API for existing > public constants as well as the ones changed here. But we don't > promise API stability for Go. I don't greatly care either way, unless someone who uses golang express

Re: [Libguestfs] [libnbd PATCH 1/7] api: Expose "qemu:" meta-context constants

2023-07-31 Thread Richard W.M. Jones
On Thu, Jul 27, 2023 at 09:33:19AM -0500, Eric Blake wrote: > On Thu, Jul 27, 2023 at 01:41:03PM +0200, Laszlo Ersek wrote: > > On 7/26/23 19:29, Eric Blake wrote: > > > Qemu has a couple of documented meta-context definitions[1]; we might > > > as well expose constants for these namespaces. > > >

[Libguestfs] [PATCH nbdkit v2 7/9] tests/test-retry-request-mirror.c: Don't assume state after connect

2023-07-28 Thread Richard W.M. Jones
After forthcoming changes to the curl plugin we cannot assume the exact mirror we will be connected to after making the NBD connection. So remove that assumption. See: commit 38dccd848bd40cccdf012df7a606e13282aaeecb --- tests/test-retry-request-mirror.c | 4 +++- 1 file changed, 3 insertions(+),

<    1   2   3   4   5   6   7   8   9   10   >