Re: [Libguestfs] libldm crashes in a linux-sandbox context

2023-06-20 Thread Richard W.M. Jones
On Mon, Jun 19, 2023 at 08:18:20PM +0900, Vincent MAILHOL wrote: > On Fri. 16 juin 2023 at 16:34, Richard W.M. Jones wrote: > (...) > > > Last thing, the segfault on ldmtool [1] still seems a valid issue. > > > Even if I now do have a workaround for my problem, that segfa

Re: [Libguestfs] [PATCH v3] ldmtool: fix NULL pointer dereference

2023-06-20 Thread Richard W.M. Jones
ool.c > @@ -746,6 +746,8 @@ cmdline(LDM * const ldm, gchar **devices, > GArray * scanned = NULL; > if (!devices) { > scanned = get_devices(); > +if (!scanned) > +return FALSE; > devices = (gchar **) scanned->data; > }

Re: [Libguestfs] [v2v PATCH] test-data/phony-guests: fix prerequisite list of "fedora-luks-on-lvm.img"

2023-06-20 Thread Richard W.M. Jones
edora-luks-on-lvm.img: make-fedora-img.pl \ > fedora-journal.tar.xz \ > - fedora.db > + fedora.db \ > + fedora-static-bin > SRCDIR=$(srcdir) LAYOUT=luks-on-lvm $(top_builddir)/run --test ./$< > > # Make a (dummy) Fedora i

Re: [Libguestfs] libldm crashes in a linux-sandbox context

2023-06-16 Thread Richard W.M. Jones
On Fri, Jun 16, 2023 at 11:17:21AM +0900, Vincent MAILHOL wrote: > Hi Richard, > > On Fri. 16 Jun. 2023 à 03:08, Richard W.M. Jones wrote: > > On Thu, Jun 15, 2023 at 09:18:38PM +0900, Vincent Mailhol wrote: > > > Hello, > > > > > > I am using libguestf

Re: [Libguestfs] [PATCH nbdkit 3/3] tar: Implement tar-limit

2023-06-15 Thread Richard W.M. Jones
I did a lot more testing and decided to push this upstream in commits 462eb6cbf..ae3fb4def. Comments still welcome of course. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs

Re: [Libguestfs] libldm crashes in a linux-sandbox context

2023-06-15 Thread Richard W.M. Jones
On Thu, Jun 15, 2023 at 09:18:38PM +0900, Vincent Mailhol wrote: > Hello, > > I am using libguestfs in a Bazel's linux-sandbox environment[1]. > > When executing in that sandbox environment, I got frequent crashes. > > Please find attached below the results of libguestfs-test-tool when > run

[Libguestfs] [PATCH nbdkit 0/3] tar: Implement tar-limit

2023-06-13 Thread Richard W.M. Jones
This patch series cleans up some issues with nbdkit-tar-filter and (in patch 3) implements a new tar-limit feature which limits how much of the tar file we will read to find the entry within the tar file that we will serve. This protects against the case where a tar file is presented that

[Libguestfs] [PATCH nbdkit 2/3] tar: Document the optional tar option in --help output

2023-06-13 Thread Richard W.M. Jones
--- filters/tar/tar.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/filters/tar/tar.c b/filters/tar/tar.c index c650a3499..f3adb2c46 100644 --- a/filters/tar/tar.c +++ b/filters/tar/tar.c @@ -96,8 +96,9 @@ tar_config_complete (nbdkit_next_config_complete *next,

[Libguestfs] [PATCH nbdkit 3/3] tar: Implement tar-limit

2023-06-13 Thread Richard W.M. Jones
This can be used to ensure that the tar filter does not read indefinite amounts of input when opening the tar file. See: https://github.com/kubevirt/containerized-data-importer/pull/2748#issuecomment-1589852102 --- filters/tar/nbdkit-tar-filter.pod | 19 + tests/Makefile.am

[Libguestfs] [PATCH nbdkit 1/3] tests/test-tar-info*: Remove bogus test requires of guestfish

2023-06-13 Thread Richard W.M. Jones
These tests don't need guestfish so don't test for it. Fixes: commit 05b49f89dccb0ca3fd0919e9502c54dbcadf0b73 --- tests/test-tar-info-xz.sh | 1 - tests/test-tar-info.sh| 1 - 2 files changed, 2 deletions(-) diff --git a/tests/test-tar-info-xz.sh b/tests/test-tar-info-xz.sh index

Re: [Libguestfs] [PATCH libnbd 2/2] generator: state machine: Be less verbose in debug messages

2023-06-13 Thread Richard W.M. Jones
On Mon, Jun 12, 2023 at 02:38:44PM -0500, Eric Blake wrote: > On Mon, Jun 12, 2023 at 07:27:53PM +0100, Richard W.M. Jones wrote: > > Logging state transitions in debug mode produces huge amounts of > > output which is not especially helpful. This change removes this >

Re: [Libguestfs] [PATCH libnbd 1/2] info: Avoid calling nbd_opt_abort if not in option negotiation mode

2023-06-13 Thread Richard W.M. Jones
On Mon, Jun 12, 2023 at 02:32:21PM -0500, Eric Blake wrote: > On Mon, Jun 12, 2023 at 07:27:52PM +0100, Richard W.M. Jones wrote: > > This change avoids the following harmless but annoying warning on the > > normal exit path: > > > > libnbd: debug: nbd1: nbd_opt_abor

[Libguestfs] [PATCH libnbd 2/2] generator: state machine: Be less verbose in debug messages

2023-06-12 Thread Richard W.M. Jones
Logging state transitions in debug mode produces huge amounts of output which is not especially helpful. This change removes this debugging output. This reduces the debug output by approximately two thirds. --- generator/state_machine_generator.ml | 12 1 file changed, 12

[Libguestfs] [PATCH libnbd 1/2] info: Avoid calling nbd_opt_abort if not in option negotiation mode

2023-06-12 Thread Richard W.M. Jones
This change avoids the following harmless but annoying warning on the normal exit path: libnbd: debug: nbd1: nbd_opt_abort: enter: libnbd: debug: nbd1: nbd_opt_abort: leave: error="nbd_opt_abort: invalid state: READY: the handle must be negotiating: Invalid argument" --- info/main.c | 7 +--

[Libguestfs] [PATCH libnbd 0/2] Two simple patches

2023-06-12 Thread Richard W.M. Jones
These patches aren't related to each other, but both are quite simple. The second one requires particular attention - it's my experience that printing out the state transitions in debug mode has never helped me to diagnose a bug, but it has made the debug logs huge and hard to follow. However

Re: [Libguestfs] [libnbd PATCH v4 1/4] states: Document our reliance on type overlaps

2023-06-12 Thread Richard W.M. Jones
On Fri, Jun 09, 2023 at 03:39:19PM -0500, Eric Blake wrote: > [Bah - I typed up a longer response, but lost it when accidentally > trying to send through the wrong SMTP server, so now I have to > remember what I had...] > > On Fri, Jun 09, 2023 at 02:45:56PM +0200, Laszlo Ersek wrote: > > On

Re: [Libguestfs] [libnbd PATCH v3 03/22] protocol: Add definitions for extended headers

2023-06-08 Thread Richard W.M. Jones
On Thu, Jun 08, 2023 at 02:38:40PM +0200, Laszlo Ersek wrote: > On 6/8/23 14:20, Richard W.M. Jones wrote: > > On Thu, Jun 08, 2023 at 01:48:41PM +0200, Laszlo Ersek wrote: > >> On 6/7/23 12:00, Richard W.M. Jones wrote: > >>> On Tue, May 30, 2023 at 05:48:

Re: [Libguestfs] [libnbd PATCH v3 07/22] generator: Add struct nbd_extent in prep for 64-bit extents

2023-06-08 Thread Richard W.M. Jones
On Wed, Jun 07, 2023 at 04:23:27PM +0200, Laszlo Ersek wrote: [...] > > diff --git a/ocaml/helpers.c b/ocaml/helpers.c > > index 3361a696..09666daf 100644 > > --- a/ocaml/helpers.c > > +++ b/ocaml/helpers.c > > @@ -133,6 +133,26 @@ nbd_internal_ocaml_alloc_i64_from_u32_array (uint32_t > > *a,

Re: [Libguestfs] [libnbd PATCH v3 17/22] ocaml: Add example for 64-bit extents

2023-06-08 Thread Richard W.M. Jones
On Thu, Jun 08, 2023 at 10:37:59AM +0100, Richard W.M. Jones wrote: > Yes, the API is nicer now we return the subelements as a list instead > of having to iterate over the list in pairs. I might change that to > an array or struct after these patches go upstream as that will be a > t

Re: [Libguestfs] [libnbd PATCH v3 03/22] protocol: Add definitions for extended headers

2023-06-08 Thread Richard W.M. Jones
On Thu, Jun 08, 2023 at 01:48:41PM +0200, Laszlo Ersek wrote: > On 6/7/23 12:00, Richard W.M. Jones wrote: > > On Tue, May 30, 2023 at 05:48:25PM +0200, Laszlo Ersek wrote: > >> BTW I'm foreseeing a problem: if the extended block descriptor can > >> provide an unsigned

[Libguestfs] [PATCH] VxFS Filesystem support to libguestfs

2023-06-08 Thread Richard W.M. Jones
From: Ravi Singh --- appliance/hostfiles.in| 4 ++ appliance/init| 2 + daemon/Makefile.am| 7 +++ daemon/guestfsd.c | 3 + daemon/listfs.ml | 17 ++ daemon/vm.ml | 68 ++ daemon/vm.mli | 24

[Libguestfs] [PATCH] VxFS Filesystem support to libguestfs

2023-06-08 Thread Richard W.M. Jones
Reposting this patch as a patch, original message here: https://listman.redhat.com/archives/libguestfs/2023-June/031722.html ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs

Re: [Libguestfs] [libnbd PATCH v3 00/22] NBD 64-bit extensions (libnbd portion)

2023-06-08 Thread Richard W.M. Jones
Sorry it took me so long to get around to this one. Hopefully the next version will need only cursory approval. I did read all of the patches, and I basically agree with Laszlo on the ones he reviewed already. Therefore I didn't add any comment except where necessary. You can assume Acked-by

Re: [Libguestfs] [libnbd PATCH v3 22/22] api: Add nbd_[aio_]block_status_filter()

2023-06-08 Thread Richard W.M. Jones
= 0; > + { > +const char *dupes[] = { "base:allocation", "base:allocation", NULL }; > +if (nbd_block_status_filter (nbd, exportsize, 0, (char **) dupes, > + (nbd_extent64_callback) { .callback = cb, > +

Re: [Libguestfs] [libnbd PATCH v3 21/22] api: Add nbd_can_block_status_payload()

2023-06-08 Thread Richard W.M. Jones
if (strcasecmp (can, "cache") == 0) > feature = nbd_can_cache (nbd); > > diff --git a/info/show.c b/info/show.c > index 920bbb0a..8914f927 100644 > --- a/info/show.c > +++ b/info/show.c > @@ -54,7 +54,7 @@ show_one_export (struct nbd_handle *nbd, cons

Re: [Libguestfs] [libnbd PATCH v3 20/22] interop: Add test of 64-bit block status

2023-06-08 Thread Richard W.M. Jones
block_status callback interface. I think it would be best to call this test "large-block-status.{c,sh}" as "large-status" is ambiguous. (Or even "block-status-64"?) The test itself is fine, so if renamed: Reviewed-by: Richard W.M. Jones > Signed-off-by: Eric

Re: [Libguestfs] [libnbd PATCH v3 19/22] api: Add nbd_[aio_]opt_extended_headers()

2023-06-08 Thread Richard W.M. Jones
gt; + check (nbd_get_structured_replies_negotiated (nbd), false); > + check (nbd_opt_structured_reply (nbd), true); > + check (nbd_get_extended_headers_negotiated (nbd), false); > + check (nbd_get_structured_replies_negotiated (nbd), true); > + check (nbd_opt_extended_headers (n

Re: [Libguestfs] [libnbd PATCH v3 18/22] generator: Actually request extended headers

2023-06-08 Thread Richard W.M. Jones
NEXT_STATE (%^OPT_STRUCTURED_REPLY.START); > + return 0; > + > +} /* END STATE MACHINE */ > diff --git a/generator/states-newstyle-opt-starttls.c > b/generator/states-newstyle-opt-starttls.c > index e497548c..1e2997a3 100644 > --- a/generator/states-newstyle-opt-starttls.c > +

Re: [Libguestfs] [libnbd PATCH v3 17/22] ocaml: Add example for 64-bit extents

2023-06-08 Thread Richard W.M. Jones
len flags; > + fetch_offset := Int64.add !fetch_offset len > + done; > +); > +0 > +) (* NBD.block_status *) > + done > + ) Yes, the API is nicer now we return the subelements as a list instead of having to iterate over t

Re: [Libguestfs] [libnbd PATCH v3 16/22] examples: Update copy-libev to use 64-bit block status

2023-06-08 Thread Richard W.M. Jones
ack=extents_completed, > .user_data=r }, > 0); > @@ -324,7 +323,7 @@ next_extent (struct request *r) > limit = MIN (REQUEST_SIZE, size - offset); > > while (length < limit) { > - DEBUG ("e%zu: offset

Re: [Libguestfs] [libnbd PATCH v3 15/22] info: Update nbdinfo --map to use 64-bit block status

2023-06-08 Thread Richard W.M. Jones
"\"type\": %" PRIu32, > + "\"type\": %" PRIu64, > c, percent, type); > if (descr) { >fprintf (fp, ", \"description\": "); > @@ -292,7 +295,7 @@ print_totals (uint

Re: [Libguestfs] [libnbd PATCH v3 14/22] info: Expose extended-headers support through nbdinfo

2023-06-08 Thread Richard W.M. Jones
otiated = nbd_get_extended_headers_negotiated (nbd); > > if (!json_output) { >if (protocol) { > @@ -314,8 +316,9 @@ main (int argc, char *argv[]) > fprintf (fp, "protocol: %s", protocol); > if (tls_negotiated >= 0) >fprintf

Re: [Libguestfs] [libnbd PATCH v3 13/22] dump: Update nbddump to use 64-bit block status

2023-06-08 Thread Richard W.M. Jones
); >} > >count_read = 0; >for (i = 0; i < entries.len; i += 2) { > -uint32_t len = entries.ptr[i]; > -uint32_t type = entries.ptr[i+1]; > +uint64_t len = entries.ptr[i]; > +uint64_t type = entries.ptr[i+1]; > > count_read += len; > -

Re: [Libguestfs] [libnbd PATCH v3 12/22] copy: Update nbdcopy to use 64-bit block status

2023-06-08 Thread Richard W.M. Jones
; > > if (extent_list_append (ret, e) == -1) { >perror ("realloc"); >exit (EXIT_FAILURE); > } > > -offset += entries[i]; > +offset += entries[i].length; >} > >return 0; > -- > 2.40.1 Reviewed-by: Richard W

[Libguestfs] Proposal to delete the mirrors github.com/libguestfs/libnbd & /nbdkit

2023-06-07 Thread Richard W.M. Jones
I don't want to actually link to them to avoid giving them link-karma, but the old repositories / now mirrors at: github.com/libguestfs/libnbd github.com/libguestfs/nbdkit stopped mirroring the true repositories: https://gitlab.com/nbdkit/libnbd https://gitlab.com/nbdkit/nbdkit some months

Re: [Libguestfs] [libnbd PATCH v3 05/22] states: Prepare to receive 64-bit replies

2023-06-07 Thread Richard W.M. Jones
On Thu, Jun 01, 2023 at 11:04:05AM +0200, Laszlo Ersek wrote: > On 5/25/23 15:00, Eric Blake wrote: > > @@ -69,11 +75,18 @@ REPLY.STRUCTURED_REPLY.RECV_REMAINING: > > REPLY.STRUCTURED_REPLY.CHECK: > >struct command *cmd = h->reply_cmd; > >uint16_t flags, type; > > - uint32_t length; >

Re: [Libguestfs] [PATCH nbdkit 2/2] curl: Fallback to GET if HEAD not supported

2023-06-07 Thread Richard W.M. Jones
On Tue, Jun 06, 2023 at 08:06:50PM +0100, Richard W.M. Jones wrote: > > Michael Henriksen pointed out an issue with this approach. > > If the web server is actually generating the content on the fly then > it may send it as chunked encoding, and in HTTP/1.1 it's not required >

Re: [Libguestfs] [libnbd PATCH v3 04/22] states: Prepare to send 64-bit requests

2023-06-07 Thread Richard W.M. Jones
", NULL > + }; > + uint32_t strict; > + > + progname = argv[0]; > + > + nbd = nbd_create (); > + if (nbd == NULL) { > +fprintf (stderr, "%s\n", nbd_get_error ()); > +exit (EXIT_FAILURE); > + } > + > + /* Connect to the ser

Re: [Libguestfs] [libnbd PATCH v3 03/22] protocol: Add definitions for extended headers

2023-06-07 Thread Richard W.M. Jones
On Tue, May 30, 2023 at 05:48:25PM +0200, Laszlo Ersek wrote: > BTW I'm foreseeing a problem: if the extended block descriptor can > provide an unsigned 64-bit length, we're going to have trouble exposing > that in OCaml, because OCaml only has signed 64-bit integers. So that's > going to

Re: [Libguestfs] [PATCH nbdkit 2/2] curl: Fallback to GET if HEAD not supported

2023-06-06 Thread Richard W.M. Jones
Michael Henriksen pointed out an issue with this approach. If the web server is actually generating the content on the fly then it may send it as chunked encoding, and in HTTP/1.1 it's not required that the Content-Length field is present (since it may not be known when the server begins

Re: [Libguestfs] [PATCH nbdkit 2/2] curl: Fallback to GET if HEAD not supported

2023-06-06 Thread Richard W.M. Jones
On Tue, Jun 06, 2023 at 06:09:09PM +0200, Laszlo Ersek wrote: > only superficial comments: > > On 6/6/23 13:22, Richard W.M. Jones wrote: [...] > > diff --git a/tests/test-curl-head-forbidden.c > > b/tests/test-curl-head-forbidden.c > > new file mode 100644 >

Re: [Libguestfs] [PATCH nbdkit 0/2] curl: Fallback to GET if HEAD not supported

2023-06-06 Thread Richard W.M. Jones
On Tue, Jun 06, 2023 at 12:22:12PM +0100, Richard W.M. Jones wrote: > I haven't been able to test it against a real server, but I modified > our internal test web server so it can behave like this and added a > test. > > Fixes: https://github.com/kubevirt/containerized-data-impor

Re: [Libguestfs] [PATCH libguestfs 1/2] ocaml/implicit_close test: collect all currently unreachable blocks

2023-06-06 Thread Richard W.M. Jones
On Sat, May 27, 2023 at 03:32:36PM +0200, Jürgen Hötzel wrote: > Fixes failing implice_close test on OCaml 5. > --- > ocaml/t/guestfs_065_implicit_close.ml | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/ocaml/t/guestfs_065_implicit_close.ml >

[Libguestfs] [PATCH nbdkit 1/2] tests/web-server.c: Ignore SIGPIPE

2023-06-06 Thread Richard W.M. Jones
If the client (curl plugin) disconnects early then the whole test can fail with SIGPIPE, and it can be unclear why the test failed -- you just get the mysterious error "FAIL test (exit status: 141)". We always check the return code from write(2) so just report EPIPE failures through that. In

[Libguestfs] [PATCH nbdkit 0/2] curl: Fallback to GET if HEAD not supported

2023-06-06 Thread Richard W.M. Jones
AWS S3 servers are mad. You can do an unclear "pre-signing" operation on them, but only for a single method (eg. just GET). This means that the way we get the size of the export, by first querying the headers with HEAD and then using GET for partial data transfers, does not work. In theory we

[Libguestfs] [PATCH nbdkit 2/2] curl: Fallback to GET if HEAD not supported

2023-06-06 Thread Richard W.M. Jones
Some servers do not support HEAD for requesting the headers. If the HEAD request fails, fallback to using the GET method, abandoning the transfer as soon as possible after the headers have been received. Fixes: https://github.com/kubevirt/containerized-data-importer/issues/2737 ---

Re: [Libguestfs] Workaround to get supermin to work on Slackware

2023-06-02 Thread Richard W.M. Jones
On Fri, Jun 02, 2023 at 04:35:06AM +0400, shiftag wrote: > > On 6/1/23 15:21, Richard W.M. Jones wrote: > >On Thu, Jun 01, 2023 at 07:07:45AM +0400, shiftag wrote: > >>Hello, > >> > >>I'm experimenting with a Software using supermin. However, it loo

Re: [Libguestfs] [PATCH libguestfs 1/2] ocaml/implicit_close test: collect all currently unreachable blocks

2023-06-01 Thread Richard W.M. Jones
On Thu, Jun 01, 2023 at 01:26:13PM +0200, Laszlo Ersek wrote: > On 6/1/23 13:18, Richard W.M. Jones wrote: > > [Sorry, not read the other replies yet, in meetings all day today] > > > > On Wed, May 31, 2023 at 05:13:58PM +0100, Richard W.M. Jones wrote: > >&g

Re: [Libguestfs] Workaround to get supermin to work on Slackware

2023-06-01 Thread Richard W.M. Jones
On Thu, Jun 01, 2023 at 07:07:45AM +0400, shiftag wrote: > Hello, > > I'm experimenting with a Software using supermin. However, it look > like supermin cannot work because of Slackware package manager. > > $ supermin --list-drivers > arch/pacman   not-detected > debian/dpkg  

Re: [Libguestfs] [PATCH libguestfs 1/2] ocaml/implicit_close test: collect all currently unreachable blocks

2023-06-01 Thread Richard W.M. Jones
[Sorry, not read the other replies yet, in meetings all day today] On Wed, May 31, 2023 at 05:13:58PM +0100, Richard W.M. Jones wrote: > To start with I have opened a discussion topic. We can think about > opening bugs later. > > https://discuss.ocaml.org/t/ocaml-heap-fsck

Re: [Libguestfs] [PATCH libguestfs 1/2] ocaml/implicit_close test: collect all currently unreachable blocks

2023-05-31 Thread Richard W.M. Jones
On Wed, May 31, 2023 at 05:33:13PM +0200, Laszlo Ersek wrote: > On 5/31/23 13:23, Richard W.M. Jones wrote: > > > > ... And while I'm scrawling my throughts into this thread ... > > > > What we intend here are two slightly different operations: > > > > (A)

Re: [Libguestfs] [PATCH libguestfs 1/2] ocaml/implicit_close test: collect all currently unreachable blocks

2023-05-31 Thread Richard W.M. Jones
On Wed, May 31, 2023 at 04:44:49PM +0200, Laszlo Ersek wrote: > On 5/31/23 13:13, Richard W.M. Jones wrote: > > On Wed, May 31, 2023 at 12:32:49PM +0200, Laszlo Ersek wrote: > >> On 5/31/23 11:12, Richard W.M. Jones wrote: > >>> On Sat, May 27, 2023 at 03:32:3

Re: [Libguestfs] [PATCH libguestfs 1/2] ocaml/implicit_close test: collect all currently unreachable blocks

2023-05-31 Thread Richard W.M. Jones
... And while I'm scrawling my throughts into this thread ... What we intend here are two slightly different operations: (A) Free every unreachable object. That's what we want in this specific place in the code. (B) Provide a soft test that the OCaml heap hasn't been screwed up because of

Re: [Libguestfs] [PATCH libguestfs 1/2] ocaml/implicit_close test: collect all currently unreachable blocks

2023-05-31 Thread Richard W.M. Jones
On Wed, May 31, 2023 at 12:32:49PM +0200, Laszlo Ersek wrote: > On 5/31/23 11:12, Richard W.M. Jones wrote: > > On Sat, May 27, 2023 at 03:32:36PM +0200, Jürgen Hötzel wrote: > >> Fixes failing implice_close test on OCaml 5. > >> --- > >> ocaml/t/guestfs_06

Re: [Libguestfs] [PATCH libguestfs 1/2] ocaml/implicit_close test: collect all currently unreachable blocks

2023-05-31 Thread Richard W.M. Jones
This bug and thread seem relevant: https://github.com/ocaml/ocaml/issues/11812 https://discuss.ocaml.org/t/ocaml-5-gc-releasing-memory-back-to-the-os/11293 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog:

Re: [Libguestfs] [PATCH libguestfs 1/2] ocaml/implicit_close test: collect all currently unreachable blocks

2023-05-31 Thread Richard W.M. Jones
On Sat, May 27, 2023 at 03:32:36PM +0200, Jürgen Hötzel wrote: > Fixes failing implice_close test on OCaml 5. > --- > ocaml/t/guestfs_065_implicit_close.ml | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/ocaml/t/guestfs_065_implicit_close.ml >

Re: [Libguestfs] [PATCH libguestfs 2/2] Only leave/enter blocking_section when OCaml lock is not held

2023-05-31 Thread Richard W.M. Jones
On Sat, May 27, 2023 at 03:35:38PM +0200, Jürgen Hötzel wrote: > Fixes deadlocks on OCaml5 when trying to get the lock that is already > held: > > Fatal error during lock: Resource deadlock avoided > --- > ocaml/guestfs-c.c | 8 ++-- > 1 file changed, 6 insertions(+), 2 deletions(-) > >

Re: [Libguestfs] [libnbd PATCH] internal: s/handle/cookie/ to match NBD spec

2023-05-31 Thread Richard W.M. Jones
On Tue, May 30, 2023 at 12:44:41PM -0500, Eric Blake wrote: > On Tue, May 30, 2023 at 01:18:55PM +0200, Laszlo Ersek wrote: > > On 5/29/23 18:24, Eric Blake wrote: > > > Externally, we have been exposing the 64-bit opaque marker for each > > > NBD packet as the "cookie", because it was less

Re: [Libguestfs] [libnbd PATCH] internal: Tweak use of attribute packed in union sbuf

2023-05-31 Thread Richard W.M. Jones
msg[NBD_MAX_STRING]; /* Common to all error types */ >uint64_t offset; /* Only used for NBD_REPLY_TYPE_ERROR_OFFSET */ > -} __attribute__ ((packed)) error; > +} NBD_ATTRIBUTE_PACKED error; >} payload; > -} __attribute__ ((packed)) sr; > +} s

Re: [Libguestfs] libnbd | Failed pipeline for master | 21f4c618

2023-05-30 Thread Richard W.M. Jones
On Mon, May 29, 2023 at 03:17:11PM +, GitLab wrote: > GitLab >✖ Pipeline #882103063 has failed! > > Project nbdkit / libnbd > Branch● master > Commit● 21f4c618 > maint: Ignore .dirstamp files from automake A ... > Commit Author ● Eric Blake

Re: [Libguestfs] [libnbd PATCH] maint: Drop useless casts

2023-05-24 Thread Richard W.M. Jones
@ -415,7 +415,7 @@ main (int argc, char *argv[]) >handles_append (, h); /* reserved above, so can't fail */ > } >} > - connections = (unsigned)nbd.len; > + connections = nbd.len; Similarly this would be size_t -> unsigned, also a truncation on 64 bit. But if there are no warnin

Re: [Libguestfs] [libnbd PATCH] maint: pick consistent spacing style for casts

2023-05-24 Thread Richard W.M. Jones
On Wed, May 24, 2023 at 07:39:20AM +0200, Laszlo Ersek wrote: > Going through the changes individually, it seems like we could eliminate > a handful of the casts altogether; for examle (char *)"string literal" > ones. It'd be interesting to remove then and compile with './configure

Re: [Libguestfs] [libnbd PATCH] maint: pick consistent spacing style for casts

2023-05-24 Thread Richard W.M. Jones
; > $ git grep '\(_t\|int\|long\|signed\|char\|\*\)) [a-zA-Z0-9&"]' -- \ > '**/*.[hc]' '**/*.ml' | grep -v '^ */\?\*' > > 'git show -w' shows that this change is whitespace only. > > [1] https://listman.redhat.com/archives/libguestfs/2023-February/030771.html >

Re: [Libguestfs] [libguestfs-common PATCH] Add support for OCaml 5.0

2023-05-22 Thread Richard W.M. Jones
On Sat, May 20, 2023 at 06:16:40PM +0200, Jürgen Hötzel wrote: > Pervasives is deprecated since Ocaml 4.08 and has been > removed in OCaml 5.0. > > https://github.com/ocaml/ocaml/pull/1605 > --- > mlstdutils/std_utils.ml | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git

Re: [Libguestfs] [guestfs-tools PATCH 3/3] inspector: test /dev/mapper/VG-LV translation in LUKS-on-LVM test

2023-05-22 Thread Richard W.M. Jones
On Mon, May 22, 2023 at 02:46:34PM +0200, Laszlo Ersek wrote: > On 5/19/23 18:31, Richard W.M. Jones wrote: > > > > For the series: > > > > Reviewed-by: Richard W.M. Jones > > Thanks! > > Commit range 67647b883e13..569bd1dd29da. > > > > &g

Re: [Libguestfs] [guestfs-tools PATCH 3/3] inspector: test /dev/mapper/VG-LV translation in LUKS-on-LVM test

2023-05-19 Thread Richard W.M. Jones
For the series: Reviewed-by: Richard W.M. Jones BTW it's usually possible to cherry pick across git repos, eg: $ git fetch ../libguestfs $ git cherry-pick -x Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization

Re: [Libguestfs] [libguestfs PATCH 3/3] LUKS-on-LVM inspection test: test /dev/mapper/VG-LV translation

2023-05-19 Thread Richard W.M. Jones
ey /dev/mapper/Volume--Group-Logical--Volume--3:key:FEDORA-LV3 > +) > +fish_ref=$("${guestfish[@]}" "${keys_by_mapper_lvname[@]}") > +eval "$fish_ref" > + > +# Repeat the test. > +check_filesystems For the series: Reviewed-by: Richard W.M. Jones R

Re: [Libguestfs] [PATCH nbdkit v2 5/6] New plugin: ones

2023-05-18 Thread Richard W.M. Jones
I pushed this series upstream, commits 3095d8bad..8627ed08b. The changes should address everything you mentioned: * nbdkit_parse_probability now only returns probabilities >= 0 and not NaN or infinite (but they can still be larger than 100%) * Add Rust bindings for parse_size, parse_bool

Re: [Libguestfs] [libguestfs-common PATCH v2 2/2] options/keys: introduce unescape_device_mapper_lvm()

2023-05-18 Thread Richard W.M. Jones
On Thu, May 18, 2023 at 05:42:03PM +0200, Laszlo Ersek wrote: > On 5/18/23 15:42, Richard W.M. Jones wrote: > > On Thu, May 18, 2023 at 03:09:42PM +0200, Laszlo Ersek wrote: > >> Libguestfs uses the > >> > >> /dev/VG/LV > >> > >>

Re: [Libguestfs] [libguestfs-common PATCH v2 2/2] options/keys: introduce unescape_device_mapper_lvm()

2023-05-18 Thread Richard W.M. Jones
On Thu, May 18, 2023 at 03:09:42PM +0200, Laszlo Ersek wrote: > Libguestfs uses the > > /dev/VG/LV > > format internally, for identifying LVM logical volumes, but the user might > want to specify the > > /dev/mapper/VG-LV ID > > format with the "--key ID:..." options. > > Introduce

Re: [Libguestfs] [PATCH nbdkit v2 6/6] New filter: evil

2023-05-18 Thread Richard W.M. Jones
On Wed, May 17, 2023 at 05:32:50PM -0500, Eric Blake wrote: > On Wed, May 17, 2023 at 11:06:59AM +0100, Richard W.M. Jones wrote: > > +static void > > +corrupt_all_bits (uint8_t *buf, uint32_t count, > > + struct random_state *rs, enum corruption_type ct) &

Re: [Libguestfs] [PATCH nbdkit v2 4/6] common/include: Make log_2_bits work on 64 bit ints

2023-05-18 Thread Richard W.M. Jones
On Wed, May 17, 2023 at 04:36:32PM -0500, Eric Blake wrote: > On Wed, May 17, 2023 at 11:06:57AM +0100, Richard W.M. Jones wrote: > > Previously it only worked for 64 bit ints on 64 bit platforms, but we > > can easily adjust the function to work on any platform. > > --

Re: [Libguestfs] [PATCH nbdkit v2 1/6] Add new public nbdkit_parse_probability function

2023-05-18 Thread Richard W.M. Jones
On Wed, May 17, 2023 at 04:05:23PM -0500, Eric Blake wrote: > On Wed, May 17, 2023 at 11:06:54AM +0100, Richard W.M. Jones wrote: > > In nbdkit-error-filter we need to parse parameters as probabilities. > > This is useful enough to add to nbdkit, since we will use it in >

Re: [Libguestfs] [PATCH nbdkit 5/5] New filter: evil

2023-05-17 Thread Richard W.M. Jones
On Wed, May 17, 2023 at 03:28:11PM +0200, Laszlo Ersek wrote: > On 5/16/23 14:12, Richard W.M. Jones wrote: > > > +/* This is the heart of the algorithm, the function which corrupts > > + * the buffer after reading it from the plugin. > > + * > > + * The observat

[Libguestfs] [PATCH nbdkit v2 4/6] common/include: Make log_2_bits work on 64 bit ints

2023-05-17 Thread Richard W.M. Jones
Previously it only worked for 64 bit ints on 64 bit platforms, but we can easily adjust the function to work on any platform. --- common/include/ispowerof2.h | 4 ++-- common/include/test-ispowerof2.c | 6 ++ 2 files changed, 4 insertions(+), 6 deletions(-) diff --git

[Libguestfs] [PATCH nbdkit v2 5/6] New plugin: ones

2023-05-17 Thread Richard W.M. Jones
+=back + +=head1 VERSION + +C first appeared in nbdkit 1.36. + +=head1 SEE ALSO + +L, +L, +L, +L, +L, +L, +L. +L, +L. + +=head1 AUTHORS + +Richard W.M. Jones + +=head1 COPYRIGHT + +Copyright Red Hat diff --git a/plugins/pattern/nbdkit-pattern-plugin.pod b/plugins/pattern/nbdkit-pattern-pl

[Libguestfs] [PATCH nbdkit v2 2/6] error: Use new nbdkit_parse_probability

2023-05-17 Thread Richard W.M. Jones
This replaces the existing ad hoc parsing with the new public function. --- filters/error/nbdkit-error-filter.pod | 12 +++- filters/error/error.c | 14 +- 2 files changed, 8 insertions(+), 18 deletions(-) diff --git a/filters/error/nbdkit-error-filter.pod

[Libguestfs] [PATCH nbdkit v2 1/6] Add new public nbdkit_parse_probability function

2023-05-17 Thread Richard W.M. Jones
In nbdkit-error-filter we need to parse parameters as probabilities. This is useful enough to add to nbdkit, since we will use it in another filter in future. --- docs/nbdkit-plugin.pod | 19 +++ plugins/python/nbdkit-python-plugin.pod | 6 ++ include/nbdkit-common.h

[Libguestfs] [PATCH nbdkit v2 3/6] common/include: Add next_power_of_2 function

2023-05-17 Thread Richard W.M. Jones
It takes a 64 bit integer and finds the next power of 2, eg. next_power_of_2 (510) => 512 (2^9) Taken from https://jameshfisher.com/2018/03/30/round-up-power-2/ with some fixes. Thanks: Eric Blake --- common/include/ispowerof2.h | 9 + common/include/test-ispowerof2.c | 15

[Libguestfs] [PATCH nbdkit v2 6/6] New filter: evil

2023-05-17 Thread Richard W.M. Jones
ck + +=head1 FILES + +=over 4 + +=item F<$filterdir/nbdkit-evil-filter.so> + +The filter. + +Use C to find the location of C<$filterdir>. + +=back + +=head1 VERSION + +C first appeared in nbdkit 1.36. + +=head1 SEE ALSO + +L, +L, +L, +L, +L. + +=head1 AUTHORS + +Richard W.M. Jones

[Libguestfs] [PATCH nbdkit v2 0/6] New ones plugin and evil filter

2023-05-17 Thread Richard W.M. Jones
This addresses most of the issues from the review of v1 here: https://listman.redhat.com/archives/libguestfs/2023-May/031520.html * The filter now only handles 1e-12 <= P <= 1/8. Probabilities outside this range are treated as 0% or 100% respectively. The reasons are given in the code and

Re: [Libguestfs] [PATCH nbdkit 1/5] Add new public nbdkit_parse_probability function

2023-05-17 Thread Richard W.M. Jones
On Tue, May 16, 2023 at 09:22:31PM +0100, Richard W.M. Jones wrote: > I'm not really sure we reached a conclusion so far, but I did want to > say that I changed the evil filter impl so that it now treats any > 0 <= P < 1e-12 as effectively 0. (P == evil-probability; P < 0 and

Re: [Libguestfs] [PATCH nbdkit 1/5] Add new public nbdkit_parse_probability function

2023-05-16 Thread Richard W.M. Jones
On Tue, May 16, 2023 at 02:35:38PM -0500, Eric Blake wrote: > > > +=head2 Parsing probabilities > > > + > > > +Use the C utility function to parse > > > +probabilities. Common formats understood include: C<"0.1">, C<"10%"> > > > +or C<"1:10">, which all mean a probability of 1 in 10. > > > + > >

Re: [Libguestfs] [PATCH nbdkit 5/5] New filter: evil

2023-05-16 Thread Richard W.M. Jones
On Tue, May 16, 2023 at 10:55:25AM -0500, Eric Blake wrote: > > +static int > > +evil_config (nbdkit_next_config *next, nbdkit_backend *nxdata, > > + const char *key, const char *value) > > +{ > > + if (strcmp (key, "evil") == 0 || strcmp (key, "evil-mode") == 0) { > > +if (strcmp

Re: [Libguestfs] virt-v2v check-valgrind?

2023-05-16 Thread Richard W.M. Jones
Or maybe we could have tests/Makefile.am remove GLIBC_TUNABLES if we're using valgrind. Something like this perhaps ... check-valgrind: -$(MAKE) VG="@VG@" check +$(MAKE) GLIBC_TUNABLES= VG="@VG@" check Rich. -- Richard Jones, Virtualization Group, Red Hat

Re: [Libguestfs] virt-v2v check-valgrind?

2023-05-16 Thread Richard W.M. Jones
On Tue, May 16, 2023 at 04:25:04PM +0200, Laszlo Ersek wrote: > Should I submit patches to remove the LD_PRELOAD and GLIBC_TUNABLES > tweaking from all the run scripts? Those do something useful - malloc perturbing occasionally finds bugs. Since you're running RHEL 9, why not install the

Re: [Libguestfs] virt-v2v check-valgrind?

2023-05-16 Thread Richard W.M. Jones
On Tue, May 16, 2023 at 02:37:00PM +0200, Laszlo Ersek wrote: > Something is not adding up. > > * I've run "ldd" on my locally built virt-v2v binary, to learn what shared > libraries it uses. Then I located all the packages (installed RPMs) providing > those libraries (symlinks in fact), using

Re: [Libguestfs] [PATCH nbdkit 3/5] common/include: Add next_power_of_2 function

2023-05-16 Thread Richard W.M. Jones
On Tue, May 16, 2023 at 07:29:45AM -0500, Eric Blake wrote: > > On Tue, May 16, 2023 at 01:12:17PM +0100, Richard W.M. Jones wrote: > > > > It takes a 64 bit integer and finds the next power of 2, > > eg. next_power_of_2 (510) => 512 (2^9) > > > > Taken f

Re: [Libguestfs] [v2v PATCH 2/2] LUKS-on-LVM conversion test: test /dev/mapper/VG-LV translation

2023-05-16 Thread Richard W.M. Jones
> + --key /dev/mapper/Volume--Group-Logical--Volume--2:key:FEDORA-LV2 > + --key /dev/mapper/Volume--Group-Logical--Volume--3:key:FEDORA-LV3) > + > +$VG virt-v2v --debug-gc -i disk $f -o null "${keys[@]}" Yup all good: Reviewed-by: Richard W.M. Jones However I think t

Re: [Libguestfs] [v2v PATCH 1/2] LUKS-on-LVM conversion test: rename VGs and LVs

2023-05-16 Thread Richard W.M. Jones
ical-Volume-3', 'Volume-Group', 64); Initially I was surprised that this didn't break a bazillion tests in libguestfs / guestfs-tools, however then I realised this patch would only go into virt-v2v and is not shared. So the change and the minor test fix below both look fine: Reviewed-by: Rich

Re: [Libguestfs] [libguestfs-common PATCH 2/2] options/keys: introduce unescape_device_mapper_lvm()

2023-05-16 Thread Richard W.M. Jones
On Mon, May 15, 2023 at 07:49:24PM +0200, Laszlo Ersek wrote: > Introduce unescape_device_mapper_lvm() for turning > > /dev/mapper/VG-LV > > key IDs into > > /dev/VG/LV > > ones, unescaping doubled hyphens to single hyphens in both VG and LV in > the process. Libguestfs uses the /dev/VG/LV

Re: [Libguestfs] [libguestfs-common PATCH 1/2] options/keys: key_store_import_key(): un-constify "key" parameter

2023-05-16 Thread Richard W.M. Jones
onst strings? However I agree it looks confusing so that's a decent reason to change it, so: Reviewed-by: Richard W.M. Jones > Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2168506 > Signed-off-by: Laszlo Ersek > --- > options/options.h | 3 ++- > options/keys.c| 2 +- >

[Libguestfs] [PATCH nbdkit 1/5] Add new public nbdkit_parse_probability function

2023-05-16 Thread Richard W.M. Jones
In nbdkit-error-filter we need to parse parameters as probabilities. This is useful enough to add to nbdkit, since we will use it in another filter in future. --- docs/nbdkit-plugin.pod | 19 +++ plugins/python/nbdkit-python-plugin.pod | 6 ++ include/nbdkit-common.h

[Libguestfs] [PATCH nbdkit 4/5] New plugin: ones

2023-05-16 Thread Richard W.M. Jones
n nbdkit 1.36. + +=head1 SEE ALSO + +L, +L, +L, +L, +L, +L, +L. +L, +L. + +=head1 AUTHORS + +Richard W.M. Jones + +=head1 COPYRIGHT + +Copyright Red Hat diff --git a/plugins/pattern/nbdkit-pattern-plugin.pod b/plugins/pattern/nbdkit-pattern-plugin.pod index c5d4f0d36..f4ae324ed 100644 --- a/plugin

[Libguestfs] [PATCH nbdkit 0/5] New ones plugin and evil filter

2023-05-16 Thread Richard W.M. Jones
This is just a bit of silliness I was thinking about on the plane last week. Yes, I know nbdkit_parse_probability uses scanf and that's bad. I still think we should fix scanf. We might call the new filter "--filter=corruption" instead, but this name stuck. Rich.

[Libguestfs] [PATCH nbdkit 2/5] error: Use new nbdkit_parse_probability

2023-05-16 Thread Richard W.M. Jones
This replaces the existing ad hoc parsing with the new public function. --- filters/error/nbdkit-error-filter.pod | 12 +++- filters/error/error.c | 14 +- 2 files changed, 8 insertions(+), 18 deletions(-) diff --git a/filters/error/nbdkit-error-filter.pod

[Libguestfs] [PATCH nbdkit 5/5] New filter: evil

2023-05-16 Thread Richard W.M. Jones
il-filter.so> + +The filter. + +Use C to find the location of C<$filterdir>. + +=back + +=head1 VERSION + +C first appeared in nbdkit 1.36. + +=head1 SEE ALSO + +L, +L, +L, +L, +L. + +=head1 AUTHORS + +Richard W.M. Jones + +=head1 COPYRIGHT + +Copyright Red Hat diff --git a/configure.ac b/

[Libguestfs] [PATCH nbdkit 3/5] common/include: Add next_power_of_2 function

2023-05-16 Thread Richard W.M. Jones
It takes a 64 bit integer and finds the next power of 2, eg. next_power_of_2 (510) => 512 (2^9) Taken from https://jameshfisher.com/2018/03/30/round-up-power-2/ with some fixes. --- common/include/ispowerof2.h | 9 + common/include/test-ispowerof2.c | 14 ++ 2 files

Re: [Libguestfs] virt-v2v check-valgrind?

2023-05-15 Thread Richard W.M. Jones
On Mon, May 15, 2023 at 07:22:28PM +0200, Laszlo Ersek wrote: > Hi Rich, > > do we expect "make check-valgrind" to succeed in virt-v2v at the moment > (at commit e83de8abe6c5)? I see there's a "valgrind-suppressions" file > in the project root, but "make check-valgrind" still fails for me, with >

Re: [Libguestfs] [libnbd PATCH 0/6] finish wrapping generated C code harder

2023-05-15 Thread Richard W.M. Jones
It all looks fine to me, barring comments that Eric has already made, so ACK series. Thanks, Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with

<    1   2   3   4   5   6   7   8   9   10   >