Re: [Libguestfs] nbdkit build failure (was: Re: jplesnik's nbdkit-1.35.5-4.fc39 failed to build)

2023-07-13 Thread Richard W.M. Jones
On Tue, Jul 11, 2023 at 09:23:35PM +0100, Richard W.M. Jones wrote: > On Tue, Jul 11, 2023 at 07:31:07PM +0100, Richard W.M. Jones wrote: > > > > https://koji.fedoraproject.org/koji/buildinfo?buildID=2230411 > > > > It's a bit of a coincidence that this failed exactly in the Perl test > > &

Re: [Libguestfs] [PATCH] Add partprobe call in the API

2023-07-13 Thread Philippe Midol-Monnet
Hello You will find in the joined file a new version of the patch including the missing code. The automatic disclaimer automatically added to end of this e-mail does not apply to the patch file as seen with our legal team. Regards Philippe On 20/06/2023 14:30, Richard W.M. Jones wrote:

Re: [Libguestfs] nbdkit build failure (was: Re: jplesnik's nbdkit-1.35.5-4.fc39 failed to build)

2023-07-13 Thread Jitka Plesnikova
Thank you very much for your investigation. Perl 5.38 is merge to rawhide now. Regards; Jitka On 7/13/23 12:55, Richard W.M. Jones wrote: On Thu, Jul 13, 2023 at 11:03:35AM +0100, Richard W.M. Jones wrote: On Thu, Jul 13, 2023 at 10:29:44AM +0100, Richard W.M. Jones wrote: It's a crash in

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Tage Johansson
On 7/13/2023 4:36 PM, Eric Blake wrote: On Thu, Jul 13, 2023 at 01:37:49PM +, Tage Johansson wrote: On 7/13/2023 3:26 PM, Richard W.M. Jones wrote: On Thu, Jul 13, 2023 at 08:01:09AM -0500, Eric Blake wrote: On Thu, Jul 13, 2023 at 07:13:37AM -0500, Eric Blake wrote: I have replaced a

Re: [Libguestfs] nbdkit build failure (was: Re: jplesnik's nbdkit-1.35.5-4.fc39 failed to build)

2023-07-13 Thread Richard W.M. Jones
It's a crash in Perl code, although called from the nbdkit perl plugin (C bindings). Still debugging this ... Core was generated by `/home/rjones/d/nbdkit/server/nbdkit -U /tmp/nbdkittPN34a/sock -P /tmp/nbdkittPN'. Program terminated with signal SIGSEGV, Segmentation fault. warning: Section

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Tage Johansson
Apologize if resending, but I'm not sure my previous email was actually delivered. On 7/12/2023 10:33 PM, Eric Blake wrote: On Wed, Jul 12, 2023 at 03:19:59PM +, Tage Johansson wrote: Hello, While writing some tests for the Rust bindings, I discovered a memory leak with Valgrind due

Re: [Libguestfs] [libnbd PATCH] tests: Test free callback in synchronous nbd_opt_* calls

2023-07-13 Thread Richard W.M. Jones
On Wed, Jul 12, 2023 at 04:56:03PM -0500, Eric Blake wrote: > The documentation guarantees that a user's .free callback is reached > exactly once for all synchronous nbd_opt_* functions that take a > callback structure (nbd_opt_list, nbd_opt_list_meta, ...), regardless > of API success or failure,

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Eric Blake
On Thu, Jul 13, 2023 at 12:03:24PM +0100, Richard W.M. Jones wrote: > On Thu, Jul 13, 2023 at 09:53:58AM +, Tage Johansson wrote: > > Apologize if resending, but I'm not sure my previous email was > > actually delivered. > > > > > > On 7/12/2023 10:33 PM, Eric Blake wrote: > > > > > > >On

Re: [Libguestfs] nbdkit build failure (was: Re: jplesnik's nbdkit-1.35.5-4.fc39 failed to build)

2023-07-13 Thread Richard W.M. Jones
On Thu, Jul 13, 2023 at 10:29:44AM +0100, Richard W.M. Jones wrote: > It's a crash in Perl code, although called from the nbdkit perl plugin > (C bindings). Still debugging this ... > > Core was generated by `/home/rjones/d/nbdkit/server/nbdkit -U > /tmp/nbdkittPN34a/sock -P /tmp/nbdkittPN'. >

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Eric Blake
On Thu, Jul 13, 2023 at 07:13:37AM -0500, Eric Blake wrote: > > > I have replaced a call to `nbd_opt_info()` with a call to > > > `nbd_aio_opt_info()` and passed in a completion callback which just > > > calls `exit(EXIT_FAILURE)`. So if the completion callback is called > > > the test should

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Richard W.M. Jones
On Wed, Jul 12, 2023 at 03:19:59PM +, Tage Johansson wrote: > While writing some tests for the Rust bindings, I discovered a > memory leak with Valgrind due to a completion callback not being > freed. A note about the valgrind tests: They are quite sensitive to versions of software

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Tage Johansson
On 7/13/2023 2:13 PM, Eric Blake wrote: On Thu, Jul 13, 2023 at 12:03:24PM +0100, Richard W.M. Jones wrote: On Thu, Jul 13, 2023 at 09:53:58AM +, Tage Johansson wrote: Apologize if resending, but I'm not sure my previous email was actually delivered. On 7/12/2023 10:33 PM, Eric Blake

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Eric Blake
On Thu, Jul 13, 2023 at 09:36:58AM -0500, Eric Blake wrote: > > The user needs a way to know if an error occurred. So the completion > > callback must be called if the asynchronous function did not fail (returned > > 0). If the completion callback should be called, with the error parameter > >

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Tage Johansson
On 7/13/2023 1:03 PM, Richard W.M. Jones wrote: On Thu, Jul 13, 2023 at 09:53:58AM +, Tage Johansson wrote: Apologize if resending, but I'm not sure my previous email was actually delivered. On 7/12/2023 10:33 PM, Eric Blake wrote: On Wed, Jul 12, 2023 at 03:19:59PM +, Tage

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Tage Johansson
On 7/13/2023 3:26 PM, Richard W.M. Jones wrote: On Thu, Jul 13, 2023 at 08:01:09AM -0500, Eric Blake wrote: On Thu, Jul 13, 2023 at 07:13:37AM -0500, Eric Blake wrote: I have replaced a call to `nbd_opt_info()` with a call to `nbd_aio_opt_info()` and passed in a completion callback which

Re: [Libguestfs] nbdkit build failure (was: Re: jplesnik's nbdkit-1.35.5-4.fc39 failed to build)

2023-07-13 Thread Richard W.M. Jones
On Thu, Jul 13, 2023 at 11:03:35AM +0100, Richard W.M. Jones wrote: > On Thu, Jul 13, 2023 at 10:29:44AM +0100, Richard W.M. Jones wrote: > > It's a crash in Perl code, although called from the nbdkit perl plugin > > (C bindings). Still debugging this ... > > > > Core was generated by

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Tage Johansson
On 7/12/2023 10:33 PM, Eric Blake wrote: On Wed, Jul 12, 2023 at 03:19:59PM +, Tage Johansson wrote: Hello, While writing some tests for the Rust bindings, I discovered a memory leak with Valgrind due to a completion callback not being freed. More specificly, the completion callback of

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Richard W.M. Jones
On Thu, Jul 13, 2023 at 09:53:58AM +, Tage Johansson wrote: > Apologize if resending, but I'm not sure my previous email was > actually delivered. > > > On 7/12/2023 10:33 PM, Eric Blake wrote: > > > >On Wed, Jul 12, 2023 at 03:19:59PM +, Tage Johansson wrote: > >>Hello, > >> > >> >

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Richard W.M. Jones
On Thu, Jul 13, 2023 at 08:01:09AM -0500, Eric Blake wrote: > On Thu, Jul 13, 2023 at 07:13:37AM -0500, Eric Blake wrote: > > > > I have replaced a call to `nbd_opt_info()` with a call to > > > > `nbd_aio_opt_info()` and passed in a completion callback which just > > > > calls

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Eric Blake
On Thu, Jul 13, 2023 at 01:37:49PM +, Tage Johansson wrote: > > On 7/13/2023 3:26 PM, Richard W.M. Jones wrote: > > On Thu, Jul 13, 2023 at 08:01:09AM -0500, Eric Blake wrote: > > > On Thu, Jul 13, 2023 at 07:13:37AM -0500, Eric Blake wrote: > > > > > > I have replaced a call to

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Richard W.M. Jones
On Thu, Jul 13, 2023 at 03:05:30PM +, Tage Johansson wrote: > > On 7/13/2023 4:36 PM, Eric Blake wrote: > >On Thu, Jul 13, 2023 at 01:37:49PM +, Tage Johansson wrote: > >>On 7/13/2023 3:26 PM, Richard W.M. Jones wrote: > >>>On Thu, Jul 13, 2023 at 08:01:09AM -0500, Eric Blake wrote: >

[Libguestfs] [libguestfs PATCH 7/7] lib/launch-direct: support networking with passt

2023-07-13 Thread Laszlo Ersek
On QEMU 7.2.0+, if "passt" is available, ask QEMU for passt ("stream") rather than SLIRP ("user") networking. For this, we need to run passt ourselves. Given that passt daemonizes by default, start it with our traditional function guestfs_int_cmd_run(). Ask passt to save its PID file, because in

[Libguestfs] [libguestfs PATCH 3/7] docs: fix broken link in the guestfs manual

2023-07-13 Thread Laszlo Ersek
Commit 55202a4d49a1 ("New API: get-sockdir", 2016-02-03) added identical language to "fish/guestfish.pod" and "src/guestfs.pod", including an internal link L. That's appropriate for "fish/guestfish.pod", but the same API description is generated with a different anchor for "src/guestfs.pod". Adapt

[Libguestfs] [libguestfs PATCH 1/7] lib: fix NETWORK_ADDRESS: make it an actual IP address, not a subnet base

2023-07-13 Thread Laszlo Ersek
Currently we #define NETWORK_ADDRESS as "169.254.0.0". That's a bug in libguestfs, but it's been invisible -- thus far it's been canceled out by *independent* bugs in *both* QEMU and libvirt. (1) With the direct backend, the current definition of NETWORK_ADDRESS results in the following QEMU

[Libguestfs] [libguestfs PATCH 6/7] lib: introduce guestfs_int_make_pid_path()

2023-07-13 Thread Laszlo Ersek
Introduce a small function for creating pathnames for PID files. guestfs_int_make_pid_path() is something of an amalgamation of guestfs_int_make_temp_path() [1] and guestfs_int_create_socketname() [2]: - it creates a pathname under sockdir, like [2], - it uses the handle's unique counter, like

[Libguestfs] [libguestfs PATCH 4/7] docs: clarify sockdir's separation

2023-07-13 Thread Laszlo Ersek
There's another reason for separating sockdir from tmpdir, beyond "shorter pathnames needed": permissions. For example, passt drops privileges such that it cannot access "/tmp", and that restricts both the unix domain socket and the PID file of passt. Bugzilla:

[Libguestfs] [libguestfs PATCH 2/7] lib/launch-libvirt: support networking with passt

2023-07-13 Thread Laszlo Ersek
We generate the element on libvirt 3.8.0+ already. For selecting passt rather than SLIRP, we only need to insert the child element . Make that child element conditional on libvirt 9.0.0+, plus "passt --help" being executable. For the latter, place the new helper function

[Libguestfs] [libguestfs PATCH 5/7] lib: move guestfs_int_create_socketname() from "launch.c" to "tmpdirs.c"

2023-07-13 Thread Laszlo Ersek
Consider the following inverted call tree (effectively a dependency tree -- callees are at the top and near the left margin): lazy_make_tmpdir() [lib/tmpdirs.c] guestfs_int_lazy_make_tmpdir()[lib/tmpdirs.c] guestfs_int_make_temp_path()[lib/tmpdirs.c]

[Libguestfs] [libguestfs PATCH 0/7] lib: support networking with passt

2023-07-13 Thread Laszlo Ersek
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2184967 This series makes both backends prefer passt over slirp for appliance networking, if QEMU or libvirt (respectively) is recent enough, and passt is installed. My test setup is: $ virt-builder fedora-38 Then, each test run looks like

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Tage Johansson
On 7/13/2023 5:37 PM, Richard W.M. Jones wrote: On Thu, Jul 13, 2023 at 03:05:30PM +, Tage Johansson wrote: On 7/13/2023 4:36 PM, Eric Blake wrote: On Thu, Jul 13, 2023 at 01:37:49PM +, Tage Johansson wrote: On 7/13/2023 3:26 PM, Richard W.M. Jones wrote: On Thu, Jul 13, 2023 at

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Richard W.M. Jones
On Thu, Jul 13, 2023 at 04:18:03PM +, Tage Johansson wrote: > > On 7/13/2023 5:37 PM, Richard W.M. Jones wrote: > >On Thu, Jul 13, 2023 at 03:05:30PM +, Tage Johansson wrote: > >>On 7/13/2023 4:36 PM, Eric Blake wrote: > >>>On Thu, Jul 13, 2023 at 01:37:49PM +, Tage Johansson wrote: >

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Richard W.M. Jones
On Thu, Jul 13, 2023 at 05:46:56PM +0100, Richard W.M. Jones wrote: > On Thu, Jul 13, 2023 at 04:18:03PM +, Tage Johansson wrote: > > > > On 7/13/2023 5:37 PM, Richard W.M. Jones wrote: > > >On Thu, Jul 13, 2023 at 03:05:30PM +, Tage Johansson wrote: > > >>On 7/13/2023 4:36 PM, Eric Blake

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Richard W.M. Jones
On Thu, Jul 13, 2023 at 07:07:33PM +, Tage Johansson wrote: > Does this mean that the call to `aio_notify_read()` or > `aio_notify_write()` which discovered the error will also return the > error? Yes! > And that the completion callback was called by that call to > `aio_notify_*`, in the

[Libguestfs] [libnbd PATCH 2/2] tests: Add coverage of nbd_aio_opt_* in wrong state

2023-07-13 Thread Eric Blake
Enhance the regression tests to prove that the completion callback is not reached if the aio call itself reports an error; only the .free callback is guaranteed. Also add some asserts to the library code that may aid future readers in seeing how we track transfer semantics of a callback. Goes

[Libguestfs] [libnbd PATCH 0/2] Fix docs and testing of completion callback

2023-07-13 Thread Eric Blake
This is my proposal for fixing the documentation to match practice (namely, that completion.callback is not invoked in the cases where the aio call itself reports errors); we could instead try to go the other direction and tweak the generator to guarantee that both completion.callback and

[Libguestfs] [libnbd PATCH 1/2] api: Tighten rules on completion.callback

2023-07-13 Thread Eric Blake
The documentation has claimed since commit 6f4dcdab that any completion callback will be called exactly once; but this is not consistent with the code: if nbd_aio_* itself returns an error, then nothing is queued and the user does not need to wait for a completion callback to know how the command

Re: [Libguestfs] [libguestfs PATCH 7/7] lib/launch-direct: support networking with passt

2023-07-13 Thread Laszlo Ersek
On 7/13/23 19:10, Laszlo Ersek wrote: > On QEMU 7.2.0+, if "passt" is available, ask QEMU for passt ("stream") > rather than SLIRP ("user") networking. > > For this, we need to run passt ourselves. Given that passt daemonizes by > default, start it with our traditional function

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Eric Blake
On Thu, Jul 13, 2023 at 04:18:03PM +, Tage Johansson wrote: > > > > So is there any safe way to get some description of the error from a > > > completion callback apart from a non-zero number? It isn't too > > > helpful to report to the user that the read operation faild with -1. > > As I

Re: [Libguestfs] libnbd: When are callbacks freed

2023-07-13 Thread Tage Johansson
On 7/13/2023 6:50 PM, Richard W.M. Jones wrote: On Thu, Jul 13, 2023 at 05:46:56PM +0100, Richard W.M. Jones wrote: On Thu, Jul 13, 2023 at 04:18:03PM +, Tage Johansson wrote: On 7/13/2023 5:37 PM, Richard W.M. Jones wrote: On Thu, Jul 13, 2023 at 03:05:30PM +, Tage Johansson wrote: