Re: [Libguestfs] [PATCH] Skip vsock-related checks if AF_VSOCK is not supported.

2021-10-28 Thread Richard W.M. Jones
On Thu, Oct 28, 2021 at 10:20:56PM +0200, Hilko Bengen wrote: > * Richard W.M. Jones: > > > Thanks Hilko, I've pushed it. > > > > Out of interest, where are you running that doesn't have AF_VSOCK > > support? Does Debian not compile it into the kernel? > > It's there on my machines, but

Re: [Libguestfs] [PATCH] Skip vsock-related checks if AF_VSOCK is not supported.

2021-10-28 Thread Hilko Bengen
* Richard W.M. Jones: > Thanks Hilko, I've pushed it. > > Out of interest, where are you running that doesn't have AF_VSOCK > support? Does Debian not compile it into the kernel? It's there on my machines, but apparently Debian's buildd's don't have it enabled, see

Re: [Libguestfs] [PATCH nbdkit v2 5/5] vddk: Implement parallel thread model

2021-10-28 Thread Richard W.M. Jones
On Thu, Oct 28, 2021 at 07:55:39PM +0200, Laszlo Ersek wrote: > On 10/28/21 16:24, Richard W.M. Jones wrote: > > +#ifdef HAVE_STDATOMIC_H > > +#include > > +#else > > +/* Some old platforms lack atomic types, but this is only used for > > + * debug messages. > > + */ > > +#define _Atomic /**/ > >

Re: [Libguestfs] [PATCH nbdkit v2 5/5] vddk: Implement parallel thread model

2021-10-28 Thread Laszlo Ersek
On 10/28/21 16:24, Richard W.M. Jones wrote: > Since VDDK 6.0, asynchronous read and write operations are available. > This commit makes use of these, allowing us to use the parallel thread > model for increased performance. > > Note that at least VDDK 6.5 is required because VDDK 6.0 had a >

Re: [Libguestfs] [PATCH libnbd] lib/poll.c: Retry poll after EINTR

2021-10-28 Thread Nir Soffer
On Thu, Oct 28, 2021 at 3:59 PM Eric Blake wrote: > > Sorry for a late reply... > > On Sat, Oct 23, 2021 at 11:10:08AM +0100, Richard W.M. Jones wrote: > > > Fixing this in nbd_unlock_poll() avoids this issue in the entire > > > library, when we wait for command completion. This seems more useful

Re: [Libguestfs] [PATCH nbdkit 5/5] vddk: Implement parallel thread model

2021-10-28 Thread Richard W.M. Jones
On Thu, Oct 28, 2021 at 05:26:06PM +0200, Laszlo Ersek wrote: > On 10/28/21 13:57, Richard W.M. Jones wrote: > > On Thu, Oct 28, 2021 at 12:39:03PM +0100, Richard W.M. Jones wrote: > >> On Wed, Oct 27, 2021 at 07:42:31PM +0200, Laszlo Ersek wrote: > >>> (4) This could be slightly optimized by

Re: [Libguestfs] [PATCH nbdkit v2 4/5] vddk: Refactor -D vddk.stats=1 into a new file

2021-10-28 Thread Laszlo Ersek
On 10/28/21 16:24, Richard W.M. Jones wrote: > --- > plugins/vddk/Makefile.am | 1 + > plugins/vddk/vddk.h | 15 + > plugins/vddk/stats.c | 118 +++ > plugins/vddk/vddk.c | 78 +- > 4 files changed, 135

Re: [Libguestfs] [PATCH nbdkit v2 3/5] vddk: Start to split VDDK over several files

2021-10-28 Thread Laszlo Ersek
On 10/28/21 16:24, Richard W.M. Jones wrote: > This change doesn't do anything except move some definitions into the > header file vddk.h, but it allows future commits to split up the very > large vddk.c file. > --- > plugins/vddk/vddk.h | 89 +++- >

Re: [Libguestfs] [PATCH nbdkit v2 2/5] vddk: Add read, write and wait asynchronous functions

2021-10-28 Thread Laszlo Ersek
On 10/28/21 16:24, Richard W.M. Jones wrote: > These functions added in VDDK 6.0 - 6.5 implement asynchronous read > and write. > --- > plugins/vddk/vddk-structs.h | 3 +++ > plugins/vddk/vddk-stubs.h | 19 ++- > 2 files changed, 21 insertions(+), 1 deletion(-) This patch is

Re: [Libguestfs] [PATCH nbdkit v2 1/5] vddk: Move minimum version to VDDK 6.5

2021-10-28 Thread Laszlo Ersek
On 10/28/21 16:24, Richard W.M. Jones wrote: > Drop support for VDDK 5.5.5 (released in 2015) and 6.0 (released the > same year). Move minimum supported version to 6.5 (released Nov > 2016). This is so we can use asynchronous operations. > --- > plugins/vddk/nbdkit-vddk-plugin.pod | 10

Re: [Libguestfs] [PATCH nbdkit v2 0/5] vddk: Implement parallel thread model

2021-10-28 Thread Laszlo Ersek
On 10/28/21 16:24, Richard W.M. Jones wrote: > This version incorporates many of the changes suggested by Laszlo and > Nir in their reviews (thanks). > > We now require VDDK 6.5 because it turned out that 6.0 is broken. > There is a reliable way to detect VDDK 6.0 by the absence of >

Re: [Libguestfs] [PATCH nbdkit 5/5] vddk: Implement parallel thread model

2021-10-28 Thread Laszlo Ersek
On 10/28/21 13:57, Richard W.M. Jones wrote: > On Thu, Oct 28, 2021 at 12:39:03PM +0100, Richard W.M. Jones wrote: >> On Wed, Oct 27, 2021 at 07:42:31PM +0200, Laszlo Ersek wrote: >>> (4) This could be slightly optimized by restricting the condvar signal >>> to "queue size is now exactly 1 (i.e.,

[Libguestfs] [PATCH nbdkit v2 3/5] vddk: Start to split VDDK over several files

2021-10-28 Thread Richard W.M. Jones
This change doesn't do anything except move some definitions into the header file vddk.h, but it allows future commits to split up the very large vddk.c file. --- plugins/vddk/vddk.h | 89 +++- plugins/vddk/vddk.c | 91

[Libguestfs] [PATCH nbdkit v2 5/5] vddk: Implement parallel thread model

2021-10-28 Thread Richard W.M. Jones
Since VDDK 6.0, asynchronous read and write operations are available. This commit makes use of these, allowing us to use the parallel thread model for increased performance. Note that at least VDDK 6.5 is required because VDDK 6.0 had a different and incompatible signature for

[Libguestfs] [PATCH nbdkit v2 0/5] vddk: Implement parallel thread model

2021-10-28 Thread Richard W.M. Jones
This version incorporates many of the changes suggested by Laszlo and Nir in their reviews (thanks). We now require VDDK 6.5 because it turned out that 6.0 is broken. There is a reliable way to detect VDDK 6.0 by the absence of VixDiskLib_Wait, so we now give an accurate error message if someone

[Libguestfs] [PATCH nbdkit v2 1/5] vddk: Move minimum version to VDDK 6.5

2021-10-28 Thread Richard W.M. Jones
Drop support for VDDK 5.5.5 (released in 2015) and 6.0 (released the same year). Move minimum supported version to 6.5 (released Nov 2016). This is so we can use asynchronous operations. --- plugins/vddk/nbdkit-vddk-plugin.pod | 10 +++--- plugins/vddk/vddk-stubs.h | 3 +--

[Libguestfs] [PATCH nbdkit v2 4/5] vddk: Refactor -D vddk.stats=1 into a new file

2021-10-28 Thread Richard W.M. Jones
--- plugins/vddk/Makefile.am | 1 + plugins/vddk/vddk.h | 15 + plugins/vddk/stats.c | 118 +++ plugins/vddk/vddk.c | 78 +- 4 files changed, 135 insertions(+), 77 deletions(-) diff --git a/plugins/vddk/Makefile.am

Re: [Libguestfs] [PATCH libnbd] lib/poll.c: Retry poll after EINTR

2021-10-28 Thread Eric Blake
Sorry for a late reply... On Sat, Oct 23, 2021 at 11:10:08AM +0100, Richard W.M. Jones wrote: > > Fixing this in nbd_unlock_poll() avoids this issue in the entire > > library, when we wait for command completion. This seems more useful > > that fixing it in all libnbd clients. > > > > +++

Re: [Libguestfs] [PATCH libnbd] lib/poll.c: Retry poll after EINTR

2021-10-28 Thread Eric Blake
On Sat, Oct 23, 2021 at 11:10:08AM +0100, Richard W.M. Jones wrote: Sorry for a delayed reply, > > > > Fixing this in nbd_unlock_poll() avoids this issue in the entire > > library, when we wait for command completion. This seems more useful > > that fixing it in all libnbd clients. > > > > -

Re: [Libguestfs] [PATCH libnbd 1/2] golang/tests: Retry syscall.Select on EINTR

2021-10-28 Thread Eric Blake
On Sat, Oct 23, 2021 at 11:47:41PM +0300, Nir Soffer wrote: > Go syscall package does not handle EINTR like higher level packages, so > we need to handle it when using syscall.Select(). > > The test always fails when running: > > $ GOLANG=go pkg=libguestfs.org/libnbd ./run-tests.sh > ...

Re: [Libguestfs] [PATCH nbdkit 5/5] vddk: Implement parallel thread model

2021-10-28 Thread Richard W.M. Jones
On Thu, Oct 28, 2021 at 12:39:03PM +0100, Richard W.M. Jones wrote: > On Wed, Oct 27, 2021 at 07:42:31PM +0200, Laszlo Ersek wrote: > > (4) This could be slightly optimized by restricting the condvar signal > > to "queue size is now exactly 1 (i.e., it was empty)". > > I think there's a possible

Re: [Libguestfs] [PATCH nbdkit 5/5] vddk: Implement parallel thread model

2021-10-28 Thread Richard W.M. Jones
On Wed, Oct 27, 2021 at 07:42:31PM +0200, Laszlo Ersek wrote: > On 10/27/21 14:21, Richard W.M. Jones wrote: > > +/* Queue of asynchronous commands sent to the background thread. */ > > +enum command_type { GET_SIZE, READ, WRITE, FLUSH, CAN_EXTENTS, EXTENTS, > > STOP }; > > (1) Shouldn't we use

[Libguestfs] [PATCH v2 9/9] qapi: Extend -compat to set policy for unstable interfaces

2021-10-28 Thread Markus Armbruster
New option parameters unstable-input and unstable-output set policy for unstable interfaces just like deprecated-input and deprecated-output set policy for deprecated interfaces (see commit 6dd75472d5 "qemu-options: New -compat to set policy for deprecated interfaces"). This is intended for

[Libguestfs] [PATCH v2 5/9] qapi: Generalize struct member policy checking

2021-10-28 Thread Markus Armbruster
The generated visitor functions call visit_deprecated_accept() and visit_deprecated() when visiting a struct member with special feature flag 'deprecated'. This makes the feature flag visible to the actual visitors. I want to make feature flag 'unstable' visible there as well, so I can add

[Libguestfs] [PATCH v2 8/9] qapi: Factor out compat_policy_input_ok()

2021-10-28 Thread Markus Armbruster
The code to check policy for handling deprecated input is triplicated. Factor it out into compat_policy_input_ok() before I mess with it in the next commit. Signed-off-by: Markus Armbruster Reviewed-by: Philippe Mathieu-Daudé --- include/qapi/compat-policy.h | 7 + qapi/qapi-visit-core.c

[Libguestfs] [PATCH v2 6/9] qapi: Generalize command policy checking

2021-10-28 Thread Markus Armbruster
The code to check command policy can see special feature flag 'deprecated' as command flag QCO_DEPRECATED. I want to make feature flag 'unstable' visible there as well, so I can add policy for it. To let me make it visible, add member @special_features (a bitset of QapiSpecialFeature) to

[Libguestfs] [PATCH v2 0/9] Configurable policy for handling unstable interfaces

2021-10-28 Thread Markus Armbruster
Option -compat lets you configure what to do when deprecated interfaces get used. This series extends this to unstable interfaces. Works the same way. Intended for testing users of the management interfaces. It is experimental. To make it possible, I replace the "x-" naming convention by

[Libguestfs] [PATCH v2 1/9] qapi: New special feature flag "unstable"

2021-10-28 Thread Markus Armbruster
By convention, names starting with "x-" are experimental. The parts of external interfaces so named may be withdrawn or changed incompatibly in future releases. The naming convention makes unstable interfaces easy to recognize. Promoting something from experimental to stable involves a name

[Libguestfs] [PATCH v2 4/9] qapi: Tools for sets of special feature flags in generated code

2021-10-28 Thread Markus Armbruster
New enum QapiSpecialFeature enumerates the special feature flags. New helper gen_special_features() returns code to represent a collection of special feature flags as a bitset. The next few commits will put them to use. Signed-off-by: Markus Armbruster Reviewed-by: John Snow ---

[Libguestfs] [PATCH v2 2/9] qapi: Mark unstable QMP parts with feature 'unstable'

2021-10-28 Thread Markus Armbruster
Add special feature 'unstable' everywhere the name starts with 'x-', except for InputBarrierProperties member x-origin and MemoryBackendProperties member x-use-canonical-path-for-ramblock-id, because these two are actually stable. Signed-off-by: Markus Armbruster Reviewed-by: Juan Quintela

[Libguestfs] [PATCH v2 3/9] qapi: Eliminate QCO_NO_OPTIONS for a slight simplification

2021-10-28 Thread Markus Armbruster
Signed-off-by: Markus Armbruster Reviewed-by: Juan Quintela Reviewed-by: John Snow --- include/qapi/qmp/dispatch.h | 1 - monitor/misc.c | 3 +-- scripts/qapi/commands.py| 5 + 3 files changed, 2 insertions(+), 7 deletions(-) diff --git a/include/qapi/qmp/dispatch.h

Re: [Libguestfs] [PATCH nbdkit 1/5] vddk: Move minimum version to VDDK 6.0

2021-10-28 Thread Laszlo Ersek
On 10/27/21 20:52, Richard W.M. Jones wrote: > It turns out that VDDK 6.0 async support is present but broken, which > makes it hard to support this version. If it was missing we could use > an alternative codepath, but present-and-broken is difficult to deal > with. (There is no way to detect

Re: [Libguestfs] [PATCH nbdkit 5/5] vddk: Implement parallel thread model

2021-10-28 Thread Laszlo Ersek
On 10/28/21 02:34, Nir Soffer wrote: > On Wed, Oct 27, 2021 at 8:45 PM Laszlo Ersek wrote: > ... >> (hopefully that means nbdkit_error is thread-safe) >> >>> + return ! cmd->error ? 0 : -1; >> >> (6) Using the ! operator with the ternary operator ?: is awkward IMO; >> how about just >> >>

Re: [Libguestfs] [PATCH 1/9] qapi: New special feature flag "unstable"

2021-10-28 Thread Markus Armbruster
Kevin Wolf writes: > Am 26.10.2021 um 11:37 hat Markus Armbruster geschrieben: >> Kevin Wolf writes: >> >> > Am 25.10.2021 um 07:25 hat Markus Armbruster geschrieben: >> >> By convention, names starting with "x-" are experimental. The parts >> >> of external interfaces so named may be

Re: [Libguestfs] [PATCH nbdkit 1/5] vddk: Move minimum version to VDDK 6.0

2021-10-28 Thread Richard W.M. Jones
On Thu, Oct 28, 2021 at 08:38:46AM +0300, Nikolay Ivanets wrote: > Rich, > Just FYI: > » The VMware policy concerning backward and forward compatibility is for VDDK > to support N-2 and N+1 releases. In other words, VDDK 6.7 and all its update > releases support vSphere 6.0, 6.5 (except for new