Re: [Libguestfs] [libnbd PATCH v4 08/11] generator: Add `modifies_fd` flag to the [call] structure

[Richard W.M. Jones]

On Thu, Aug 03, 2023 at 03:30:36PM +, Tage Johansson wrote:
> 
> On 8/2/2023 6:39 PM, Richard W.M. Jones wrote:
> >On Wed, Aug 02, 2023 at 12:40:53PM +, Tage Johansson wrote:
> >>Add a flag (`modifies_fd`) to the [call] structure in generator/API.ml
> >>which is set to [true] if the handle call may do something with the
> >>file descriptor. That is, it is [true] for all calls which are or may
> >>call [aio_notify_*], including all synchronous commands like
> >>[nbd_connect] or [nbd_opt_go].
> >>
> >>The motivation for this is that the asynchronous handle in the Rust
> >>bindings uses its own loop for polling, modifying the file descriptor
> >>outside of this loop may cause unexpected behaviour. Including that the
> >>handle may hang. All commands which set this flag to [true] will be
> >>excluded from that handle. The asynchronous (`aio_*`) functions will be
> >>used instead.
> >Isn't this just saying that the call is synchronous?
> >
> >Rich.
> 
> 
> Kind of, it is true for all synchronous commands and the functions
> aio_notify_read() and aio_notify_write(). I could mention those
> explicitly though and rename the flag to something like
> "is_synchronous", if you want?

No it'll be fine, thanks.

Rich.

> 
> Best regards,
> 
> Tage
> 
> 
> >>---
> >>  generator/API.ml  | 32 
> >>  generator/API.mli |  7 +++
> >>  2 files changed, 39 insertions(+)
> >>
> >>diff --git a/generator/API.ml b/generator/API.ml
> >>index 42b9eec..6e1670d 100644
> >>--- a/generator/API.ml
> >>+++ b/generator/API.ml
> >>@@ -33,6 +33,7 @@ type call = {
> >>is_locked : bool;
> >>may_set_error : bool;
> >>async_kind : async_kind option;
> >>+  modifies_fd: bool;
> >>mutable first_version : int * int;
> >>  }
> >>  and arg =
> >>@@ -274,6 +275,7 @@ let default_call = { args = []; optargs = []; ret = 
> >>RErr;
> >>   permitted_states = [];
> >>   is_locked = true; may_set_error = true;
> >>   async_kind = None;
> >>+ modifies_fd = false;
> >>   first_version = (0, 0) }
> >>  (* Calls.
> >>@@ -1181,6 +1183,7 @@ Return true if option negotiation mode was enabled on 
> >>this handle.";
> >>  default_call with
> >>  args = []; ret = RErr;
> >>  permitted_states = [ Negotiating ];
> >>+modifies_fd = true;
> >>  shortdesc = "end negotiation and move on to using an export";
> >>  longdesc = "\
> >>  Request that the server finish negotiation and move on to serving the
> >>@@ -1208,6 +1211,7 @@ although older servers will instead have killed the 
> >>connection.";
> >>  default_call with
> >>  args = []; ret = RErr;
> >>  permitted_states = [ Negotiating ];
> >>+modifies_fd = true;
> >>  shortdesc = "end negotiation and close the connection";
> >>  longdesc = "\
> >>  Request that the server finish negotiation, gracefully if possible, then
> >>@@ -1221,6 +1225,7 @@ enabled option mode.";
> >>  default_call with
> >>  args = []; ret = RBool;
> >>  permitted_states = [ Negotiating ];
> >>+modifies_fd = true;
> >>  shortdesc = "request the server to initiate TLS";
> >>  longdesc = "\
> >>  Request that the server initiate a secure TLS connection, by
> >>@@ -1259,6 +1264,7 @@ established, as reported by 
> >>L.";
> >>  default_call with
> >>  args = []; ret = RBool;
> >>  permitted_states = [ Negotiating ];
> >>+modifies_fd = true;
> >>  shortdesc = "request the server to enable structured replies";
> >>  longdesc = "\
> >>  Request that the server use structured replies, by sending
> >>@@ -1285,6 +1291,7 @@ later calls to this function return false.";
> >>  default_call with
> >>  args = [ Closure list_closure ]; ret = RInt;
> >>  permitted_states = [ Negotiating ];
> >>+modifies_fd = true;
> >>  shortdesc = "request the server to list all exports during 
> >> negotiation";
> >>  longdesc = "\
> >>  Request that the server list all exports that it supports.  This can
> >>@@ -1326,6 +1333,7 @@ description is set with I<-D>.";
> >>  default_call with
> >>  args = []; ret = RErr;
> >>  permitted_states = [ Negotiating ];
> >>+modifies_fd = true;
> >>  shortdesc = "request the server for information about an export";
> >>  longdesc = "\
> >>  Request that the server supply information about the export name
> >>@@ -1357,6 +1365,7 @@ corresponding L would succeed.";
> >>  default_call with
> >>  args = [ Closure context_closure ]; ret = RInt;
> >>  permitted_states = [ Negotiating ];
> >>+modifies_fd = true;
> >>  shortdesc = "list available meta contexts, using implicit query list";
> >>  longdesc = "\
> >>  Request that the server list available meta contexts associated with
> >>@@ -1412,6 +1421,7 @@ a server may send a lengthy list.";
> >>  default_call with
> >>  args = [ StringList "queries"; Closure context_closure 

Re: [Libguestfs] [libnbd PATCH v4 08/11] generator: Add `modifies_fd` flag to the [call] structure

[Tage Johansson]

On 8/2/2023 6:39 PM, Richard W.M. Jones wrote:

On Wed, Aug 02, 2023 at 12:40:53PM +, Tage Johansson wrote:

Add a flag (`modifies_fd`) to the [call] structure in generator/API.ml
which is set to [true] if the handle call may do something with the
file descriptor. That is, it is [true] for all calls which are or may
call [aio_notify_*], including all synchronous commands like
[nbd_connect] or [nbd_opt_go].

The motivation for this is that the asynchronous handle in the Rust
bindings uses its own loop for polling, modifying the file descriptor
outside of this loop may cause unexpected behaviour. Including that the
handle may hang. All commands which set this flag to [true] will be
excluded from that handle. The asynchronous (`aio_*`) functions will be
used instead.

Isn't this just saying that the call is synchronous?

Rich.



Kind of, it is true for all synchronous commands and the functions 
aio_notify_read() and aio_notify_write(). I could mention those 
explicitly though and rename the flag to something like 
"is_synchronous", if you want?



Best regards,

Tage



---
  generator/API.ml  | 32 
  generator/API.mli |  7 +++
  2 files changed, 39 insertions(+)

diff --git a/generator/API.ml b/generator/API.ml
index 42b9eec..6e1670d 100644
--- a/generator/API.ml
+++ b/generator/API.ml
@@ -33,6 +33,7 @@ type call = {
is_locked : bool;
may_set_error : bool;
async_kind : async_kind option;
+  modifies_fd: bool;
mutable first_version : int * int;
  }
  and arg =
@@ -274,6 +275,7 @@ let default_call = { args = []; optargs = []; ret = RErr;
   permitted_states = [];
   is_locked = true; may_set_error = true;
   async_kind = None;
+ modifies_fd = false;
   first_version = (0, 0) }
  
  (* Calls.

@@ -1181,6 +1183,7 @@ Return true if option negotiation mode was enabled on this 
handle.";
  default_call with
  args = []; ret = RErr;
  permitted_states = [ Negotiating ];
+modifies_fd = true;
  shortdesc = "end negotiation and move on to using an export";
  longdesc = "\
  Request that the server finish negotiation and move on to serving the
@@ -1208,6 +1211,7 @@ although older servers will instead have killed the 
connection.";
  default_call with
  args = []; ret = RErr;
  permitted_states = [ Negotiating ];
+modifies_fd = true;
  shortdesc = "end negotiation and close the connection";
  longdesc = "\
  Request that the server finish negotiation, gracefully if possible, then
@@ -1221,6 +1225,7 @@ enabled option mode.";
  default_call with
  args = []; ret = RBool;
  permitted_states = [ Negotiating ];
+modifies_fd = true;
  shortdesc = "request the server to initiate TLS";
  longdesc = "\
  Request that the server initiate a secure TLS connection, by
@@ -1259,6 +1264,7 @@ established, as reported by 
L.";
  default_call with
  args = []; ret = RBool;
  permitted_states = [ Negotiating ];
+modifies_fd = true;
  shortdesc = "request the server to enable structured replies";
  longdesc = "\
  Request that the server use structured replies, by sending
@@ -1285,6 +1291,7 @@ later calls to this function return false.";
  default_call with
  args = [ Closure list_closure ]; ret = RInt;
  permitted_states = [ Negotiating ];
+modifies_fd = true;
  shortdesc = "request the server to list all exports during negotiation";
  longdesc = "\
  Request that the server list all exports that it supports.  This can
@@ -1326,6 +1333,7 @@ description is set with I<-D>.";
  default_call with
  args = []; ret = RErr;
  permitted_states = [ Negotiating ];
+modifies_fd = true;
  shortdesc = "request the server for information about an export";
  longdesc = "\
  Request that the server supply information about the export name
@@ -1357,6 +1365,7 @@ corresponding L would succeed.";
  default_call with
  args = [ Closure context_closure ]; ret = RInt;
  permitted_states = [ Negotiating ];
+modifies_fd = true;
  shortdesc = "list available meta contexts, using implicit query list";
  longdesc = "\
  Request that the server list available meta contexts associated with
@@ -1412,6 +1421,7 @@ a server may send a lengthy list.";
  default_call with
  args = [ StringList "queries"; Closure context_closure ]; ret = RInt;
  permitted_states = [ Negotiating ];
+modifies_fd = true;
  shortdesc = "list available meta contexts, using explicit query list";
  longdesc = "\
  Request that the server list available meta contexts associated with
@@ -1462,6 +1472,7 @@ a server may send a lengthy list.";
  default_call with
  args = [ Closure context_closure ]; ret = RInt;
  permitted_states = [ Negotiating ];
+modifies_fd = true;
  shortdesc = "select specific meta contexts, using implicit query 

[Libguestfs] [libnbd PATCH v5 12/12] rust: async: Add an example

[Tage Johansson]

This patch adds an example using the asynchronous Rust bindings.
---
 rust/Cargo.toml|   1 +
 rust/examples/concurrent-read-write.rs | 135 +
 rust/run-tests.sh.in   |   2 +
 3 files changed, 138 insertions(+)
 create mode 100644 rust/examples/concurrent-read-write.rs

diff --git a/rust/Cargo.toml b/rust/Cargo.toml
index d001248..4332783 100644
--- a/rust/Cargo.toml
+++ b/rust/Cargo.toml
@@ -54,5 +54,6 @@ default = ["log", "tokio"]
 anyhow = "1.0.72"
 once_cell = "1.18.0"
 pretty-hex = "0.3.0"
+rand = { version = "0.8.5", default-features = false, features = ["small_rng", 
"min_const_gen"] }
 tempfile = "3.6.0"
 tokio = { version = "1.29.1", default-features = false, features = 
["rt-multi-thread", "macros"] }
diff --git a/rust/examples/concurrent-read-write.rs 
b/rust/examples/concurrent-read-write.rs
new file mode 100644
index 000..a1c3e8a
--- /dev/null
+++ b/rust/examples/concurrent-read-write.rs
@@ -0,0 +1,135 @@
+//! Example usage with nbdkit:
+//!
+//! nbdkit -U - memory 100M \
+//!   --run 'cargo run --example concurrent-read-write -- $unixsocket'
+//!
+//! This will read and write randomly over the first megabyte of the
+//! plugin using multi-conn, multiple threads and multiple requests in
+//! flight on each thread.
+
+#![deny(warnings)]
+use rand::prelude::*;
+use std::env;
+use std::path::PathBuf;
+use std::sync::Arc;
+use tokio::task::JoinSet;
+
+/// Number of simultaneous connections to the NBD server.
+///
+/// Note that some servers only support a limited number of
+/// simultaneous connections, and/or have a configurable thread pool
+/// internally, and if you exceed those limits then something will break.
+const NR_MULTI_CONN: usize = 8;
+
+/// Number of commands that can be "in flight" at the same time on each
+/// connection.  (Therefore the total number of requests in flight may
+/// be up to NR_MULTI_CONN * MAX_IN_FLIGHT).
+const MAX_IN_FLIGHT: usize = 16;
+
+/// The size of large reads and writes, must be > 512.
+const BUFFER_SIZE: usize = 1024;
+
+/// Number of commands we issue (per [task][tokio::task]).
+const NR_CYCLES: usize = 32;
+
+/// Statistics gathered during the run.
+#[derive(Debug, Default)]
+struct Stats {
+/// The total number of requests made.
+requests: usize,
+}
+
+#[tokio::main]
+async fn main() -> anyhow::Result<()> {
+let args = env::args_os().collect::>();
+if args.len() != 2 {
+anyhow::bail!("Usage: {:?} socket", args[0]);
+}
+let socket = [1];
+
+// We begin by making a connection to the server to get the export size
+// and ensure that it supports multiple connections and is writable.
+let nbd = libnbd::Handle::new()?;
+nbd.connect_unix()?;
+let export_size = nbd.get_size()?;
+anyhow::ensure!(
+(BUFFER_SIZE as u64) < export_size,
+"export is {export_size}B, must be larger than {BUFFER_SIZE}B"
+);
+anyhow::ensure!(
+!nbd.is_read_only()?,
+"error: this NBD export is read-only"
+);
+anyhow::ensure!(
+nbd.can_multi_conn()?,
+"error: this NBD export does not support multi-conn"
+);
+drop(nbd); // Close the connection.
+
+// Start the worker tasks, one per connection.
+let mut tasks = JoinSet::new();
+for i in 0..NR_MULTI_CONN {
+tasks.spawn(run_thread(i, socket.clone().into(), export_size));
+}
+
+// Wait for the tasks to complete.
+let mut stats = Stats::default();
+while !tasks.is_empty() {
+let this_stats = tasks.join_next().await.unwrap().unwrap()?;
+stats.requests += this_stats.requests;
+}
+
+// Make sure the number of requests that were required matches what
+// we expect.
+assert_eq!(stats.requests, NR_MULTI_CONN * NR_CYCLES);
+
+Ok(())
+}
+
+async fn run_thread(
+task_idx: usize,
+socket: PathBuf,
+export_size: u64,
+) -> anyhow::Result {
+// Start a new connection to the server.
+// We shall spawn many commands concurrently on different tasks and those
+// futures must be `'static`, hence we wrap the handle in an [Arc].
+let nbd = Arc::new(libnbd::AsyncHandle::new()?);
+nbd.connect_unix(socket).await?;
+
+let mut rng = SmallRng::seed_from_u64(44 as u64);
+
+// Issue commands.
+let mut stats = Stats::default();
+let mut join_set = JoinSet::new();
+//tokio::time::sleep(std::time::Duration::from_secs(1)).await;
+while stats.requests < NR_CYCLES || !join_set.is_empty() {
+while stats.requests < NR_CYCLES && join_set.len() < MAX_IN_FLIGHT {
+// If we want to issue another request, do so.  Note that we reuse
+// the same buffer for multiple in-flight requests.  It doesn't
+// matter here because we're just trying to write random stuff,
+// but that would be Very Bad in a real application.
+// Simulate a mix of large and small requests.
+let size = if 

Re: [Libguestfs] [libnbd PATCH v4 04/11] rust: Use more specific closure traits

[Tage Johansson]

On 8/2/2023 6:32 PM, Richard W.M. Jones wrote:

On Wed, Aug 02, 2023 at 12:40:49PM +, Tage Johansson wrote:

For closures with `cb,count = CBOnce`, `FnOnce` will be used instead of

I think there's an extra ',' here.

Previous comments about use of markdown apply too.


`FnMut`. Moreover, closures in synchronous commands with
`cblifetime = CBCommand` will not need to live for the static lifetime.
See [here](https://doc.rust-lang.org/std/ops/trait.FnOnce.html) for more
information about the advantages of using `FnOnce` when possible.
---
  generator/Rust.ml  | 44 
  rust/src/handle.rs |  2 ++
  rust/src/types.rs  |  2 ++
  3 files changed, 32 insertions(+), 16 deletions(-)

diff --git a/generator/Rust.ml b/generator/Rust.ml
index 3117980..d3225eb 100644
--- a/generator/Rust.ml
+++ b/generator/Rust.ml
@@ -111,7 +111,7 @@ let rust_cbarg_name : cbarg -> string = function
| CBArrayAndLen (arg, _) | CBMutable arg -> rust_arg_name arg
  
  (* Get the Rust type for an argument. *)

-let rec rust_arg_type : arg -> string = function
+let rec rust_arg_type ?(async_kind = None) : arg -> string = function
| Bool _ -> "bool"
| Int _ -> "c_int"
| UInt _ -> "c_uint"
@@ -131,15 +131,18 @@ let rec rust_arg_type : arg -> string = function
| BytesOut _ -> " [u8]"
| BytesPersistIn _ -> "&'static [u8]"
| BytesPersistOut _ -> "&'static mut [u8]"
-  | Closure { cbargs } -> "impl " ^ rust_closure_trait cbargs
+  | Closure { cbargs; cbcount } -> (
+  match async_kind with
+  | Some _ -> "impl " ^ rust_closure_trait cbargs cbcount
+  | None -> "impl " ^ rust_closure_trait cbargs cbcount ~lifetime:None)
  
  (* Get the Rust closure trait for a callback, That is `Fn*(...) -> ...)`. *)

-and rust_closure_trait ?(lifetime = Some "'static") cbargs : string =
+and rust_closure_trait ?(lifetime = Some "'static") cbargs cbcount : string =
let rust_cbargs = String.concat ", " (List.map rust_cbarg_type cbargs)
-  and lifetime_constraint =
-match lifetime with None -> "" | Some x -> " + " ^ x
-  in
-  "FnMut(" ^ rust_cbargs ^ ") -> c_int + Send + Sync" ^ lifetime_constraint
+  and closure_type =
+match cbcount with CBOnce -> "FnOnce" | CBMany -> "FnMut"
+  and lifetime = match lifetime with None -> "" | Some x -> " + " ^ x in
+  sprintf "%s(%s) -> c_int + Send + Sync%s" closure_type rust_cbargs lifetime
  
  (* Get the Rust type for a callback argument. *)

  and rust_cbarg_type : cbarg -> string = function
@@ -153,8 +156,8 @@ and rust_cbarg_type : cbarg -> string = function
| CBMutable arg -> " " ^ rust_arg_type arg
  
  (* Get the type of a rust optional argument. *)

-let rust_optarg_type : optarg -> string = function
-  | OClosure x -> sprintf "Option<%s>" (rust_arg_type (Closure x))
+let rust_optarg_type ?(async_kind = None) : optarg -> string = function
+  | OClosure x -> sprintf "Option<%s>" (rust_arg_type (Closure x) ~async_kind)
| OFlags (name, flags, _) ->
sprintf "Option<%s>" (rust_arg_type (Flags (name, flags)))
  
@@ -419,8 +422,8 @@ let ffi_ret_to_rust (call : call) =

 closure data, and a free function for the closure data. This struct is what
 will be sent to a C function taking the closure as an argument. In fact,
 the struct itself is generated by rust-bindgen. *)
-let print_rust_closure_to_raw_fn ({ cbname; cbargs } : closure) =
-  let closure_trait = rust_closure_trait cbargs ~lifetime:None in
+let print_rust_closure_to_raw_fn ({ cbname; cbargs; cbcount } : closure) =
+  let closure_trait = rust_closure_trait cbargs cbcount ~lifetime:None in
let ffi_cbargs_names = List.flatten (List.map ffi_cbarg_names cbargs) in
let ffi_cbargs_types = List.flatten (List.map ffi_cbarg_types cbargs) in
let rust_cbargs_names = List.map rust_cbarg_name cbargs in
@@ -435,16 +438,24 @@ let print_rust_closure_to_raw_fn ({ cbname; cbargs } : 
closure) =
 (List.map2 (sprintf "%s: %s") ffi_cbargs_names ffi_cbargs_types));
pr "  where F: %s\n" closure_trait;
pr "{\n";
-  pr "let callback_ptr = data as *mut F;\n";
-  pr "let callback =  *callback_ptr;\n";
+  (match cbcount with
+  | CBMany ->
+  pr "let callback_ptr = data as *mut F;\n";
+  pr "let callback =  *callback_ptr;\n"
+  | CBOnce ->
+  pr "let callback_ptr = data as *mut Option;\n";
+  pr "let callback_option:  Option =  *callback_ptr;\n";
+  pr "let callback: F = callback_option.take().unwrap();\n");
List.iter ffi_cbargs_to_rust cbargs;
pr "callback(%s)\n" (String.concat ", " rust_cbargs_names);
pr "}\n";
-  pr "let callback_data = Box::into_raw(Box::new(f));\n";
+  pr "let callback_data = Box::into_raw(Box::new(%s));\n"
+(match cbcount with CBMany -> "f" | CBOnce -> "Some(f)");
pr "sys::nbd_%s_callback {\n" cbname;
pr "callback: Some(call_closure::),\n";
pr "user_data: callback_data 

Re: [Libguestfs] [libnbd PATCH v4 01/11] rust: create basic Rust bindings

[Richard W.M. Jones]

On Thu, Aug 03, 2023 at 03:20:04PM +, Tage Johansson wrote:
> On 8/2/2023 5:42 PM, Richard W.M. Jones wrote:
> >Here we're using markdown, I guess, but ocamldoc comments prefer to
> >use [ ... ] instead of ` ... ` (although we don't use ocamldoc really).
> 
> 
> But should [ ... ] (brackets) be used even for pseudo code? When I
> write `impl Handle {` I am just writing some pseudo Rust code which
> would make no sense for ocamldoc to interpret. So my strategy has
> been to use `...` (backticks) for pseudo code and [...] (brackets)
> for actual OCaml items.

With the proviso that we don't in fact use ocamldoc, there is an
ocamldoc markup for verbatim blocks:

  {v

  v}

but it's probably a bit too heavyweight to use in general comments.

I would just stick to whatever other parts of the code use and be
consistent with them.

> >>+version = "0.1.0"
> >If you wanted to (and it may or may not be a good idea) you could
> >include the actual version of libnbd here.  You'd need to move
> >rust/Cargo.toml to rust/Cargo.toml.in and add an autoconf
> >AC_CONFIG_FILES directive to near the end of configure.in.
> 
> I am not sure about this. I think most "wrapper crates" don't follow
> the exact same versioning as the library they are binding to. It
> would make it less flexible to make breaking changes to the Rust
> bindings alone without bumping Libnbd's version.

Sure, nbdkit rust bindings also use their own versioning scheme.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs

Re: [Libguestfs] [libnbd PATCH v4 11/11] rust: Add some examples

[Tage Johansson]

On 8/2/2023 6:53 PM, Richard W.M. Jones wrote:

On Wed, Aug 02, 2023 at 12:40:56PM +, Tage Johansson wrote:

This patch adds a few examples in rust/examples/. The examples are
compiled and run as part of the test suite.
---
  rust/Makefile.am   |   3 +
  rust/examples/concurrent-read-write.rs | 135 +
  rust/examples/connect-command.rs   |  39 +++
  rust/examples/fetch-first-sector.rs|  38 +++
  rust/examples/get-size.rs  |  29 ++
  rust/run-tests.sh  |   7 ++
  6 files changed, 251 insertions(+)
  create mode 100644 rust/examples/concurrent-read-write.rs
  create mode 100644 rust/examples/connect-command.rs
  create mode 100644 rust/examples/fetch-first-sector.rs
  create mode 100644 rust/examples/get-size.rs

diff --git a/rust/Makefile.am b/rust/Makefile.am
index b954b22..d75163d 100644
--- a/rust/Makefile.am
+++ b/rust/Makefile.am
@@ -32,6 +32,9 @@ source_files = \
src/types.rs \
src/utils.rs \
src/async_handle.rs \
+   examples/connect-command.rs \
+   examples/get-size.rs \
+   examples/fetch-first-sector.rs \

This doesn't list all the source files, it is missing
examples/concurrent-read-write.rs.

If you split out examples/connect-command.rs, examples/get-size.rs and
examples/fetch-first-sector.rs into a separate patch (since those
don't depend on asynch), and moved that patch earlier in the sequence,
then it could go upstream earlier.



Yes, I have done that.


Best regards,

Tage



libnbd-sys/Cargo.toml \
libnbd-sys/build.rs \
$(NULL)
diff --git a/rust/examples/concurrent-read-write.rs 
b/rust/examples/concurrent-read-write.rs
new file mode 100644
index 000..a1c3e8a
--- /dev/null
+++ b/rust/examples/concurrent-read-write.rs
@@ -0,0 +1,135 @@
+//! Example usage with nbdkit:
+//!
+//! nbdkit -U - memory 100M \
+//!   --run 'cargo run --example concurrent-read-write -- $unixsocket'
+//!
+//! This will read and write randomly over the first megabyte of the
+//! plugin using multi-conn, multiple threads and multiple requests in
+//! flight on each thread.
+
+#![deny(warnings)]
+use rand::prelude::*;
+use std::env;
+use std::path::PathBuf;
+use std::sync::Arc;
+use tokio::task::JoinSet;
+
+/// Number of simultaneous connections to the NBD server.
+///
+/// Note that some servers only support a limited number of
+/// simultaneous connections, and/or have a configurable thread pool
+/// internally, and if you exceed those limits then something will break.
+const NR_MULTI_CONN: usize = 8;
+
+/// Number of commands that can be "in flight" at the same time on each
+/// connection.  (Therefore the total number of requests in flight may
+/// be up to NR_MULTI_CONN * MAX_IN_FLIGHT).
+const MAX_IN_FLIGHT: usize = 16;
+
+/// The size of large reads and writes, must be > 512.
+const BUFFER_SIZE: usize = 1024;
+
+/// Number of commands we issue (per [task][tokio::task]).
+const NR_CYCLES: usize = 32;
+
+/// Statistics gathered during the run.
+#[derive(Debug, Default)]
+struct Stats {
+/// The total number of requests made.
+requests: usize,
+}
+
+#[tokio::main]
+async fn main() -> anyhow::Result<()> {
+let args = env::args_os().collect::>();
+if args.len() != 2 {
+anyhow::bail!("Usage: {:?} socket", args[0]);
+}
+let socket = [1];
+
+// We begin by making a connection to the server to get the export size
+// and ensure that it supports multiple connections and is writable.
+let nbd = libnbd::Handle::new()?;
+nbd.connect_unix()?;
+let export_size = nbd.get_size()?;
+anyhow::ensure!(
+(BUFFER_SIZE as u64) < export_size,
+"export is {export_size}B, must be larger than {BUFFER_SIZE}B"
+);
+anyhow::ensure!(
+!nbd.is_read_only()?,
+"error: this NBD export is read-only"
+);
+anyhow::ensure!(
+nbd.can_multi_conn()?,
+"error: this NBD export does not support multi-conn"
+);
+drop(nbd); // Close the connection.
+
+// Start the worker tasks, one per connection.
+let mut tasks = JoinSet::new();
+for i in 0..NR_MULTI_CONN {
+tasks.spawn(run_thread(i, socket.clone().into(), export_size));
+}
+
+// Wait for the tasks to complete.
+let mut stats = Stats::default();
+while !tasks.is_empty() {
+let this_stats = tasks.join_next().await.unwrap().unwrap()?;
+stats.requests += this_stats.requests;
+}
+
+// Make sure the number of requests that were required matches what
+// we expect.
+assert_eq!(stats.requests, NR_MULTI_CONN * NR_CYCLES);
+
+Ok(())
+}
+
+async fn run_thread(
+task_idx: usize,
+socket: PathBuf,
+export_size: u64,
+) -> anyhow::Result {
+// Start a new connection to the server.
+// We shall spawn many commands concurrently on different tasks and those
+// futures must be `'static`, hence we wrap the handle in an [Arc].
+let 

[Libguestfs] [libnbd PATCH v5 00/12] Rust Bindings for Libnbd

[Tage Johansson]

A new version of the Rust bindings for Libnbd. Most changes have been
mentioned as replies to the comments I got on v4. But the most significant
change is probably that the patches have been reordered.

Best regards,
Tage


Tage Johansson (12):
  rust: create basic Rust bindings
  rust: Add a couple of integration tests
  rust: Make it possible to run tests with Valgrind
  rust: Add some examples
  generator: Add information about asynchronous handle calls
  generator: Add information about the lifetime of closures
  rust: Use more specific closure traits
  rust: async: Create an async friendly handle type
  generator: Add `modifies_fd` flag to the [call] structure
  rust: async: Use the modifies_fd flag to exclude calls
  rust: async: Add a couple of integration tests
  rust: async: Add an example

 .gitignore|  10 +
 .ocamlformat  |   4 +
 Makefile.am   |   2 +
 configure.ac  |  30 +
 generator/API.ml  |  84 ++
 generator/API.mli |  35 +
 generator/Makefile.am |   4 +
 generator/Rust.ml | 794 ++
 generator/Rust.mli|  22 +
 generator/RustSys.ml  | 167 
 generator/RustSys.mli |  19 +
 generator/generator.ml|   4 +
 rust/Cargo.toml   |  59 ++
 rust/Makefile.am  | 106 +++
 rust/cargo_test/Cargo.toml|  23 +
 rust/cargo_test/README.md |   3 +
 rust/cargo_test/src/lib.rs|  31 +
 rust/examples/concurrent-read-write.rs| 135 +++
 rust/examples/connect-command.rs  |  39 +
 rust/examples/fetch-first-sector.rs   |  38 +
 rust/examples/get-size.rs |  29 +
 rust/libnbd-sys/Cargo.toml|  32 +
 rust/libnbd-sys/build.rs  |  26 +
 rust/libnbd-sys/src/.keep |   0
 rust/run-tests.sh.in  |  39 +
 rust/src/async_handle.rs  | 268 ++
 rust/src/error.rs | 157 
 rust/src/handle.rs|  67 ++
 rust/src/lib.rs   |  36 +
 rust/src/types.rs |  20 +
 rust/src/utils.rs |  23 +
 rust/tests/nbdkit_pattern/mod.rs  |  28 +
 rust/tests/test_100_handle.rs |  25 +
 rust/tests/test_110_defaults.rs   |  33 +
 rust/tests/test_120_set_non_defaults.rs   |  53 ++
 rust/tests/test_130_private_data.rs   |  28 +
 rust/tests/test_140_explicit_close.rs |  31 +
 rust/tests/test_200_connect_command.rs|  32 +
 rust/tests/test_210_opt_abort.rs  |  31 +
 rust/tests/test_220_opt_list.rs   |  86 ++
 rust/tests/test_230_opt_info.rs   | 120 +++
 rust/tests/test_240_opt_list_meta.rs  | 147 
 rust/tests/test_245_opt_list_meta_queries.rs  |  93 ++
 rust/tests/test_250_opt_set_meta.rs   | 123 +++
 rust/tests/test_255_opt_set_meta_queries.rs   | 109 +++
 rust/tests/test_300_get_size.rs   |  35 +
 rust/tests/test_400_pread.rs  |  39 +
 rust/tests/test_405_pread_structured.rs   |  79 ++
 rust/tests/test_410_pwrite.rs |  58 ++
 rust/tests/test_460_block_status.rs   |  92 ++
 rust/tests/test_620_stats.rs  |  75 ++
 rust/tests/test_async_100_handle.rs   |  25 +
 rust/tests/test_async_200_connect_command.rs  |  33 +
 rust/tests/test_async_210_opt_abort.rs|  32 +
 rust/tests/test_async_220_opt_list.rs |  81 ++
 rust/tests/test_async_230_opt_info.rs | 122 +++
 rust/tests/test_async_240_opt_list_meta.rs| 147 
 .../test_async_245_opt_list_meta_queries.rs   |  91 ++
 rust/tests/test_async_250_opt_set_meta.rs | 122 +++
 .../test_async_255_opt_set_meta_queries.rs| 107 +++
 rust/tests/test_async_400_pread.rs|  40 +
 rust/tests/test_async_405_pread_structured.rs |  84 ++
 rust/tests/test_async_410_pwrite.rs   |  59 ++
 rust/tests/test_async_460_block_status.rs |  92 ++
 rust/tests/test_async_620_stats.rs|  76 ++
 rust/tests/test_log/mod.rs|  86 ++
 rustfmt.toml  |  19 +
 scripts/git.orderfile |  12 +
 68 files changed, 4851 insertions(+)
 create mode 100644 .ocamlformat
 create mode 100644 generator/Rust.ml
 create mode 100644 generator/Rust.mli
 create mode 100644 generator/RustSys.ml
 create mode 100644 generator/RustSys.mli
 create mode 100644 rust/Cargo.toml
 create mode 100644 rust/Makefile.am
 create mode 100644 rust/cargo_test/Cargo.toml
 create mode 100644 

[Libguestfs] [libnbd PATCH v5 10/12] rust: async: Use the modifies_fd flag to exclude calls

[Tage Johansson]

All handle calls which has the modifies_fd flag set to true will be
excluded from AsyncHandle (the asynchronous handle in the rust
bindings). This is a better approach then listing all calls that should
be excluded in Rust.ml explicetly.
---
 generator/Rust.ml | 60 +--
 1 file changed, 27 insertions(+), 33 deletions(-)

diff --git a/generator/Rust.ml b/generator/Rust.ml
index f6016d4..fad991a 100644
--- a/generator/Rust.ml
+++ b/generator/Rust.ml
@@ -568,18 +568,17 @@ let generate_rust_bindings () =
 
 let excluded_handle_calls : NameSet.t =
   NameSet.of_list
-[
-  "aio_get_fd";
-  "aio_get_direction";
-  "aio_notify_read";
-  "aio_notify_write";
-  "clear_debug_callback";
-  "get_debug";
-  "poll";
-  "poll2";
-  "set_debug";
-  "set_debug_callback";
-]
+  @@ [
+   "aio_get_fd";
+   "aio_get_direction";
+   "clear_debug_callback";
+   "get_debug";
+   "set_debug";
+   "set_debug_callback";
+ ]
+  @ (handle_calls
+|> List.filter (fun (_, { modifies_fd }) -> modifies_fd)
+|> List.map (fun (name, _) -> name))
 
 (* A mapping with names as keys. *)
 module NameMap = Map.Make (String)
@@ -590,16 +589,16 @@ let strip_aio name : string =
 String.sub name 4 (String.length name - 4)
   else failwithf "Asynchronous call %s must begin with aio_" name
 
-(* A map with all asynchronous handle calls. The keys are names with "aio_"
-   stripped, the values are a tuple with the actual name (with "aio_"), the
-   [call] and the [async_kind]. *)
+(* A map with all asynchronous handle calls. The keys are names, the values
+   are tuples of: the name with "aio_" stripped, the [call] and the
+   [async_kind]. *)
 let async_handle_calls : ((string * call) * async_kind) NameMap.t =
   handle_calls
   |> List.filter (fun (n, _) -> not (NameSet.mem n excluded_handle_calls))
   |> List.filter_map (fun (name, call) ->
  call.async_kind
  |> Option.map (fun async_kind ->
-(strip_aio name, ((name, call), async_kind
+(name, ((strip_aio name, call), async_kind
   |> List.to_seq |> NameMap.of_seq
 
 (* A mapping with all synchronous (not asynchronous) handle calls. Excluded
@@ -609,11 +608,7 @@ let async_handle_calls : ((string * call) * async_kind) 
NameMap.t =
 let sync_handle_calls : call NameMap.t =
   handle_calls
   |> List.filter (fun (n, _) -> not (NameSet.mem n excluded_handle_calls))
-  |> List.filter (fun (name, _) ->
- (not (NameMap.mem name async_handle_calls))
- && not
-  (String.starts_with ~prefix:"aio_" name
-  && NameMap.mem (strip_aio name) async_handle_calls))
+  |> List.filter (fun (n, _) -> not (NameMap.mem n async_handle_calls))
   |> List.to_seq |> NameMap.of_seq
 
 (* Get the Rust type for an argument in the asynchronous API. Like
@@ -658,7 +653,7 @@ let print_rust_sync_handle_call name call =
 (* Print the Rust function for an asynchronous handle call with a completion
callback. (Note that "callback" might be abbreviated with "cb" in the
following code. *)
-let print_rust_async_handle_call_with_completion_cb name (aio_name, call) =
+let print_rust_async_handle_call_with_completion_cb aio_name (name, call) =
   (* An array of all optional arguments. Useful because we need to deel with
  the index of the completion callback. *)
   let optargs = Array.of_list call.optargs in
@@ -738,7 +733,7 @@ let print_rust_async_handle_call_with_completion_cb name 
(aio_name, call) =
completion by changing state. The predicate is a call like
"aio_is_connecting" which should get the value (like false) for the call to
be complete. *)
-let print_rust_async_handle_call_changing_state name (aio_name, call)
+let print_rust_async_handle_call_changing_state aio_name (name, call)
 (predicate, value) =
   let value = if value then "true" else "false" in
   print_rust_handle_call_comment call;
@@ -767,16 +762,15 @@ let print_rust_async_handle_call_changing_state name 
(aio_name, call)
 (* Print an impl with all handle calls. *)
 let print_rust_async_handle_impls () =
   pr "impl AsyncHandle {\n";
-  NameMap.iter print_rust_sync_handle_call sync_handle_calls;
-  NameMap.iter
-(fun name (call, async_kind) ->
-  match async_kind with
-  | WithCompletionCallback ->
-  print_rust_async_handle_call_with_completion_cb name call
-  | ChangesState (predicate, value) ->
-  print_rust_async_handle_call_changing_state name call
-(predicate, value))
-async_handle_calls;
+  sync_handle_calls |> NameMap.iter print_rust_sync_handle_call;
+  async_handle_calls
+  |> NameMap.iter (fun name (call, async_kind) ->
+ match async_kind with
+ | WithCompletionCallback ->
+ print_rust_async_handle_call_with_completion_cb name call
+ | ChangesState (predicate, value) ->
+ print_rust_async_handle_call_changing_state 

[Libguestfs] [libnbd PATCH v5 04/12] rust: Add some examples

[Tage Johansson]

This patch adds a few examples in rust/examples/. The examples are
compiled and run as part of the test suite.
---
 rust/Cargo.toml |  2 ++
 rust/Makefile.am|  3 +++
 rust/examples/connect-command.rs| 39 +
 rust/examples/fetch-first-sector.rs | 38 
 rust/examples/get-size.rs   | 29 +
 rust/run-tests.sh.in|  7 ++
 scripts/git.orderfile   |  1 +
 7 files changed, 119 insertions(+)
 create mode 100644 rust/examples/connect-command.rs
 create mode 100644 rust/examples/fetch-first-sector.rs
 create mode 100644 rust/examples/get-size.rs

diff --git a/rust/Cargo.toml b/rust/Cargo.toml
index b498930..04e371e 100644
--- a/rust/Cargo.toml
+++ b/rust/Cargo.toml
@@ -49,5 +49,7 @@ libc = "0.2.147"
 default = ["log"]
 
 [dev-dependencies]
+anyhow = "1.0.72"
 once_cell = "1.18.0"
+pretty-hex = "0.3.0"
 tempfile = "3.6.0"
diff --git a/rust/Makefile.am b/rust/Makefile.am
index f9830d0..bdfe2ca 100644
--- a/rust/Makefile.am
+++ b/rust/Makefile.am
@@ -30,6 +30,9 @@ source_files = \
src/handle.rs \
src/types.rs \
src/utils.rs \
+   examples/connect-command.rs \
+   examples/get-size.rs \
+   examples/fetch-first-sector.rs \
libnbd-sys/Cargo.toml \
libnbd-sys/build.rs \
libnbd-sys/src/.keep \
diff --git a/rust/examples/connect-command.rs b/rust/examples/connect-command.rs
new file mode 100644
index 000..db4adbe
--- /dev/null
+++ b/rust/examples/connect-command.rs
@@ -0,0 +1,39 @@
+//! This example shows how to run an NBD server
+//! (nbdkit) as a subprocess of libnbd.
+
+
+fn main() -> libnbd::Result<()> {
+// Create the libnbd handle.
+let handle = libnbd::Handle::new()?;
+
+// Run nbdkit as a subprocess.
+let args = [
+"nbdkit",
+// You must use ‘-s’ (which tells nbdkit to serve
+// a single connection on stdin/stdout).
+"-s",
+// It is recommended to use ‘--exit-with-parent’
+// to ensure nbdkit is always cleaned up even
+// if the main program crashes.
+"--exit-with-parent",
+// Use this to enable nbdkit debugging.
+"-v",
+// The nbdkit plugin name - this is a RAM disk.
+"memory",
+"size=1M",
+];
+handle.connect_command()?;
+
+// Write some random data to the first sector.
+let wbuf: Vec = (0..512).into_iter().map(|i| (i % 13) as u8).collect();
+handle.pwrite(, 0, None)?;
+
+// Read the first sector back.
+let mut rbuf = [0; 512];
+handle.pread( rbuf, 0, None)?;
+
+// What was read must be exactly the same as what was written.
+assert_eq!(wbuf.as_slice(), rbuf.as_slice());
+
+Ok(())
+}
diff --git a/rust/examples/fetch-first-sector.rs 
b/rust/examples/fetch-first-sector.rs
new file mode 100644
index 000..9efb47a
--- /dev/null
+++ b/rust/examples/fetch-first-sector.rs
@@ -0,0 +1,38 @@
+//! This example shows how to connect to an NBD server
+//! and fetch and print the first sector (usually the
+//! boot sector or partition table or filesystem
+//! superblock).
+//!
+//! You can test it with nbdkit like this:
+//!
+//! nbdkit -U - floppy . \
+//!   --run 'cargo run --example fetch-first-sector -- $unixsocket'
+//!
+//! The nbdkit floppy plugin creates an MBR disk so the
+//! first sector is the partition table.
+
+use pretty_hex::pretty_hex;
+use std::env;
+
+fn main() -> anyhow::Result<()> {
+let nbd = libnbd::Handle::new()?;
+
+let args = env::args_os().collect::>();
+if args.len() != 2 {
+anyhow::bail!("Usage: {:?} socket", args[0]);
+}
+let socket = [1];
+
+// Connect to the NBD server over a
+// Unix domain socket.
+nbd.connect_unix(socket)?;
+
+// Read the first sector synchronously.
+let mut buf = [0; 512];
+nbd.pread( buf, 0, None)?;
+
+// Print the sector in hexdump like format.
+print!("{}", pretty_hex());
+
+Ok(())
+}
diff --git a/rust/examples/get-size.rs b/rust/examples/get-size.rs
new file mode 100644
index 000..7f31df5
--- /dev/null
+++ b/rust/examples/get-size.rs
@@ -0,0 +1,29 @@
+//! This example shows how to connect to an NBD
+//! server and read the size of the disk.
+//!
+//! You can test it with nbdkit like this:
+//!
+//! nbdkit -U - memory 1M \
+//!   --run 'cargo run --example get-size -- $unixsocket'
+
+use std::env;
+
+fn main() -> anyhow::Result<()> {
+let nbd = libnbd::Handle::new()?;
+
+let args = env::args_os().collect::>();
+if args.len() != 2 {
+anyhow::bail!("Usage: {:?} socket", args[0]);
+}
+let socket = [1];
+
+// Connect to the NBD server over a
+// Unix domain socket.
+nbd.connect_unix(socket)?;
+
+// Read the size in bytes and print it.
+let size = nbd.get_size()?;
+println!("{:?}: size = {size} bytes", socket);
+
+Ok(())
+}
diff --git a/rust/run-tests.sh.in 

[Libguestfs] [libnbd PATCH v5 05/12] generator: Add information about asynchronous handle calls

[Tage Johansson]

A new field (async_kind) is added to the call data type in
generator/API.ml*. The purpose is to tell if a certain handle call is
an asynchronous command and if so how one can know when it is
completed. The motivation for this is that all asynchronous commands
on the AsyncHandle in the Rust bindings makes use of Rust's
[`async fn`s](https://doc.rust-lang.org/std/keyword.async.html).
But to implement such an async fn, the API needs to know when the
command completed, either by a completion callback or via a change
of state.
---
 generator/API.ml  | 32 
 generator/API.mli | 11 +++
 2 files changed, 43 insertions(+)

diff --git a/generator/API.ml b/generator/API.ml
index 5fcb0e1..f90a6fa 100644
--- a/generator/API.ml
+++ b/generator/API.ml
@@ -32,6 +32,7 @@ type call = {
   permitted_states : permitted_state list;
   is_locked : bool;
   may_set_error : bool;
+  async_kind : async_kind option;
   mutable first_version : int * int;
 }
 and arg =
@@ -102,6 +103,9 @@ and permitted_state =
 | Negotiating
 | Connected
 | Closed | Dead
+and async_kind =
+| WithCompletionCallback
+| ChangesState of string * bool
 and link =
 | Link of string
 | SectionLink of string
@@ -249,6 +253,7 @@ let default_call = { args = []; optargs = []; ret = RErr;
  see_also = [];
  permitted_states = [];
  is_locked = true; may_set_error = true;
+ async_kind = None;
  first_version = (0, 0) }
 
 (* Calls.
@@ -2798,6 +2803,7 @@ wait for an L.";
 default_call with
 args = [ SockAddrAndLen ("addr", "addrlen") ]; ret = RErr;
 permitted_states = [ Created ];
+async_kind = Some (ChangesState ("aio_is_connecting", false));
 shortdesc = "connect to the NBD server";
 longdesc = "\
 Begin connecting to the NBD server.  The C and C
@@ -2810,6 +2816,7 @@ parameters specify the address of the socket to connect 
to.
 default_call with
 args = [ String "uri" ]; ret = RErr;
 permitted_states = [ Created ];
+async_kind = Some (ChangesState ("aio_is_connecting", false));
 shortdesc = "connect to an NBD URI";
 longdesc = "\
 Begin connecting to the NBD URI C.  Parameters behave as
@@ -2823,6 +2830,7 @@ documented in L.
 default_call with
 args = [ Path "unixsocket" ]; ret = RErr;
 permitted_states = [ Created ];
+async_kind = Some (ChangesState ("aio_is_connecting", false));
 shortdesc = "connect to the NBD server over a Unix domain socket";
 longdesc = "\
 Begin connecting to the NBD server over Unix domain socket
@@ -2837,6 +2845,7 @@ L.
 default_call with
 args = [ UInt32 "cid"; UInt32 "port" ]; ret = RErr;
 permitted_states = [ Created ];
+async_kind = Some (ChangesState ("aio_is_connecting", false));
 shortdesc = "connect to the NBD server over AF_VSOCK socket";
 longdesc = "\
 Begin connecting to the NBD server over the C
@@ -2850,6 +2859,7 @@ L.
 default_call with
 args = [ String "hostname"; String "port" ]; ret = RErr;
 permitted_states = [ Created ];
+async_kind = Some (ChangesState ("aio_is_connecting", false));
 shortdesc = "connect to the NBD server over a TCP port";
 longdesc = "\
 Begin connecting to the NBD server listening on C.
@@ -2862,6 +2872,7 @@ Parameters behave as documented in L.
 default_call with
 args = [ Fd "sock" ]; ret = RErr;
 permitted_states = [ Created ];
+async_kind = Some (ChangesState ("aio_is_connecting", false));
 shortdesc = "connect directly to a connected socket";
 longdesc = "\
 Begin connecting to the connected socket C.
@@ -2874,6 +2885,7 @@ Parameters behave as documented in 
L.
 default_call with
 args = [ StringList "argv" ]; ret = RErr;
 permitted_states = [ Created ];
+async_kind = Some (ChangesState ("aio_is_connecting", false));
 shortdesc = "connect to the NBD server";
 longdesc = "\
 Run the command as a subprocess and begin connecting to it over
@@ -2887,6 +2899,7 @@ L.
 default_call with
 args = [ StringList "argv" ]; ret = RErr;
 permitted_states = [ Created ];
+async_kind = Some (ChangesState ("aio_is_connecting", false));
 shortdesc = "connect using systemd socket activation";
 longdesc = "\
 Run the command as a subprocess and begin connecting to it using
@@ -2903,6 +2916,7 @@ L.
 optargs = [ OClosure completion_closure ];
 ret = RErr;
 permitted_states = [ Negotiating ];
+async_kind = Some WithCompletionCallback;
 shortdesc = "end negotiation and move on to using an export";
 longdesc = "\
 Request that the server finish negotiation and move on to serving the
@@ -2926,6 +2940,7 @@ when L returns true.";
 default_call with
 args = []; ret = RErr;
 permitted_states = [ Negotiating ];
+async_kind = Some (ChangesState ("aio_is_connecting", false));
 shortdesc = "end negotiation and close the connection";
 longdesc = "\
 

[Libguestfs] [libnbd PATCH v5 01/12] rust: create basic Rust bindings

[Tage Johansson]

This commit creates basic Rust bindings in the rust directory.
The bindings are generated by generator/Rust.ml and
generator/RustSys.ml.
---
 .gitignore |  10 +
 .ocamlformat   |   4 +
 Makefile.am|   2 +
 configure.ac   |  30 ++
 generator/Makefile.am  |   4 +
 generator/Rust.ml  | 548 +
 generator/Rust.mli |  20 ++
 generator/RustSys.ml   | 167 +++
 generator/RustSys.mli  |  19 ++
 generator/generator.ml |   3 +
 rust/Cargo.toml|  49 
 rust/Makefile.am   |  76 +
 rust/cargo_test/Cargo.toml |  23 ++
 rust/cargo_test/README.md  |   3 +
 rust/cargo_test/src/lib.rs |  31 +++
 rust/libnbd-sys/Cargo.toml |  32 +++
 rust/libnbd-sys/build.rs   |  26 ++
 rust/libnbd-sys/src/.keep  |   0
 rust/run-tests.sh.in   |  24 ++
 rust/src/error.rs  | 157 +++
 rust/src/handle.rs |  65 +
 rust/src/lib.rs|  28 ++
 rust/src/types.rs  |  18 ++
 rust/src/utils.rs  |  23 ++
 rustfmt.toml   |  19 ++
 scripts/git.orderfile  |  10 +
 26 files changed, 1391 insertions(+)
 create mode 100644 .ocamlformat
 create mode 100644 generator/Rust.ml
 create mode 100644 generator/Rust.mli
 create mode 100644 generator/RustSys.ml
 create mode 100644 generator/RustSys.mli
 create mode 100644 rust/Cargo.toml
 create mode 100644 rust/Makefile.am
 create mode 100644 rust/cargo_test/Cargo.toml
 create mode 100644 rust/cargo_test/README.md
 create mode 100644 rust/cargo_test/src/lib.rs
 create mode 100644 rust/libnbd-sys/Cargo.toml
 create mode 100644 rust/libnbd-sys/build.rs
 create mode 100644 rust/libnbd-sys/src/.keep
 create mode 100755 rust/run-tests.sh.in
 create mode 100644 rust/src/error.rs
 create mode 100644 rust/src/handle.rs
 create mode 100644 rust/src/lib.rs
 create mode 100644 rust/src/types.rs
 create mode 100644 rust/src/utils.rs
 create mode 100644 rustfmt.toml

diff --git a/.gitignore b/.gitignore
index efe3080..ac514b0 100644
--- a/.gitignore
+++ b/.gitignore
@@ -174,6 +174,16 @@ Makefile.in
 /python/nbd.py
 /python/run-python-tests
 /run
+/rust/Cargo.lock
+/rust/libnbd-sys/Cargo.lock
+/rust/libnbd-sys/libnbd_version
+/rust/libnbd-sys/src/lib.rs
+/rust/src/async_bindings.rs
+/rust/src/bindings.rs
+/rust/target
+/rust/cargo_test/Cargo.lock
+/rust/cargo_test/target
+/rust/run-tests.sh
 /sh/nbdsh
 /sh/nbdsh.1
 /stamp-h1
diff --git a/.ocamlformat b/.ocamlformat
new file mode 100644
index 000..7bfe155
--- /dev/null
+++ b/.ocamlformat
@@ -0,0 +1,4 @@
+profile = default
+version = 0.25.1
+wrap-comments = true
+margin = 78
diff --git a/Makefile.am b/Makefile.am
index 243fabd..9f7707a 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -28,6 +28,7 @@ EXTRA_DIST = \
README.md \
scripts/git.orderfile \
SECURITY \
+   rustfmt.toml \
$(NULL)
 
 CLEANFILES += m4/*~
@@ -55,6 +56,7 @@ SUBDIRS = \
ocaml/tests \
golang \
golang/examples \
+   rust \
interop \
fuzzing \
bash-completion \
diff --git a/configure.ac b/configure.ac
index 0b94f5e..816b59e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -613,6 +613,32 @@ AS_IF([test "x$enable_golang" != "xno"],[
 ],[GOLANG=no])
 AM_CONDITIONAL([HAVE_GOLANG],[test "x$GOLANG" != "xno"])
 
+dnl Rust.
+AC_ARG_ENABLE([rust],
+AS_HELP_STRING([--disable-rust], [disable Rust language bindings]),
+[],
+[enable_rust=yes])
+AS_IF([test "x$enable_rust" != "xno"],[
+AC_CHECK_PROG([CARGO],[cargo],[cargo],[no])
+AC_CHECK_PROG([RUSTFMT],[rustfmt],[rustfmt],[no])
+AS_IF([test "x$CARGO" != "xno"],[
+AC_MSG_CHECKING([if $CARGO is usable])
+AS_IF([ (
+  cd $srcdir/rust/cargo_test &&
+   $CARGO test 2>_MESSAGE_LOG_FD 1>&2 &&
+   $CARGO doc 2>_MESSAGE_LOG_FD 1>&2 &&
+   $CARGO fmt 2>_MESSAGE_LOG_FD 1>&2
+   ) ],[
+AC_MSG_RESULT([yes])
+],[
+AC_MSG_RESULT([no])
+AC_MSG_WARN([Rust ($CARGO) is installed but not usable])
+CARGO=no
+])
+])
+],[CARGO=no])
+AM_CONDITIONAL([HAVE_RUST],[test "x$CARGO" != "xno" -a "x$RUSTFMT" != "xno"])
+
 AC_MSG_CHECKING([for how to mark DSO non-deletable at runtime])
 NODELETE=
 `$LD --help 2>&1 | grep -- "-z nodelete" >/dev/null` && \
@@ -643,6 +669,8 @@ AC_CONFIG_FILES([run],
 [chmod +x,-w run])
 AC_CONFIG_FILES([sh/nbdsh],
 [chmod +x,-w sh/nbdsh])
+AC_CONFIG_FILES([rust/run-tests.sh],
+[chmod +x,-w rust/run-tests.sh])
 
 AC_CONFIG_FILES([Makefile
  bash-completion/Makefile
@@ -657,6 +685,7 @@ AC_CONFIG_FILES([Makefile
  generator/Makefile
  golang/Makefile
  golang/examples/Makefile
+ rust/Makefile
  include/Makefile
  info/Makefile
  

[Libguestfs] [libnbd PATCH v5 02/12] rust: Add a couple of integration tests

[Tage Johansson]

A couple of integration tests are added in rust/tests. They are mostly
ported from the OCaml tests.
---
 rust/Cargo.toml  |   4 +
 rust/Makefile.am |  22 +++
 rust/run-tests.sh.in |   4 +-
 rust/tests/nbdkit_pattern/mod.rs |  28 
 rust/tests/test_100_handle.rs|  25 
 rust/tests/test_110_defaults.rs  |  33 +
 rust/tests/test_120_set_non_defaults.rs  |  53 +++
 rust/tests/test_130_private_data.rs  |  28 
 rust/tests/test_140_explicit_close.rs|  31 
 rust/tests/test_200_connect_command.rs   |  32 
 rust/tests/test_210_opt_abort.rs |  31 
 rust/tests/test_220_opt_list.rs  |  86 +++
 rust/tests/test_230_opt_info.rs  | 120 +++
 rust/tests/test_240_opt_list_meta.rs | 147 +++
 rust/tests/test_245_opt_list_meta_queries.rs |  93 
 rust/tests/test_250_opt_set_meta.rs  | 123 
 rust/tests/test_255_opt_set_meta_queries.rs  | 109 ++
 rust/tests/test_300_get_size.rs  |  35 +
 rust/tests/test_400_pread.rs |  39 +
 rust/tests/test_405_pread_structured.rs  |  79 ++
 rust/tests/test_410_pwrite.rs|  58 
 rust/tests/test_460_block_status.rs  |  92 
 rust/tests/test_620_stats.rs |  75 ++
 rust/tests/test_log/mod.rs   |  86 +++
 24 files changed, 1432 insertions(+), 1 deletion(-)
 create mode 100644 rust/tests/nbdkit_pattern/mod.rs
 create mode 100644 rust/tests/test_100_handle.rs
 create mode 100644 rust/tests/test_110_defaults.rs
 create mode 100644 rust/tests/test_120_set_non_defaults.rs
 create mode 100644 rust/tests/test_130_private_data.rs
 create mode 100644 rust/tests/test_140_explicit_close.rs
 create mode 100644 rust/tests/test_200_connect_command.rs
 create mode 100644 rust/tests/test_210_opt_abort.rs
 create mode 100644 rust/tests/test_220_opt_list.rs
 create mode 100644 rust/tests/test_230_opt_info.rs
 create mode 100644 rust/tests/test_240_opt_list_meta.rs
 create mode 100644 rust/tests/test_245_opt_list_meta_queries.rs
 create mode 100644 rust/tests/test_250_opt_set_meta.rs
 create mode 100644 rust/tests/test_255_opt_set_meta_queries.rs
 create mode 100644 rust/tests/test_300_get_size.rs
 create mode 100644 rust/tests/test_400_pread.rs
 create mode 100644 rust/tests/test_405_pread_structured.rs
 create mode 100644 rust/tests/test_410_pwrite.rs
 create mode 100644 rust/tests/test_460_block_status.rs
 create mode 100644 rust/tests/test_620_stats.rs
 create mode 100644 rust/tests/test_log/mod.rs

diff --git a/rust/Cargo.toml b/rust/Cargo.toml
index c745972..b498930 100644
--- a/rust/Cargo.toml
+++ b/rust/Cargo.toml
@@ -47,3 +47,7 @@ libc = "0.2.147"
 
 [features]
 default = ["log"]
+
+[dev-dependencies]
+once_cell = "1.18.0"
+tempfile = "3.6.0"
diff --git a/rust/Makefile.am b/rust/Makefile.am
index 8615794..db8fc66 100644
--- a/rust/Makefile.am
+++ b/rust/Makefile.am
@@ -36,6 +36,27 @@ source_files = \
cargo_test/Cargo.toml \
cargo_test/src/lib.rs \
cargo_test/README.md \
+   tests/nbdkit_pattern/mod.rs \
+   tests/test_100_handle.rs \
+   tests/test_110_defaults.rs \
+   tests/test_120_set_non_defaults.rs \
+   tests/test_130_private_data.rs \
+   tests/test_140_explicit_close.rs \
+   tests/test_200_connect_command.rs \
+   tests/test_210_opt_abort.rs \
+   tests/test_220_opt_list.rs \
+   tests/test_230_opt_info.rs \
+   tests/test_240_opt_list_meta.rs \
+   tests/test_245_opt_list_meta_queries.rs \
+   tests/test_250_opt_set_meta.rs \
+   tests/test_255_opt_set_meta_queries.rs \
+   tests/test_300_get_size.rs \
+   tests/test_400_pread.rs \
+   tests/test_405_pread_structured.rs \
+   tests/test_410_pwrite.rs \
+   tests/test_460_block_status.rs \
+   tests/test_620_stats.rs \
+   tests/test_log/mod.rs \
run-tests.sh.in \
$(NULL)
 
@@ -63,6 +84,7 @@ TESTS_ENVIRONMENT = \
LIBNBD_DEBUG=1 \
$(MALLOC_CHECKS) \
abs_top_srcdir=$(abs_top_srcdir) \
+   CARGO=$(CARGO) \
$(NULL)
 LOG_COMPILER = $(top_builddir)/run
 TESTS = run-tests.sh
diff --git a/rust/run-tests.sh.in b/rust/run-tests.sh.in
index f4d220c..d45b1bf 100755
--- a/rust/run-tests.sh.in
+++ b/rust/run-tests.sh.in
@@ -1,6 +1,6 @@
 #!/bin/bash -
 # nbd client library in userspace
-# Copyright Red Hat
+# Copyright Tage Johansson
 #
 # This library is free software; you can redistribute it and/or
 # modify it under the terms of the GNU Lesser General Public
@@ -21,4 +21,6 @@
 set -e
 set -x
 
+requires nbdkit --version
+
 @CARGO@ test -- --nocapture
diff --git a/rust/tests/nbdkit_pattern/mod.rs b/rust/tests/nbdkit_pattern/mod.rs
new file mode 100644
index 000..5f4069e

[Libguestfs] [libnbd PATCH v5 03/12] rust: Make it possible to run tests with Valgrind

[Tage Johansson]

Make it possible to run Rust tests with Valgrind with
`make check-valgrind` in the rust directory.
---
 rust/Makefile.am | 3 +++
 rust/run-tests.sh.in | 6 +-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/rust/Makefile.am b/rust/Makefile.am
index db8fc66..f9830d0 100644
--- a/rust/Makefile.am
+++ b/rust/Makefile.am
@@ -89,6 +89,9 @@ TESTS_ENVIRONMENT = \
 LOG_COMPILER = $(top_builddir)/run
 TESTS = run-tests.sh
 
+check-valgrind:
+   LIBNBD_VALGRIND=1 $(MAKE) check
+
 clean-local:
$(CARGO) clean
$(CARGO) clean --manifest-path cargo_test/Cargo.toml
diff --git a/rust/run-tests.sh.in b/rust/run-tests.sh.in
index d45b1bf..f7db344 100755
--- a/rust/run-tests.sh.in
+++ b/rust/run-tests.sh.in
@@ -23,4 +23,8 @@ set -x
 
 requires nbdkit --version
 
-@CARGO@ test -- --nocapture
+if [ -z "$VG" ]; then
+@CARGO@ test -- --nocapture
+else
+@CARGO@ test --config "target.'cfg(all())'.runner = \"$VG\"" -- --nocapture
+fi
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs

[Libguestfs] [libnbd PATCH v5 06/12] generator: Add information about the lifetime of closures

[Tage Johansson]

Add two new fields, cblifetime and cbcount, to the `closure` type
in generator/API.ml*. cblifetime tells if the closure may only be used
for as long as the command is in flight or if the closure may be used
until the handle is destructed. cbcount tells whether the closure may
be called many times or just once.

This information is needed in the Rust bindings for:
a) Knowing if the closure trait should be FnMut or FnOnce
   (see ).
b) Knowing for what lifetime the closure should be valid. A closure that
   may be called after the function invokation has returned must live
   for the `'static` lietime. But static closures are inconveniant for
   the user since they can't effectively borrow any local data. So it is
   good if this restriction is relaxed when it is not needed.
---
 generator/API.ml  | 20 
 generator/API.mli | 17 +
 2 files changed, 37 insertions(+)

diff --git a/generator/API.ml b/generator/API.ml
index f90a6fa..42b9eec 100644
--- a/generator/API.ml
+++ b/generator/API.ml
@@ -77,6 +77,8 @@ and ret =
 and closure = {
   cbname : string;
   cbargs : cbarg list;
+  cblifetime : cblifetime;
+  cbcount : cbcount
 }
 and cbarg =
 | CBArrayAndLen of arg * string
@@ -87,6 +89,12 @@ and cbarg =
 | CBString of string
 | CBUInt of string
 | CBUInt64 of string
+and cblifetime =
+| CBCommand
+| CBHandle
+and cbcount =
+| CBOnce
+| CBMany
 and enum = {
   enum_prefix : string;
   enums : (string * int) list
@@ -141,20 +149,28 @@ the handle from the NBD protocol handshake."
 (* Closures. *)
 let chunk_closure = {
   cbname = "chunk";
+  cblifetime = CBCommand;
+  cbcount = CBMany;
   cbargs = [ CBBytesIn ("subbuf", "count");
  CBUInt64 "offset"; CBUInt "status";
  CBMutable (Int "error") ]
 }
 let completion_closure = {
   cbname = "completion";
+  cblifetime = CBCommand;
+  cbcount = CBOnce;
   cbargs = [ CBMutable (Int "error") ]
 }
 let debug_closure = {
   cbname = "debug";
+  cblifetime = CBHandle;
+  cbcount = CBMany;
   cbargs = [ CBString "context"; CBString "msg" ]
 }
 let extent_closure = {
   cbname = "extent";
+  cblifetime = CBCommand;
+  cbcount = CBMany;
   cbargs = [ CBString "metacontext";
  CBUInt64 "offset";
  CBArrayAndLen (UInt32 "entries",
@@ -163,10 +179,14 @@ let extent_closure = {
 }
 let list_closure = {
   cbname = "list";
+  cblifetime = CBCommand;
+  cbcount = CBMany;
   cbargs = [ CBString "name"; CBString "description" ]
 }
 let context_closure = {
   cbname = "context";
+  cblifetime = CBCommand;
+  cbcount = CBMany;
   cbargs = [ CBString "name" ]
 }
 let all_closures = [ chunk_closure; completion_closure;
diff --git a/generator/API.mli b/generator/API.mli
index 361132d..ff85849 100644
--- a/generator/API.mli
+++ b/generator/API.mli
@@ -94,6 +94,12 @@ and ret =
 and closure = {
   cbname : string; (** name of callback function *)
   cbargs : cbarg list; (** all closures return int for now *)
+  (** An upper bound of the lifetime of the closure. Either it will be used for
+  as long as the command is in flight or it may be used until the handle
+  is destructed. *)
+  cblifetime : cblifetime;
+  (** Whether the callback may only be called once or many times. *)
+  cbcount : cbcount;
 }
 and cbarg =
 | CBArrayAndLen of arg * string (** array + number of entries *)
@@ -104,6 +110,17 @@ and cbarg =
 | CBString of string   (** like String *)
 | CBUInt of string (** like UInt *)
 | CBUInt64 of string   (** like UInt64 *)
+and cblifetime =
+| CBCommand (** The closure may only be used until the command is retired.
+(E.G., completion callback or list callback.) *)
+| CBHandle  (** The closure might be used until the handle is descructed.
+(E.G., debug callback.) *)
+and cbcount =
+| CBOnce (** The closure will be used 0 or 1 time if the aio_* call returned an
+ error and exactly once if the call succeeded.
+ (E.g., completion callback.) *)
+| CBMany (** The closure may be used any number of times.
+ (E.g., list callback.) *)
 and enum = {
   enum_prefix : string;(** prefix of each enum variant *)
   enums : (string * int) list (** enum names and their values in C *)
-- 
2.41.0

___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs

[Libguestfs] [libnbd PATCH v5 09/12] generator: Add `modifies_fd` flag to the [call] structure

[Tage Johansson]

Add a flag (modifies_fd) to the call structure in generator/API.ml
which is set to true if the handle call may do something with the
file descriptor. That is, it is true for all calls which are or may
call aio_notify_*, including all synchronous commands like
nbd_connect or nbd_opt_go.

The motivation for this is that the asynchronous handle in the Rust
bindings uses its own loop for polling, modifying the file descriptor
outside of this loop may cause unexpected behaviour. Including that the
handle may hang. All commands which set this flag to true will be
excluded from that handle. The asynchronous (aio_*) functions will be
used instead.
---
 generator/API.ml  | 32 
 generator/API.mli |  7 +++
 2 files changed, 39 insertions(+)

diff --git a/generator/API.ml b/generator/API.ml
index 42b9eec..6e1670d 100644
--- a/generator/API.ml
+++ b/generator/API.ml
@@ -33,6 +33,7 @@ type call = {
   is_locked : bool;
   may_set_error : bool;
   async_kind : async_kind option;
+  modifies_fd: bool;
   mutable first_version : int * int;
 }
 and arg =
@@ -274,6 +275,7 @@ let default_call = { args = []; optargs = []; ret = RErr;
  permitted_states = [];
  is_locked = true; may_set_error = true;
  async_kind = None;
+ modifies_fd = false;
  first_version = (0, 0) }
 
 (* Calls.
@@ -1181,6 +1183,7 @@ Return true if option negotiation mode was enabled on 
this handle.";
 default_call with
 args = []; ret = RErr;
 permitted_states = [ Negotiating ];
+modifies_fd = true;
 shortdesc = "end negotiation and move on to using an export";
 longdesc = "\
 Request that the server finish negotiation and move on to serving the
@@ -1208,6 +1211,7 @@ although older servers will instead have killed the 
connection.";
 default_call with
 args = []; ret = RErr;
 permitted_states = [ Negotiating ];
+modifies_fd = true;
 shortdesc = "end negotiation and close the connection";
 longdesc = "\
 Request that the server finish negotiation, gracefully if possible, then
@@ -1221,6 +1225,7 @@ enabled option mode.";
 default_call with
 args = []; ret = RBool;
 permitted_states = [ Negotiating ];
+modifies_fd = true;
 shortdesc = "request the server to initiate TLS";
 longdesc = "\
 Request that the server initiate a secure TLS connection, by
@@ -1259,6 +1264,7 @@ established, as reported by 
L.";
 default_call with
 args = []; ret = RBool;
 permitted_states = [ Negotiating ];
+modifies_fd = true;
 shortdesc = "request the server to enable structured replies";
 longdesc = "\
 Request that the server use structured replies, by sending
@@ -1285,6 +1291,7 @@ later calls to this function return false.";
 default_call with
 args = [ Closure list_closure ]; ret = RInt;
 permitted_states = [ Negotiating ];
+modifies_fd = true;
 shortdesc = "request the server to list all exports during negotiation";
 longdesc = "\
 Request that the server list all exports that it supports.  This can
@@ -1326,6 +1333,7 @@ description is set with I<-D>.";
 default_call with
 args = []; ret = RErr;
 permitted_states = [ Negotiating ];
+modifies_fd = true;
 shortdesc = "request the server for information about an export";
 longdesc = "\
 Request that the server supply information about the export name
@@ -1357,6 +1365,7 @@ corresponding L would succeed.";
 default_call with
 args = [ Closure context_closure ]; ret = RInt;
 permitted_states = [ Negotiating ];
+modifies_fd = true;
 shortdesc = "list available meta contexts, using implicit query list";
 longdesc = "\
 Request that the server list available meta contexts associated with
@@ -1412,6 +1421,7 @@ a server may send a lengthy list.";
 default_call with
 args = [ StringList "queries"; Closure context_closure ]; ret = RInt;
 permitted_states = [ Negotiating ];
+modifies_fd = true;
 shortdesc = "list available meta contexts, using explicit query list";
 longdesc = "\
 Request that the server list available meta contexts associated with
@@ -1462,6 +1472,7 @@ a server may send a lengthy list.";
 default_call with
 args = [ Closure context_closure ]; ret = RInt;
 permitted_states = [ Negotiating ];
+modifies_fd = true;
 shortdesc = "select specific meta contexts, using implicit query list";
 longdesc = "\
 Request that the server supply all recognized meta contexts
@@ -1521,6 +1532,7 @@ no contexts are reported, or may fail but have a 
non-empty list.";
 default_call with
 args = [ StringList "queries"; Closure context_closure ]; ret = RInt;
 permitted_states = [ Negotiating ];
+modifies_fd = true;
 shortdesc = "select specific meta contexts, using explicit query list";
 longdesc = "\
 Request that the server supply all recognized meta contexts
@@ -1750,6 

[Libguestfs] [libnbd PATCH v5 08/12] rust: async: Create an async friendly handle type

[Tage Johansson]

Create another handle type: AsyncHandle, which makes use of Rust's
builtin asynchronous functions (see
) and runs on top of
the Tokio runtime (see ). For every asynchronous
command, like aio_connect(), a corresponding `async` method is created
on the handle. In this case it would be:
async fn connect(...) -> Result<(), ...>
When called, it will poll the file descriptor until the command is
complete, and then return with a result. All the synchronous
counterparts (like nbd_connect()) are excluded from this handle type
as they are unnecessary and since they might interfear with the polling
made by the Tokio runtime. For more details about how the asynchronous
commands are executed, please see the comments in
rust/src/async_handle.rs.
---
 generator/Rust.ml| 240 +++
 generator/Rust.mli   |   2 +
 generator/generator.ml   |   1 +
 rust/Cargo.toml  |   4 +-
 rust/Makefile.am |   2 +
 rust/src/async_handle.rs | 268 +++
 rust/src/lib.rs  |   8 ++
 scripts/git.orderfile|   1 +
 8 files changed, 525 insertions(+), 1 deletion(-)
 create mode 100644 rust/src/async_handle.rs

diff --git a/generator/Rust.ml b/generator/Rust.ml
index cd31b60..f6016d4 100644
--- a/generator/Rust.ml
+++ b/generator/Rust.ml
@@ -558,3 +558,243 @@ let generate_rust_bindings () =
   pr "impl Handle {\n";
   List.iter print_rust_handle_method handle_calls;
   pr "}\n\n"
+
+(*)
+(* The rest of the file conserns the asynchronous API.   *)
+(*   *)
+(* See the comments in rust/src/async_handle.rs for more *)
+(* information about how it works.   *)
+(*)
+
+let excluded_handle_calls : NameSet.t =
+  NameSet.of_list
+[
+  "aio_get_fd";
+  "aio_get_direction";
+  "aio_notify_read";
+  "aio_notify_write";
+  "clear_debug_callback";
+  "get_debug";
+  "poll";
+  "poll2";
+  "set_debug";
+  "set_debug_callback";
+]
+
+(* A mapping with names as keys. *)
+module NameMap = Map.Make (String)
+
+(* Strip "aio_" from the beginning of a string. *)
+let strip_aio name : string =
+  if String.starts_with ~prefix:"aio_" name then
+String.sub name 4 (String.length name - 4)
+  else failwithf "Asynchronous call %s must begin with aio_" name
+
+(* A map with all asynchronous handle calls. The keys are names with "aio_"
+   stripped, the values are a tuple with the actual name (with "aio_"), the
+   [call] and the [async_kind]. *)
+let async_handle_calls : ((string * call) * async_kind) NameMap.t =
+  handle_calls
+  |> List.filter (fun (n, _) -> not (NameSet.mem n excluded_handle_calls))
+  |> List.filter_map (fun (name, call) ->
+ call.async_kind
+ |> Option.map (fun async_kind ->
+(strip_aio name, ((name, call), async_kind
+  |> List.to_seq |> NameMap.of_seq
+
+(* A mapping with all synchronous (not asynchronous) handle calls. Excluded
+   are also all synchronous calls that has an asynchronous counterpart. So if
+   "foo" is the name of a handle call and an asynchronous call "aio_foo"
+   exists, then "foo" will not b in this map. *)
+let sync_handle_calls : call NameMap.t =
+  handle_calls
+  |> List.filter (fun (n, _) -> not (NameSet.mem n excluded_handle_calls))
+  |> List.filter (fun (name, _) ->
+ (not (NameMap.mem name async_handle_calls))
+ && not
+  (String.starts_with ~prefix:"aio_" name
+  && NameMap.mem (strip_aio name) async_handle_calls))
+  |> List.to_seq |> NameMap.of_seq
+
+(* Get the Rust type for an argument in the asynchronous API. Like
+   [rust_arg_type] but no static lifetime on some closures and buffers. *)
+let rust_async_arg_type : arg -> string = function
+  | Closure { cbargs; cbcount; cblifetime } ->
+  let lifetime =
+match cblifetime with CBCommand -> None | CBHandle -> Some "'static"
+  in
+  "impl " ^ rust_closure_trait ~lifetime cbargs cbcount
+  | BytesPersistIn _ -> "&[u8]"
+  | BytesPersistOut _ -> " [u8]"
+  | x -> rust_arg_type x
+
+(* Get the Rust type for an optional argument in the asynchronous API. Like
+   [rust_optarg_type] but no static lifetime on some closures. *)
+let rust_async_optarg_type : optarg -> string = function
+  | OClosure x -> sprintf "Option<%s>" (rust_async_arg_type (Closure x))
+  | x -> rust_optarg_type x
+
+(* A string of the argument list for a method on the handle, with both
+   mandotory and optional arguments. *)
+let rust_async_handle_call_args { args; optargs } : string =
+  let rust_args_names =
+List.map rust_arg_name args @ List.map rust_optarg_name optargs
+  and rust_args_types =
+List.map rust_async_arg_type args
+@ List.map rust_async_optarg_type optargs
+  in
+  

[Libguestfs] [libnbd PATCH v5 11/12] rust: async: Add a couple of integration tests

[Tage Johansson]

Add a couple of integration tests as rust/tests/test_async_*.rs. They
are very similar to the tests for the synchronous API.
---
 rust/Cargo.toml   |   1 +
 rust/tests/test_async_100_handle.rs   |  25 +++
 rust/tests/test_async_200_connect_command.rs  |  33 
 rust/tests/test_async_210_opt_abort.rs|  32 
 rust/tests/test_async_220_opt_list.rs |  81 ++
 rust/tests/test_async_230_opt_info.rs | 122 +++
 rust/tests/test_async_240_opt_list_meta.rs| 147 ++
 .../test_async_245_opt_list_meta_queries.rs   |  91 +++
 rust/tests/test_async_250_opt_set_meta.rs | 122 +++
 .../test_async_255_opt_set_meta_queries.rs| 107 +
 rust/tests/test_async_400_pread.rs|  40 +
 rust/tests/test_async_405_pread_structured.rs |  84 ++
 rust/tests/test_async_410_pwrite.rs   |  59 +++
 rust/tests/test_async_460_block_status.rs |  92 +++
 rust/tests/test_async_620_stats.rs|  76 +
 15 files changed, 1112 insertions(+)
 create mode 100644 rust/tests/test_async_100_handle.rs
 create mode 100644 rust/tests/test_async_200_connect_command.rs
 create mode 100644 rust/tests/test_async_210_opt_abort.rs
 create mode 100644 rust/tests/test_async_220_opt_list.rs
 create mode 100644 rust/tests/test_async_230_opt_info.rs
 create mode 100644 rust/tests/test_async_240_opt_list_meta.rs
 create mode 100644 rust/tests/test_async_245_opt_list_meta_queries.rs
 create mode 100644 rust/tests/test_async_250_opt_set_meta.rs
 create mode 100644 rust/tests/test_async_255_opt_set_meta_queries.rs
 create mode 100644 rust/tests/test_async_400_pread.rs
 create mode 100644 rust/tests/test_async_405_pread_structured.rs
 create mode 100644 rust/tests/test_async_410_pwrite.rs
 create mode 100644 rust/tests/test_async_460_block_status.rs
 create mode 100644 rust/tests/test_async_620_stats.rs

diff --git a/rust/Cargo.toml b/rust/Cargo.toml
index e076826..d001248 100644
--- a/rust/Cargo.toml
+++ b/rust/Cargo.toml
@@ -55,3 +55,4 @@ anyhow = "1.0.72"
 once_cell = "1.18.0"
 pretty-hex = "0.3.0"
 tempfile = "3.6.0"
+tokio = { version = "1.29.1", default-features = false, features = 
["rt-multi-thread", "macros"] }
diff --git a/rust/tests/test_async_100_handle.rs 
b/rust/tests/test_async_100_handle.rs
new file mode 100644
index 000..e50bad9
--- /dev/null
+++ b/rust/tests/test_async_100_handle.rs
@@ -0,0 +1,25 @@
+// libnbd Rust test case
+// Copyright Tage Johansson
+//
+// This library is free software; you can redistribute it and/or
+// modify it under the terms of the GNU Lesser General Public
+// License as published by the Free Software Foundation; either
+// version 2 of the License, or (at your option) any later version.
+//
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+// Lesser General Public License for more details.
+//
+// You should have received a copy of the GNU Lesser General Public
+// License along with this library; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+
+//! Just check that we can link with libnbd and create a handle.
+
+#![deny(warnings)]
+
+#[tokio::test]
+async fn test_async_nbd_handle_new() {
+let _ = libnbd::AsyncHandle::new().unwrap();
+}
diff --git a/rust/tests/test_async_200_connect_command.rs 
b/rust/tests/test_async_200_connect_command.rs
new file mode 100644
index 000..8a3f497
--- /dev/null
+++ b/rust/tests/test_async_200_connect_command.rs
@@ -0,0 +1,33 @@
+// libnbd Rust test case
+// Copyright Tage Johansson
+//
+// This library is free software; you can redistribute it and/or
+// modify it under the terms of the GNU Lesser General Public
+// License as published by the Free Software Foundation; either
+// version 2 of the License, or (at your option) any later version.
+//
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+// Lesser General Public License for more details.
+//
+// You should have received a copy of the GNU Lesser General Public
+// License along with this library; if not, write to the Free Software
+// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+
+#![deny(warnings)]
+
+
+#[tokio::test]
+async fn test_async_connect_command() {
+let nbd = libnbd::AsyncHandle::new().unwrap();
+nbd.connect_command(&[
+"nbdkit",
+"-s",
+"--exit-with-parent",
+"-v",
+"null",
+])
+.await
+.unwrap();
+}
diff --git a/rust/tests/test_async_210_opt_abort.rs 
b/rust/tests/test_async_210_opt_abort.rs
new file mode 100644
index 000..e85fa0c
--- /dev/null

[Libguestfs] [libnbd PATCH v5 07/12] rust: Use more specific closure traits

[Tage Johansson]

For closures with cbcount = CBOnce, FnOnce will be used instead of
FnMut. Moreover, closures in synchronous commands with
cblifetime = CBCommand will not need to live for the static lifetime.
See [here](https://doc.rust-lang.org/std/ops/trait.FnOnce.html) for more
information about the advantages of using `FnOnce` when possible.
---
 generator/Rust.ml  | 44 
 rust/src/handle.rs |  2 ++
 rust/src/types.rs  |  2 ++
 3 files changed, 32 insertions(+), 16 deletions(-)

diff --git a/generator/Rust.ml b/generator/Rust.ml
index 88434c3..cd31b60 100644
--- a/generator/Rust.ml
+++ b/generator/Rust.ml
@@ -111,7 +111,7 @@ let rust_cbarg_name : cbarg -> string = function
   | CBArrayAndLen (arg, _) | CBMutable arg -> rust_arg_name arg
 
 (* Get the Rust type for an argument. *)
-let rec rust_arg_type : arg -> string = function
+let rec rust_arg_type ?(async_kind = None) : arg -> string = function
   | Bool _ -> "bool"
   | Int _ -> "c_int"
   | UInt _ -> "c_uint"
@@ -131,15 +131,18 @@ let rec rust_arg_type : arg -> string = function
   | BytesOut _ -> " [u8]"
   | BytesPersistIn _ -> "&'static [u8]"
   | BytesPersistOut _ -> "&'static mut [u8]"
-  | Closure { cbargs } -> "impl " ^ rust_closure_trait cbargs
+  | Closure { cbargs; cbcount } -> (
+  match async_kind with
+  | Some _ -> "impl " ^ rust_closure_trait cbargs cbcount
+  | None -> "impl " ^ rust_closure_trait cbargs cbcount ~lifetime:None)
 
 (* Get the Rust closure trait for a callback, That is `Fn*(...) -> ...)`. *)
-and rust_closure_trait ?(lifetime = Some "'static") cbargs : string =
+and rust_closure_trait ?(lifetime = Some "'static") cbargs cbcount : string =
   let rust_cbargs = String.concat ", " (List.map rust_cbarg_type cbargs)
-  and lifetime_constraint =
-match lifetime with None -> "" | Some x -> " + " ^ x
-  in
-  "FnMut(" ^ rust_cbargs ^ ") -> c_int + Send + Sync" ^ lifetime_constraint
+  and closure_type =
+match cbcount with CBOnce -> "FnOnce" | CBMany -> "FnMut"
+  and lifetime = match lifetime with None -> "" | Some x -> " + " ^ x in
+  sprintf "%s(%s) -> c_int + Send + Sync%s" closure_type rust_cbargs lifetime
 
 (* Get the Rust type for a callback argument. *)
 and rust_cbarg_type : cbarg -> string = function
@@ -153,8 +156,8 @@ and rust_cbarg_type : cbarg -> string = function
   | CBMutable arg -> " " ^ rust_arg_type arg
 
 (* Get the type of a rust optional argument. *)
-let rust_optarg_type : optarg -> string = function
-  | OClosure x -> sprintf "Option<%s>" (rust_arg_type (Closure x))
+let rust_optarg_type ?(async_kind = None) : optarg -> string = function
+  | OClosure x -> sprintf "Option<%s>" (rust_arg_type (Closure x) ~async_kind)
   | OFlags (name, flags, _) ->
   sprintf "Option<%s>" (rust_arg_type (Flags (name, flags)))
 
@@ -419,8 +422,8 @@ let ffi_ret_to_rust call =
closure data, and a free function for the closure data. This struct is what
will be sent to a C function taking the closure as an argument. In fact,
the struct itself is generated by rust-bindgen. *)
-let print_rust_closure_to_raw_fn { cbname; cbargs } =
-  let closure_trait = rust_closure_trait cbargs ~lifetime:None in
+let print_rust_closure_to_raw_fn { cbname; cbargs; cbcount } =
+  let closure_trait = rust_closure_trait cbargs cbcount ~lifetime:None in
   let ffi_cbargs_names = List.flatten (List.map ffi_cbarg_names cbargs) in
   let ffi_cbargs_types = List.flatten (List.map ffi_cbarg_types cbargs) in
   let rust_cbargs_names = List.map rust_cbarg_name cbargs in
@@ -435,16 +438,24 @@ let print_rust_closure_to_raw_fn { cbname; cbargs } =
(List.map2 (sprintf "%s: %s") ffi_cbargs_names ffi_cbargs_types));
   pr "  where F: %s\n" closure_trait;
   pr "{\n";
-  pr "let callback_ptr = data as *mut F;\n";
-  pr "let callback =  *callback_ptr;\n";
+  (match cbcount with
+  | CBMany ->
+  pr "let callback_ptr = data as *mut F;\n";
+  pr "let callback =  *callback_ptr;\n"
+  | CBOnce ->
+  pr "let callback_ptr = data as *mut Option;\n";
+  pr "let callback_option:  Option =  *callback_ptr;\n";
+  pr "let callback: F = callback_option.take().unwrap();\n");
   List.iter ffi_cbargs_to_rust cbargs;
   pr "callback(%s)\n" (String.concat ", " rust_cbargs_names);
   pr "}\n";
-  pr "let callback_data = Box::into_raw(Box::new(f));\n";
+  pr "let callback_data = Box::into_raw(Box::new(%s));\n"
+(match cbcount with CBMany -> "f" | CBOnce -> "Some(f)");
   pr "sys::nbd_%s_callback {\n" cbname;
   pr "callback: Some(call_closure::),\n";
   pr "user_data: callback_data as *mut _,\n";
-  pr "free: Some(utils::drop_data::),\n";
+  pr "free: Some(utils::drop_data::<%s>),\n"
+(match cbcount with CBMany -> "F" | CBOnce -> "Option");
   pr "}\n";
   pr "}\n";
   pr "\n"
@@ -501,7 +512,8 @@ let print_rust_handle_method (name, call) =
   let