[Libguestfs] [PATCH nbdkit 5/6] curl: Use the parallel thread model

2023-02-04 Thread Richard W.M. Jones
After previous changes, it is now safe to use the parallel thread model in this plugin. The locking in pool.c protects a single curl handle from being used from multiple threads. An advantage of this is we can now combine the curl plugin with filters such as readahead and scan. ---

[Libguestfs] [PATCH nbdkit 4/6] curl: Introduce concept of getting/putting handles from a common pool

2023-02-04 Thread Richard W.M. Jones
Introduce the concept of a pool of curl handles from which the main code has to take a handle (get_handle()), and later return it to the pool (put_handle()). When a handle is taken from the pool it can be used exclusively by the main code until it is returned. The "pool" only has at most one

[Libguestfs] [PATCH nbdkit 6/6] curl: Complete implementation of the curl handle pool

2023-02-04 Thread Richard W.M. Jones
This commit implements the curl handle pool. By default it can grow to up to 4 curl handles, shared between all NBD connections. You can change this using the new connections=N parameter. --- plugins/curl/nbdkit-curl-plugin.pod | 7 +++ plugins/curl/curldefs.h | 2 +

[Libguestfs] [PATCH nbdkit 2/6] curl: Split out the libcurl handle from the plugin handle

2023-02-04 Thread Richard W.M. Jones
In a future commit we will create a pool of libcurl handles. In order to prepare for that, split out the libcurl handle and associated fields and buffers, from the plugin handle. The plugin handle (for the moment) contains a pointer to the libcurl handle so there is still a 1-1 relationship

[Libguestfs] [PATCH nbdkit 3/6] curl: Fix declarations of globals in "curldefs.h"

2023-02-04 Thread Richard W.M. Jones
Tidy up previous work on this plugin so that every global is declared in "curldefs.h". Move the HAVE_CURLINFO_CONTENT_LENGTH_DOWNLOAD_T macro to this header file too. --- plugins/curl/curldefs.h | 16 plugins/curl/curl.c | 11 +-- 2 files changed, 17 insertions(+),

[Libguestfs] [PATCH nbdkit 1/6] curl: Rename curl_handle -> handle

2023-02-04 Thread Richard W.M. Jones
Trivial refactoring to make future changes easier. --- plugins/curl/curldefs.h | 4 ++-- plugins/curl/curl.c | 18 +- plugins/curl/scripts.c | 14 +++--- 3 files changed, 18 insertions(+), 18 deletions(-) diff --git a/plugins/curl/curldefs.h

[Libguestfs] [PATCH nbdkit 0/6] curl: Use a curl handle pool

2023-02-04 Thread Richard W.M. Jones
This experimental series changes the way that the curl plugin deals with libcurl handles. It also changes the thread model of the plugin from SERIALIZE_REQUESTS to PARALLEL. Currently one NBD connection opens one libcurl handle. This also implies one TCP connection to the web server. If you