After previous changes, it is now safe to use the parallel thread
model in this plugin. The locking in pool.c protects a single curl
handle from being used from multiple threads.
An advantage of this is we can now combine the curl plugin with
filters such as readahead and scan.
---
Introduce the concept of a pool of curl handles from which the main
code has to take a handle (get_handle()), and later return it to the
pool (put_handle()). When a handle is taken from the pool it can be
used exclusively by the main code until it is returned.
The "pool" only has at most one
This commit implements the curl handle pool. By default it can grow
to up to 4 curl handles, shared between all NBD connections. You can
change this using the new connections=N parameter.
---
plugins/curl/nbdkit-curl-plugin.pod | 7 +++
plugins/curl/curldefs.h | 2 +
In a future commit we will create a pool of libcurl handles. In order
to prepare for that, split out the libcurl handle and associated
fields and buffers, from the plugin handle. The plugin handle (for
the moment) contains a pointer to the libcurl handle so there is still
a 1-1 relationship
Tidy up previous work on this plugin so that every global is declared
in "curldefs.h". Move the HAVE_CURLINFO_CONTENT_LENGTH_DOWNLOAD_T
macro to this header file too.
---
plugins/curl/curldefs.h | 16
plugins/curl/curl.c | 11 +--
2 files changed, 17 insertions(+),
Trivial refactoring to make future changes easier.
---
plugins/curl/curldefs.h | 4 ++--
plugins/curl/curl.c | 18 +-
plugins/curl/scripts.c | 14 +++---
3 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/plugins/curl/curldefs.h
This experimental series changes the way that the curl plugin deals
with libcurl handles. It also changes the thread model of the plugin
from SERIALIZE_REQUESTS to PARALLEL.
Currently one NBD connection opens one libcurl handle. This also
implies one TCP connection to the web server. If you