On 7/22/20 7:42 AM, Richard W.M. Jones wrote:
See also:
https://www.redhat.com/archives/libguestfs/2020-July/msg00090.html
---
docs/nbdkit-plugin.pod | 57 +++++++++++++++++++++++++++++++++++++---
docs/nbdkit-protocol.pod | 7 +++--
2 files changed, 58 insertions(+), 6 deletions(-)
I'm playing with code for an initial implementation of this, and quickly
ran into a design issue. For plugins, your proposal is straightforward.
But for filters, we have several cases:
1. Filter doesn't care about export name, so it passes through
.list_exports to the plugin and advertises the plugin's name; then it
further passes through the client's export name to .open()
2. Filter is providing its own export names, so it skips the plugin's
.list_exports altogether, and eventually calls .open("") to get the
plugin's default export. But where does the filter get the information
on what exports exist? For example, the tar filter would want to use
next_ops->pread to determine what files are contained within the tar
served by the underlying plugin. Except that .list_exports is called
prior to .open, so we have no plugin handle or next_ops to use for pread.
3. The filter wants to provide a cartesian product of export names (for
each export offered by the plugin, the filter then determines a sublist
of items from that plugin content), which implies multiple .plugin opens
(in this mode, we probably DO want to pursue my proposal in an earlier
thread about enhancing the NBD protocol to add an NBD_OPT_LIST_HIER and
NBD_REP_SERVER_DIR to allow hierarchical traversal rather than flat
listing of every possible export at once).
For 1), the interface for the filter is like any other interface -
provide a hook for filters to call, along with a next_ops to defer into
the plugin. But for 2) I'm thinking we need some way to make it easy to
write a filter that shares a single underlying connection to a plugin
among multiple connections to the plugin; and for 3), that extends even
further into more than one plugin .open for a single filter .list_exports.
For comparison, I'm envisioning something like the current nbd plugin
having a --shared mode; by default (--shared=false), we get:
client1 -> nbdkit nbd .open() -> server connection 1
client2 -> nbdkit nbd .open() -> server connection 2
but with --shared=true, we get:
client1 -\
> nbdkit nbd .after_fork() -> single server connection
client2 -/
We need something similar for a filter to be able to request a single
shared .open of the underlying plugin (shared by all connections into
the filter), and which has a lifetime visible between the filter's
.after_fork and .unload regardless of how many .open/.prepare/.close
client connections come and go, rather than having to repeat a
next_ops->open for each filter .open. Kind of the opposite of the
reopen filter (which uses multiple calls into next_ops->open from a
single .open to the filter).
While filters like tar and ext2 will probably benefit from doing
explicit sharing of a single underlying plugin, it would also allow us
to create a new --filter=shared that can be thrown in front of any
plugin (rendering the existing 'nbdkit nbd --shared' obsolete in favor
of 'nbdkit nbd --filter=shared').
Basically, I'm thinking we need the following filter-only functions:
int nbdkit_plugin_open(int readonly, struct nbdkit_next_ops **next_ops,
void **nxdata);
which a filter can call at any point between .after_fork and .unload to
open a distinct connection into the plugin independent of any filter
clients, and which returns by reference a next_ops/nxdata pair that the
filter can then use in place of waiting for a per-connection
next_ops/nxdata passed through .prepare/.can_write/.pread/etc.
void nbdkit_plugin_close (struct nbdkit_next_ops *next_ops,
void *nxdata);
which a filter later calls to close its own connection into the
underlying plugin. Any plugin that uses these new functions would
probably implement a .open that does NOT call next_open(nxdata), because
it instead reuses the shared handle that it opened globally.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
_______________________________________________
Libguestfs mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/libguestfs
