This plugin lets you use an external subprocess, like nbdkit-sh-plugin except that the subprocess is persistent across calls. There is one subprocess handling all connections. Think of this as "FastCGI for plugins" - because the subprocess is persistent instead of having to be forked on every request, it may be faster for some use cases.
Communication between nbdkit and the subprocess uses a simple remote procedure call (RPC) over stdin/stdout, which is string based and similar to the shell commands used by nbdkit-sh-plugin. The new plugin shares most code with nbdkit-sh-plugin / nbdkit-eval-plugin. --- plugins/cc/nbdkit-cc-plugin.pod | 1 + plugins/eval/nbdkit-eval-plugin.pod | 1 + plugins/proc/nbdkit-proc-plugin.pod | 304 +++++++++++++++++++++++++++ plugins/sh/nbdkit-sh-plugin.pod | 4 + configure.ac | 3 + plugins/proc/Makefile.am | 95 +++++++++ plugins/proc/examples/Makefile.am | 49 +++++ plugins/proc/proc-protocol.h | 81 +++++++ plugins/proc/examples/simple.c | 125 +++++++++++ plugins/proc/proc-protocol.c | 290 +++++++++++++++++++++++++ plugins/proc/proc.c | 314 ++++++++++++++++++++++++++++ .gitignore | 2 + 12 files changed, 1269 insertions(+) diff --git a/plugins/cc/nbdkit-cc-plugin.pod b/plugins/cc/nbdkit-cc-plugin.pod index c05d6a72b5..26a525fe81 100644 --- a/plugins/cc/nbdkit-cc-plugin.pod +++ b/plugins/cc/nbdkit-cc-plugin.pod @@ -210,6 +210,7 @@ L<nbdkit(1)>, L<nbdkit-plugin(3)>, L<nbdkit-eval-plugin(3)>, L<nbdkit-ocaml-plugin(3)>, +L<nbdkit-proc-plugin(3)>, L<nbdkit-sh-plugin(3)>. =head1 AUTHORS diff --git a/plugins/eval/nbdkit-eval-plugin.pod b/plugins/eval/nbdkit-eval-plugin.pod index 2807955b24..cff4cc6916 100644 --- a/plugins/eval/nbdkit-eval-plugin.pod +++ b/plugins/eval/nbdkit-eval-plugin.pod @@ -201,6 +201,7 @@ C<nbdkit-eval-plugin> first appeared in nbdkit 1.18. L<nbdkit(1)>, L<nbdkit-plugin(3)>, L<nbdkit-sh-plugin(3)>, +L<nbdkit-proc-plugin(1)>, L<nbdkit-cc-plugin(1)>. =head1 AUTHORS diff --git a/plugins/proc/nbdkit-proc-plugin.pod b/plugins/proc/nbdkit-proc-plugin.pod new file mode 100644 index 0000000000..35e879e1e8 --- /dev/null +++ b/plugins/proc/nbdkit-proc-plugin.pod @@ -0,0 +1,304 @@ +=head1 NAME + +nbdkit-proc-plugin - use an external program to implement a plugin + +=head1 SYNOPSIS + + nbdkit proc program [arguments...] + +=head1 DESCRIPTION + +C<nbdkit-proc-plugin> allows you to write plugins for L<nbdkit(1)> +using an external program written in any programming or scripting +language. + +It is similar to L<nbdkit-sh-plugin(3)>, except that the subprocess is +persistent across calls. If you think of nbdkit-sh-plugin as being +like CGI, then this is like +L<FastCGI|https://en.wikipedia.org/wiki/FastCGI>. Because the +subprocess is persistent instead of having to be forked on every +request, it may be faster than nbdkit-sh-plugin in some cases. + +Alternatives to this plugin include L<nbdkit-sh-plugin(3)>, +L<nbdkit-eval-plugin(1)>, or writing a regular nbdkit plugin which is +dynamically loaded into nbdkit (L<nbdkit-plugin(3)>). + +=head2 If you have been given an nbdkit proc plugin + +You can use it like this: + + nbdkit proc program + +You may have to add further C<key=value> arguments to the command +line. The program must be executable (C<chmod +x>). + +=head1 WRITING AN NBDKIT PROC PLUGIN + +For example plugins, see: +L<https://gitlab.com/nbdkit/nbdkit/blob/master/plugins/proc/examples/> + +As this plugin works like L<nbdkit-sh-plugin(3)> and shares the +implementation, reading the documentation for that plugin first as +well as L<nbdkit-plugin(3)> is recommended. + +=head2 The external program + +C<nbdkit proc program> starts the external C<program> when nbdkit +starts, and kills it when nbdkit shuts down. + +While nbdkit is running, requests from nbdkit are forwarded to the +external program over its stdin, and the external program should write +replies to its stdout. The protocol is a simple string-based remote +procedure call (RPC) described below. + +It is possible to configure the RPC to allow only one, or multiple +requests to be in flight at once. This gives you flexibility: you can +choose to write a simple read request + write reply loop that handles +requests one at a time; or you can write something more sophisticated +that keeps track of multiple requests in parallel. + +This plugin only ever starts one external program (per instance of +nbdkit) across all nbdkit clients, so if your program needs +parallelism it should start its own threads. + +=head2 Remote procedure call protocol + +The remote procedure call protocol is a simple string-based encoding +which is very similar to the parameters used by +L<nbdkit-sh-plugin(3)>, but serialized over the stdin and stdout of +your program. + +There is a simple implementation available in +L<https://gitlab.com/nbdkit/nbdkit/blob/master/plugins/proc/proc-protocol.c> +and +L<https://gitlab.com/nbdkit/nbdkit/blob/master/plugins/proc/proc-protocol.h> +or you can write your own by following the specification below. + +Requests and replies are lists of NUL-terminated (C-like) strings. +For example, this sequence of bytes: + + 'R', '9', '9', '\0', + '0', '\0', + '1', '\0', + 'c', 'l', 'o', 's', 'e', '\0' + +represents the list: + + [ "R99", "0", "1", "close" ] + +=over 4 + +=item request ID + +The first element is the request identifier, a string, which is used +to correlate requests with replies when there are multiple outstanding +requests in flight. + +=item optional binary buffer length + +The second element is used to indicate that an optional binary buffer +follows the list. It is currently non-C<"0"> only in the request +message of C<"pwrite"> and the reply message of C<"pread">. + +If this element is C<"0">, then the buffer is zero length / not sent, +else it is the length of the buffer in bytes. + +The binary buffer is written to the stream directly following the end +of the payload. There are no delimiting bytes before or after the +buffer. + +=item payload length + +The third element of the list is always the number of elements in the +payload, encoded as a string. + +=item payload + +There follow payload length list elements, which contain the request +or reply. + +=item binary buffer + +If the second list element is not C<"0"> then immediately after the +payload a binary buffer of the given length in bytes is sent. + +=back + +=head3 Security + +The third element (payload length) is always in the range +S<["1"..."64"]>. Individual strings in the list always have length +E<le> 4096 bytes (not including the terminating NUL byte). If +present, the binary buffer has length E<le> 67108864 bytes. + +List elements that are meant to be numbers should parse without any +leading or trailing garbage, and every number in the basic protocol is +representable as a C C<unsigned>. (Note this does not apply to +certain request and reply payloads, and there are list elements like +the request ID which are not numbers at all.) + +Messages which do not conform to this indicate a fatal, unrecoverable +error. The external program should exit. If nbdkit sees the external +program write a non-conforming message, it will kill the external +program and any further NBD client requests will receive C<EIO> error. + +=head3 Requests + +Requests are sent as the list: + + [ request ID, buffer length, payload length, method [, arguments...] ] + +The method and arguments (fourth and subsequent elements of the list) +are the same set of strings that are passed to L<nbdkit-sh-plugin(3)> +scripts. Examples: + + [ "R1", "0", "3", "config", "size", "1048576" ] + [ "R2", "0", "1", "thread_model" ] + [ "R3", "0", "1", "get_ready" ] + [ "R4", "0", "4", "open", "false", "", "true" ] + [ "R5", "0", "2", "get_size", "myhandle" ] + [ "R6", "0", "2", "flush", "myhandle" ] + +=head3 pwrite request + +The C<"pwrite"> request message is special because the second element +may be non-zero, in which case a binary buffer immediately follows the +request message (containing the data to be written). + +=head3 Replies + +Replies are sent as the list: + + [ request ID, buffer length, payload length, exit code [, reply] ] + +or for errors: + + [ request ID, buffer length, payload length, "1", [, errno [, error]] ] + +The request ID should usually match the identifier of the +corresponding request (but in some cases can be set to anything, see +L</Thread models> below). The buffer length is only used in replies +to C<"pread"> requests, see next section. The exit code should +correspond to one of the codes in L<nbdkit-sh-plugin(3)/Exit codes>. + +There may be an additional list element for some methods. For +C<"get_size"> it would be the returned size of the disk, encoded as a +string, in the same formats as supported by L<nbdkit-sh-plugin(3)>. + +Examples of replies corresponding to the request messages above: + + [ "R1", "0", "1", "0" ] # reply to "config" + [ "R2", "0", "2", "0", "serialize_all_requests" ] # "thread_model" + [ "R3", "0", "1", "2" ] # reply to "get_ready" + [ "R4", "0", "2", "0", "myhandle" ] # reply to "open" + [ "R5", "0" ,"2", "0", "1048576" ] # reply to "get_size" + [ "R6", "0", "3", "1", "EIO", "I/O error" ] # reply to "flush" + +=head3 pread reply + +The reply message associated with an earlier C<"pread"> request is +special (in the non-error case). The second element of the reply +message may be non-zero and the reply message is immediately followed +by a binary buffer (containing the data read). + +=head3 Requests that the program does not understand + +If a program does not understand a request, it must send a reply with +exit code 2, ie: + + [ <request ID>, "0", "1", "2" ] + +=head3 Thread models + +The C<"thread_model"> request message is used by nbdkit to ask the +program what thread model it supports, and all of the thread model +strings as described in L<nbdkit-sh-plugin(3)> are supported. + +In addition, to make it easier to write external programs that use +this plugin, if the thread model that you return is +C<"serialize_connections"> or C<"serialize_all_requests">, then this +plugin will operate in a simpler mode. + +For C<"serialize_all_requests">: + +=over 4 + +=item * + +There is never more than one request in flight. + +=item * + +The request-ID element in replies is ignored (so you can put anything +there). + +=back + +For C<"serialize_connections">, the above, plus: + +=over 4 + +=item * + +There is only one client, so the handle may be ignored. + +=back + +This allows you to write external programs which are a simple read +request + write reply loop that don't have to consider request-IDs or +deal with parallelism, although they may be slower. + +=head1 PARAMETERS + +=over 4 + +=item [B<script=>]PROGRAM + +The name or path of the external program to run. + +=back + +All other parameters are passed as C<"config"> requests to the program. + +=head1 ENVIRONMENT VARIABLES + +=over 4 + +=item C<tmpdir> + +This contains the name of a temporary directory which can be used by +the external program. It is deleted when nbdkit exits. + +=back + +=head1 FILES + +=over 4 + +=item F<$plugindir/nbdkit-proc-plugin.so> + +The plugin. + +Use C<nbdkit --dump-config> to find the location of C<$plugindir>. + +=back + +=head1 VERSION + +C<nbdkit-proc-plugin> first appeared in nbdkit 1.40. + +=head1 SEE ALSO + +L<nbdkit(1)>, +L<nbdkit-plugin(3)>, +L<nbdkit-sh-plugin(3)>, +L<nbdkit-eval-plugin(3)>, +L<nbdkit-cc-plugin(1)>. + +=head1 AUTHORS + +Richard W.M. Jones + +=head1 COPYRIGHT + +Copyright Red Hat diff --git a/plugins/sh/nbdkit-sh-plugin.pod b/plugins/sh/nbdkit-sh-plugin.pod index 8b045547b5..5836ba2647 100644 --- a/plugins/sh/nbdkit-sh-plugin.pod +++ b/plugins/sh/nbdkit-sh-plugin.pod @@ -26,6 +26,9 @@ those will be more efficient (see L<nbdkit(1)> for a complete list). To use shell script fragments from the nbdkit command line (rather than a separate script) see L<nbdkit-eval-plugin(1)>. +To use a persistent subprocess instead of forking on every request see +L<nbdkit-proc-plugin(1)>. + =head2 If you have been given an nbdkit sh plugin Assuming you have a shell script which is an nbdkit plugin, you run it @@ -607,6 +610,7 @@ C<nbdkit-sh-plugin> first appeared in nbdkit 1.8. L<nbdkit(1)>, L<nbdkit-plugin(3)>, L<nbdkit-eval-plugin(1)>, +L<nbdkit-proc-plugin(1)>, L<nbdkit-cc-plugin(1)>. =head1 AUTHORS diff --git a/configure.ac b/configure.ac index 241a869f31..955ee6c956 100644 --- a/configure.ac +++ b/configure.ac @@ -98,6 +98,7 @@ non_lang_plugins="\ ones \ partitioning \ pattern \ + proc \ random \ S3 \ sparse-random \ @@ -1596,6 +1597,8 @@ AC_CONFIG_FILES([Makefile plugins/partitioning/Makefile plugins/pattern/Makefile plugins/perl/Makefile + plugins/proc/Makefile + plugins/proc/examples/Makefile plugins/python/Makefile plugins/random/Makefile plugins/rust/Makefile diff --git a/plugins/proc/Makefile.am b/plugins/proc/Makefile.am new file mode 100644 index 0000000000..9d96f3f2f8 --- /dev/null +++ b/plugins/proc/Makefile.am @@ -0,0 +1,95 @@ +# nbdkit +# Copyright Red Hat +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# * Neither the name of Red Hat nor the names of its contributors may be +# used to endorse or promote products derived from this software without +# specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, +# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR +# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF +# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +include $(top_srcdir)/common-rules.mk + +SUBDIRS = . examples + +EXTRA_DIST = nbdkit-proc-plugin.pod + +plugin_LTLIBRARIES = nbdkit-proc-plugin.la + +# This plugin shares most of the same sources as nbdkit-sh-plugin. In +# RHEL 7 we cannot add the C files from ../sh directly to SOURCES +# because subdir-objects is broken. Instead we create symlinks to +# them. +BUILT_SOURCES = \ + methods.c \ + tmpdir.c \ + $(NULL) +methods.c: $(srcdir)/../sh/methods.c + ln -f -s $(srcdir)/../sh/$@ +tmpdir.c: $(srcdir)/../sh/tmpdir.c + ln -f -s $(srcdir)/../sh/$@ +CLEANFILES += $(BUILT_SOURCES) + +nbdkit_proc_plugin_la_SOURCES = \ + proc.c \ + proc-protocol.c \ + proc-protocol.h \ + $(BUILT_SOURCES) \ + $(srcdir)/../sh/methods.h \ + $(srcdir)/../sh/subplugin.h \ + $(top_srcdir)/include/nbdkit-plugin.h \ + $(NULL) +nbdkit_proc_plugin_la_CPPFLAGS = \ + -I$(top_srcdir)/include \ + -I$(top_builddir)/include \ + -I$(top_srcdir)/plugins/sh \ + -I$(top_srcdir)/common/include \ + -I$(top_srcdir)/common/utils \ + -DIN_PROC_PLUGIN=1 \ + $(NULL) +nbdkit_proc_plugin_la_CFLAGS = $(WARNINGS_CFLAGS) +nbdkit_proc_plugin_la_LIBADD = \ + $(top_builddir)/common/utils/libutils.la \ + $(IMPORT_LIBRARY_ON_WINDOWS) \ + $(NULL) +nbdkit_proc_plugin_la_LDFLAGS = \ + -module -avoid-version -shared $(NO_UNDEFINED_ON_WINDOWS) \ + $(NULL) +if USE_LINKER_SCRIPT +nbdkit_proc_plugin_la_LDFLAGS += \ + -Wl,--version-script=$(top_srcdir)/plugins/plugins.syms +endif + +if HAVE_POD + +man_MANS = nbdkit-proc-plugin.1 +CLEANFILES += $(man_MANS) + +nbdkit-proc-plugin.1: nbdkit-proc-plugin.pod \ + $(top_builddir)/podwrapper.pl + $(PODWRAPPER) --section=1 --man $@ \ + --html $(top_builddir)/html/$@.html \ + $< + +endif HAVE_POD diff --git a/plugins/proc/examples/Makefile.am b/plugins/proc/examples/Makefile.am new file mode 100644 index 0000000000..de4e2eeca4 --- /dev/null +++ b/plugins/proc/examples/Makefile.am @@ -0,0 +1,49 @@ +# nbdkit +# Copyright Red Hat +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# * Neither the name of Red Hat nor the names of its contributors may be +# used to endorse or promote products derived from this software without +# specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, +# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR +# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF +# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +include $(top_srcdir)/common-rules.mk + +noinst_PROGRAMS = simple + +simple_SOURCES = \ + simple.c \ + ../proc-protocol.c \ + ../proc-protocol.h \ + $(NULL) +simple_CPPFLAGS = \ + -I$(srcdir)/.. \ + -I$(top_srcdir)/common/include \ + -I$(top_srcdir)/common/utils \ + $(NULL) +simple_CFLAGS = $(WARNINGS_CFLAGS) +simple_LDADD = \ + $(top_builddir)/common/utils/libutils.la \ + $(NULL) diff --git a/plugins/proc/proc-protocol.h b/plugins/proc/proc-protocol.h new file mode 100644 index 0000000000..582a004f39 --- /dev/null +++ b/plugins/proc/proc-protocol.h @@ -0,0 +1,81 @@ +/* nbdkit + * Copyright Red Hat + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#ifndef NBDKIT_PROC_PROTOCOL_H +#define NBDKIT_PROC_PROTOCOL_H + +#include <stdbool.h> + +struct proc_list { + char *request_id; /* Request-ID */ + char **payload; /* Request or reply payload. */ + char *buffer; /* Optional binary buffer. */ + unsigned buffer_len; +}; + +/* Free all fields of proc_list and the struct itself. */ +extern void proc_list_free (struct proc_list *); + +/* Read a list of strings and optional binary buffer from the file, + * decoding the binary format that nbdkit-proc-plugin uses. The file + * is usually 'stdin'. + * + * The first element is copied into the 'request_id' field. The + * second element is used to determine if a binary buffer follows the + * input, and if so that is read into 'buffer' + 'buffer_len'. The + * third element of the incoming list is parsed and checked and used + * to determine how many further payload elements to read. The + * subsequent elements are copied into the 'payload' list followed by + * NULL. + * + * This function never returns an error. If there is a problem with + * the input then it exits (see "Security" section in + * nbdkit-proc-plugin(1)). + * + * The caller should call 'proc_list_free' on the return value after + * use. + */ +extern struct proc_list *proc_read_list (FILE *fp); + +/* Write a list of strings and optional binary buffer to the file, + * encoding them in the binary format that nbdkit-proc-plugin uses. + * The file is usually 'stdout. + * + * This is the reverse of 'proc_read_list'. + * + * This function always returns 0. If there is a problem with the + * list of strings or the file, then it exits (see "Security" in + * nbdkit-proc-plugin(1)). + */ +extern int proc_write_list (FILE *fp, const struct proc_list *list); + +#endif /* NBDKIT_PROC_PROTOCOL_H */ diff --git a/plugins/proc/examples/simple.c b/plugins/proc/examples/simple.c new file mode 100644 index 0000000000..f3376be875 --- /dev/null +++ b/plugins/proc/examples/simple.c @@ -0,0 +1,125 @@ +/* nbdkit + * Copyright Red Hat + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +/* A very simple example proc plugin which acts like a RAM disk. + * + * This example won't make any sense unless you read + * nbdkit-proc-plugin(1) and plugins/proc/proc-protocol.h first. + */ + +#include <stdio.h> +#include <stdlib.h> +#include <stdbool.h> +#include <string.h> +#include <assert.h> + +#include "proc-protocol.h" + +static char *disk; +static size_t size; + +static void +ok_reply (const char *request_id) +{ + const struct proc_list reply = { + .request_id = request_id, + .payload = { "0", NULL }, + }; + proc_write_list (stdout, &reply); +} + +static void +error_reply (const char *request_id, const char *errno, const char *error) +{ + const struct proc_list reply = { + .request_id = request_id, + .payload = { "1", errno, error, NULL }, + }; + proc_write_list (stdout, &reply); +} + +static void +missing_command_reply (const char *request_id) +{ + const struct proc_list reply = { + .request_id = request_id, + .payload = { "2", NULL }, + }; + proc_write_list (stdout, &reply); +} + +static void +do_config (const char *request_id, + const char *key, const char *value) +{ + if (strcmp (key, "size") == 0) { + if (sscanf (value, "%zu", &size) != 1) { + error_reply (request_id, "EINVAL", "cannot parse size parameter"); + return; + } + ok_reply (request_id); + return; + } + else { + error_reply (request_id, "EINVAL", "unknown command"); + } +} + +int +main (int argc, char *argv[]) +{ + for (;;) { + struct proc_list *request; + + /* Get the request from nbdkit. */ + request = proc_read_list (stdin); + assert (request->payload[0]); /* proc_read_list ensures this */ + + /* Pick the request command. */ + if (strcmp (request->payload[0], "config") == 0) { + /* Handle .config commands. */ + assert (request->payload[1]); + assert (request->payload[2]); + do_config (request->request_id, + request->payload[1], request->payload[2]); + } + else { + /* It's some other request we don't understand, reply with the + * missing command code. + */ + missing_command_reply (request->request_id); + } + + /* Free up the request message. */ + proc_list_free (request); + } +} diff --git a/plugins/proc/proc-protocol.c b/plugins/proc/proc-protocol.c new file mode 100644 index 0000000000..d11b01cbc1 --- /dev/null +++ b/plugins/proc/proc-protocol.c @@ -0,0 +1,290 @@ +/* nbdkit + * Copyright Red Hat + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <stdio.h> +#include <stdlib.h> +#include <stdbool.h> +#include <stdarg.h> +#include <limits.h> +#include <errno.h> + +#include "string-vector.h" +#include "nbdkit-string.h" + +#include "proc-protocol.h" + +/* If IN_PROC_PLUGIN is defined then we are running inside the plugin + * (ie. in nbdkit) and unlike what the documentation says, we do + * return an error on failures. On errors (fatal or not) we call + * nbdkit_error, either the one defined in nbdkit, or a replacement + * supplied here. + */ +#if IN_PROC_PLUGIN +extern void nbdkit_error (const char *msg, ...); +#else /* !IN_PROC_PLUGIN */ +void +nbdkit_error (const char *msg, ...) +{ + va_list args; + va_start (args, msg); + vfprintf (stderr, msg, args); + va_end (args); + fprintf (stderr, "\n"); +} +#endif /* !IN_PROC_PLUGIN */ + +void +proc_list_free (struct proc_list *list) +{ + size_t i; + + if (list) { + free (list->request_id); + for (i = 0; list->payload[i] != NULL; ++i) + free (list->payload[i]); + free (list->payload); + free (list->buffer); + free (list); + } +} + +static int +read_a_string (FILE *fp, string *ret) +{ + ret->len = 0; + while (ret->len <= 4096 /* allow it to grow to 4096+1 */) { + int c = fgetc (fp); + if (c == EOF) { + nbdkit_error ("proc-protocol: unexpected end of input"); + return -1; + } + if (string_append (ret, (char)c) == -1) { + nbdkit_error ("realloc: %m"); + return -1; + } + if (c == '\0') + return 0; + } + nbdkit_error ("proc-protocol: input string exceeded 4096 bytes"); + return -1; +} + +/* Try to copy what nbdkit_parse_unsigned does, although this is more strict. */ +static int +parse_unsigned (const char *str, unsigned *u) +{ + unsigned long r; + char *end; + + if (str[0] < '0' || str[0] > '9') { + nbdkit_error ("proc-protocol: parse_unsigned: " + "first character is not a number"); + return -1; + } + errno = 0; + r = strtoul (str, &end, 0); +#if UINT_MAX != ULONG_MAX + if (r > UINT_MAX) + errno = ERANGE; +#endif + if (errno != 0) { + nbdkit_error ("proc-protocol: parse_unsigned: " + "could not parse number: \"%s\": %m", str); + return -1; + } + if (end == str) { + nbdkit_error ("proc-protocol: parse_unsigned: " + "empty string where we expected a number"); + return -1; + } + if (*end) { + nbdkit_error ("proc-protocol: parse_unsigned: " + "could not parse number: \"%s\": trailing garbage", str); + return -1; + } + *u = r; + return 0; +} + +static int +read_buffer (FILE *fp, string *ret, unsigned len) +{ + ret->len = 0; + if (string_reserve (ret, len) == -1) { + nbdkit_error ("realloc: %m"); + return -1; + } + if (fread (ret, 1, len, fp) < len) { + nbdkit_error ("proc-protocol: unexpected end of input"); + return -1; + } + ret->len = len; + return 0; +} + +struct proc_list * +proc_read_list (FILE *fp) +{ + struct proc_list *list; + string input = empty_vector; + string_vector payload = empty_vector; + unsigned payload_len, buffer_len, i; + + list = calloc (1, sizeof *list); + if (list == NULL) { + nbdkit_error ("calloc: %m"); + goto error; + } + + /* Read the request-ID. */ + if (read_a_string (fp, &input) == -1) + goto error; + list->request_id = input.ptr; + input = (string) empty_vector; + + /* Read the binary buffer length. */ + if (read_a_string (fp, &input) == -1) + goto error; + if (parse_unsigned (input.ptr, &buffer_len) == -1) + goto error; + if (buffer_len > 67108864) { + nbdkit_error ("proc-protocol: buffer length is too long"); + goto error; + } + + /* Read the payload length. */ + if (read_a_string (fp, &input) == -1) + goto error; + if (parse_unsigned (input.ptr, &payload_len) == -1) + goto error; + if (payload_len < 1 || payload_len > 64) { + nbdkit_error ("proc-protocol: " + "payload length must be in range [1..64]"); + goto error; + } + + /* Read the payload elements. */ + if (string_vector_reserve (&payload, payload_len + 1 /* + NULL */) == -1) { + nbdkit_error ("realloc: %m"); + goto error; + } + for (i = 0; i < payload_len; ++i) { + if (read_a_string (fp, &input) == -1) + goto error; + payload.ptr[i] = input.ptr; + input = (string) empty_vector; + } + payload.ptr[payload_len] = NULL; + list->payload = payload.ptr; + string_vector_reset (&payload); + + /* Read the optional binary buffer. */ + if (buffer_len > 0 && + read_buffer (fp, &input, buffer_len) == -1) + goto error; + list->buffer = input.ptr; + list->buffer_len = buffer_len; + input = (string) empty_vector; + + return list; + + error: + if (list) proc_list_free (list); + string_reset (&input); + string_vector_reset (&payload); +#if IN_PROC_PLUGIN + return NULL; +#else + exit (EXIT_FAILURE); +#endif +} + +static int +write_buffer (FILE *fp, const void *data, size_t len) +{ + if (fwrite (data, 1, len, fp) < len) { + nbdkit_error ("proc-protocol: write_buffer: %m"); + return -1; + } + return 0; +} + +static int +write_a_string (FILE *fp, const char *str) +{ + return write_buffer (fp, str, strlen (str) + 1 /* also write \0 */); +} + +int +proc_write_list (FILE *fp, const struct proc_list *list) +{ + size_t i; + unsigned len; + char str[64]; + + /* Request-ID. */ + if (write_a_string (fp, list->request_id) == -1) + goto error; + + /* Optional binary buffer length. */ + len = list->buffer_len; + snprintf (str, sizeof str, "%u", len); + if (write_a_string (fp, str) == -1) + goto error; + + /* Number of payload elements. */ + for (len = 0; list->payload[len] != NULL; ++len) + ; + snprintf (str, sizeof str, "%u", len); + if (write_a_string (fp, str) == -1) + goto error; + + /* Payload. */ + for (i = 0; i < len; ++i) { + if (write_a_string (fp, list->payload[i]) == -1) + goto error; + } + + /* Optional binary buffer. */ + if (list->buffer_len > 0 && + write_buffer (fp, list->buffer, list->buffer_len) == -1) + goto error; + + return 0; + + error: +#if IN_PROC_PLUGIN + return -1; +#else + exit (EXIT_FAILURE); +#endif +} diff --git a/plugins/proc/proc.c b/plugins/proc/proc.c new file mode 100644 index 0000000000..5a444fc599 --- /dev/null +++ b/plugins/proc/proc.c @@ -0,0 +1,314 @@ +/* nbdkit + * Copyright Red Hat + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <stdbool.h> +#include <inttypes.h> +#include <string.h> +#include <unistd.h> +#include <errno.h> +#include <sys/stat.h> + +#define NBDKIT_API_VERSION 2 + +#include <nbdkit-plugin.h> + +#include "cleanup.h" +#include "vector.h" + +#include "methods.h" +#include "subplugin.h" +#include "tmpdir.h" + +static char *script; +static char *magic_config_key; + +static const char *get_script (const char *method); + +static exit_code invoke (const char **argv) + __attribute__ ((__nonnull__ (1))); +static exit_code invoke_read (string *rbuf, const char **argv) + __attribute__ ((__nonnull__ (1, 2))); +static exit_code invoke_write (const char *wbuf, size_t wbuflen, + const char **argv) + __attribute__ ((__nonnull__ (1, 3))); + +/* This abstracts the nbdkit-proc-plugin sub-plugin. */ +struct subplugin sub = { + get_script, + invoke, + invoke_read, + invoke_write, +}; + +static const char * +get_script (const char *method) +{ + return script; /* It's never actually used. */ +} + +static exit_code +invoke (const char **argv) +{ + XXX; +} + +static exit_code +invoke_read (string *rbuf, const char **argv) +{ + XXX; +} + +static exit_code +invoke_write (const char *wbuf, size_t wbuflen, + const char **argv) +{ + XXX; +} + +static void +proc_load (void) +{ + tmpdir_load (); +} + +static void +proc_unload (void) +{ + const char *method = "unload"; + + /* Run the unload method. Ignore all errors. */ + if (script) { + const char *args[] = { script, method, NULL }; + + sub.call (args); + } + + tmpdir_unload (); + free (script); + free (magic_config_key); +} + +static int +proc_config (const char *key, const char *value) +{ + if (!script) { + /* The first parameter MUST be "script". */ + if (strcmp (key, "script") != 0) { + nbdkit_error ("the first parameter must be script=/path/to/script"); + return -1; + } + + script = nbdkit_realpath (value); + if (script == NULL) + return -1; + + /* Call the load method. */ + const char *args[] = { script, "load", NULL }; + switch (sub.call (args)) { + case OK: + case MISSING: + break; + + case ERROR: + return -1; + + case RET_FALSE: + nbdkit_error ("%s: %s method returned unexpected code (3/false)", + script, "load"); + errno = EIO; + return -1; + + default: abort (); + } + + /* Call the magic_config_key method if it exists. */ + const char *args2[] = { script, "magic_config_key", NULL }; + CLEANUP_FREE_STRING string s = empty_vector; + switch (sub.call_read (&s, args2)) { + case OK: + if (s.len > 0 && s.ptr[s.len-1] == '\n') + s.ptr[s.len-1] = '\0'; + magic_config_key = strdup (s.ptr); + if (magic_config_key == NULL) { + nbdkit_error ("strdup: %m"); + return -1; + } + break; + + case MISSING: + break; + + case ERROR: + return -1; + + case RET_FALSE: + nbdkit_error ("%s: %s method returned unexpected code (3/false)", + script, "magic_config_key"); + errno = EIO; + return -1; + + default: abort (); + } + } + else { + /* If the script sets a magic_config_key then it's possible that + * we will be called here with key == "script" (which is the + * plugin.magic_config_key). If that happens then swap in the + * script magic_config_key as the key. However if the script + * didn't define a magic_config_key then it's an error, emulating + * the behaviour of the core server. + */ + if (strcmp (key, "script") == 0) { + if (magic_config_key) + key = magic_config_key; + else { + nbdkit_error ("%s: expecting key=value on the command line but got: " + "%s\n", + script, value); + return -1; + } + } + + const char *args[] = { script, "config", key, value, NULL }; + switch (sub.call (args)) { + case OK: + return 0; + + case MISSING: + /* Emulate what core nbdkit does if a config callback is NULL. */ + nbdkit_error ("%s: this plugin does not need command line configuration", + script); + return -1; + + case ERROR: + return -1; + + case RET_FALSE: + nbdkit_error ("%s: %s method returned unexpected code (3/false)", + script, "config"); + errno = EIO; + return -1; + + default: abort (); + } + } + + return 0; +} + +static int +proc_config_complete (void) +{ + const char *args[] = { script, "config_complete", NULL }; + + if (!script) { + nbdkit_error ("missing script parameter"); + return -1; + } + + switch (sub.call (args)) { + case OK: + case MISSING: + return 0; + + case ERROR: + return -1; + + case RET_FALSE: + nbdkit_error ("%s: %s method returned unexpected code (3/false)", + script, "config_complete"); + errno = EIO; + return -1; + + default: abort (); + } +} + +#define proc_config_help \ + "script=<FILENAME> (required) The shell script to run.\n" \ + "[other arguments may be used by the plugin that you load]" + +/* Default to simple mode. External programs can override this. */ +#define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS + +static struct nbdkit_plugin plugin = { + .name = "proc", + .version = PACKAGE_VERSION, + .load = proc_load, + .unload = proc_unload, + + .dump_plugin = sh_dump_plugin, + + .config = proc_config, + .config_complete = proc_config_complete, + .config_help = proc_config_help, + .magic_config_key = "script", + .thread_model = sh_thread_model, + .get_ready = sh_get_ready, + .after_fork = sh_after_fork, + + .preconnect = sh_preconnect, + .list_exports = sh_list_exports, + .default_export = sh_default_export, + .open = sh_open, + .close = sh_close, + + .export_description = sh_export_description, + .get_size = sh_get_size, + .block_size = sh_block_size, + .can_write = sh_can_write, + .can_flush = sh_can_flush, + .is_rotational = sh_is_rotational, + .can_trim = sh_can_trim, + .can_zero = sh_can_zero, + .can_extents = sh_can_extents, + .can_fua = sh_can_fua, + .can_multi_conn = sh_can_multi_conn, + .can_cache = sh_can_cache, + .can_fast_zero = sh_can_fast_zero, + + .pread = sh_pread, + .pwrite = sh_pwrite, + .flush = sh_flush, + .trim = sh_trim, + .zero = sh_zero, + .extents = sh_extents, + .cache = sh_cache, + + .errno_is_preserved = 1, +}; + +NBDKIT_REGISTER_PLUGIN (plugin) diff --git a/.gitignore b/.gitignore index a05231b859..c59cc5f262 100644 --- a/.gitignore +++ b/.gitignore @@ -106,6 +106,8 @@ plugins/*/*.3 /plugins/golang/examples/*/nbdkit-*-plugin.so /plugins/ocaml/nbdkit-ocamlexample-plugin.so /plugins/ondemand/default-command.c +/plugins/proc/methods.c +/plugins/proc/tmpdir.c /plugins/rust/Cargo.lock /plugins/rust/target /plugins/S3/nbdkit-S3-plugin -- 2.44.0 _______________________________________________ Libguestfs mailing list -- guestfs@lists.libguestfs.org To unsubscribe send an email to guestfs-le...@lists.libguestfs.org
