Re: [Libguestfs] [libvirt] Support for qemu snapshot=on drives in libvirt

2012-10-05 Thread Eric Blake
implement something ... We definitely need to tie it into the XML that has already been designated for this purpose: transient/. -- Eric Blake ebl...@redhat.com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature

Re: [Libguestfs] [libvirt] Support for qemu snapshot=on drives in libvirt

2012-10-05 Thread Eric Blake
On 10/05/2012 06:29 AM, Eric Blake wrote: On 10/05/2012 05:53 AM, Richard W.M. Jones wrote: And looking even more closely, I see disk-transient (not implemented in the qemu driver ... why?) Lack of developer time :) [My usual excuse?] By the way, this is: https://bugzilla.redhat.com

Re: [Libguestfs] [libvirt] Support for qemu snapshot=on drives in libvirt

2012-10-05 Thread Eric Blake
On 10/05/2012 05:53 AM, Richard W.M. Jones wrote: And looking even more closely, I see disk-transient (not implemented in the qemu driver ... why?) Lack of developer time :) [My usual excuse?] -- Eric Blake ebl...@redhat.com+1-919-301-3266 Libvirt virtualization library http

Re: [Libguestfs] Request to relicense hash gnulib module to LGPLv2+

2013-09-13 Thread Eric Blake
of not having to involve the FSF. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature ___ Libguestfs mailing list Libguestfs@redhat.com https

Re: [Libguestfs] [Qemu-devel] Why is virt-resize designed to involve two disks?

2014-09-23 Thread Eric Blake
that virt-resize is doing MUCH more than resizing the block device - it is also copying entire filesystems from the old location to the new resized location for optimal initial placement. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org

Re: [Libguestfs] Point-in-time snapshots

2015-03-27 Thread Eric Blake
because the guest hasn't touched it in the meantime; or because even though the guest touched it after the point in time, you were given the original contents through your NBD target). -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org

Re: [Libguestfs] Point-in-time snapshots

2015-03-27 Thread Eric Blake
reads take priority over linear streaming of dirty blocks, but it might help. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature ___ Libguestfs

[Libguestfs] [PATCH] protocol: Ignore rest of option when replying with error

2016-05-20 Thread Eric Blake
We must consume the payload of any option received, whether or not we can answer the option successfully, in order to be in sync for reading the next option. This is particularly true once clients start using NBD_OPT_GO, as long as we don't happen to service that option. Signed-off-by: Eric

Re: [Libguestfs] [PATCH v3 libguestfs] launch: Implement a safer getumask.

2016-04-15 Thread Eric Blake
algrind is somehow calling lseek on stdin, wiping out the buffer that it previously read. Since it doesn't happen when the program is run standalone, I'm thinking this is a valgrind bug. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signatu

Re: [Libguestfs] [PATCH] Add safe wrapper around waitpid which deals with EINTR correctly.

2016-04-14 Thread Eric Blake
On 04/14/2016 11:26 AM, Richard W.M. Jones wrote: > Thanks: Eric Blake. > --- > src/Makefile.am| 1 + > src/command.c | 7 ++ > src/guestfs-internal.h | 4 +++ > src/launch-direct.c| 11 +++-- > src/launch-uml.c | 11 +++-- > src

Re: [Libguestfs] [PATCH v3 libguestfs] launch: Implement a safer getumask.

2016-04-14 Thread Eric Blake
On 04/14/2016 08:11 AM, Richard W.M. Jones wrote: > On Thu, Apr 14, 2016 at 08:04:39AM -0600, Eric Blake wrote: >> On 04/14/2016 07:57 AM, Richard W.M. Jones wrote: >>> On Thu, Apr 14, 2016 at 07:38:23AM -0600, Eric Blake wrote: >>>>> + /* Read the umask. */ &

Re: [Libguestfs] [PATCH v3 libguestfs] launch: Implement a safer getumask.

2016-04-14 Thread Eric Blake
On 04/14/2016 07:57 AM, Richard W.M. Jones wrote: > On Thu, Apr 14, 2016 at 07:38:23AM -0600, Eric Blake wrote: >>> + /* Read the umask. */ >>> + if (read (fd[0], , sizeof mask) != sizeof mask) { >>> +perrorf (g, "read"); >>> +close (fd[0

Re: [Libguestfs] [PATCH v3 libguestfs] launch: Implement a safer getumask.

2016-04-14 Thread Eric Blake
1). Return a plain int > instead. > > Thanks: Josh Stone, Jiri Jaburek, Eric Blake. > --- > +guestfs_int_getumask (guestfs_h *g) > +{ > + if (pid == 0) { > +/* The child process must ONLY call async-safe functions. */ > +close (fd[0]); > + > +/* umask

Re: [Libguestfs] [PATCH libguestfs] launch: Implement a safer getumask.

2016-04-13 Thread Eric Blake
1). Return a plain int > instead. > > Thanks: Josh Stone, Jiri Jaburek, Eric Blake. > --- > +/** > + * glibc documents, but does not actually implement, a L<getumask(3)> > + * call. > + * > + * This function implements an expensive, but thread-safe way to get > + * t

Re: [Libguestfs] [PATCH v2 libguestfs] launch: Implement a safer getumask.

2016-04-13 Thread Eric Blake
ther you successfully write the umask to the parent and exit with status 0, or you fail and exit with status non-zero. Yeah, you've lost the ability to report the errno of why the write() failed, but what are you really losing in that EXTREMELY unlikely sc

Re: [Libguestfs] [PATCH libguestfs] launch: Implement a safer getumask.

2016-04-13 Thread Eric Blake
On 04/13/2016 03:53 PM, Eric Blake wrote: >> +mask = umask (0); >> +if (mask == -1) { >> + perror ("umask"); > > perror() is NOT async-safe (it manipulates stdio, and could permanently > wedgelock the child if you happened to fork() while som

[Libguestfs] [nbdkit PATCH 1/2] ruby: Expose nbdkit_set_error to ruby script

2017-02-02 Thread Eric Blake
, so that a user can write any of: err = Errno::EINVAL::Errno Nbdkit.set_error(err) Nbdkit.set_error(Errno::EINVAL) begin #... rescue SystemCallError => err Nbdkit.set_error(err) end Signed-off-by: Eric Blake <ebl...@redhat.com> --- plugins/ruby/nbdkit-ruby-plugin

[Libguestfs] [nbdkit PATCH 0/2] Ruby bindings for .zero

2017-02-02 Thread Eric Blake
Similar to python and perl. But MUCH easier (especially considering that this is the first time I've every tried to run Ruby). I even had fun making set_error() polymorphic. Eric Blake (2): ruby: Expose nbdkit_set_error to ruby script ruby: Support zero callback plugins/ruby/example.rb

[Libguestfs] [nbdkit PATCH 2/2] ruby: Support zero callback

2017-02-02 Thread Eric Blake
, and to serve as a test of Nbdkit.set_error(). Signed-off-by: Eric Blake <ebl...@redhat.com> --- plugins/ruby/example.rb | 11 +++ plugins/ruby/nbdkit-ruby-plugin.pod | 20 plugins/ruby/ruby.c | 28 3 files c

[Libguestfs] [nbdkit PATCH] tests: Test in-tree rather than installed header

2017-02-01 Thread Eric Blake
The previous commit broke the build during 'make check'; we want to test the in-tree header, not some (potentially stale or missing) installed header. Signed-off-by: Eric Blake <ebl...@redhat.com> --- tests/Makefile.am | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a

Re: [Libguestfs] [PATCH 1/2] Define .errno_is_preserved constant instead of a .errno_is_reliable callback.

2017-02-06 Thread Eric Blake
d against newer headers). Thus, such plugins will now fail with EIO where they used to fail with reasonable errno values. But I guess we can live with that. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Descrip

[Libguestfs] [nbdkit PATCH 1/2] perl: Fix previous patches

2017-01-31 Thread Eric Blake
Avoid calling nbdkit_set_error with a possibly stale value from a previous API call; and fix inadvertent hard TABs and malformed POD text. Signed-off-by: Eric Blake <ebl...@redhat.com> --- docs/nbdkit-plugin.pod | 2 +- plugins/perl/perl.c| 9 + 2 files changed, 6 insertions

[Libguestfs] [nbdkit PATCH 2/2] build: Kill build on POD error

2017-01-31 Thread Eric Blake
. Lame, but it merely means that someone developing on RHEL 5 won't catch pod errors until they rebuild on a newer platform. Signed-off-by: Eric Blake <ebl...@redhat.com> --- .gitignore| 2 ++ configure.ac | 10 +- docs/Makefile.am

[Libguestfs] [nbdkit PATCH 0/2] fix pod and other errors in recent patches

2017-01-31 Thread Eric Blake
POD errors are not nice, and 'pod2man --stderr' does not do what we want. Eric Blake (2): perl: Fix previous patches build: Kill build on POD error .gitignore| 2 ++ configure.ac | 10 +- docs/Makefile.am | 8 ++-- docs/nbdkit

[Libguestfs] [nbdkit PATCH] tls: Fix build on RHEL 5

2017-01-31 Thread Eric Blake
Don't assume that errno will be implicitly available. Signed-off-by: Eric Blake <ebl...@redhat.com> --- src/tls.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/tls.c b/src/tls.c index 5d380e3..0e9ea11 100644 --- a/src/tls.c +++ b/src/tls.c @@ -38,6 +38,7 @@ #include #i

Re: [Libguestfs] [nbdkit PATCH 0/2] bind .zero to more languages

2017-01-24 Thread Eric Blake
t would be worth exploring how easy it would be to let the plugin's error message be reflected through to the client. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature

Re: [Libguestfs] [nbdkit PATCH 0/2] bind .zero to more languages

2017-01-24 Thread Eric Blake
till marked as an experimental feature rather an official part of the NBD spec in part because there is no implementation yet (but I'm trying to get qemu to be the first such implementation, and getting nbdkit to do it as well to prove interoperability can't hurt). -- Eric Blake eblake redhat

[Libguestfs] [nbdkit PATCH 1/2] perl: Support zero callback

2017-01-23 Thread Eric Blake
the binding can turn it into EOPNOTSUPP), so success requires returning true. Signed-off-by: Eric Blake <ebl...@redhat.com> --- plugins/perl/nbdkit-perl-plugin.pod | 25 + plugins/perl/perl.c | 37 + 2 files chang

[Libguestfs] [nbdkit PATCH 2/2] python: Support zero callback

2017-01-23 Thread Eric Blake
so the binding can turn it into EOPNOTSUPP), so success requires returning true. Signed-off-by: Eric Blake <ebl...@redhat.com> --- plugins/python/nbdkit-python-plugin.pod | 18 + plugins/python/python.c | 35 + 2 files chang

[Libguestfs] [nbdkit PATCH 0/2] bind .zero to more languages

2017-01-23 Thread Eric Blake
with those. I'm less familiar with ruby and ocaml, so I've left those for tomorrow (it will rely heavily on copy-and-paste); it is also a chance to review if my decision of requiring a return value in order to trigger the graceful fallback is the best approach. Eric Blake (2): perl: Support zero

Re: [Libguestfs] [nbdkit PATCH v2 4/6] plugins: Add new nbdkit_set_error() utility function

2017-01-26 Thread Eric Blake
mmit the second half of this series. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature ___ Libguestfs mailing list Libguestfs@redhat.com https:/

Re: [Libguestfs] [nbdkit PATCH 2/5] protocol: Validate request flags

2017-01-26 Thread Eric Blake
On 01/26/2017 04:18 AM, Richard W.M. Jones wrote: > On Wed, Jan 25, 2017 at 08:55:18PM -0600, Eric Blake wrote: >> On 01/20/2017 02:16 PM, Eric Blake wrote: >>> Reject rather than silently ignoring unknown client request flags. >>> >> >>> &g

Re: [Libguestfs] [nbdkit PATCH v2 4/6] plugins: Add new nbdkit_set_error() utility function

2017-01-26 Thread Eric Blake
On 01/26/2017 04:13 AM, Richard W.M. Jones wrote: > On Wed, Jan 25, 2017 at 08:42:34PM -0600, Eric Blake wrote: >> +eg. NULL or -1. If the call to C is omitted, then >> +the value of C will be used instead. > [...] >> +/* Grab the appropriate error value. >> + */

Re: [Libguestfs] [nbdkit PATCH v3 1/4] plugins: Don't use bogus errno from non-C plugins

2017-01-27 Thread Eric Blake
nbdkit_error ("%s", caml_format_exception (Extract_exception (rv))); > caml_enter_blocking_section (); > + errno = saved_errno; > CAMLreturnT (int, -1); Okay, I'll respin the patch along these lines. -- Eric Blake eblake redhat com+1-919-301-3266 Libvi

[Libguestfs] [nbdkit PATCH v3 4/4] python: Support zero callback

2017-01-26 Thread Eric Blake
, and to serve as a test of nbdkit.set_error(). Signed-off-by: Eric Blake <ebl...@redhat.com> --- v2: rebase to .errno_is_reliable --- plugins/python/example.py | 11 plugins/python/nbdkit-python-plugin.pod | 20 ++ plugins/python/python.c

[Libguestfs] [nbdkit PATCH v3 3/4] python: Expose nbdkit_set_error to python script

2017-01-26 Thread Eric Blake
In addition to calling python functions from C, we want to make script writing easier by exposing C functions to python. For now, just wrap nbdkit_set_error(), as that will be needed for an optimal implementation of a zero() callback. Signed-off-by: Eric Blake <ebl...@redhat.com> --- v2:

[Libguestfs] [nbdkit PATCH v3 0/4] bind .zero to Python

2017-01-26 Thread Eric Blake
This cleans up the existing code base with regards to implicit use of errno from language bindings, then rebases the previous work in python on top of that. I'm still playing with the perl bindings, but got further after reading 'perldoc perlembed'. Eric Blake (4): plugins: Don't use bogus

[Libguestfs] [nbdkit PATCH v3 2/4] plugins: Add new nbdkit_set_error() utility function

2017-01-26 Thread Eric Blake
. Signed-off-by: Eric Blake <ebl...@redhat.com> --- v2: rebase on top of .errno_is_implicit --- docs/nbdkit-plugin.pod | 59 - include/nbdkit-plugin.h | 1 + src/connections.c | 8 +-- src/errors.c| 6 + src/inte

[Libguestfs] [nbdkit PATCH v3 1/4] plugins: Don't use bogus errno from non-C plugins

2017-01-26 Thread Eric Blake
callback which determines whether we can trust errno; defaulting to true for C plugins, where we can trace all code in between to ensure errno is not clobbered, and always false for other language bindings. Signed-off-by: Eric Blake <ebl...@redhat.com> --- docs/nbdkit-plug

[Libguestfs] [nbdkit PATCH v3 1/3] perl: Allow use of modules

2017-01-30 Thread Eric Blake
'perldoc perlembed', the solution is simple: statically incorporate the DynaLoader module into the embedded interpreter. Signed-off-by: Eric Blake <ebl...@redhat.com> --- plugins/perl/perl.c | 12 +++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/plugins/perl/pe

[Libguestfs] [nbdkit PATCH v3 3/3] perl: Support zero callback

2017-01-30 Thread Eric Blake
, and to serve as a test of Nbdkit::set_error(). Signed-off-by: Eric Blake <ebl...@redhat.com> --- plugins/perl/example.pl | 16 + plugins/perl/nbdkit-perl-plugin.pod | 26 + plugins/perl/perl.c | 45 ++

[Libguestfs] [nbdkit PATCH v3 0/3] bind .zero to Perl

2017-01-30 Thread Eric Blake
can commit that patch as-is now and then touch things up further when we actually get set_error and zero working in OCaml. Eric Blake (3): perl: Allow use of modules perl: Expose nbdkit_set_error to perl script perl: Support zero callback plugins/perl/example.pl | 16

[Libguestfs] [nbdkit PATCH v3 2/3] perl: Expose nbdkit_set_error to perl script

2017-01-30 Thread Eric Blake
is very picky: I found that including after , even with no other changes in the file, caused the embedded interpreter to segfault during perl_open(); but listing first worked for me (tested on Fedora 25). Signed-off-by: Eric Blake <ebl...@redhat.com> --- plugins/perl/nbdkit-perl-plugin.po

Re: [Libguestfs] [nbdkit PATCH 0/2] bind .zero to more languages

2017-01-25 Thread Eric Blake
rl5/POSIX.pm line 17. Compilation failed in require at plugins/perl/example.pl line 2. BEGIN failed--compilation aborted at plugins/perl/example.pl line 2. nbdkit: error: plugins/perl/example.pl: one of the required callbacks 'open', 'get_size' or 'pread' is not defined by this Perl script. nbdkit requires t

Re: [Libguestfs] [nbdkit PATCH 2/5] protocol: Validate request flags

2017-01-25 Thread Eric Blake
On 01/20/2017 02:16 PM, Eric Blake wrote: > Reject rather than silently ignoring unknown client request flags. > > > + /* Validate flags */ > + if (flags & ~NBD_CMD_FLAG_FUA) { > +nbdkit_error ("invalid request: unknown flag (0x%x)", flags); > +

[Libguestfs] [nbdkit PATCH v2 6/6] python: Support zero callback

2017-01-25 Thread Eric Blake
, and to serve as a test of nbdkit.set_error(). Signed-off-by: Eric Blake <ebl...@redhat.com> --- plugins/python/example.py | 11 plugins/python/nbdkit-python-plugin.pod | 20 ++ plugins/python/python.c | 46 + 3

[Libguestfs] [nbdkit PATCH v2 5/6] python: Expose nbdkit_set_error to python script

2017-01-25 Thread Eric Blake
In addition to calling python functions from C, we want to make script writing easier by exposing C functions to python. For now, just wrap nbdkit_set_error(), as that will be needed for an optimal implementation of a zero() callback. Signed-off-by: Eric Blake <ebl...@redhat.com> --- p

[Libguestfs] [nbdkit PATCH v2 2/6] protocol: Map EROFS to EPERM

2017-01-25 Thread Eric Blake
The NBD Protocol requests an EPERM failure on attempts to write to a read-only export. We were internally using EROFS, but our mapping was slamming that to EINVAL. Signed-off-by: Eric Blake <ebl...@redhat.com> --- src/connections.c | 1 + 1 file changed, 1 insertion(+) diff --git

[Libguestfs] [nbdkit PATCH v2 4/6] plugins: Add new nbdkit_set_error() utility function

2017-01-25 Thread Eric Blake
-off-by: Eric Blake <ebl...@redhat.com> --- docs/nbdkit-plugin.pod | 16 +--- include/nbdkit-plugin.h | 3 ++- src/connections.c | 32 +--- src/errors.c| 6 ++ src/internal.h | 2 ++ src/plugins.c

[Libguestfs] [nbdkit PATCH v2 0/6] bind .zero to Python

2017-01-25 Thread Eric Blake
will take a similar approach. Maybe we want to also make nbdkit.error() and nbdkit.debug() callable from Python (so that we have a more complete binding of all the utility functions that C plugins can use); this can be used as a starting point for that. Eric Blake (6): file: Optimize writing zeroes

[Libguestfs] [nbdkit PATCH 2/5] protocol: Validate request flags

2017-01-20 Thread Eric Blake
Reject rather than silently ignoring unknown client request flags. Signed-off-by: Eric Blake <ebl...@redhat.com> --- src/connections.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/connections.c b/src/connections.c index 16c6584..44b7530 100644 ---

[Libguestfs] [nbdkit PATCH 0/5] Add WRITE_ZEROES support

2017-01-20 Thread Eric Blake
, the server benefits from less network traffic from the client. Eric Blake (5): protocol: Support NBD_FLAG_NO_ZEROES protocol: Validate request flags plugins: Add callback for writing zeroes protocol: Implement NBD_CMD_WRITE_ZEROES file: Support punching holes for write zero docs/nbdkit

[Libguestfs] [nbdkit PATCH 3/5] plugins: Add callback for writing zeroes

2017-01-20 Thread Eric Blake
can try to use fallocate(), then gracefully use normal writes if the underlying file system doesn't support what we need. Signed-off-by: Eric Blake <ebl...@redhat.com> --- docs/nbdkit-plugin.pod | 19 +++ include/nbdkit-plugin.h | 1 + src/internal.h | 1

[Libguestfs] [nbdkit PATCH 4/5] protocol: Implement NBD_CMD_WRITE_ZEROES

2017-01-20 Thread Eric Blake
We always advertise this to the client (for writable exports), even when the plugin does not have any optimized implementation, because it allows for more efficient network traffic. Signed-off-by: Eric Blake <ebl...@redhat.com> --- src/connections.c | 26 -

[Libguestfs] [nbdkit PATCH 1/5] protocol: Support NBD_FLAG_NO_ZEROES

2017-01-20 Thread Eric Blake
The upstream NBD protocol allows a reduction in the length of the handshake by both sides agreeing to skip the 124 bytes of padding in the final server reply. Signed-off-by: Eric Blake <ebl...@redhat.com> --- src/connections.c | 14 +++--- src/protocol.h| 1 + 2 files chang

Re: [Libguestfs] [PATCH] daemon: Move the useful 'is_zero' function into common code.

2017-04-19 Thread Eric Blake
ce alignments), by manually checking that the first 16 bytes are zero, then letting memcmp do the rest: size_t i; size_t limit = MIN(size, 16); for (i = 0; i < limit; ++i) if (buffer[i]) return 0; if (size != limit) return !memcmp(buffer, buffer + 16, size - 16); return 1; -- Eric Bl

Re: [Libguestfs] [PATCH v2 1/2] appliance: search all types of appliances for each path separately

2017-05-05 Thread Eric Blake
I mean >> the standard header where #define NULL (0), but as an extra precaution >> I can offer a patch (attached). Not worth worrying about in practice. All modern libc define NULL according to POSIX rules, rather than the weaker C99 rules. And gnulib guarantees the stronger defin

Re: [Libguestfs] [PATCH v2 1/2] appliance: search all types of appliances for each path separately

2017-05-05 Thread Eric Blake
On 05/05/2017 12:18 PM, Eric Blake wrote: > On 05/05/2017 12:07 PM, Richard W.M. Jones wrote: >> Eric, what do you think of Pavel's analysis and/or suggested fix >> below? It seems all too plausible to me unfortunately :-( >> > >>> There NULL is macros whic

Re: [Libguestfs] [PATCH miniexpect 2/2] Add debugging capability at runtime.

2017-10-12 Thread Eric Blake
/3369588/pass-va-list-or-pointer-to-va-list > has a piece of advice on that. Here's a recent explanation on the qemu list, where va_list* is involved, and thus the va_copy is necessary: https://lists.gnu.org/archive/html/qemu-devel/2017-10/msg00171.html -- Eric Blake, Principal Software Engineer

[Libguestfs] [nbdkit PATCH 0/2] Better response to bogus NBD_CMD_READ

2017-11-15 Thread Eric Blake
When facing a malicious client that is sending bogus NBD_CMD_READ, we should make sure that we never end up in a situation where we could try to treat the tail from a command that we diagnosed as bad as being further commands. Eric Blake (2): connections: Report mid-message EOF as fatal

[Libguestfs] [nbdkit PATCH 1/2] connections: Report mid-message EOF as fatal

2017-11-15 Thread Eric Blake
our behavior on the contents of that random payload. Signed-off-by: Eric Blake <ebl...@redhat.com> --- src/connections.c | 30 ++ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/src/connections.c b/src/connections.c index 0cbf54a..d0ef6a5 100644 ---

[Libguestfs] [nbdkit PATCH 2/2] connections: Hang up early on insanely large WRITE requests

2017-11-15 Thread Eric Blake
of sync for further reads, so the simplest option is to just silently disconnect. Signed-off-by: Eric Blake <ebl...@redhat.com> --- src/connections.c | 5 + 1 file changed, 5 insertions(+) diff --git a/src/connections.c b/src/connections.c index d0ef6a5..8dc1925 100644 --- a/src/connect

[Libguestfs] [nbdkit PATCH] connections: Improve error responses

2017-11-15 Thread Eric Blake
return -1. Make these functions return bool instead. And finally, fix a comment typo. Signed-off-by: Eric Blake <ebl...@redhat.com> --- src/connections.c | 53 - 1 file changed, 24 insertions(+), 29 deletions(-) diff --git a/src/connect

[Libguestfs] [nbdkit PATCH] connections: Extract common export flag computation code

2017-11-15 Thread Eric Blake
No need to duplicate maintenance of export flag computation between old and new style handshakes. Signed-off-by: Eric Blake <ebl...@redhat.com> --- src/connections.c | 120 +- 1 file changed, 47 insertions(+), 73 deletions(-) diff --git

Re: [Libguestfs] [PATCH 1/1] Switch from YAJL to Jansson

2017-11-23 Thread Eric Blake
the RFC > behaviour, then change one of the yajl_tests checks accordingly. Which RFC? RFC 4627 stated that only objects or arrays could occur at the top level. But it was superseded by: RFC 7159, which states that ANY json value can be the top level of the grammar. So maybe it's better to se

[Libguestfs] [nbdkit PATCH] nbd: Fix sporadic use-after-free

2017-12-04 Thread Eric Blake
detect. Reported-by: Richard W.M. Jones <rjo...@redhat.com> Fixes: cb189648f11df6c4f7287dcaed4bc856650e2c3b Signed-off-by: Eric Blake <ebl...@redhat.com> --- Perhaps I should also add more comments to the code about transfer of ownership of 'trans' between threads? plugins/nb

Re: [Libguestfs] [nbdkit PATCH] maint: Rebuild nbdkit if nbdkit.in changes

2017-11-16 Thread Eric Blake
On 11/16/2017 05:49 AM, Richard W.M. Jones wrote: > On Wed, Nov 15, 2017 at 10:50:06AM -0600, Eric Blake wrote: >> Tell automake that we want ./nbdkit to be kept up-to-date in the >> developer's tree. Otherwise, after touching nbdkit.in, you would >> have to remember to manu

Re: [Libguestfs] [nbdkit PATCH v2 0/2] add nbd plugin

2017-11-16 Thread Eric Blake
the forwarder about TLS connections after all, someday. (The whole idea of a plugin that lets you add arbitrary features on top of another existing NBD endpoint is cool!) -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org

Re: [Libguestfs] [nbdkit PATCH v2 1/2] nbd: Add new nbd forwarding plugin

2017-11-16 Thread Eric Blake
On 11/14/2017 01:23 PM, Eric Blake wrote: > This is a minimal implementation of an NBD forwarder; it lets us > convert between old and newstyle connections (great if a client > expects one style but the real server only provides the other), > or add TLS safety on top of a server wi

Re: [Libguestfs] [nbdkit PATCH v2 2/2] nbd: Split reading into separate thread

2017-11-16 Thread Eric Blake
On 11/14/2017 01:23 PM, Eric Blake wrote: > In preparation for allowing interleaved response, refactor the > nbd forwarder so that writes are still done from the thread > handling the original request from the client, but all reads are > done by a dedicated reader thread. Control bet

Re: [Libguestfs] [nbdkit PATCH 3/6] connections: Add read/write lock over client I/O

2017-11-17 Thread Eric Blake
On 11/17/2017 02:59 AM, Richard W.M. Jones wrote: > On Thu, Nov 16, 2017 at 09:26:54PM -0600, Eric Blake wrote: >> In preparation for parallel processing, we need to be sure that >> two threads belonging to the same connection cannot interleave >> their I/O except at mes

Re: [Libguestfs] [nbdkit PATCH 1/4] errors: Avoid interleaved errors from parallel threads

2017-11-17 Thread Eric Blake
On 11/17/2017 02:50 AM, Richard W.M. Jones wrote: > On Thu, Nov 16, 2017 at 09:13:55PM -0600, Eric Blake wrote: >> Since we construct our error/debug messages via multiple calls >> to stdio primitives, we are at risk of multiple threads interleaving >> their output if they

Re: [Libguestfs] [nbdkit PATCH 2/4] threadlocal: Copy thread name

2017-11-17 Thread Eric Blake
On 11/16/2017 09:13 PM, Eric Blake wrote: > We can't guarantee what storage duration the caller's request > for a thread name has; and in fact, if the caller uses > plugin_name() for their thread name, then the moment .unload > is called, our threadlocal storage is pointing to la-la-l

Re: [Libguestfs] [nbdkit PATCH] nbd: Add new nbd forwarding plugin

2017-11-11 Thread Eric Blake
On 11/11/2017 08:33 PM, Eric Blake wrote: > This is a minimal implementation of an NBD forwarder; it lets us > convert between old and newstyle connections (great if a client > expects one style but the real server only provides the other), > or add TLS safety on top of a server wi

Re: [Libguestfs] [nbdkit PATCH] nbd: Add new nbd forwarding plugin

2017-11-11 Thread Eric Blake
On 11/11/2017 08:33 PM, Eric Blake wrote: > This is a minimal implementation of an NBD forwarder; it lets us > convert between old and newstyle connections (great if a client > expects one style but the real server only provides the other), > or add TLS safety on top of a server wi

[Libguestfs] [nbdkit PATCH 0/3] various nbdkit patches

2017-11-13 Thread Eric Blake
source block device. Eric Blake (3): maint: Add emacs hint file maint: Add NBDKIT_GDB support to in-tree nbdkit script connections: Don't send NBD_CMD_READ payload on error .dir-locals.el| 1 + nbdkit.in | 9 +++-- src/connections.c | 2 +- 3 files changed, 9 insertions(+), 3

[Libguestfs] [nbdkit PATCH 3/3] connections: Don't send NBD_CMD_READ payload on error

2017-11-13 Thread Eric Blake
The NBD spec was clarified to state that a payload for NBD_CMD_READ must NOT be sent unless the error field is zero (see https://github.com/NetworkBlockDevice/nbd/commit/e6c6fb39); doing this makes us compatible with nbd-server and qemu-nbd. Signed-off-by: Eric Blake <ebl...@redhat.com> --

Re: [Libguestfs] question on nbdkit shutdown behavior

2017-11-14 Thread Eric Blake
On 11/14/2017 04:25 AM, Richard W.M. Jones wrote: > On Mon, Nov 13, 2017 at 12:42:48PM -0600, Eric Blake wrote: >> I'm observing a difference in timing on nbdkit shutdown that is >> dependent on client behavior, and I wonder if that's a bug, but I can't >> figure out where to

[Libguestfs] question on nbdkit shutdown behavior

2017-11-13 Thread Eric Blake
le-plugin.so: unload term2$ qemu-io> r 0 1 read 1/1 bytes at offset 0 1 bytes, 1 ops; 0.0002 sec (4.138 KiB/sec and 4237.2881 ops/sec) Why does current traffic from the client cause the plugin to be torn down faster? Does it matter? -- Eric Blake, Principal Software Engineer Red Hat, Inc.

Re: [Libguestfs] [nbdkit PATCH] nbd: Add new nbd forwarding plugin

2017-11-13 Thread Eric Blake
On 11/11/2017 08:33 PM, Eric Blake wrote: > This is a minimal implementation of an NBD forwarder; it lets us > convert between old and newstyle connections (great if a client > expects one style but the real server only provides the other), > or add TLS safety on top of a server wi

[Libguestfs] [nbdkit PATCH 1/3] maint: Add emacs hint file

2017-11-13 Thread Eric Blake
Let emacs know that this project prefers space rather than TAB indentation. Signed-off-by: Eric Blake <ebl...@redhat.com> --- .dir-locals.el | 1 + 1 file changed, 1 insertion(+) create mode 100644 .dir-locals.el diff --git a/.dir-locals.el b/.dir-locals.el new file mode 100644 index 0

[Libguestfs] [nbdkit PATCH 0/2] Add nbd forwarder test coverage

2017-11-20 Thread Eric Blake
a for-loop of user-controlled depth for how deep you want to nest the forwarding tree, to see where the bottlenecks like in trying to improve nbdkit performance. Eric Blake (2): tests: Prepare for running multiple nbdkits in one test tests: Test nbd forwarder plugin .gitignore| 1

[Libguestfs] [nbdkit PATCH 2/2] tests: Test nbd forwarder plugin

2017-11-20 Thread Eric Blake
-off-by: Eric Blake <ebl...@redhat.com> --- .gitignore| 1 + tests/Makefile.am | 8 tests/test-nbd.c | 119 ++ 3 files changed, 128 insertions(+) create mode 100644 tests/test-nbd.c diff --git a/.gitignore b/.gitignore

[Libguestfs] [nbdkit PATCH 1/2] tests: Prepare for running multiple nbdkits in one test

2017-11-20 Thread Eric Blake
, with the cleanup function destroying servers in LIFO order. To minimize changes to existing tests, existing global variables are changed to track the details of the most-recently created server. A new global sock variable will also prove useful. Signed-off-by: Eric Blake <ebl...@redhat.com> ---

[Libguestfs] [nbdkit PATCH v2 1/8] sockets: Use pipe-to-self to avoid hanging in poll

2017-11-20 Thread Eric Blake
. Signed-off-by: Eric Blake <ebl...@redhat.com> --- src/internal.h | 4 ++-- src/main.c | 19 ++- src/sockets.c | 14 +++--- 3 files changed, 31 insertions(+), 6 deletions(-) diff --git a/src/internal.h b/src/internal.h index 1fc5d69..5953185 100644 --- a/src/inte

[Libguestfs] [nbdkit PATCH v2 7/8] tests: Annotate Makefile.am conditionals

2017-11-20 Thread Eric Blake
We have a lot of nested conditionals controlling which tests to run; I'm about to add more, and it's easier to do correctly if we exploit automake's ability to warn us about improper nesting by annotating which 'if' each 'endif' belongs to. Signed-off-by: Eric Blake <ebl...@redhat.com> ---

[Libguestfs] [nbdkit PATCH v2 4/8] connections: Add thread-safe status indicator

2017-11-20 Thread Eric Blake
; which will come in handy later as it will avoid trying to coordinate a value out of multiple threads. Signed-off-by: Eric Blake <ebl...@redhat.com> --- src/connections.c | 82 +-- 1 file changed, 56 insertions(+), 26 deletions(-) diff

[Libguestfs] [nbdkit PATCH v2 0/8] Support parallel transactions within single connection

2017-11-20 Thread Eric Blake
forwarder wort with the parallel thread model. Eric Blake (8): sockets: Use pipe-to-self to avoid hanging in poll cleanup: Add ACQUIRE_LOCK_FOR_CURRENT_SCOPE() connections: Add read/write lock over client I/O connections: Add thread-safe status indicator connections: Set up thread pool

[Libguestfs] [nbdkit PATCH v2 6/8] core: Add --threads option for supporting true parallel requests

2017-11-20 Thread Eric Blake
emu-io -f raw nbd://localhost:10809/ --trace='nbd_*' \ -c 'aio_read 0 1' -c 'aio_write -P 0x6c 2 2' -c 'aio_flush' If the write completes before the read, then nbdkit was properly handling things in parallel with out-of-order replies. Signed-off-by: Eric Blake <ebl...@redhat.com>

[Libguestfs] [nbdkit PATCH v2 3/8] connections: Add read/write lock over client I/O

2017-11-20 Thread Eric Blake
. Signed-off-by: Eric Blake <ebl...@redhat.com> --- src/connections.c | 151 +- 1 file changed, 82 insertions(+), 69 deletions(-) diff --git a/src/connections.c b/src/connections.c index 27d685e..f779903 100644 --- a/src/connections.c +++

[Libguestfs] [nbdkit PATCH v2 2/8] cleanup: Add ACQUIRE_LOCK_FOR_CURRENT_SCOPE()

2017-11-20 Thread Eric Blake
compiled with cleanup support is immune). Based on an idea in libguestfs, courtesy of Richard Jones. Signed-off-by: Eric Blake <ebl...@redhat.com> --- configure.ac | 7 +++ src/cleanup.c | 8 +++- src/internal.h | 9 + 3 files changed, 15 insertions(+), 9 deletions(-) diff

Re: [Libguestfs] [nbdkit PATCH 3/6] connections: Add read/write lock over client I/O

2017-11-18 Thread Eric Blake
On 11/17/2017 12:28 PM, Eric Blake wrote: >> There's nothing wrong with this patch, but it might be easier to use >> an attribute((cleanup)) handler to deal with the unlocking. See these >> links for how we do it in libguestfs: > > Oh cool! Yes, that looks nicer. Althoug

Re: [Libguestfs] [nbdkit PATCH 3/6] connections: Add read/write lock over client I/O

2017-11-18 Thread Eric Blake
On 11/18/2017 12:54 PM, Eric Blake wrote: > On 11/17/2017 12:28 PM, Eric Blake wrote: >>> There's nothing wrong with this patch, but it might be easier to use >>> an attribute((cleanup)) handler to deal with the unlocking. See these >>> links for how we do it in

[Libguestfs] [nbdkit PATCH] nbd: Silence compiler warning

2017-11-18 Thread Eric Blake
ngs being treated as errors The problem is that gcc can't tell that nbd_reply_raw()'s early return path via nbd_mark_dead() prior to initializing trans will always set err to a negative value. Help it out by manually initializing trans. Signed-off-by: Eric Blake <ebl...@redhat.com> ---

Re: [Libguestfs] [nbdkit PATCH v2 0/8] Support parallel transactions within single connection

2017-11-21 Thread Eric Blake
n't plan to write them this week, so they aren't a reason to hold up an upstream release. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature __

[Libguestfs] [nbdkit PATCH v2 1/4] nbd: Split reading into separate thread

2017-11-21 Thread Eric Blake
the transaction information, coupled with a pipe for the reader thread to send the final status back to the handler thread. Signed-off-by: Eric Blake <ebl...@redhat.com> --- plugins/nbd/nbd.c | 145 -- 1 file changed, 119 insertions(+), 26 del

[Libguestfs] [nbdkit PATCH v2 0/4] enable parallel nbd forwarding

2017-11-21 Thread Eric Blake
With this, I am finally able to get the nbd plugin to do out-of-order responses to the client. Once this series goes in, we should be ready for Rich to cut a release. Eric Blake (4): nbd: Split reading into separate thread nbd: Protect writes with mutex nbd: Enable parallel handling

[Libguestfs] [nbdkit PATCH v2 2/4] nbd: Protect writes with mutex

2017-11-21 Thread Eric Blake
Once we allow parallel threads, we need to ensure that at most one thread is writing to the wrapped NBD server at a time. Reads are already protected from concurrent access by virtue of being limited to a single reader thread. Signed-off-by: Eric Blake <ebl...@redhat.com> --- plugins/nbd

[Libguestfs] [nbdkit PATCH v2 4/4] tests: Test parallel nbd behavior

2017-11-21 Thread Eric Blake
. Signed-off-by: Eric Blake <ebl...@redhat.com> --- tests/Makefile.am | 1 + tests/test-parallel-nbd.sh | 81 ++ 2 files changed, 82 insertions(+) create mode 100755 tests/test-parallel-nbd.sh diff --git a/tests/Makefile.am b/tests/Makefile.am

[Libguestfs] [nbdkit PATCH v2 3/4] nbd: Enable parallel handling

2017-11-21 Thread Eric Blake
quits, we make sure to mop up any remaining in-flight operations. Signed-off-by: Eric Blake <ebl...@redhat.com> --- plugins/nbd/nbd.c | 95 ++- 1 file changed, 59 insertions(+), 36 deletions(-) diff --git a/plugins/nbd/nbd.c b/plugins/nbd

  1   2   3   4   5   6   7   8   9   10   >