[Libguestfs] [PATCH nbdkit] python: Drop support for Python 2.
Starting with nbdkit 1.16, Python >= 3.3 will be required.
Python 2 reaches end of life on 2020-01-01:
https://python3statement.org/
https://pythonclock.org/
Debian oldoldstable and RHEL 7 have Python 3.4 and 3.6 respectively,
so it seems pointless to try to support Python < 3.3 which lacked
support for PyUnicode_AsUTF8.
---
README | 13 +++--
configure.ac| 36 ++--
plugins/python/python.c | 31 +--
3 files changed, 22 insertions(+), 58 deletions(-)
diff --git a/README b/README
index 187da49..9752e0b 100644
--- a/README
+++ b/README
@@ -125,8 +125,7 @@ For the Perl, example4 and tar plugins:
For the Python plugin:
- - python interpreter
- (either version 2 or 3 may be used)
+ - python interpreter (version 3 only)
- python development libraries
@@ -201,12 +200,14 @@ Optionally run this command as root to install everything:
Python
--
+Since nbdkit >= 1.16, only Python >= 3.3 is supported.
+
By default nbdkit uses the Python version of the Python interpreter
-called “python” on the current $PATH. To use another version of
-Python you may need to set the PYTHON variable when configuring. For
-example:
+called “python” on the current $PATH. If you have parallel versions
+of Python installed then you can choose a different version by setting
+the PYTHON variable when configuring. For example:
-./configure PYTHON=/usr/bin/python3
+./configure PYTHON=/usr/bin/python3.8
Running the tests
-
diff --git a/configure.ac b/configure.ac
index d326377..e9d2b1f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -470,7 +470,7 @@ AC_SUBST([PERL_ARCHLIB])
AC_SUBST([PERL_CFLAGS])
AC_SUBST([PERL_LDOPTS])
-dnl Check for Python, for embedding in the python plugin.
+dnl Check for Python 3, for embedding in the python plugin.
AC_CHECK_PROG([PYTHON],[python],[python],[no])
AC_ARG_ENABLE([python],
[AS_HELP_STRING([--disable-python], [disable Python embed plugin])],
@@ -488,6 +488,19 @@ AS_IF([test "x$PYTHON" != "xno" && test "x$enable_python"
!= "xno"],[
enable_python=no
])
+AC_MSG_CHECKING([Python major version is 3])
+AS_IF([test "x$PYTHON_VERSION_MAJOR" = "x3"],[
+AC_MSG_RESULT([yes])
+],[
+AC_MSG_RESULT([no])
+AC_MSG_ERROR([Python $PYTHON_VERSION_MAJOR <> 3 is no longer supported.
+
+Python 2 end of life is 2020-01-01 and nbdkit >= 1.16 no longer
+supports it.
+
+If you want to use Python 2, you will need to use nbdkit 1.14.])
+])
+
dnl Check for Python CFLAGS, libraries.
dnl For Python >= 3.8 we have to use python--embed.pc, see:
dnl
https://docs.python.org/3.8/whatsnew/3.8.html#debug-build-uses-the-same-abi-as-release-build
@@ -515,27 +528,6 @@ AS_IF([test "x$PYTHON" != "xno" && test "x$enable_python"
!= "xno"],[
AC_MSG_WARN([python $PYTHON_VERSION not found])
enable_python=no
])])])
-
-dnl Check for various functions needed by the bindings.
-old_LIBS="$LIBS"
-
-PYTHON_BLDLIBRARY=`$PYTHON -c "import distutils.sysconfig; \
-print (distutils.sysconfig.get_config_var('BLDLIBRARY'))"`
-AC_CHECK_LIB([c],[PyString_FromString],
- [AC_DEFINE([HAVE_PYSTRING_FROMSTRING],1,
-[Found PyString_FromString in libpython.])],
- [],[$PYTHON_BLDLIBRARY])
-AC_CHECK_LIB([c],[PyString_AsString],
- [AC_DEFINE([HAVE_PYSTRING_ASSTRING],1,
-[Found PyString_AsString in libpython.])],
- [],[$PYTHON_BLDLIBRARY])
-AC_CHECK_LIB([c],[PyUnicode_AsUTF8],
- [AC_DEFINE([HAVE_PYUNICODE_ASUTF8],1,
-[Found PyUnicode_AsUTF8 in libpython.])],
- [],[$PYTHON_BLDLIBRARY])
-
-LIBS="$old_LIBS"
-
])
AM_CONDITIONAL([HAVE_PYTHON],[test "x$enable_python" != "xno" && test
"x$PYTHON" != "xno"])
AC_SUBST([PYTHON_CFLAGS])
diff --git a/plugins/python/python.c b/plugins/python/python.c
index 20232f4..2e1a53f 100644
--- a/plugins/python/python.c
+++ b/plugins/python/python.c
@@ -117,17 +117,9 @@ static char *
python_to_string (PyObject *str)
{
if (str) {
-#ifdef HAVE_PYUNICODE_ASUTF8
if (PyUnicode_Check (str))
return strdup (PyUnicode_AsUTF8 (str));
-else
-#endif
-#ifdef HAVE_PYSTRING_ASSTRING
-if (PyString_Check (str))
- return strdup (PyString_AsString (str));
-else
-#endif
-if (PyBytes_Check (str))
+else if (PyBytes_Check (str))
return strdup (PyBytes_AS_STRING (str));
}
return NULL;
@@ -159,11 +151,7 @@ print_python_traceback (const char *callback,
*traceback_str;
CLEANUP_FREE char *traceback_cstr = NULL;
-#ifdef HAVE_PYSTRING_FROMSTRING
- module_name = PyString_FromString ("traceback");
-#else
module_name = PyUnicode_FromString ("traceback");
-#endif
traceback_module = PyImport_Import (module_name);
Py_DECREF (module_name);
@@
[Libguestfs] [PATCH nbdkit] python: Drop support for Python 2.
This patch proposes to drop support for Python 2 in nbdkit. Rather than abruptly drop it everywhere, my proposal is that we point people to nbdkit 1.14 (the current stable version) if they want to continue with Python 2 plugins, while gently reminding them of the upcoming Python 2.7 end of life announcement. Libnbd never supported Python 2. Libguestfs in theory supports Python 2 but I dropped the bindings from Fedora back in March so it's basically never tested now. Libvirt also started to prefer Python 3 over Python 2 back in March. Rich. ___ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
Re: [Libguestfs] [PATCH nbdkit] tests: Convert some tests to use nbdsh instead of qemu-io.
On Wed, Sep 11, 2019 at 02:57:50PM +0100, Richard W.M. Jones wrote: > On Wed, Sep 11, 2019 at 08:48:28AM -0500, Eric Blake wrote: > > On 9/11/19 5:40 AM, Richard W.M. Jones wrote: > > > Very much a work in progress as there are still many tests using > > > qemu-io which are candidates for conversion. > > > > > > You'll notice at the end of test-full.sh that the new test has some > > > duplicated code which looks as if it ought to be refactored into a > > > Python function. When I tried to do that, I got loads of strange > > > Python problems which may indicate bugs in nbdsh itself or problems > > > with my understanding of how Python works, but I wasn't sure how to > > > fix it. The current implementation works. > > > > I'm not sure if I'd spot the problem, but I'd least need to see your > > alternative that didn't work before I can tell for sure :) > > Attached is my alternative (on top of the current patch), which seems > to me straightforward, but actually fails with the attached error. > > I tried variations inserting ‘global h’ in various places but couldn't > make it work. > > Also note there are two related errors, because ‘nbd’ somehow get > "unimported" from the function too. > > Needs help from someone who knows more Python :-) This turned out to be a bug in nbdsh: https://github.com/libguestfs/libnbd/commit/d6cbd130101add28431bd6e67aa2ea0430a9234e Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW ___ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
Re: [Libguestfs] [PATCH nbdkit] python: Drop support for Python 2.
On 9/11/19 2:42 PM, Richard W.M. Jones wrote: > Starting with nbdkit 1.16, Python >= 3.3 will be required. > > Python 2 reaches end of life on 2020-01-01: > https://python3statement.org/ > https://pythonclock.org/ > > Debian oldoldstable and RHEL 7 have Python 3.4 and 3.6 respectively, > so it seems pointless to try to support Python < 3.3 which lacked > support for PyUnicode_AsUTF8. > --- > README | 13 +++-- > configure.ac| 36 ++-- > plugins/python/python.c | 31 +-- > 3 files changed, 22 insertions(+), 58 deletions(-) Progress! Reminds me that I should revisit the patches I originally wrote around Apr 2018 to bump the python plugin to v2 API (but at the time, I had not tested it on python 3, because I was still on a machine using python 2). Those patches may be easier to revive now that my main dev machine is on a newer Fedora version where python3 is now default, and where I don't have to worry about testing the patches for python2 support. ACK. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature ___ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
[Libguestfs] [nbdkit PATCH 2/2] server: Add --mask-handshake option for integration testing
Similar to --no-sr, it can be handy for testing a client
implementation to have a server that can easily be forced into older
behaviors, without having to recompile a one-off hack into a server or
dig up an older server binary that lacked a newer feature.
To see the patch in action, try things like:
$ ./nbdkit -U - -fv --mask-handshake=0 null \
--run 'qemu-nbd --list -k $unixsocket'
Signed-off-by: Eric Blake
---
docs/nbdkit-protocol.pod | 25 ++-
docs/synopsis.txt| 2 +-
server/internal.h| 1 +
server/options.h | 2 ++
server/main.c| 30 ++--
server/protocol-handshake-newstyle.c | 4 ++--
6 files changed, 45 insertions(+), 19 deletions(-)
diff --git a/docs/nbdkit-protocol.pod b/docs/nbdkit-protocol.pod
index 3ae89063..272f4e5b 100644
--- a/docs/nbdkit-protocol.pod
+++ b/docs/nbdkit-protocol.pod
@@ -4,8 +4,8 @@ nbdkit - which parts of the NBD protocol nbdkit supports
=head1 SYNOPSIS
- nbdkit [-n|--newstyle] [--no-sr] [-o|--oldstyle] [-e|--exportname EXPORTNAME]
-[...]
+ nbdkit [-n|--newstyle] [--mask-handshake MASK] [--no-sr] [-o|--oldstyle]
+[-e|--exportname EXPORTNAME] [...]
=head1 DESCRIPTION
@@ -24,15 +24,30 @@ newstyle protocol is better in every respect than the
oldstyle
protocol and you should prefer it if possible. The newstyle protocol
also includes an extension where a client may request structured
replies for even more capabilities, such as sparse reads or obtaining
-block status.
+block status. By default, nbdkit advertises as many features as it
+can support (in some cases, this can be limited by what callbacks the
+plugin handles), even if the client does not negotiate to use all
+advertised features.
Use the I<-e> or I<--exportname> flag to set the optional exportname
for the newstyle protocol.
-Use the I<--no-sr> flag to force the newstyle protocol to decline any
-client request for structured replies.
+Nbdkit also includes some options that are useful mainly when
+performing integration tests, for proving whether clients have sane
+fallback behavior when dealing various older servers permitted by the
+NBD protocol. Use the I<--no-sr> flag to force the newstyle protocol
+to decline any client request for structured replies. Use the
+I<--mask-handshake> parameter to mask off particular global features
+which are advertised during new-style handshake (defaulting to all
+supported bits set). Clearing bit 0 (the low order bit) limits a
+client to using just C (and is incompatible with
+TLS or structured replies); clearing bit 1 causes the handshake to
+send more padding bytes in response to C. Other
+bits in the mask will only have an effect if the NBD protocol is
+extended in the future to define other global bits.
Use the I<-o> or I<--oldstyle> flag to force the oldstyle protocol.
+In this mode, I<--no-sr> and I<--mask-handshake> have no effect.
=head2 Common clients and the protocol they require
diff --git a/docs/synopsis.txt b/docs/synopsis.txt
index 04cd136d..5fc57fd1 100644
--- a/docs/synopsis.txt
+++ b/docs/synopsis.txt
@@ -3,7 +3,7 @@ nbdkit [-D|--debug PLUGIN|FILTER.FLAG=N]
[--filter FILTER ...] [-f|--foreground]
[-g|--group GROUP] [-i|--ipaddr IPADDR]
[--log stderr|syslog]
- [-n|--newstyle] [--no-sr] [-o|--oldstyle]
+ [-n|--newstyle] [--mask-handshake MASK] [--no-sr] [-o|--oldstyle]
[-P|--pidfile PIDFILE]
[-p|--port PORT] [-r|--readonly]
[--run CMD] [-s|--single] [--selinux-label LABEL]
diff --git a/server/internal.h b/server/internal.h
index 9314e8ff..5da3e3c3 100644
--- a/server/internal.h
+++ b/server/internal.h
@@ -90,6 +90,7 @@ extern const char *exportname;
extern bool foreground;
extern const char *ipaddr;
extern enum log_to log_to;
+extern unsigned mask_handshake;
extern bool newstyle;
extern bool no_sr;
extern const char *port;
diff --git a/server/options.h b/server/options.h
index a69f413a..c74e0b8b 100644
--- a/server/options.h
+++ b/server/options.h
@@ -46,6 +46,7 @@ enum {
FILTER_OPTION,
LOG_OPTION,
LONG_OPTIONS_OPTION,
+ MASK_HANDSHAKE_OPTION,
NO_SR_OPTION,
RUN_OPTION,
SELINUX_LABEL_OPTION,
@@ -74,6 +75,7 @@ static const struct option long_options[] = {
{ "ipaddr", required_argument, NULL, 'i' },
{ "log", required_argument, NULL, LOG_OPTION },
{ "long-options", no_argument, NULL, LONG_OPTIONS_OPTION },
+ { "mask-handshake", required_argument, NULL, MASK_HANDSHAKE_OPTION },
{ "new-style",no_argument, NULL, 'n' },
{ "newstyle", no_argument, NULL, 'n' },
{ "no-sr",no_argument, NULL, NO_SR_OPTION },
diff --git a/server/main.c b/server/main.c
index 22cf8d33..d433c1fa 100644
--- a/server/main.c
+++ b/server/main.c
@@ -67,6 +67,7 @@ const char *exportname; /* -e */
bool foreground;/*
[Libguestfs] [nbdkit PATCH 1/2] server: Skip option haggling from client lacking fixed newstyle
The NBD protocol states that servers may still choose to honor various
NBD_OPT_* from a client that did not reply with
NBD_FLAG_C_FIXED_NEWSTYLE; however, for integration testing purposes,
it's a lot nicer if we reject everything except NBD_OPT_EXPORT_NAME
from such a client (for example, with this in place, we might have
spotted the bug fixed in commit e03b34d6 a bit sooner). Thus, a
client that does not claim to understand fixed newstyle can now no
longer trigger TLS, structured replies, meta contexts, or the nicer
handling of NBD_OPT_GO. All well-known clients listed in
nbdkit-protocol.pod default to requesting fixed newstyle, so this
shouldn't affect normal usage.
Signed-off-by: Eric Blake
---
server/protocol-handshake-newstyle.c | 10 ++
1 file changed, 10 insertions(+)
diff --git a/server/protocol-handshake-newstyle.c
b/server/protocol-handshake-newstyle.c
index 9ddc3198..486d416f 100644
--- a/server/protocol-handshake-newstyle.c
+++ b/server/protocol-handshake-newstyle.c
@@ -259,6 +259,16 @@ negotiate_handshake_newstyle_options (struct connection
*conn)
option = be32toh (new_option.option);
optname = name_of_nbd_opt (option);
+/* If the client lacks fixed newstyle support, it should only send
+ * NBD_OPT_EXPORT_NAME.
+ */
+if (!(conn->cflags & NBD_FLAG_FIXED_NEWSTYLE) &&
+option != NBD_OPT_EXPORT_NAME) {
+ if (send_newstyle_option_reply (conn, option, NBD_REP_ERR_INVALID))
+return -1;
+ continue;
+}
+
/* In --tls=require / FORCEDTLS mode the only options allowed
* before TLS negotiation are NBD_OPT_ABORT and NBD_OPT_STARTTLS.
*/
--
2.21.0
___
Libguestfs mailing list
Libguestfs@redhat.com
https://www.redhat.com/mailman/listinfo/libguestfs
[Libguestfs] [nbdkit PATCH 0/2] Make client fallback testing easier
This is similar to the recent --no-sr option - it's a change that is unlikely to ever be used except by someone testing whether a client is compliant to the protocol, but in that niche case, it can be quite handy (it's a lot nicer to be able to purposefully cripple a server from the command line than from a one-off compile, when testing if a client's fallback for a spec-compliant but older server is correct). I'm planning on writing a counterpart patch for libnbd to allow the user to mask out bits that the client does not reply with (so you can choose to cripple the protocol from either the server or the client side, to see how the other side reacts, similar to the recently-added nbd_set_request_structured_replies). I'm open to naming suggestions on the command-line option. Eric Blake (2): server: Skip option haggling from client lacking fixed newstyle server: Add --mask-handshake option for integration testing docs/nbdkit-protocol.pod | 25 ++- docs/synopsis.txt| 2 +- server/internal.h| 1 + server/options.h | 2 ++ server/main.c| 30 ++-- server/protocol-handshake-newstyle.c | 14 +++-- 6 files changed, 55 insertions(+), 19 deletions(-) -- 2.21.0 ___ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
[Libguestfs] [libnbd PATCH] nbdsh: Start adding unit tests
Ultimately, I'd like to add a unit test covering our recent fix for
handling globals from consecutive -c, but let's start with a simple
test that --help output is sane. Sanity includes a mention of the man
page for more details (similar to nbdkit --help, although I couldn't
figure out how to make python's argparse output that sentence last).
---
python/nbdsh.py | 3 ++-
sh/Makefile.am | 9 ++---
sh/test-help.sh | 32
3 files changed, 40 insertions(+), 4 deletions(-)
create mode 100755 sh/test-help.sh
diff --git a/python/nbdsh.py b/python/nbdsh.py
index 553e632..117c824 100644
--- a/python/nbdsh.py
+++ b/python/nbdsh.py
@@ -23,7 +23,8 @@ def shell():
import nbd
-description = '''Network Block Device (NBD) shell'''
+description = '''Network Block Device (NBD) shell. Please read the
+nbdsh(1) manual page for full usage.'''
parser = argparse.ArgumentParser (prog='nbdsh', description=description)
parser.add_argument ('--connect',
help="connect to NBD URI")
diff --git a/sh/Makefile.am b/sh/Makefile.am
index 2e04e46..2e9a047 100644
--- a/sh/Makefile.am
+++ b/sh/Makefile.am
@@ -21,6 +21,7 @@ EXTRA_DIST = \
nbdsh.pod \
examples/LICENSE-FOR-EXAMPLES \
examples/hexdump.sh \
+ test-help.sh \
$(NULL)
if HAVE_PYTHON
@@ -40,8 +41,10 @@ nbdsh.1: nbdsh.pod $(top_builddir)/podwrapper.pl
endif HAVE_POD
-if HAVE_NBDKIT
-
TESTS_ENVIRONMENT = LIBNBD_DEBUG=1
LOG_COMPILER = $(top_builddir)/run
-TESTS =
+TESTS = test-help.sh
+
+if HAVE_NBDKIT
+
+TESTS +=
endif HAVE_NBDKIT
diff --git a/sh/test-help.sh b/sh/test-help.sh
new file mode 100755
index 000..6ed1c9c
--- /dev/null
+++ b/sh/test-help.sh
@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+# nbd client library in userspace
+# Copyright (C) 2019 Red Hat Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+
+# Test that nbdsh --help looks sane.
+
+fail=0
+output=$(nbdsh --help)
+if [ $? != 0 ]; then
+echo "$0: unexpected exit status"
+fail=1
+fi
+if [[ ! ( "$output" =~ "usage:" && "$output" =~ "manual page" ) ]]; then
+echo "$0: unexpected output"
+fail=1
+fi
+echo "$output"
+exit $fail
--
2.21.0
___
Libguestfs mailing list
Libguestfs@redhat.com
https://www.redhat.com/mailman/listinfo/libguestfs
[Libguestfs] [PATCH nbdkit] tests: Convert some tests to use nbdsh instead of qemu-io.
nbdsh has some advantages over qemu-io:
- scriptable
- allows us to more finely control NBD commands, such as
making subsector-sized requests and controlling how
many commands are sent on the wire
- can write controlled patterns
- can read NBD export flags
---
README | 2 ++
tests/Makefile.am | 5 +++--
tests/test-error0.sh | 13 +++--
tests/test-error100.sh | 19 ---
tests/test-full.sh | 32 +++-
5 files changed, 47 insertions(+), 24 deletions(-)
diff --git a/README b/README
index b78f490..187da49 100644
--- a/README
+++ b/README
@@ -164,6 +164,8 @@ For non-essential enhancements to the test suite:
- mke2fs (from e2fsprogs)
+ - nbdsh (from libnbd)
+
- qemu-img, qemu-io, qemu-nbd (usually shipped with qemu)
- sfdisk (from util-linux)
diff --git a/tests/Makefile.am b/tests/Makefile.am
index b581cf6..b5806bb 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -355,12 +355,13 @@ if HAVE_LIBGUESTFS
check_PROGRAMS += $(LIBGUESTFS_TESTS)
TESTS += $(LIBGUESTFS_TESTS)
-# Use the 'direct' backend, and ensure maximum libguestfs debugging is
-# written to the *.log files in case there is a problem.
+# Use the 'direct' backend, and ensure maximum libguestfs and libnbd
+# debugging is written to the *.log files in case there is a problem.
TESTS_ENVIRONMENT += \
LIBGUESTFS_ATTACH_METHOD=appliance \
LIBGUESTFS_DEBUG=1 \
LIBGUESTFS_TRACE=1 \
+ LIBNBD_DEBUG=1 \
$(NULL)
# Common test library.
diff --git a/tests/test-error0.sh b/tests/test-error0.sh
index fe97354..de9dca2 100755
--- a/tests/test-error0.sh
+++ b/tests/test-error0.sh
@@ -34,7 +34,7 @@ source ./functions.sh
set -e
set -x
-requires qemu-io --version
+requires nbdsh --version
sock=`mktemp -u`
files="$sock error0.pid"
@@ -47,8 +47,9 @@ start_nbdkit -P error0.pid -U $sock \
pattern 1G error-rate=0%
# Because error rate is 0%, reads should never fail.
-qemu-io -r -f raw "nbd+unix://?socket=$sock" \
--c "r 0M 10M" \
--c "r 20M 10M" \
--c "r 40M 10M" \
--c "r 60M 10M"
+nbdsh --connect "nbd+unix://?socket=$sock" \
+ -c 'mbytes = 2**20' \
+ -c 'h.pread(10*mbytes, 0)' \
+ -c 'h.pread(10*mbytes, 20*mbytes)' \
+ -c 'h.pread(10*mbytes, 40*mbytes)' \
+ -c 'h.pread(10*mbytes, 60*mbytes)'
diff --git a/tests/test-error100.sh b/tests/test-error100.sh
index 64056ce..a2d3bdc 100755
--- a/tests/test-error100.sh
+++ b/tests/test-error100.sh
@@ -34,7 +34,7 @@ source ./functions.sh
set -e
set -x
-requires qemu-io --version
+requires nbdsh --version
sock=`mktemp -u`
files="$sock error100.pid"
@@ -46,11 +46,16 @@ start_nbdkit -P error100.pid -U $sock \
--filter=error \
pattern 1G error-rate=100%
-# The error rate is 100% so every operation must fail.
+# The error rate is 100% so every operation must fail with error EIO.
for i in {1..100}; do
-if qemu-io -r -f raw "nbd+unix://?socket=$sock" \
- -c "r 0 512"; then
-echo "$0: expected qemu-io command to fail"
-exit 1
-fi
+nbdsh --connect "nbd+unix://?socket=$sock" \
+ -c '
+try:
+h.pread (512, 0)
+# This should not happen.
+exit (1)
+except nbd.Error as ex:
+# Check the errno is expected.
+assert ex.errno == "EIO"
+'
done
diff --git a/tests/test-full.sh b/tests/test-full.sh
index 47bff5e..8f237dd 100755
--- a/tests/test-full.sh
+++ b/tests/test-full.sh
@@ -36,7 +36,7 @@
source ./functions.sh
set -e
-requires qemu-io --version
+requires nbdsh --version
sock=`mktemp -u`
files="full.pid $sock full.out"
@@ -47,13 +47,27 @@ cleanup_fn rm -f $files
start_nbdkit -P full.pid -U $sock full 1M
# All reads should succeed.
-qemu-io -f raw "nbd+unix://?socket=$sock" \
--c 'r -v 0 512' \
--c 'r -v 512 512' \
--c 'r -v 1048064 512'
+nbdsh --connect "nbd+unix://?socket=$sock" \
+ -c 'h.pread (512, 0)' \
+ -c 'h.pread (512, 512)' \
+ -c 'h.pread (512, 1048064)'
# All writes should fail with the ENOSPC error.
-! LANG=C qemu-io -f raw "nbd+unix://?socket=$sock" \
--c 'w -P 1 0 512' \
--c 'w -P 2 1048064 512' >& full.out
-grep "No space left on device" full.out
+nbdsh --connect "nbd+unix://?socket=$sock" \
+ -c '
+try:
+h.pwrite (bytearray (512), 0)
+# This should not happen.
+exit (1)
+except nbd.Error as ex:
+# Check the errno is expected.
+assert ex.errno == "ENOSPC"
+
+try:
+h.pwrite (bytearray (512), 1048064)
+# This should not happen.
+exit (1)
+except nbd.Error as ex:
+# Check the errno is expected.
+assert ex.errno == "ENOSPC"
+'
--
2.23.0
___
Libguestfs mailing list
Libguestfs@redhat.com
https://www.redhat.com/mailman/listinfo/libguestfs
[Libguestfs] [PATCH nbdkit] tests: Convert some tests to use nbdsh instead of qemu-io.
Very much a work in progress as there are still many tests using qemu-io which are candidates for conversion. You'll notice at the end of test-full.sh that the new test has some duplicated code which looks as if it ought to be refactored into a Python function. When I tried to do that, I got loads of strange Python problems which may indicate bugs in nbdsh itself or problems with my understanding of how Python works, but I wasn't sure how to fix it. The current implementation works. Rich. ___ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
Re: [Libguestfs] [PATCH nbdkit] tests: Convert some tests to use nbdsh instead of qemu-io.
On Wed, Sep 11, 2019 at 08:48:28AM -0500, Eric Blake wrote: > On 9/11/19 5:40 AM, Richard W.M. Jones wrote: > > Very much a work in progress as there are still many tests using > > qemu-io which are candidates for conversion. > > > > You'll notice at the end of test-full.sh that the new test has some > > duplicated code which looks as if it ought to be refactored into a > > Python function. When I tried to do that, I got loads of strange > > Python problems which may indicate bugs in nbdsh itself or problems > > with my understanding of how Python works, but I wasn't sure how to > > fix it. The current implementation works. > > I'm not sure if I'd spot the problem, but I'd least need to see your > alternative that didn't work before I can tell for sure :) Attached is my alternative (on top of the current patch), which seems to me straightforward, but actually fails with the attached error. I tried variations inserting ‘global h’ in various places but couldn't make it work. Also note there are two related errors, because ‘nbd’ somehow get "unimported" from the function too. Needs help from someone who knows more Python :-) Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top >From d88957e9572e69490e109526f90574fe8589 Mon Sep 17 00:00:00 2001 From: "Richard W.M. Jones" Date: Wed, 11 Sep 2019 14:54:49 +0100 Subject: [PATCH] tests/test-full.sh: Refactor to tests into a single function (NOT WORKING). --- tests/test-full.sh | 24 ++-- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/tests/test-full.sh b/tests/test-full.sh index 8f237dd..f060f4f 100755 --- a/tests/test-full.sh +++ b/tests/test-full.sh @@ -55,19 +55,15 @@ nbdsh --connect "nbd+unix://?socket=$sock" \ # All writes should fail with the ENOSPC error. nbdsh --connect "nbd+unix://?socket=$sock" \ -c ' -try: -h.pwrite (bytearray (512), 0) -# This should not happen. -exit (1) -except nbd.Error as ex: -# Check the errno is expected. -assert ex.errno == "ENOSPC" +def test (offset): +try: +h.pwrite (bytearray (512), offset) +# This should not happen. +exit (1) +except nbd.Error as ex: +# Check the errno is expected. +assert ex.errno == "ENOSPC" -try: -h.pwrite (bytearray (512), 1048064) -# This should not happen. -exit (1) -except nbd.Error as ex: -# Check the errno is expected. -assert ex.errno == "ENOSPC" +test (0) +test (1048064) ' -- 2.23.0 /home/rjones/d/nbdkit/server/nbdkit -v -P full.pid -U /tmp/tmp.sHet2Luahj -- /home/rjones/d/nbdkit/plugins/full/.libs/nbdkit-full-plugin.so 1M nbdkit: debug: TLS disabled: could not load TLS certificates nbdkit: debug: registering plugin /home/rjones/d/nbdkit/plugins/full/.libs/nbdkit-full-plugin.so nbdkit: debug: registered plugin /home/rjones/d/nbdkit/plugins/full/.libs/nbdkit-full-plugin.so (name full) nbdkit: debug: full: load nbdkit: debug: full: config key=size, value=1M nbdkit: debug: full: config_complete nbdkit: debug: using thread model: parallel nbdkit: debug: bound to unix socket /tmp/tmp.sHet2Luahj nbdkit: debug: forked into background (new pid = 422301) nbdkit: debug: written pidfile /home/rjones/d/nbdkit/tests/full.pid libnbd: debug: nbd1: nbd_create: opening handle libnbd: debug: nbd1: nbd_close: closing handle libnbd: debug: nbd2: nbd_create: opening handle libnbd: debug: nbd2: nbd_close: closing handle libnbd: debug: nbd3: nbd_create: opening handle libnbd: debug: nbd3: nbd_close: closing handle libnbd: debug: nbd4: nbd_create: opening handle libnbd: debug: nbd4: nbd_close: closing handle libnbd: debug: nbd5: nbd_create: opening handle libnbd: debug: nbd5: nbd_connect_uri: enter: uri="nbd+unix://?socket=/tmp/tmp.sHet2Luahj" libnbd: debug: nbd5: nbd_connect_uri: event CmdConnectUnix: START -> CONNECT_UNIX.START libnbd: debug: nbd5: nbd_connect_uri: transition: CONNECT_UNIX.START -> CONNECT.START libnbd: debug: nbd5: nbd_connect_uri: poll start: events=4 libnbd: debug: nbd5: nbd_connect_uri: poll end: r=1 revents=4 libnbd: debug: nbd5: nbd_connect_uri: event NotifyWrite: CONNECT.START -> CONNECT.CONNECTING libnbd: debug: nbd5: nbd_connect_uri: transition: CONNECT.CONNECTING -> MAGIC.START libnbd: debug: nbd5: nbd_connect_uri: transition: MAGIC.START -> MAGIC.RECV_MAGIC libnbd: debug: nbd5: nbd_connect_uri: poll start: events=1 nbdkit: debug: accepted connection nbdkit: full[1]: debug: newstyle negotiation: flags: global 0x3 libnbd: debug: nbd5: nbd_connect_uri: poll end: r=1 revents=1 libnbd: debug: nbd5: nbd_connect_uri: transition: MAGIC.RECV_MAGIC -> MAGIC.CHECK_MAGIC libnbd: debug: nbd5: nbd_connect_uri: transition: MAGIC.CHECK_MAGIC -> NEWSTYLE.START libnbd:
Re: [Libguestfs] [PATCH nbdkit] tests: Convert some tests to use nbdsh instead of qemu-io.
On 9/11/19 5:40 AM, Richard W.M. Jones wrote: > Very much a work in progress as there are still many tests using > qemu-io which are candidates for conversion. > > You'll notice at the end of test-full.sh that the new test has some > duplicated code which looks as if it ought to be refactored into a > Python function. When I tried to do that, I got loads of strange > Python problems which may indicate bugs in nbdsh itself or problems > with my understanding of how Python works, but I wasn't sure how to > fix it. The current implementation works. I'm not sure if I'd spot the problem, but I'd least need to see your alternative that didn't work before I can tell for sure :) -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature ___ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
Re: [Libguestfs] [PATCH nbdkit] tests: Convert some tests to use nbdsh instead of qemu-io.
On 9/11/19 5:40 AM, Richard W.M. Jones wrote: > nbdsh has some advantages over qemu-io: > > - scriptable > > - allows us to more finely control NBD commands, such as >making subsector-sized requests and controlling how >many commands are sent on the wire > > - can write controlled patterns > > - can read NBD export flags - can more easily get at exact errno response returned by server > # Because error rate is 0%, reads should never fail. > -qemu-io -r -f raw "nbd+unix://?socket=$sock" \ > --c "r 0M 10M" \ > --c "r 20M 10M" \ > --c "r 40M 10M" \ > --c "r 60M 10M" > +nbdsh --connect "nbd+unix://?socket=$sock" \ > + -c 'mbytes = 2**20' \ > + -c 'h.pread(10*mbytes, 0)' \ > + -c 'h.pread(10*mbytes, 20*mbytes)' \ > + -c 'h.pread(10*mbytes, 40*mbytes)' \ > + -c 'h.pread(10*mbytes, 60*mbytes)' A bit annoying that qemu-io and libnbd picked opposite ordering for length vs. offset, but such is life. LGTM. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature ___ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
