Re: [PATCH 3/3] nbd: Add 'qemu-nbd -A' to expose allocation depth

2020-09-28 Thread Eric Blake

On 9/26/20 8:32 AM, Vladimir Sementsov-Ogievskiy wrote:

25.09.2020 23:32, Eric Blake wrote:

Allow the server to expose an additional metacontext to be requested
by savvy clients.  qemu-nbd adds a new option -A to expose the
qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this
can also be set via QMP when using nbd-server-add.

qemu as client can be hacked into viewing this new context by using
the now-misnamed x-dirty-bitmap option when creating an NBD blockdev;


may be rename it to x-block-status ?


I thought about it.  We don't promise any back-compat with x- prefix, 
but at the same time, if there's not a strong reason to change it, 
scripts written against older qemu will not break as long as we still 
have the hack.





although it is worth noting the decoding of how such context
information will appear in 'qemu-img map --output=json':

NBD_STATE_DEPTH_UNALLOC => "zero":false, "data":true
NBD_STATE_DEPTH_LOCAL => "zero":false, "data":false
NBD_STATE_DEPTH_BACKING => "zero":true, "data":true


It wouldn't be so simple if we decide to export exact depth number..


It's still simple to do tri-state results if we separate the exact depth 
number to bits 31-4, leaving bits 1-0 as the tri-state summary...


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org




Re: [PATCH 3/3] nbd: Add 'qemu-nbd -A' to expose allocation depth

2020-09-26 Thread Vladimir Sementsov-Ogievskiy

25.09.2020 23:32, Eric Blake wrote:

Allow the server to expose an additional metacontext to be requested
by savvy clients.  qemu-nbd adds a new option -A to expose the
qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this
can also be set via QMP when using nbd-server-add.

qemu as client can be hacked into viewing this new context by using
the now-misnamed x-dirty-bitmap option when creating an NBD blockdev;


may be rename it to x-block-status ?


although it is worth noting the decoding of how such context
information will appear in 'qemu-img map --output=json':

NBD_STATE_DEPTH_UNALLOC => "zero":false, "data":true
NBD_STATE_DEPTH_LOCAL => "zero":false, "data":false
NBD_STATE_DEPTH_BACKING => "zero":true, "data":true


It wouldn't be so simple if we decide to export exact depth number..


--
Best regards,
Vladimir



Re: [PATCH 3/3] nbd: Add 'qemu-nbd -A' to expose allocation depth

2020-09-26 Thread Richard W.M. Jones
On Fri, Sep 25, 2020 at 03:32:49PM -0500, Eric Blake wrote:
> Allow the server to expose an additional metacontext to be requested
> by savvy clients.  qemu-nbd adds a new option -A to expose the
> qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this
> can also be set via QMP when using nbd-server-add.
> 
> qemu as client can be hacked into viewing this new context by using
> the now-misnamed x-dirty-bitmap option when creating an NBD blockdev;
> although it is worth noting the decoding of how such context
> information will appear in 'qemu-img map --output=json':
> 
> NBD_STATE_DEPTH_UNALLOC => "zero":false, "data":true
> NBD_STATE_DEPTH_LOCAL => "zero":false, "data":false
> NBD_STATE_DEPTH_BACKING => "zero":true, "data":true
> 
> libnbd as client is probably a nicer way to get at the information
> without having to decipher such hacks in qemu as client. ;)

I've been meaning to add extents information to nbdinfo, or
perhaps a new tool ("nbdmap").

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html




[PATCH 3/3] nbd: Add 'qemu-nbd -A' to expose allocation depth

2020-09-25 Thread Eric Blake
Allow the server to expose an additional metacontext to be requested
by savvy clients.  qemu-nbd adds a new option -A to expose the
qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this
can also be set via QMP when using nbd-server-add.

qemu as client can be hacked into viewing this new context by using
the now-misnamed x-dirty-bitmap option when creating an NBD blockdev;
although it is worth noting the decoding of how such context
information will appear in 'qemu-img map --output=json':

NBD_STATE_DEPTH_UNALLOC => "zero":false, "data":true
NBD_STATE_DEPTH_LOCAL => "zero":false, "data":false
NBD_STATE_DEPTH_BACKING => "zero":true, "data":true

libnbd as client is probably a nicer way to get at the information
without having to decipher such hacks in qemu as client. ;)

Signed-off-by: Eric Blake 
---
 docs/tools/qemu-nbd.rst|  6 
 qapi/block-core.json   |  7 ++--
 include/block/nbd.h|  7 ++--
 blockdev-nbd.c |  3 +-
 nbd/server.c   |  8 +++--
 qemu-nbd.c | 16 ++---
 tests/qemu-iotests/309 | 73 ++
 tests/qemu-iotests/309.out | 22 
 tests/qemu-iotests/group   |  1 +
 9 files changed, 129 insertions(+), 14 deletions(-)
 create mode 100755 tests/qemu-iotests/309
 create mode 100644 tests/qemu-iotests/309.out

diff --git a/docs/tools/qemu-nbd.rst b/docs/tools/qemu-nbd.rst
index 667861cb22e9..0e545a97cfa3 100644
--- a/docs/tools/qemu-nbd.rst
+++ b/docs/tools/qemu-nbd.rst
@@ -72,6 +72,12 @@ driver options if ``--image-opts`` is specified.

   Export the disk as read-only.

+.. option:: -A, --allocation-depth
+
+  Expose allocation depth information via the
+  ``qemu:allocation-depth`` context accessible through
+  NBD_OPT_SET_META_CONTEXT.
+
 .. option:: -B, --bitmap=NAME

   If *filename* has a qcow2 persistent bitmap *NAME*, expose
diff --git a/qapi/block-core.json b/qapi/block-core.json
index b3ec30a83cd7..84f7a7a34dc1 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3874,9 +3874,12 @@
 #
 # @tls-creds: TLS credentials ID
 #
-# @x-dirty-bitmap: A "qemu:dirty-bitmap:NAME" string to query in place of
+# @x-dirty-bitmap: A metacontext name such as "qemu:dirty-bitmap:NAME" or
+#  "qemu:allocation-depth" to query in place of the
 #  traditional "base:allocation" block status (see
-#  NBD_OPT_LIST_META_CONTEXT in the NBD protocol) (since 3.0)
+#  NBD_OPT_LIST_META_CONTEXT in the NBD protocol; and
+#  yes, naming this option x-context would have made
+#  more sense) (since 3.0)
 #
 # @reconnect-delay: On an unexpected disconnect, the nbd client tries to
 #   connect again until succeeding or encountering a serious
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 71a2623f7842..f660e0274ce3 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -336,9 +336,10 @@ typedef struct NBDClient NBDClient;

 NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
   uint64_t size, const char *name, const char *desc,
-  const char *bitmap, bool readonly, bool shared,
-  void (*close)(NBDExport *), bool writethrough,
-  BlockBackend *on_eject_blk, Error **errp);
+  bool alloc_depth, const char *bitmap, bool readonly,
+  bool shared, void (*close)(NBDExport *),
+  bool writethrough, BlockBackend *on_eject_blk,
+  Error **errp);
 void nbd_export_close(NBDExport *exp);
 void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp);
 void nbd_export_get(NBDExport *exp);
diff --git a/blockdev-nbd.c b/blockdev-nbd.c
index 1a95d89f0096..562ea3751b85 100644
--- a/blockdev-nbd.c
+++ b/blockdev-nbd.c
@@ -203,7 +203,8 @@ void qmp_nbd_server_add(BlockExportNbd *arg, Error **errp)
 arg->writable = false;
 }

-exp = nbd_export_new(bs, 0, len, arg->name, arg->description, arg->bitmap,
+exp = nbd_export_new(bs, 0, len, arg->name, arg->description,
+ arg->alloc, arg->bitmap,
  !arg->writable, !arg->writable,
  NULL, false, on_eject_blk, errp);
 if (!exp) {
diff --git a/nbd/server.c b/nbd/server.c
index 02d5fb375b24..f5e0e115b703 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -1450,9 +1450,10 @@ static void nbd_eject_notifier(Notifier *n, void *data)

 NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset,
   uint64_t size, const char *name, const char *desc,
-  const char *bitmap, bool readonly, bool shared,
-  void (*close)(NBDExport *), bool writethrough,
-  BlockBackend *on_eject_blk, Error **errp)
+  bool alloc_depth, const