Re: [Qemu-devel] [PATCH v4 1/3] block: add bdrv_get_format_alloc_stat format interface

[Markus Armbruster]

Cc: Eric Blake for additional schema review expertise.

Vladimir Sementsov-Ogievskiy  writes:

> The function should collect statistics, about used/unused by top-level
> format driver space (in its .file) and allocation status
> (data/zero/discarded/after-eof) of corresponding areas in this .file.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy 
> ---
>  block.c   | 16 +++
>  include/block/block.h |  3 ++
>  include/block/block_int.h |  2 ++
>  qapi/block-core.json  | 72 
> +++
>  4 files changed, 93 insertions(+)
>
> diff --git a/block.c b/block.c
> index 50ba264143..7d720ae0c2 100644
> --- a/block.c
> +++ b/block.c
> @@ -3407,6 +3407,22 @@ int64_t bdrv_get_allocated_file_size(BlockDriverState 
> *bs)
>  }
>  
>  /**
> + * Collect format allocation info. See BlockFormatAllocInfo definition in
> + * qapi/block-core.json.
> + */

I'd prefer 

   /**
* Collect format allocation info.
* See BlockFormatAllocInfo definition in qapi/block-core.json.
*/

Admittedly a matter of taste.

> +int bdrv_get_format_alloc_stat(BlockDriverState *bs, BlockFormatAllocInfo 
> *bfai)

bdrv_get_format_alloc_info(), please, for symmetry with
BlockFormatAllocInfo.

> +{
> +BlockDriver *drv = bs->drv;
> +if (!drv) {
> +return -ENOMEDIUM;
> +}
> +if (drv->bdrv_get_format_alloc_stat) {
> +return drv->bdrv_get_format_alloc_stat(bs, bfai);
> +}
> +return -ENOTSUP;
> +}
> +
> +/**
>   * Return number of sectors on success, -errno on error.
>   */
>  int64_t bdrv_nb_sectors(BlockDriverState *bs)
> diff --git a/include/block/block.h b/include/block/block.h
> index 9b355e92d8..646376a772 100644
> --- a/include/block/block.h
> +++ b/include/block/block.h
> @@ -335,6 +335,9 @@ typedef enum {
>  
>  int bdrv_check(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode 
> fix);
>  
> +int bdrv_get_format_alloc_stat(BlockDriverState *bs,
> +   BlockFormatAllocInfo *bfai);
> +
>  /* The units of offset and total_work_size may be chosen arbitrarily by the
>   * block driver; total_work_size may change during the course of the 
> amendment
>   * operation */
> diff --git a/include/block/block_int.h b/include/block/block_int.h
> index 8d3724cce6..458c715e99 100644
> --- a/include/block/block_int.h
> +++ b/include/block/block_int.h
> @@ -208,6 +208,8 @@ struct BlockDriver {
>  int64_t (*bdrv_getlength)(BlockDriverState *bs);
>  bool has_variable_length;
>  int64_t (*bdrv_get_allocated_file_size)(BlockDriverState *bs);
> +int (*bdrv_get_format_alloc_stat)(BlockDriverState *bs,
> +  BlockFormatAllocInfo *bfai);
>  
>  int coroutine_fn (*bdrv_co_pwritev_compressed)(BlockDriverState *bs,
>  uint64_t offset, uint64_t bytes, QEMUIOVector *qiov);
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index ea0b3e8b13..93f6995381 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -139,6 +139,78 @@
> '*format-specific': 'ImageInfoSpecific' } }
>  
>  ##
> +# @BlockFormatAllocInfo:
> +#
> +#

Just one blank like, please.

> +# Allocation information of an underlying protocol file, as partitioned by a
> +# format driver's utilization of said allocations.
> +# All fields are in bytes.
> +#
> +# Regions of the underlying protocol file may be considered used or unused by
> +# the format driver interpreting these regions. It is at the discretion of 
> the
> +# format driver (e.g. qcow2) which regions of its backing storage are 
> considered

Long line.  Please wrap your comment lines around column 70 for legibility.

> +# in-use or not.
> +#
> +# For now, the only format driver supporting this feature is Qcow2 which is a

"is QCow2, which"

> +# cluster based format. Clusters considered in-use by qcow2 are those with a
> +# non-zero refcount in the format metadata. All other clusters, if present, 
> are
> +# considered unused. Examples of unused allocations for the Qcow2 format are
> +# leaked clusters, pre-allocated clusters, and recently freed clusters.
> +#
> +# Note: the whole underlying protocol file is described as well as all format
> +# file allocations, not only virtual disk data (metadata, internal snapshots,
> +# etc. are included).

I'm not sure I get this note.

> +#
> +# For the underlying protocol file there are native block-status types of the
> +# regions:

Comma after "protocol file"?  The sentence doesn't feel right to me even
with the comma, though.  What's a "native block-status type"?

> +#  - data: allocated data
> +#  - zero: reported as zero (for example, this type corresponds to holes for
> +#  POSIX files on sparce file-system)
> +#  - discarded: not allocated
> +# 4th additional type is 'overrun', is data referenced by the format driver
> +# located beyond EOF of the underlying protocol file. For example, a 
> partially

[Qemu-devel] [PATCH v4 1/3] block: add bdrv_get_format_alloc_stat format interface

[Vladimir Sementsov-Ogievskiy]

The function should collect statistics, about used/unused by top-level
format driver space (in its .file) and allocation status
(data/zero/discarded/after-eof) of corresponding areas in this .file.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c   | 16 +++
 include/block/block.h |  3 ++
 include/block/block_int.h |  2 ++
 qapi/block-core.json  | 72 +++
 4 files changed, 93 insertions(+)

diff --git a/block.c b/block.c
index 50ba264143..7d720ae0c2 100644
--- a/block.c
+++ b/block.c
@@ -3407,6 +3407,22 @@ int64_t bdrv_get_allocated_file_size(BlockDriverState 
*bs)
 }
 
 /**
+ * Collect format allocation info. See BlockFormatAllocInfo definition in
+ * qapi/block-core.json.
+ */
+int bdrv_get_format_alloc_stat(BlockDriverState *bs, BlockFormatAllocInfo 
*bfai)
+{
+BlockDriver *drv = bs->drv;
+if (!drv) {
+return -ENOMEDIUM;
+}
+if (drv->bdrv_get_format_alloc_stat) {
+return drv->bdrv_get_format_alloc_stat(bs, bfai);
+}
+return -ENOTSUP;
+}
+
+/**
  * Return number of sectors on success, -errno on error.
  */
 int64_t bdrv_nb_sectors(BlockDriverState *bs)
diff --git a/include/block/block.h b/include/block/block.h
index 9b355e92d8..646376a772 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -335,6 +335,9 @@ typedef enum {
 
 int bdrv_check(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode fix);
 
+int bdrv_get_format_alloc_stat(BlockDriverState *bs,
+   BlockFormatAllocInfo *bfai);
+
 /* The units of offset and total_work_size may be chosen arbitrarily by the
  * block driver; total_work_size may change during the course of the amendment
  * operation */
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 8d3724cce6..458c715e99 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -208,6 +208,8 @@ struct BlockDriver {
 int64_t (*bdrv_getlength)(BlockDriverState *bs);
 bool has_variable_length;
 int64_t (*bdrv_get_allocated_file_size)(BlockDriverState *bs);
+int (*bdrv_get_format_alloc_stat)(BlockDriverState *bs,
+  BlockFormatAllocInfo *bfai);
 
 int coroutine_fn (*bdrv_co_pwritev_compressed)(BlockDriverState *bs,
 uint64_t offset, uint64_t bytes, QEMUIOVector *qiov);
diff --git a/qapi/block-core.json b/qapi/block-core.json
index ea0b3e8b13..93f6995381 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -139,6 +139,78 @@
'*format-specific': 'ImageInfoSpecific' } }
 
 ##
+# @BlockFormatAllocInfo:
+#
+#
+# Allocation information of an underlying protocol file, as partitioned by a
+# format driver's utilization of said allocations.
+# All fields are in bytes.
+#
+# Regions of the underlying protocol file may be considered used or unused by
+# the format driver interpreting these regions. It is at the discretion of the
+# format driver (e.g. qcow2) which regions of its backing storage are 
considered
+# in-use or not.
+#
+# For now, the only format driver supporting this feature is Qcow2 which is a
+# cluster based format. Clusters considered in-use by qcow2 are those with a
+# non-zero refcount in the format metadata. All other clusters, if present, are
+# considered unused. Examples of unused allocations for the Qcow2 format are
+# leaked clusters, pre-allocated clusters, and recently freed clusters.
+#
+# Note: the whole underlying protocol file is described as well as all format
+# file allocations, not only virtual disk data (metadata, internal snapshots,
+# etc. are included).
+#
+# For the underlying protocol file there are native block-status types of the
+# regions:
+#  - data: allocated data
+#  - zero: reported as zero (for example, this type corresponds to holes for
+#  POSIX files on sparce file-system)
+#  - discarded: not allocated
+# 4th additional type is 'overrun', is data referenced by the format driver
+# located beyond EOF of the underlying protocol file. For example, a partially
+# allocated cluster at the end of a QCOW2 file, where Qcow2 generally operates
+# on complete clusters.
+#
+# So, the fields are:
+#
+# @used-data: used by the format file and backed by data in the underlying
+# protocol file
+#
+# @used-zero: used by the format file and backed by zeroes in the underlying
+# protocol file; which may be a filesystem hole for POSIX files.
+#
+# @used-discarded: used by the format file but actually unallocated in the
+#  underlying protocol file
+#
+# @used-overrun: used by the format file beyond the end of the underlying
+#protocol file
+#
+# @unused-data: allocated data in the underlying protocol file not used by the
+#   format file
+#
+# @unused-zero: reported-as-zero regions in the underlying protocol file not
+#   used by the format file
+#
+# @unused-discarded: