Re: [RFC PATCH v2 06/25] include/block/block_int: split header into I/O and global state API

2021-10-07 Thread Stefan Hajnoczi
On Thu, Oct 07, 2021 at 01:30:42PM +0200, Emanuele Giuseppe Esposito wrote:
> 
> 
> On 07/10/2021 12:52, Stefan Hajnoczi wrote:
> > On Tue, Oct 05, 2021 at 10:31:56AM -0400, Emanuele Giuseppe Esposito wrote:
> > > +int coroutine_fn bdrv_co_copy_range_from(BdrvChild *src, int64_t 
> > > src_offset,
> > > + BdrvChild *dst, int64_t 
> > > dst_offset,
> > > + int64_t bytes,
> > > + BdrvRequestFlags read_flags,
> > > + BdrvRequestFlags write_flags);
> > > +int coroutine_fn bdrv_co_copy_range_to(BdrvChild *src, int64_t 
> > > src_offset,
> > > +   BdrvChild *dst, int64_t 
> > > dst_offset,
> > > +   int64_t bytes,
> > > +   BdrvRequestFlags read_flags,
> > > +   BdrvRequestFlags write_flags);
> > 
> > These look like I/O APIs to me. Are they in the GS API because only
> > qemu-img.c call copy_range? I thought SCSI emulation might call it too,
> > but grep says otherwise.
> 
> SCSI (iscsi.c) implements the function pointer
> (*bdrv_co_copy_range_from/to), but does not call the function itself.
> However, later in the patches I put the function pointer as I/O.

I meant the SCSI target emulation in hw/scsi/ where the SCSI commands
that require copy_range could be handled.

> These two functions are only tested by test-replication and thus are always
> under BQL when tested. But after looking at them again, and since they
> internally use only I/O APIs, I think we can move them to the I/O API.

Okay, great.

Stefan


signature.asc
Description: PGP signature


Re: [RFC PATCH v2 06/25] include/block/block_int: split header into I/O and global state API

2021-10-07 Thread Emanuele Giuseppe Esposito




On 07/10/2021 12:52, Stefan Hajnoczi wrote:

On Tue, Oct 05, 2021 at 10:31:56AM -0400, Emanuele Giuseppe Esposito wrote:

+int coroutine_fn bdrv_co_copy_range_from(BdrvChild *src, int64_t src_offset,
+ BdrvChild *dst, int64_t dst_offset,
+ int64_t bytes,
+ BdrvRequestFlags read_flags,
+ BdrvRequestFlags write_flags);
+int coroutine_fn bdrv_co_copy_range_to(BdrvChild *src, int64_t src_offset,
+   BdrvChild *dst, int64_t dst_offset,
+   int64_t bytes,
+   BdrvRequestFlags read_flags,
+   BdrvRequestFlags write_flags);


These look like I/O APIs to me. Are they in the GS API because only
qemu-img.c call copy_range? I thought SCSI emulation might call it too,
but grep says otherwise.


SCSI (iscsi.c) implements the function pointer 
(*bdrv_co_copy_range_from/to), but does not call the function itself. 
However, later in the patches I put the function pointer as I/O.


These two functions are only tested by test-replication and thus are 
always under BQL when tested. But after looking at them again, and since 
they internally use only I/O APIs, I think we can move them to the I/O API.


Thank you,
Emanuele




Re: [RFC PATCH v2 06/25] include/block/block_int: split header into I/O and global state API

2021-10-07 Thread Stefan Hajnoczi
On Tue, Oct 05, 2021 at 10:31:56AM -0400, Emanuele Giuseppe Esposito wrote:
> +int coroutine_fn bdrv_co_copy_range_from(BdrvChild *src, int64_t src_offset,
> + BdrvChild *dst, int64_t dst_offset,
> + int64_t bytes,
> + BdrvRequestFlags read_flags,
> + BdrvRequestFlags write_flags);
> +int coroutine_fn bdrv_co_copy_range_to(BdrvChild *src, int64_t src_offset,
> +   BdrvChild *dst, int64_t dst_offset,
> +   int64_t bytes,
> +   BdrvRequestFlags read_flags,
> +   BdrvRequestFlags write_flags);

These look like I/O APIs to me. Are they in the GS API because only
qemu-img.c call copy_range? I thought SCSI emulation might call it too,
but grep says otherwise.

Otherwise:

Reviewed-by: Stefan Hajnoczi 


signature.asc
Description: PGP signature


[RFC PATCH v2 06/25] include/block/block_int: split header into I/O and global state API

2021-10-05 Thread Emanuele Giuseppe Esposito
Similarly to the previous patch, split block_int.h
in block_int-io.h and block_int-global-state.h

block_int-common.h contains the structures shared between
the two headers, and the functions that can't be categorized as
I/O or global state.

Assertions are added in the next patch.

Signed-off-by: Emanuele Giuseppe Esposito 
---
 blockdev.c |5 +
 include/block/block_int-common.h   | 1122 +++
 include/block/block_int-global-state.h |  346 ++
 include/block/block_int-io.h   |  124 +++
 include/block/block_int.h  | 1412 +---
 5 files changed, 1600 insertions(+), 1409 deletions(-)
 create mode 100644 include/block/block_int-common.h
 create mode 100644 include/block/block_int-global-state.h
 create mode 100644 include/block/block_int-io.h

diff --git a/blockdev.c b/blockdev.c
index 44c419545c..75407cbf67 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -64,6 +64,7 @@
 #include "qemu/main-loop.h"
 #include "qemu/throttle-options.h"
 
+/* Protected by BQL lock */
 QTAILQ_HEAD(, BlockDriverState) monitor_bdrv_states =
 QTAILQ_HEAD_INITIALIZER(monitor_bdrv_states);
 
@@ -1208,6 +1209,8 @@ typedef struct BlkActionState BlkActionState;
  *
  * Only prepare() may fail. In a single transaction, only one of commit() or
  * abort() will be called. clean() will always be called if it is present.
+ *
+ * Always run under BQL.
  */
 typedef struct BlkActionOps {
 size_t instance_size;
@@ -2317,6 +2320,8 @@ static TransactionProperties *get_transaction_properties(
 /*
  * 'Atomic' group operations.  The operations are performed as a set, and if
  * any fail then we roll back all operations in the group.
+ *
+ * Always run under BQL.
  */
 void qmp_transaction(TransactionActionList *dev_list,
  bool has_props,
diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
new file mode 100644
index 00..23f0d9c090
--- /dev/null
+++ b/include/block/block_int-common.h
@@ -0,0 +1,1122 @@
+/*
+ * QEMU System Emulator block driver
+ *
+ * Copyright (c) 2003 Fabrice Bellard
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to 
deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+#ifndef BLOCK_INT_COMMON_H
+#define BLOCK_INT_COMMON_H
+
+#include "block/accounting.h"
+#include "block/block.h"
+#include "block/aio-wait.h"
+#include "qemu/queue.h"
+#include "qemu/coroutine.h"
+#include "qemu/stats64.h"
+#include "qemu/timer.h"
+#include "qemu/hbitmap.h"
+#include "block/snapshot.h"
+#include "qemu/throttle.h"
+
+#define BLOCK_FLAG_LAZY_REFCOUNTS   8
+
+#define BLOCK_OPT_SIZE  "size"
+#define BLOCK_OPT_ENCRYPT   "encryption"
+#define BLOCK_OPT_ENCRYPT_FORMAT"encrypt.format"
+#define BLOCK_OPT_COMPAT6   "compat6"
+#define BLOCK_OPT_HWVERSION "hwversion"
+#define BLOCK_OPT_BACKING_FILE  "backing_file"
+#define BLOCK_OPT_BACKING_FMT   "backing_fmt"
+#define BLOCK_OPT_CLUSTER_SIZE  "cluster_size"
+#define BLOCK_OPT_TABLE_SIZE"table_size"
+#define BLOCK_OPT_PREALLOC  "preallocation"
+#define BLOCK_OPT_SUBFMT"subformat"
+#define BLOCK_OPT_COMPAT_LEVEL  "compat"
+#define BLOCK_OPT_LAZY_REFCOUNTS"lazy_refcounts"
+#define BLOCK_OPT_ADAPTER_TYPE  "adapter_type"
+#define BLOCK_OPT_REDUNDANCY"redundancy"
+#define BLOCK_OPT_NOCOW "nocow"
+#define BLOCK_OPT_EXTENT_SIZE_HINT  "extent_size_hint"
+#define BLOCK_OPT_OBJECT_SIZE   "object_size"
+#define BLOCK_OPT_REFCOUNT_BITS "refcount_bits"
+#define BLOCK_OPT_DATA_FILE "data_file"
+#define BLOCK_OPT_DATA_FILE_RAW "data_file_raw"
+#define BLOCK_OPT_COMPRESSION_TYPE  "compression_type"
+#define BLOCK_OPT_EXTL2 "extended_l2"
+
+#define BLOCK_PROBE_BUF_SIZE512
+
+enum BdrvTrackedRequestType {
+BDRV_TRACKED_READ,
+BDRV_TRACKED_WRITE,
+BDRV_TRACKED_DISCARD,
+BDRV_TRACKED_TRUNCATE,
+};
+
+/*
+ * That is not quite good that BdrvT