[PATCH 06/29] qemu-nbd: Use raw block driver for --offset

2020-09-07 Thread Kevin Wolf
Instead of implementing qemu-nbd --offset in the NBD code, just put a raw block node with the requested offset on top of the user image and rely on that doing the job. This does not only simplify the nbd_export_new() interface and bring it closer to the set of options that the nbd-server-add QMP

[PATCH 07/29] block/export: Remove magic from block-export-add

2020-09-07 Thread Kevin Wolf
nbd-server-add tries to be convenient and adds two questionable features that we don't want to share in block-export-add, even for NBD exports: 1. When requesting a writable export of a read-only device, the export is silently downgraded to read-only. This should be an error in the context

Re: [PATCH v5 08/11] osdep: These function are only available on Non-Win32 system.

2020-09-07 Thread Yonggang Luo
On Sat, Sep 5, 2020 at 4:33 PM Thomas Huth wrote: > On 05/09/2020 08.23, Yonggang Luo wrote: > > int qemu_lock_fd(int fd, int64_t start, int64_t len, bool exclusive); > > int qemu_unlock_fd(int fd, int64_t start, int64_t len); > > int qemu_lock_fd_test(int fd, int64_t start, int64_t len, bool >

[PATCH 20/29] block/export: Add block-export-del

2020-09-07 Thread Kevin Wolf
Implement a new QMP command block-export-del and make nbd-server-remove a wrapper around it. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- qapi/block-export.json | 32 - include/block/export.h | 1 + include/block/nbd.h| 1 -

[PATCH 28/29] iotests: Factor out qemu_tool_pipe_and_status()

2020-09-07 Thread Kevin Wolf
We have three almost identical functions that call an external process and return its output and return code. Refactor them into small wrappers around a common function. Signed-off-by: Kevin Wolf --- tests/qemu-iotests/iotests.py | 50 +-- 1 file changed, 24

[PATCH 29/29] iotests: Test block-export-* QMP interface

2020-09-07 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- tests/qemu-iotests/iotests.py | 11 +++- tests/qemu-iotests/307| 117 ++ tests/qemu-iotests/307.out| 111 tests/qemu-iotests/group | 1 + 4 files changed, 239 insertions(+), 1

[PATCH 27/29] nbd: Deprecate nbd-server-add/remove

2020-09-07 Thread Kevin Wolf
These QMP commands are replaced by block-export-add/del. Signed-off-by: Kevin Wolf --- qapi/block-export.json | 11 +-- docs/system/deprecated.rst | 8 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/qapi/block-export.json b/qapi/block-export.json index

Re: [PULL 00/64] Block layer patches

2020-09-07 Thread Peter Maydell
On Mon, 7 Sep 2020 at 12:09, Kevin Wolf wrote: > > The following changes since commit 7c37270b3fbe3d034ba80e488761461676e21eb4: > > Merge remote-tracking branch 'remotes/kraxel/tags/ui-20200904-pull-request' > into staging (2020-09-06 16:23:55 +0100) > > are available in the Git repository at:

[PATCH 19/29] block/export: Move strong user reference to block_exports

2020-09-07 Thread Kevin Wolf
The reference owned by the user/monitor that is created when adding the export and dropped when removing it was tied to the 'exports' list in nbd/server.c. Every block export will have a user reference, so move it to the block export level and tie it to the 'block_exports' list in

[PATCH 15/29] block/export: Add node-name to BlockExportOptions

2020-09-07 Thread Kevin Wolf
Every block export needs a block node to export, so add a 'node-name' option to BlockExportOptions and remove the replaced option 'device' from BlockExportOptionsNbd. To maintain compatibility in nbd-server-add, BlockExportOptionsNbd needs to be wrapped by a new type NbdServerAddOptions that adds

[PATCH 17/29] block/export: Add blk_exp_close_all(_type)

2020-09-07 Thread Kevin Wolf
This adds a function to shut down all block exports, and another one to shut down the block exports of a single type. The latter is used for now when stopping the NBD server. As soon as we implement support for multiple NBD servers, we'll need a per-server list of exports and it will be replaced

[PATCH 16/29] block/export: Allocate BlockExport in blk_exp_add()

2020-09-07 Thread Kevin Wolf
Instead of letting the driver allocate and return the BlockExport object, allocate it already in blk_exp_add() and pass it. This allows us to initialise the generic part before calling into the driver so that the driver can just use these values instead of having to parse the options a second

[PATCH 22/29] block/export: Move blk to BlockExport

2020-09-07 Thread Kevin Wolf
Every block export has a BlockBackend representing the disk that is exported. It should live in BlockExport therefore. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- include/block/export.h | 3 +++ block/export/export.c | 3 +++ nbd/server.c | 43

[PATCH 18/29] block/export: Add 'id' option to block-export-add

2020-09-07 Thread Kevin Wolf
We'll need an id to identify block exports in monitor commands. This adds one. Note that this is different from the 'name' option in the NBD server, which is the externally visible export name. While block export ids need to be unique in the whole process, export names must be unique only for the

[PATCH 09/29] nbd: Add writethrough to block-export-add

2020-09-07 Thread Kevin Wolf
qemu-nbd allows use of writethrough cache modes, which mean that write requests made through NBD will cause a flush before they complete. Expose the same functionality in block-export-add. Signed-off-by: Kevin Wolf --- qapi/block-export.json | 7 ++- blockdev-nbd.c | 6 +- 2

[PATCH 14/29] block/export: Move AioContext from NBDExport to BlockExport

2020-09-07 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- include/block/export.h | 3 +++ block/export/export.c | 2 ++ nbd/server.c | 26 +- 3 files changed, 18 insertions(+), 13 deletions(-) diff --git a/include/block/export.h b/include/block/export.h index 5236a35e12..e6f96f4e1e

[PATCH] ci: fixes msys2 build by upgrading capstone to 4.0.2

2020-09-07 Thread Yonggang Luo
Signed-off-by: Yonggang Luo --- capstone | 2 +- configure | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/capstone b/capstone index 22ead3e0bf..1d23053284 16 --- a/capstone +++ b/capstone @@ -1 +1 @@ -Subproject commit 22ead3e0bfdb87516656453336160e0a37b066bf

[PATCH 05/29] qemu-storage-daemon: Use qmp_block_export_add()

2020-09-07 Thread Kevin Wolf
No reason to duplicate the functionality locally, we can now just reuse the QMP command block-export-add for --export. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: Eric Blake --- storage-daemon/qemu-storage-daemon.c | 13 + 1 file changed, 1 insertion(+), 12

[PATCH 01/29] nbd: Remove unused nbd_export_get_blockdev()

2020-09-07 Thread Kevin Wolf
Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: Eric Blake --- include/block/nbd.h | 2 -- nbd/server.c| 5 - 2 files changed, 7 deletions(-) diff --git a/include/block/nbd.h b/include/block/nbd.h index 9bc3bfaeec..0451683d03 100644 --- a/include/block/nbd.h +++

[PATCH 02/29] qapi: Create block-export module

2020-09-07 Thread Kevin Wolf
Move all block export related types and commands from block-core to the new QAPI module block-export. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: Eric Blake --- qapi/block-core.json | 166 - qapi/block-export.json | 175

[PATCH 00/29] block/export: Add infrastructure and QAPI for block exports

2020-09-07 Thread Kevin Wolf
We are planning to add more block export types than just NBD in the near future (e.g. vhost-user-blk and FUSE). This series lays the ground for this with some generic block export infrastructure and QAPI interfaces that will allow managing all of them (for now add/remove/query). As a side effect,

[PATCH 03/29] qapi: Rename BlockExport to BlockExportOptions

2020-09-07 Thread Kevin Wolf
The name BlockExport will be used for the struct containing the runtime state of block exports, so change the name of export creation options. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: Eric Blake --- qapi/block-export.json | 12 ++--

[PATCH 13/29] block/export: Move refcount from NBDExport to BlockExport

2020-09-07 Thread Kevin Wolf
Having a refcount makes sense for all types of block exports. It is also a prerequisite for keeping a list of all exports at the BlockExport level. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- include/block/export.h | 15 + include/block/nbd.h| 2 -- block/export/export.c

[PATCH 04/29] block/export: Add BlockExport infrastructure and block-export-add

2020-09-07 Thread Kevin Wolf
We want to have a common set of commands for all types of block exports. Currently, this is only NBD, but we're going to add more types. This patch adds the basic BlockExport and BlockExportDriver structs and a QMP command block-export-add that creates a new export based on the given

[PATCH 11/29] qemu-nbd: Use blk_exp_add() to create the export

2020-09-07 Thread Kevin Wolf
With this change, NBD exports are now only created through the BlockExport interface. This allows us finally to move things from the NBD layer to the BlockExport layer if they make sense for other export types, too. blk_exp_add() returns only a weak reference, so the explicit nbd_export_put()

[PATCH 10/29] nbd: Remove NBDExport.close callback

2020-09-07 Thread Kevin Wolf
The export close callback is unused by the built-in NBD server. qemu-nbd uses it only during shutdown to wait for the unrefed export to actually go away. It can just use nbd_export_close_all() instead and do without the callback. This removes the close callback from nbd_export_new() and makes

[PATCH 08/29] nbd: Add max-connections to nbd-server-start

2020-09-07 Thread Kevin Wolf
This is a QMP equivalent of qemu-nbd's --shared option, limiting the maximum number of clients that can attach at the same time. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: Eric Blake --- qapi/block-export.json | 10 +++-- include/block/nbd.h

[PATCH] block: Fixes nfs on msys2/mingw

2020-09-07 Thread Yonggang Luo
Signed-off-by: Yonggang Luo --- block/nfs.c | 26 +- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/block/nfs.c b/block/nfs.c index 61a249a9fc..34b2cd5708 100644 --- a/block/nfs.c +++ b/block/nfs.c @@ -24,7 +24,9 @@ #include "qemu/osdep.h" +#if

[PATCH 24/29] block/export: Add query-block-exports

2020-09-07 Thread Kevin Wolf
This adds a simple QMP command to query the list of block exports. Signed-off-by: Kevin Wolf --- qapi/block-export.json | 32 block/export/export.c | 23 +++ 2 files changed, 55 insertions(+) diff --git a/qapi/block-export.json

[PATCH 23/29] block/export: Create BlockBackend in blk_exp_add()

2020-09-07 Thread Kevin Wolf
Every export type will need a BlockBackend, so creating it centrally in blk_exp_add() instead of the .create driver callback avoids duplication. Signed-off-by: Kevin Wolf --- include/block/nbd.h | 4 ++-- block/export/export.c | 49 +++ blockdev-nbd.c

[PATCH 25/29] block/export: Move writable to BlockExportOptions

2020-09-07 Thread Kevin Wolf
The 'writable' option is a basic option that will probably be applicable to most if not all export types that we will implement. Move it from NBD to the generic BlockExport layer. Signed-off-by: Kevin Wolf --- qapi/block-export.json | 15 ++- block/export/export.c | 16

[PATCH 21/29] block/export: Add BLOCK_EXPORT_DELETED event

2020-09-07 Thread Kevin Wolf
Clients may want to know when an export has finally disappeard (block-export-del returns earlier than that in the general case), so add a QAPI event for it. Signed-off-by: Kevin Wolf --- qapi/block-export.json | 12 block/export/export.c | 2 ++ tests/qemu-iotests/140.out

[PATCH 26/29] nbd: Merge nbd_export_new() and nbd_export_create()

2020-09-07 Thread Kevin Wolf
There is no real reason any more why nbd_export_new() and nbd_export_create() should be separate functions. The latter only performs a few checks before it calls the former. Move all the checks to nbd/server.c and make the resulting function static. Signed-off-by: Kevin Wolf ---

Re: [PATCH 02/17] hw/block/nvme: handle dma errors

2020-09-07 Thread Klaus Jensen
On Sep 7 04:34, Philippe Mathieu-Daudé wrote: > Hi Klaus, > > On 9/4/20 4:19 PM, Klaus Jensen wrote: > > From: Klaus Jensen > > > > Handling DMA errors gracefully is required for the device to pass the > > block/011 test ("disable PCI device while doing I/O") in the blktests > > suite. > > >

Re: [PATCH 17/17] hw/block/nvme: change controller pci id

2020-09-07 Thread Klaus Jensen
On Sep 7 10:58, Klaus Jensen wrote: > On Sep 7 10:36, Philippe Mathieu-Daudé wrote: > > On 9/7/20 9:23 AM, Klaus Jensen wrote: > > > On Sep 7 04:28, Philippe Mathieu-Daudé wrote: > > >> +David in case > > >> > > >> On 9/4/20 4:19 PM, Klaus Jensen wrote: > > >>> From: Klaus Jensen > > >>> > >

Re: [PATCH 00/13] dma: Let the DMA API take MemTxAttrs argument and propagate MemTxResult

2020-09-07 Thread Edgar E. Iglesias
On Fri, Sep 04, 2020 at 05:44:26PM +0200, Philippe Mathieu-Daudé wrote: > Salvaging cleanups patches from the RFC series "Forbid DMA write > accesses to MMIO regions" [*], propagating MemTxResult and > adding documentation. > > [*] https://www.mail-archive.com/qemu-block@nongnu.org/msg72924.html

Re: [PATCH 03/13] dma: Document address_space_map/address_space_unmap() prototypes

2020-09-07 Thread Edgar E. Iglesias
On Fri, Sep 04, 2020 at 05:44:29PM +0200, Philippe Mathieu-Daudé wrote: > Add documentation based on address_space_map / address_space_unmap. > > Signed-off-by: Philippe Mathieu-Daudé > --- > include/sysemu/dma.h | 30 +- > 1 file changed, 29 insertions(+), 1

Re: [PATCH] iotests: Skip test_stream_parallel in test 030 when doing "make check"

2020-09-07 Thread Max Reitz
On 05.09.20 13:24, Thomas Huth wrote: > The test_stream_parallel test still occasionally fails in the CI. > Thus let's disable it during "make check" for now so that it does > not cause trouble during merge tests. We can enable it again once > the problem has been resolved. > > Signed-off-by:

Re: [PATCH 17/17] hw/block/nvme: change controller pci id

2020-09-07 Thread Philippe Mathieu-Daudé
On 9/7/20 9:23 AM, Klaus Jensen wrote: > On Sep 7 04:28, Philippe Mathieu-Daudé wrote: >> +David in case >> >> On 9/4/20 4:19 PM, Klaus Jensen wrote: >>> From: Klaus Jensen >>> >>> There are two reasons for changing this: >>> >>> 1. The nvme device currently uses an internal Intel device id.

Re: [PATCH 17/17] hw/block/nvme: change controller pci id

2020-09-07 Thread Klaus Jensen
On Sep 7 10:36, Philippe Mathieu-Daudé wrote: > On 9/7/20 9:23 AM, Klaus Jensen wrote: > > On Sep 7 04:28, Philippe Mathieu-Daudé wrote: > >> +David in case > >> > >> On 9/4/20 4:19 PM, Klaus Jensen wrote: > >>> From: Klaus Jensen > >>> > >>> There are two reasons for changing this: > >>> > >>>

Re: [PATCH 17/17] hw/block/nvme: change controller pci id

2020-09-07 Thread Klaus Jensen
On Sep 7 04:28, Philippe Mathieu-Daudé wrote: > +David in case > > On 9/4/20 4:19 PM, Klaus Jensen wrote: > > From: Klaus Jensen > > > > There are two reasons for changing this: > > > > 1. The nvme device currently uses an internal Intel device id. > > > > 2. Since commits "nvme: fix

Re: [PATCH 1/2] tests: Trying fixes test-replication.c on msys2/mingw.

2020-09-07 Thread Daniel P . Berrangé
On Sun, Sep 06, 2020 at 08:13:19PM +0800, 罗勇刚(Yonggang Luo) wrote: > On Sun, Sep 6, 2020 at 5:49 AM Stefan Weil wrote: > > > Am 05.09.20 um 23:10 schrieb Yonggang Luo: > > > > > On Windows there is no path like /tmp/s_local_disk.XX > > > > > > Signed-off-by: Yonggang Luo > > > --- > > >

Re: [PATCH v2] qemu-img: Explicit number replaced by a constant

2020-09-07 Thread Max Reitz
On 19.08.20 03:36, Yi Li wrote: > Signed-off-by: Yi Li > --- > qemu-img.c | 10 +- > 1 file changed, 5 insertions(+), 5 deletions(-) Thanks, applied to my block branch: https://git.xanclic.moe/XanClic/qemu/commits/branch/block signature.asc Description: OpenPGP digital signature

Re: make -i check resut for msys2

2020-09-07 Thread Kevin Wolf
Am 05.09.2020 um 08:21 hat 罗勇刚(Yonggang Luo) geschrieben: > On Fri, Sep 4, 2020 at 4:51 PM Kevin Wolf wrote: > > > Am 04.09.2020 um 08:03 hat Thomas Huth geschrieben: > > > On 04/09/2020 00.53, 罗勇刚(Yonggang Luo) wrote: > > > > > > > > > > > > On Thu, Sep 3, 2020 at 10:33 PM Thomas Huth > > >

[PATCH] file-win32: Fix "locking" option

2020-09-07 Thread Kevin Wolf
The intended behaviour was that locking=off/auto work and have no effect (to remain compatible with file-posix), whereas locking=on would return an error. Unfortunately, the code forgot to remove "locking" from the options QDict, so any attempt to use the option would fail. Replace the option

Re: [PATCH 17/17] hw/block/nvme: change controller pci id

2020-09-07 Thread Dr. David Alan Gilbert
* Philippe Mathieu-Daudé (phi...@redhat.com) wrote: > +David in case > > On 9/4/20 4:19 PM, Klaus Jensen wrote: > > From: Klaus Jensen > > > > There are two reasons for changing this: > > > > 1. The nvme device currently uses an internal Intel device id. > > > > 2. Since commits "nvme:

[PULL 17/64] block: Add child access functions

2020-09-07 Thread Kevin Wolf
From: Max Reitz There are BDS children that the general block layer code can access, namely bs->file and bs->backing. Since the introduction of filters and external data files, their meaning is not quite clear. bs->backing can be a COW source, or it can be a filtered child; bs->file can be a

[PULL 10/64] block/nvme: Replace qemu_try_blockalign0 by qemu_try_blockalign/memset

2020-09-07 Thread Kevin Wolf
From: Philippe Mathieu-Daudé In the next commit we'll get rid of qemu_try_blockalign(). To ease review, first replace qemu_try_blockalign0() by explicit calls to qemu_try_blockalign() and memset(). Reviewed-by: Stefan Hajnoczi Reviewed-by: Stefano Garzarella Signed-off-by: Philippe

[PULL 07/64] block/nvme: Use common error path in nvme_add_io_queue()

2020-09-07 Thread Kevin Wolf
From: Philippe Mathieu-Daudé Rearrange nvme_add_io_queue() by using a common error path. This will be proven useful in few commits where we add IRQ notification to the IO queues. Reviewed-by: Stefan Hajnoczi Reviewed-by: Stefano Garzarella Signed-off-by: Philippe Mathieu-Daudé Message-Id:

[PULL 18/64] block: Add chain helper functions

2020-09-07 Thread Kevin Wolf
From: Max Reitz Add some helper functions for skipping filters in a chain of block nodes. Signed-off-by: Max Reitz --- include/block/block_int.h | 3 ++ block.c | 62 +++ 2 files changed, 65 insertions(+) diff --git

[PULL 06/64] block/nvme: Improve error message when IO queue creation failed

2020-09-07 Thread Kevin Wolf
From: Philippe Mathieu-Daudé Do not use the same error message for different failures. Display a different error whether it is the CQ or the SQ. Reviewed-by: Stefan Hajnoczi Reviewed-by: Stefano Garzarella Signed-off-by: Philippe Mathieu-Daudé Message-Id:

[PULL 19/64] block: bdrv_cow_child() for bdrv_has_zero_init()

2020-09-07 Thread Kevin Wolf
From: Max Reitz bdrv_has_zero_init() should use bdrv_cow_child() if it wants to check whether the given BDS has a COW backing file. Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- block.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

[PULL 09/64] block/nvme: Use union of NvmeIdCtrl / NvmeIdNs structures

2020-09-07 Thread Kevin Wolf
From: Philippe Mathieu-Daudé We allocate an unique chunk of memory then use it for two different structures. By using an union, we make it clear the data is overlapping (and we can remove the casts). Suggested-by: Stefan Hajnoczi Reviewed-by: Stefan Hajnoczi Reviewed-by: Stefano Garzarella

[PULL 35/64] block: Use CAFs in bdrv_refresh_filename()

2020-09-07 Thread Kevin Wolf
From: Max Reitz bdrv_refresh_filename() and the kind of related bdrv_dirname() should look to the primary child when they wish to copy the underlying file's filename. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf --- block.c | 29 + 1 file changed, 21

[PULL 45/64] mirror: Deal with filters

2020-09-07 Thread Kevin Wolf
From: Max Reitz This includes some permission limiting (for example, we only need to take the RESIZE permission for active commits where the base is smaller than the top). base_overlay is introduced so we can query bdrv_is_allocated_above() on it - we cannot do that with base itself, because a

[PULL 26/64] block: Use bdrv_filter_(bs|child) where obvious

2020-09-07 Thread Kevin Wolf
From: Max Reitz Places that use patterns like if (bs->drv->is_filter && bs->file) { ... something about bs->file->bs ... } should be BlockDriverState *filtered = bdrv_filter_bs(bs); if (filtered) { ... something about @filtered ... } instead.

[PULL 46/64] backup: Deal with filters

2020-09-07 Thread Kevin Wolf
From: Max Reitz Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf --- block/backup-top.c | 2 +- block/backup.c | 9 + blockdev.c | 19 +++ 3 files changed, 21 insertions(+), 9 deletions(-) diff --git a/block/backup-top.c b/block/backup-top.c index

[PULL 49/64] qemu-img: Use child access functions

2020-09-07 Thread Kevin Wolf
From: Max Reitz This changes iotest 204's output, because blkdebug on top of a COW node used to make qemu-img map disregard the rest of the backing chain (the backing chain was broken by the filter). With this patch, the allocation in the base image is reported correctly. Signed-off-by: Max

[PULL 34/64] block: Iterate over children in refresh_limits

2020-09-07 Thread Kevin Wolf
From: Max Reitz Instead of looking at just bs->file and bs->backing, we should look at all children that could end up receiving forwarded requests. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf --- block/io.c | 32 1 file changed, 16 insertions(+), 16

[PULL 53/64] block: Leave BDS.backing_{file,format} constant

2020-09-07 Thread Kevin Wolf
From: Max Reitz Parts of the block layer treat BDS.backing_file as if it were whatever the image header says (i.e., if it is a relative path, it is relative to the overlay), other parts treat it like a cache for bs->backing->bs->filename (relative paths are relative to the CWD). Considering

[PULL 51/64] blockdev: Fix active commit choice

2020-09-07 Thread Kevin Wolf
From: Max Reitz We have to perform an active commit whenever the top node has a parent that has taken the WRITE permission on it. This means that block-commit's @backing-file parameter is no longer allowed for such nodes, and that users will have to issue a block-job-complete command. Neither

[PULL 41/64] blockdev: Use CAF in external_snapshot_prepare()

2020-09-07 Thread Kevin Wolf
From: Max Reitz This allows us to differentiate between filters and nodes with COW backing files: Filters cannot be used as overlays at all (for this function). Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- blockdev.c | 7 ++- 1 file changed, 6

[PULL 36/64] block: Use CAF in bdrv_co_rw_vmstate()

2020-09-07 Thread Kevin Wolf
From: Max Reitz If a node whose driver does not provide VM state functions has a metadata child, the VM state should probably go there; if it is a filter, the VM state should probably go there. It follows that we should generally go down to the primary child. Signed-off-by: Max Reitz

[PATCH] MAINTAINERS: add Stefan Hajnoczi as block/nvme.c maintainer

2020-09-07 Thread Stefan Hajnoczi
Development of the userspace NVMe block driver picked up again recently. After talking with Fam I am stepping up as block/nvme.c maintainer. Patches will be merged through my 'block' tree. Cc: Kevin Wolf Cc: Klaus Jensen Cc: Fam Zheng Signed-off-by: Stefan Hajnoczi --- MAINTAINERS | 4 +++-

[PULL 44/64] block-copy: Use CAF to find sync=top base

2020-09-07 Thread Kevin Wolf
From: Max Reitz Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- block/block-copy.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index a30b9097ef..cd9bc47c8f 100644 --- a/block/block-copy.c

[PULL 59/64] iotests: Test committing to overridden backing

2020-09-07 Thread Kevin Wolf
From: Max Reitz Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- tests/qemu-iotests/040 | 61 ++ tests/qemu-iotests/040.out | 4 +-- 2 files changed, 63 insertions(+), 2 deletions(-) diff --git

[PULL 60/64] iotests: Allow running from different directory

2020-09-07 Thread Kevin Wolf
It is convenient to be able to edit the tests and run them without changing the current working directory back and forth. Instead of assuming that $PWD is the qemu-iotests build directory, derive the build directory from the executed script. This allows 'check' to find the required files even

[PULL 52/64] block: Inline bdrv_co_block_status_from_*()

2020-09-07 Thread Kevin Wolf
From: Max Reitz With bdrv_filter_bs(), we can easily handle this default filter behavior in bdrv_co_block_status(). blkdebug wants to have an additional assertion, so it keeps its own implementation, except bdrv_co_block_status_from_file() needs to be inlined there. Suggested-by: Eric Blake

[PULL 56/64] iotests: Add filter commit test cases

2020-09-07 Thread Kevin Wolf
From: Max Reitz This patch adds some tests on how commit copes with filter nodes. Signed-off-by: Max Reitz --- tests/qemu-iotests/040 | 177 + tests/qemu-iotests/040.out | 4 +- 2 files changed, 179 insertions(+), 2 deletions(-) diff --git

[PULL 58/64] iotests: Add test for commit in sub directory

2020-09-07 Thread Kevin Wolf
From: Max Reitz Add a test for committing an overlay in a sub directory to one of the images in its backing chain, using both relative and absolute filenames. Signed-off-by: Max Reitz --- tests/qemu-iotests/020 | 44 ++ tests/qemu-iotests/020.out | 10

Re: [PATCH 17/17] hw/block/nvme: change controller pci id

2020-09-07 Thread Philippe Mathieu-Daudé
On 9/7/20 11:20 AM, Klaus Jensen wrote: > On Sep 7 10:58, Klaus Jensen wrote: >> On Sep 7 10:36, Philippe Mathieu-Daudé wrote: >>> On 9/7/20 9:23 AM, Klaus Jensen wrote: On Sep 7 04:28, Philippe Mathieu-Daudé wrote: > +David in case > > On 9/4/20 4:19 PM, Klaus Jensen wrote:

[PATCH 3/3] iotests: add 306 to test reconnect on nbd open

2020-09-07 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev --- tests/qemu-iotests/306| 46 +++ tests/qemu-iotests/306.out| 5 tests/qemu-iotests/group | 1 + tests/qemu-iotests/iotests.py | 11 + 4 files changed, 63

[PULL 02/64] block/nvme: Replace magic value by SCALE_MS definition

2020-09-07 Thread Kevin Wolf
From: Philippe Mathieu-Daudé Use self-explicit SCALE_MS definition instead of magic value. Reviewed-by: Stefan Hajnoczi Reviewed-by: Stefano Garzarella Signed-off-by: Philippe Mathieu-Daudé Message-Id: <20200821195359.1285345-2-phi...@redhat.com> Signed-off-by: Kevin Wolf --- block/nvme.c

[PULL 23/64] block: Add bdrv_supports_compressed_writes()

2020-09-07 Thread Kevin Wolf
From: Max Reitz Filters cannot compress data themselves but they have to implement .bdrv_co_pwritev_compressed() still (or they cannot forward compressed writes). Therefore, checking whether bs->drv->bdrv_co_pwritev_compressed is non-NULL is not sufficient to know whether the node can actually

[PULL 33/64] vmdk: Drop vmdk_co_flush()

2020-09-07 Thread Kevin Wolf
From: Max Reitz Before HEAD^, we needed this because bdrv_co_flush() by itself would only flush bs->file. With HEAD^, bdrv_co_flush() will flush all children on which a WRITE or WRITE_UNCHANGED permission has been taken. Thus, vmdk no longer needs to do it itself. Signed-off-by: Max Reitz

[PULL 31/64] block: Re-evaluate backing file handling in reopen

2020-09-07 Thread Kevin Wolf
From: Max Reitz Reopening a node's backing child needs a bit of special handling because the "backing" child has different defaults than all other children (among other things). Adding filter support here is a bit more difficult than just using the child access functions. In fact, we often

[PULL 32/64] block: Flush all children in generic code

2020-09-07 Thread Kevin Wolf
From: Max Reitz If the driver does not support .bdrv_co_flush() so bdrv_co_flush() itself has to flush the children of the given node, it should not flush just bs->file->bs, but in fact all children that might have been written to (judging from the permissions taken on them). This is a bug fix

[PULL 21/64] block: Include filters when freezing backing chain

2020-09-07 Thread Kevin Wolf
From: Max Reitz In order to make filters work in backing chains, the associated functions must be able to deal with them and freeze both COW and filter child links. While at it, add some comments that note which functions require their caller to ensure that a given child link is not frozen, and

[PULL 39/64] block: Improve get_allocated_file_size's default

2020-09-07 Thread Kevin Wolf
From: Max Reitz There are two practical problems with bdrv_get_allocated_file_size()'s default right now: (1) For drivers with children, we should generally sum all their sizes instead of just passing the request through to bs->file. The latter is good for filters, but not so much for

[PULL 40/64] block/null: Implement bdrv_get_allocated_file_size

2020-09-07 Thread Kevin Wolf
From: Max Reitz It is trivial, so we might as well do it. Remove _filter_actual_image_size from iotest 184, so we get to see the result in its reference output. Signed-off-by: Max Reitz --- block/null.c | 7 +++ tests/qemu-iotests/153.out | 2 +- tests/qemu-iotests/184

[PULL 64/64] block/nvme: Pair doorbell registers

2020-09-07 Thread Kevin Wolf
From: Philippe Mathieu-Daudé For each queue doorbell registers are paired as: - Submission Queue Tail Doorbell - Completion Queue Head Doorbell Reflect that in the NVMeRegs structure, and adapt nvme_create_queue_pair() accordingly. Signed-off-by: Philippe Mathieu-Daudé Message-Id:

[PULL 63/64] block/nvme: Use generic NvmeBar structure

2020-09-07 Thread Kevin Wolf
From: Philippe Mathieu-Daudé Commit f3c507adcd7 ("NVMe: Initial commit for new storage interface") introduced the NvmeBar structure. Unfortunately in commit bdd6a90a9e5 ("block: Add VFIO based NVMe driver") we duplicated it. Apparently in commit a3d9a352d48 ("block: Move NVMe constants to a

[PULL 43/64] block: Use child access functions for QAPI queries

2020-09-07 Thread Kevin Wolf
From: Max Reitz query-block, query-named-block-nodes, and query-blockstats now return any filtered child under "backing", not just bs->backing or COW children. This is so that filters do not interrupt the reported backing chain. This changes the output for iotest 184, as the throttled node now

[PULL 62/64] block/nvme: Group controller registers in NVMeRegs structure

2020-09-07 Thread Kevin Wolf
From: Philippe Mathieu-Daudé We want to use the NvmeBar structure from "block/nvme.h" in the next commit. As a preliminary step, group all the NVMe controller registers in the 'ctrl' field, keeping the doorbells registers out of it. Signed-off-by: Philippe Mathieu-Daudé Message-Id:

[PULL 55/64] iotests: Let complete_and_wait() work with commit

2020-09-07 Thread Kevin Wolf
From: Max Reitz complete_and_wait() and wait_ready() currently only work for mirror jobs. Let them work for active commit jobs, too. Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf --- tests/qemu-iotests/iotests.py | 10 +++--- 1 file changed, 7

[PULL 50/64] block: Drop backing_bs()

2020-09-07 Thread Kevin Wolf
From: Max Reitz We want to make it explicit where bs->backing is used, and we have done so. The old role of backing_bs() is now effectively taken by bdrv_cow_bs(). Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf ---

[PULL 42/64] block: Report data child for query-blockstats

2020-09-07 Thread Kevin Wolf
From: Max Reitz It makes no sense to report the block stats of a purely metadata-storing child in query-blockstats. So if the primary child does not have any data, try to find a unique data-storing child. Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich Reviewed-by: Kevin Wolf ---

[PULL 61/64] file-win32: Fix "locking" option

2020-09-07 Thread Kevin Wolf
The intended behaviour was that locking=off/auto work and have no effect (to remain compatible with file-posix), whereas locking=on would return an error. Unfortunately, the code forgot to remove "locking" from the options QDict, so any attempt to use the option would fail. Replace the option

[PULL 57/64] iotests: Add filter mirror test cases

2020-09-07 Thread Kevin Wolf
From: Max Reitz This patch adds some test cases how mirroring relates to filters. One of them tests what happens when you mirror off a filtered COW node, two others use the mirror filter node as basically our only example of an implicitly created filter node so far (besides the commit filter).

Re: [PATCH] MAINTAINERS: add Stefan Hajnoczi as block/nvme.c maintainer

2020-09-07 Thread Kevin Wolf
Am 07.09.2020 um 13:16 hat Stefan Hajnoczi geschrieben: > Development of the userspace NVMe block driver picked up again recently. > After talking with Fam I am stepping up as block/nvme.c maintainer. > Patches will be merged through my 'block' tree. > > Cc: Kevin Wolf > Cc: Klaus Jensen > Cc:

Re: [PATCH] MAINTAINERS: add Stefan Hajnoczi as block/nvme.c maintainer

2020-09-07 Thread Klaus Jensen
On Sep 7 12:16, Stefan Hajnoczi wrote: > Development of the userspace NVMe block driver picked up again recently. > After talking with Fam I am stepping up as block/nvme.c maintainer. > Patches will be merged through my 'block' tree. > > Cc: Kevin Wolf > Cc: Klaus Jensen > Cc: Fam Zheng >

Re: [PATCH] file-win32: Fix "locking" option

2020-09-07 Thread Max Reitz
On 07.09.20 11:27, Kevin Wolf wrote: > The intended behaviour was that locking=off/auto work and have no > effect (to remain compatible with file-posix), whereas locking=on would > return an error. Unfortunately, the code forgot to remove "locking" from > the options QDict, so any attempt to use

Re: [PATCH 17/17] hw/block/nvme: change controller pci id

2020-09-07 Thread Dr. David Alan Gilbert
* Klaus Jensen (i...@irrelevant.dk) wrote: > On Sep 7 11:37, Dr. David Alan Gilbert wrote: > > * Philippe Mathieu-Daudé (phi...@redhat.com) wrote: > > > +David in case > > > > > > On 9/4/20 4:19 PM, Klaus Jensen wrote: > > > > From: Klaus Jensen > > > > > > > > There are two reasons for

Re: [PATCH 17/17] hw/block/nvme: change controller pci id

2020-09-07 Thread Klaus Jensen
On Sep 7 11:37, Dr. David Alan Gilbert wrote: > * Philippe Mathieu-Daudé (phi...@redhat.com) wrote: > > +David in case > > > > On 9/4/20 4:19 PM, Klaus Jensen wrote: > > > From: Klaus Jensen > > > > > > There are two reasons for changing this: > > > > > > 1. The nvme device currently uses

[PULL 13/64] block/nvme: Replace BDRV_POLL_WHILE by AIO_WAIT_WHILE

2020-09-07 Thread Kevin Wolf
From: Philippe Mathieu-Daudé BDRV_POLL_WHILE() is defined as: #define BDRV_POLL_WHILE(bs, cond) ({ \ BlockDriverState *bs_ = (bs); \ AIO_WAIT_WHILE(bdrv_get_aio_context(bs_), \ cond); }) As we will remove the BlockDriverState use in the

[PULL 05/64] block/nvme: Define INDEX macros to ease code review

2020-09-07 Thread Kevin Wolf
From: Philippe Mathieu-Daudé Use definitions instead of '0' or '1' indexes. Also this will be useful when using multi-queues later. Reviewed-by: Stefan Hajnoczi Reviewed-by: Stefano Garzarella Signed-off-by: Philippe Mathieu-Daudé Message-Id: <20200821195359.1285345-5-phi...@redhat.com>

[PULL 14/64] block/nvme: Simplify nvme_create_queue_pair() arguments

2020-09-07 Thread Kevin Wolf
From: Philippe Mathieu-Daudé nvme_create_queue_pair() doesn't require BlockDriverState anymore. Replace it by BDRVNVMeState and AioContext to simplify. Reviewed-by: Stefan Hajnoczi Reviewed-by: Stefano Garzarella Signed-off-by: Philippe Mathieu-Daudé Message-Id:

[PULL 08/64] block/nvme: Rename local variable

2020-09-07 Thread Kevin Wolf
From: Philippe Mathieu-Daudé We are going to modify the code in the next commit. Renaming the 'resp' variable to 'id' first makes the next commit easier to review. No logical changes. Reviewed-by: Stefan Hajnoczi Reviewed-by: Stefano Garzarella Signed-off-by: Philippe Mathieu-Daudé

[PULL 15/64] block/nvme: Extract nvme_poll_queue()

2020-09-07 Thread Kevin Wolf
From: Philippe Mathieu-Daudé As we want to do per-queue polling, extract the nvme_poll_queue() method which operates on a single queue. Reviewed-by: Stefan Hajnoczi Reviewed-by: Stefano Garzarella Signed-off-by: Philippe Mathieu-Daudé Message-Id: <20200821195359.1285345-15-phi...@redhat.com>

[PULL 27/64] block: Use CAFs in block status functions

2020-09-07 Thread Kevin Wolf
From: Max Reitz Use the child access functions in the block status inquiry functions as appropriate. Signed-off-by: Max Reitz --- block/io.c | 19 ++- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/block/io.c b/block/io.c index 01e3477a77..4ee8fe5465 100644 ---

  1   2   >