Re: [Qemu-block] [PATCH v2 0/6] nbd export qmp interface

On 01/18/2018 12:11 PM, Vladimir Sementsov-Ogievskiy wrote: > v2: > 01: tweak comment > add Eric's r-b > 02: new patch > 03: rewritten, to move form 'bool force' flag to 'enum mode' parameter > 04: add Eric's r-b > 05: improve commit message > tweak comment > 06: rebase on 03 changes >

Re: [Qemu-block] [PATCH v2 6/6] iotest 201: new test for qmp nbd-server-remove

On 01/18/2018 12:11 PM, Vladimir Sementsov-Ogievskiy wrote: > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > tests/qemu-iotests/201 | 159 > + > tests/qemu-iotests/201.out | 5 ++ > tests/qemu-iotests/group | 1

Re: [Qemu-block] [PATCH v2 3/6] qapi: add nbd-server-remove

On 01/18/2018 12:11 PM, Vladimir Sementsov-Ogievskiy wrote: > Add command for export removing. It is needed for cases when we s/export removing/removing an export/ > don't want to keep export after the operation on it was completed. > The other example is temporary node, created with

Re: [Qemu-block] [PATCH v2 2/6] hmp: add name parameter to nbd_server_add

On 01/18/2018 12:11 PM, Vladimir Sementsov-Ogievskiy wrote: > Support name parameter for HMP too. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > hmp.c | 3 ++- > hmp-commands.hx | 9 + > 2 files changed, 7 insertions(+), 5 deletions(-) > >

[Qemu-block] [PULL v4 03/29] vhost-user-blk: introduce a new vhost-user-blk host device

From: Changpeng Liu This commit introduces a new vhost-user device for block, it uses a chardev to connect with the backend, same with Qemu virito-blk device, Guest OS still uses the virtio-blk frontend driver. To use it, start QEMU with command line like this:

Re: [Qemu-block] [Qemu-devel] [PATCH v3 0/7] Call check and invalidate_cache from coroutine context

On 01/18/2018 06:43 AM, Paolo Bonzini wrote: > Check and invalidate_cache share some parts of the implementation > with the regular I/O path. This is sometimes complicated because the > I/O path wants to use a CoMutex but that is not possible outside coroutine > context. By moving things to

Re: [Qemu-block] [Qemu-devel] [PATCH 4/7] qed: make bdrv_qed_do_open a coroutine_fn

On 01/18/2018 06:43 AM, Paolo Bonzini wrote: > It is called from qcow2_invalidate_cache in coroutine context (incoming Why is a qcow2 function calling qed code? Definitely a copy-paste bug, but I'm not sure of the right fix. > migration runs in a coroutine), so it's cleaner if metadata is

Re: [Qemu-block] [Qemu-devel] [PATCH] block: implement the bdrv_reopen_prepare helper for LUKS driver

On 01/18/2018 04:31 AM, Daniel P. Berrange wrote: > If the bdrv_reopen_prepare helper isn't provided, the qemu-img commit > command fails to re-open the base layer after committing changes into > it. Provide a no-op implementation for the LUKS driver, since there > is not any custom work that

[Qemu-block] [PATCH v2 1/6] qapi: add name parameter to nbd-server-add

Allow user to specify name for new export, to not reuse internal node name and to not show it to clients. This also allows creating several exports per device. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- qapi/block.json

[Qemu-block] [PATCH v2 0/6] nbd export qmp interface

v2: 01: tweak comment add Eric's r-b 02: new patch 03: rewritten, to move form 'bool force' flag to 'enum mode' parameter 04: add Eric's r-b 05: improve commit message tweak comment 06: rebase on 03 changes make PEP8 happy some other tweaks I've left nbd_port variable hard-set

[Qemu-block] [PATCH v2 3/6] qapi: add nbd-server-remove

Add command for export removing. It is needed for cases when we don't want to keep export after the operation on it was completed. The other example is temporary node, created with blockdev-add. If we want to delete it we should firstly remove corresponding NBD export. Signed-off-by: Vladimir

[Qemu-block] [PATCH v2 4/6] iotest 147: add cases to test new @name parameter of nbd-server-add

Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- tests/qemu-iotests/147 | 68 +- tests/qemu-iotests/147.out | 4 +-- 2 files changed, 57 insertions(+), 15 deletions(-) diff

[Qemu-block] [PATCH v2 2/6] hmp: add name parameter to nbd_server_add

Support name parameter for HMP too. Signed-off-by: Vladimir Sementsov-Ogievskiy --- hmp.c | 3 ++- hmp-commands.hx | 9 + 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/hmp.c b/hmp.c index 37972f8322..964cff8aed 100644 --- a/hmp.c +++

[Qemu-block] [PATCH v2 5/6] iotests: implement QemuIoInteractive class

Implement QemuIoInteractive to test nbd-server-remove command when there are active connections. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 38 ++ 1 file changed, 38 insertions(+) diff --git

[Qemu-block] [PATCH v2 6/6] iotest 201: new test for qmp nbd-server-remove

Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/201 | 159 + tests/qemu-iotests/201.out | 5 ++ tests/qemu-iotests/group | 1 + 3 files changed, 165 insertions(+) create mode 100644

[Qemu-block] [PATCH v7 2/9] blkverify: set supported write/zero flags

Signed-off-by: Anton Nefedov Reviewed-by: Alberto Garcia --- block/blkverify.c | 9 + 1 file changed, 9 insertions(+) diff --git a/block/blkverify.c b/block/blkverify.c index 06369f9..9ba65d0 100644 --- a/block/blkverify.c +++

[Qemu-block] [PATCH v7 9/9] iotest 134: test cluster-misaligned encrypted write

COW (even empty/zero) areas require encryption too Signed-off-by: Anton Nefedov Reviewed-by: Eric Blake --- tests/qemu-iotests/134 | 9 + tests/qemu-iotests/134.out | 10 ++ 2 files changed, 19 insertions(+) diff --git

[Qemu-block] [PATCH v7 4/9] block: treat BDRV_REQ_ALLOCATE as serialising

The idea is that ALLOCATE requests may overlap with other requests. Reuse the existing block layer infrastructure for serialising requests. Use the following approach: - mark ALLOCATE serialising, so subsequent requests to the area wait - ALLOCATE request itself must never wait if another

[Qemu-block] [PATCH v7 3/9] block: introduce BDRV_REQ_ALLOCATE flag

The flag is supposed to indicate that the region of the disk image has to be sufficiently allocated so it reads as zeroes. The call with the flag set must return -ENOTSUP if allocation cannot be done efficiently. This has to be made sure of by both - the drivers that support the flag - and

[Qemu-block] [PATCH v7 6/9] block: support BDRV_REQ_ALLOCATE in passthrough drivers

Support the flag if the underlying BDS supports it Signed-off-by: Anton Nefedov Reviewed-by: Alberto Garcia --- block/blkdebug.c | 3 ++- block/blkverify.c | 2 +- block/mirror.c | 2 +- block/raw-format.c | 3 ++- 4 files changed, 6

[Qemu-block] [PATCH v7 1/9] mirror: inherit supported write/zero flags

Signed-off-by: Anton Nefedov Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia --- block/mirror.c | 5 + 1 file changed, 5 insertions(+) diff --git a/block/mirror.c b/block/mirror.c index c9badc1..d18ec65 100644 ---

[Qemu-block] [PATCH v7 0/9] qcow2: cluster space preallocation

(used to be 'qcow2: preallocation and COW improvements') v7: patch 8/9: - is_zero_cow() function reworked - blkdbg event added - write-zeroes errors handled - iotest 60 fixed properly v6: http://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg03463.html Anton Nefedov (9):

[Qemu-block] [PATCH v7 8/9] qcow2: skip writing zero buffers to empty COW areas

If COW areas of the newly allocated clusters are zeroes on the backing image, efficient bdrv_write_zeroes(flags=BDRV_REQ_ALLOCATE) can be used on the whole cluster instead of writing explicit zero buffers later in perform_cow(). iotest 060: write to the discarded cluster does not trigger COW

[Qemu-block] [PATCH v7 7/9] qcow2: move is_zero() up

To be used in the following commit without a forward declaration. Signed-off-by: Anton Nefedov Reviewed-by: Alberto Garcia --- block/qcow2.c | 35 +-- 1 file changed, 17 insertions(+), 18 deletions(-) diff --git

[Qemu-block] [PATCH v7 5/9] file-posix: support BDRV_REQ_ALLOCATE

Current write_zeroes implementation is good enough to satisfy this flag too Signed-off-by: Anton Nefedov Reviewed-by: Alberto Garcia --- block/file-posix.c | 8 1 file changed, 8 insertions(+) diff --git a/block/file-posix.c

[Qemu-block] [PATCH v4] ssh: switch from libssh2 to libssh

Rewrite the implementation of the ssh block driver to use libssh instead of libssh2. The libssh library has various advantages over libssh2: - easier API for authentication (for example for using ssh-agent) - easier API for known_hosts handling - supports newer types of keys in known_hosts

Re: [Qemu-block] [Qemu-devel] [PATCH v3] ssh: switch from libssh2 to libssh

On Monday, 18 December 2017 20:55:19 CET Jeff Cody wrote: > On Wed, Nov 15, 2017 at 05:26:48PM +0100, Pino Toscano wrote: > > Rewrite the implementation of the ssh block driver to use libssh instead > > of libssh2. The libssh library has various advantages over libssh2: > > - easier API for

Re: [Qemu-block] [Qemu-devel] [PATCH v2 31/32] qcow2: Allow configuring the L2 slice size

On Wed 17 Jan 2018 12:18:25 AM CET, Eric Blake wrote: > Is there a QMP counterpart to the command-line option? Not in this revision, but I'll add one. Berto

Re: [Qemu-block] [Qemu-devel] [PATCH v2 24/32] qcow2: Update qcow2_update_snapshot_refcount() to support L2 slices

On Wed 17 Jan 2018 12:00:59 AM CET, Eric Blake wrote: >> -l2_table = NULL; >> +l2_slice = NULL; >> l1_table = NULL; >> l1_size2 = l1_size * sizeof(uint64_t); >> >> +slice_size = s->l2_slice_size * sizeof(uint64_t); > > Again, better naming on s->l2_slice_size in an earlier

Re: [Qemu-block] [Qemu-devel] [PATCH v2 16/32] qcow2: Update l2_allocate() to support L2 slices

On Wed 17 Jan 2018 05:57:32 PM CET, Anton Nefedov wrote: > On 17/1/2018 6:42 PM, Alberto Garcia wrote: >> On Tue 16 Jan 2018 05:52:36 PM CET, Anton Nefedov wrote: >>> >>> Cosmetic: maybe this l2_load() better be merged with the copied L2 case? >>> e.g. >>> >>> if (!(l1_table[l1_index] &

[Qemu-block] [PATCH 5/7] block: convert bdrv_invalidate_cache callback to coroutine_fn

QED's bdrv_invalidate_cache implementation would like to reuse functions that acquire/release the metadata locks. Call it from coroutine context to simplify the logic. Signed-off-by: Paolo Bonzini --- block.c | 41 +

[Qemu-block] [PATCH 3/7] qcow2: make qcow2_do_open a coroutine_fn

It is called from qcow2_invalidate_cache in coroutine context (incoming migration runs in a coroutine), so it's cleaner if metadata is always loaded from a coroutine. Signed-off-by: Paolo Bonzini --- block/qcow2.c | 46 +- 1 file

[Qemu-block] [PATCH 7/7] block: convert bdrv_check callback to coroutine_fn

Suggested-by: Kevin Wolf Signed-off-by: Paolo Bonzini --- block.c | 43 --- block/parallels.c | 17 +++-- block/qcow2.c | 23 +++

[Qemu-block] [PATCH 6/7] qcow2: introduce qcow2_write_caches and qcow2_flush_caches

Avoid recursively taking s->lock during bdrv_check. Signed-off-by: Paolo Bonzini --- block/qcow2-refcount.c | 28 block/qcow2.c | 20 block/qcow2.h | 2 ++ 3 files changed, 34 insertions(+), 16

[Qemu-block] [PATCH v3 0/7] Call check and invalidate_cache from coroutine context

Check and invalidate_cache share some parts of the implementation with the regular I/O path. This is sometimes complicated because the I/O path wants to use a CoMutex but that is not possible outside coroutine context. By moving things to coroutine context, we can remove special cases. In fact,

[Qemu-block] [PATCH 1/7] block: rename .bdrv_create() to .bdrv_co_create_opts()

From: Stefan Hajnoczi BlockDriver->bdrv_create() has been called from coroutine context since commit 5b7e1542cfa41a281af9629d31cef03704d976e6 ("block: make bdrv_create adopt coroutine"). Make this explicit by renaming to .bdrv_co_create_opts() and add the coroutine_fn

[Qemu-block] [PATCH 2/7] qcow2: make qcow2_co_create2() a coroutine_fn

From: Stefan Hajnoczi qcow2_create2() calls qemu_co_mutex_lock(). Only a coroutine_fn may call another coroutine_fn. In fact, qcow2_create2 is always called from coroutine context. Rename the function to add the "co" moniker and add coroutine_fn. Reported-by: Marc-André

[Qemu-block] [PATCH 4/7] qed: make bdrv_qed_do_open a coroutine_fn

It is called from qcow2_invalidate_cache in coroutine context (incoming migration runs in a coroutine), so it's cleaner if metadata is always loaded from a coroutine. Signed-off-by: Paolo Bonzini --- block/qed.c | 40 +--- 1 file changed,

Re: [Qemu-block] [PATCH 1/1] nbd: implement bdrv_get_info callback

On 18/01/2018 12:51, Edgar Kaziakhmedov wrote: > > +static int nbd_get_info(BlockDriverState *bs, BlockDriverInfo *bdi) > +{ > +if (bs->supported_zero_flags & BDRV_REQ_MAY_UNMAP) { > +bdi->can_write_zeroes_with_unmap = true; > +} > +return 0; > +} > + Other drivers set the

[Qemu-block] [PATCH 1/1] nbd: implement bdrv_get_info callback

Since mirror job supports efficient zero out target mechanism (see in mirror_dirty_init()), implement bdrv_get_info to make it work over NBD. Such improvement will allow using the largest chunk possible and will decrease the number of NBD_CMD_WRITE_ZEROES requests on the wire. Signed-off-by:

Re: [Qemu-block] [PATCH] block: add block_set_io_throttle virtio-blk-pci QMP example

On Wed, Jan 17, 2018 at 09:07:00AM +, Stefan Hajnoczi wrote: > The block_set_io_throttle command can look up BlockBackends by the > attached qdev device ID. virtio-blk-pci is a special case because the > actual VirtIOBlock device is the "/virtio-backend" child of the PCI > adapter device. >

[Qemu-block] [PATCH] block: implement the bdrv_reopen_prepare helper for LUKS driver

If the bdrv_reopen_prepare helper isn't provided, the qemu-img commit command fails to re-open the base layer after committing changes into it. Provide a no-op implementation for the LUKS driver, since there is not any custom work that needs doing to re-open it. Signed-off-by: Daniel P. Berrange

Re: [Qemu-block] [Qemu-devel] [PATCH v9 03/13] block/dirty-bitmap: add _locked version of bdrv_reclaim_dirty_bitmap

On 18/01/2018 10:55, Vladimir Sementsov-Ogievskiy wrote: >> >> Most functions that looks at the list are "called with BQL taken". >> Functions that write to the list are "called with BQL taken" and call >> bdrv_dirty_bitmaps_lock/bdrv_dirty_bitmaps_unlock themselves. > > Paolo, could you please

Re: [Qemu-block] [Qemu-devel] [PATCH v9 13/13] iotests: add dirty bitmap postcopy test

17.01.2018 21:30, John Snow wrote: On 12/28/2017 06:49 AM, Vladimir Sementsov-Ogievskiy wrote: Thank you for reviewing my code! Time for the re-spin? There's pretty good pressure to get this into 2.12 and say the non-nbd workflow model is feature complete. Yes, you are right. Hope to do it

Re: [Qemu-block] [Qemu-devel] [PATCH v9 03/13] block/dirty-bitmap: add _locked version of bdrv_reclaim_dirty_bitmap

18.01.2018 11:43, Paolo Bonzini wrote: On 29/12/2017 02:31, Fam Zheng wrote: we have the following comment:     /* Writing to the list requires the BQL _and_ the dirty_bitmap_mutex. * Reading from the list can be done with either the BQL or the * dirty_bitmap_mutex.  Modifying a

Re: [Qemu-block] [Qemu-devel] [PATCH v9 03/13] block/dirty-bitmap: add _locked version of bdrv_reclaim_dirty_bitmap

On 29/12/2017 02:31, Fam Zheng wrote: >> we have the following comment: >> >>     /* Writing to the list requires the BQL _and_ the dirty_bitmap_mutex. >> * Reading from the list can be done with either the BQL or the >> * dirty_bitmap_mutex.  Modifying a bitmap only requires >> *