Re: [Qemu-block] [PATCH v2 resend] block/mirror: change the semantic of 'force' of block-job-cancel

2018-03-02 Thread John Snow
On 03/02/2018 10:39 AM, Eric Blake wrote: > On 02/26/2018 08:05 PM, Liang Li wrote: >> When doing drive mirror to a low speed shared storage, if there was heavy >> BLK IO write workload in VM after the 'ready' event, drive mirror >> block job >> can't be canceled immediately, it would keep

Re: [Qemu-block] [RFC PATCH 0/2] Increase usability of external snapshots

2018-03-02 Thread Roman Kagan
On Tue, Feb 27, 2018 at 12:56:49PM +0100, Richard Palethorpe wrote: > Following on from the discussion about creating savevm/loadvm QMP > equivalents. I decided to take the advice given that we should use external > snapshots. However reverting to a snapshot currently requires QEMU to be >

Re: [Qemu-block] [PATCH] virtio-blk: notify guest directly

2018-03-02 Thread Sergio Lopez
On Fri, Mar 02, 2018 at 01:38:52PM +, Stefan Hajnoczi wrote: > On Tue, Dec 19, 2017 at 1:33 PM, sochin.jiang wrote: > > Till now, we've already notify guest as a batch mostly, an > > extra BH won't decrease guest interrupts much, but cause a > > significant

[Qemu-block] [PULL 16/37] vdi: Avoid bitrot of debugging code

2018-03-02 Thread Kevin Wolf
From: Eric Blake Rework the debug define so that we always get -Wformat checking, even when debugging is disabled. Signed-off-by: Eric Blake Reviewed-by: Stefan Weil Reviewed-by: Philippe Mathieu-Daudé Reviewed-by:

[Qemu-block] [PULL 18/37] vmdk: Switch to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the vmdk driver accordingly. Drop the now-unused vmdk_find_index_in_cluster(). Also, fix a pre-existing bug: if find_extent() fails (unlikely, since the block layer did a

[Qemu-block] [PULL 19/37] vpc: Switch to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the vpc driver accordingly. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Fam Zheng

[Qemu-block] [PULL 21/37] block: Drop unused .bdrv_co_get_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Now that all drivers have been updated to provide the byte-based .bdrv_co_block_status(), we can delete the sector-based interface. Signed-off-by: Eric Blake

[Qemu-block] [PULL 32/37] qcow2: make qcow2_co_create2() a coroutine_fn

2018-03-02 Thread Kevin Wolf
From: Stefan Hajnoczi qcow2_create2() calls qemu_co_mutex_lock(). Only a coroutine_fn may call another coroutine_fn. In fact, qcow2_create2 is always called from coroutine context. Rename the function to add the "co" moniker and add coroutine_fn. Reported-by: Marc-André

[Qemu-block] [PULL 34/37] block/ssh: Pull ssh_grow_file() from ssh_create()

2018-03-02 Thread Kevin Wolf
From: Max Reitz If we ever want to offer even rudimentary truncation functionality for ssh, we should put the respective code into a reusable function. Signed-off-by: Max Reitz Message-id: 20180214204915.7980-2-mre...@redhat.com Reviewed-by: Eric Blake

[Qemu-block] [PULL 06/37] iscsi: Switch cluster_sectors to byte-based

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Convert all uses of the cluster size in sectors, along with adding assertions that we are not dividing by zero. Improve some comment grammar while in

[Qemu-block] [PULL 04/37] file-posix: Switch to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the file protocol driver accordingly. In want_zero mode, we continue to report fine-grained hole information (the caller wants as much mapping detail as possible); but when

[Qemu-block] [PULL 02/37] nvme: Drop pointless .bdrv_co_get_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake Commit bdd6a90 has a bug: drivers should never directly set BDRV_BLOCK_ALLOCATED, but only io.c should do that (as needed). Instead, drivers should report BDRV_BLOCK_DATA if it knows that data comes from this BDS. But let's look at the bigger picture:

[Qemu-block] [PULL 14/37] raw: Switch to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the raw driver accordingly. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Fam Zheng

[Qemu-block] [PULL 33/37] qemu-img: Make resize error message more general

2018-03-02 Thread Kevin Wolf
From: Max Reitz The issue: $ qemu-img resize -f qcow2 foo.qcow2 qemu-img: Expecting one image file name Try 'qemu-img --help' for more information So we gave an image file name, but we omitted the length. qemu-img thinks the last argument is always the size and

[Qemu-block] [PULL 30/37] Revert "IDE: Do not flush empty CDROM drives"

2018-03-02 Thread Kevin Wolf
From: Stefan Hajnoczi This reverts commit 4da97120d51a4383aa96d741a2b837f8c4bbcd0b. blk_aio_flush() now handles the blk->root == NULL case, so we no longer need this workaround. Cc: John Snow Signed-off-by: Stefan Hajnoczi

[Qemu-block] [PULL 36/37] block/ssh: Add basic .bdrv_truncate()

2018-03-02 Thread Kevin Wolf
From: Max Reitz libssh2 does not seem to offer real truncation support, so we can only grow files -- but that is better than nothing. Signed-off-by: Max Reitz Message-id: 20180214204915.7980-4-mre...@redhat.com Reviewed-by: Eric Blake

[Qemu-block] [PULL 09/37] null: Switch to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the null driver accordingly. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Fam Zheng

[Qemu-block] [PULL 17/37] vdi: Switch to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the vdi driver accordingly. Note that the TODO is already covered (the block layer guarantees bounds of its requests), and that we can remove the now-unused

[Qemu-block] [PULL 12/37] qcow2: Switch to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the qcow2 driver accordingly. For now, we are ignoring the 'want_zero' hint. However, it should be relatively straightforward to honor the hint as a way to return larger

[Qemu-block] [PULL 13/37] qed: Switch to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the qed driver accordingly, taking the opportunity to inline qed_is_allocated_cb() into its lone caller (the callback used to be important, until we switched qed to

[Qemu-block] [PULL 37/37] qcow2: Replace align_offset() with ROUND_UP()

2018-03-02 Thread Kevin Wolf
From: Alberto Garcia The align_offset() function is equivalent to the ROUND_UP() macro so there's no need to use the former. The ROUND_UP() name is also a bit more explicit. This patch uses ROUND_UP() instead of the slower QEMU_ALIGN_UP() because align_offset() already

[Qemu-block] [PULL 01/37] block: Add .bdrv_co_block_status() callback

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Now that the block layer exposes byte-based allocation, it's time to tackle the drivers. Add a new callback that operates on as small as byte boundaries. Subsequent patches will

[Qemu-block] [PULL 00/37] Block layer patches

2018-03-02 Thread Kevin Wolf
The following changes since commit 86f4c7e05b1c44dbe1b329a51f311f10aef6ff34: Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20180302' into staging (2018-03-02 14:37:10 +) are available in the git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream

[Qemu-block] [PULL 03/37] block: Switch passthrough drivers to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the generic helpers, and all passthrough clients (blkdebug, commit, mirror, throttle) accordingly. Signed-off-by: Eric Blake Reviewed-by: Vladimir

[Qemu-block] [PULL 07/37] iscsi: Switch iscsi_allocmap_update() to byte-based

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Convert all uses of the allocmap (no semantic change). Callers that already had bytes available are simpler, and callers that now scale to bytes will

[Qemu-block] [PULL 05/37] gluster: Switch to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the gluster driver accordingly. In want_zero mode, we continue to report fine-grained hole information (the caller wants as much mapping detail as possible); but when not

[Qemu-block] [PULL 22/37] block: fix write with zero flag set and iovector provided

2018-03-02 Thread Kevin Wolf
From: Anton Nefedov The normal bdrv_co_pwritev() use is either - BDRV_REQ_ZERO_WRITE clear and iovector provided - BDRV_REQ_ZERO_WRITE set and iovector == NULL while - the flag clear and iovector == NULL is an assertion failure in bdrv_co_do_zero_pwritev()

[Qemu-block] [PULL 29/37] block: test blk_aio_flush() with blk->root == NULL

2018-03-02 Thread Kevin Wolf
This patch adds test cases for the scenario where blk_aio_flush() is called on a BlockBackend with no root. Calling drain afterwards should complete the requests with -ENOMEDIUM. Signed-off-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi Reviewed-by: Eric

[Qemu-block] [PULL 08/37] iscsi: Switch to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the iscsi driver accordingly. In this case, it is handy to teach iscsi_co_block_status() to handle a NULL map and file parameter, even though the block layer passes

[Qemu-block] [PULL 11/37] qcow: Switch to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the qcow driver accordingly. There is no intent to optimize based on the want_zero flag for this format. Signed-off-by: Eric Blake Reviewed-by:

[Qemu-block] [PULL 25/37] docs: document how to use the l2-cache-entry-size parameter

2018-03-02 Thread Kevin Wolf
From: Alberto Garcia This patch updates docs/qcow2-cache.txt explaining how to use the new l2-cache-entry-size parameter. Here's a more detailed technical description of this feature: https://lists.gnu.org/archive/html/qemu-block/2017-09/msg00635.html And here are some

[Qemu-block] [PULL 28/37] block: add BlockBackend->in_flight counter

2018-03-02 Thread Kevin Wolf
From: Stefan Hajnoczi BlockBackend currently relies on BlockDriverState->in_flight to track requests for blk_drain(). There is a corner case where BlockDriverState->in_flight cannot be used though: blk->root can be NULL when there is no medium. This results in a segfault

[Qemu-block] [PULL 27/37] block: extract AIO_WAIT_WHILE() from BlockDriverState

2018-03-02 Thread Kevin Wolf
From: Stefan Hajnoczi BlockDriverState has the BDRV_POLL_WHILE() macro to wait on event loop activity while a condition evaluates to true. This is used to implement synchronous operations where it acts as a condvar between the IOThread running the operation and the main

[Qemu-block] [PULL 10/37] parallels: Switch to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the parallels driver accordingly. Note that the internal function block_status() is still sector-based, because it is still in use by other sector-based functions; but

[Qemu-block] [PULL 24/37] specs/qcow2: Fix documentation of the compressed cluster descriptor

2018-03-02 Thread Kevin Wolf
From: Alberto Garcia This patch fixes several mistakes in the documentation of the compressed cluster descriptor: 1) the documentation claims that the cluster descriptor contains the number of sectors used to store the compressed data, but what it actually contains is

[Qemu-block] [PULL 15/37] sheepdog: Switch to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the sheepdog driver accordingly. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Fam Zheng

[Qemu-block] [PULL 23/37] iotest 033: add misaligned write-zeroes test via truncate

2018-03-02 Thread Kevin Wolf
From: Anton Nefedov This new test case only makes sense for qcow2 while iotest 033 is generic; however it matches the test purpose perfectly and also 033 contains those do_test() tricks to pass the alignment, which won't look nice being duplicated in other tests or

[Qemu-block] [PULL 31/37] block: rename .bdrv_create() to .bdrv_co_create_opts()

2018-03-02 Thread Kevin Wolf
From: Stefan Hajnoczi BlockDriver->bdrv_create() has been called from coroutine context since commit 5b7e1542cfa41a281af9629d31cef03704d976e6 ("block: make bdrv_create adopt coroutine"). Make this explicit by renaming to .bdrv_co_create_opts() and add the coroutine_fn

[Qemu-block] [PULL 20/37] vvfat: Switch to .bdrv_co_block_status()

2018-03-02 Thread Kevin Wolf
From: Eric Blake We are gradually moving away from sector-based interfaces, towards byte-based. Update the vvfat driver accordingly. Note that we can rely on the block driver having already clamped limits to our block size, and simplify accordingly. Signed-off-by: Eric

[Qemu-block] [PULL 26/37] aio: rename aio_context_in_iothread() to in_aio_context_home_thread()

2018-03-02 Thread Kevin Wolf
From: Stefan Hajnoczi The name aio_context_in_iothread() is misleading because it also returns true when called on the main AioContext from the main loop thread, which is not an IOThread. This patch renames it to in_aio_context_home_thread() and expands the doc comment to

[Qemu-block] [PULL 35/37] block/ssh: Make ssh_grow_file() blocking

2018-03-02 Thread Kevin Wolf
From: Max Reitz At runtime (that is, during a future ssh_truncate()), the SSH session is non-blocking. However, ssh_truncate() (or rather, bdrv_truncate() in general) is not a coroutine, so this resize operation needs to block. For ssh_create(), that is fine, too; the

Re: [Qemu-block] [PATCH v2 0/4] vl: introduce vm_shutdown()

2018-03-02 Thread Stefan Hajnoczi
On Fri, Mar 02, 2018 at 08:58:26AM +0800, Fam Zheng wrote: > On Thu, 03/01 14:54, Stefan Hajnoczi wrote: > > On Thu, Mar 01, 2018 at 09:15:17AM +0800, Fam Zheng wrote: > > > On Wed, 02/28 18:19, Stefan Hajnoczi wrote: > > > > v2: > > > > * Tackle the .ioeventfd_stop() vs vq handler race by

Re: [Qemu-block] [PATCH 3/9] nbd: BLOCK_STATUS for standard get_block_status function: server part

2018-03-02 Thread Vladimir Sementsov-Ogievskiy
16.02.2018 16:21, Eric Blake wrote: On 02/15/2018 07:51 AM, Vladimir Sementsov-Ogievskiy wrote: Minimal realization: only one extent in server answer is supported. Signed-off-by: Vladimir Sementsov-Ogievskiy --- [...] +    meta = _meta; +    } + +   

[Qemu-block] ping Re: [PATCH v2 0/2] block latency histogram

2018-03-02 Thread Vladimir Sementsov-Ogievskiy
ping 07.02.2018 15:50, Vladimir Sementsov-Ogievskiy wrote: v2: 01: add block_latency_histogram_clear() 02: fix spelling (sorry =() some rewordings remove histogram if latency parameter unspecified Vladimir Sementsov-Ogievskiy (2): block/accounting: introduce latency histogram

[Qemu-block] [PATCH] iotests: Mark all tests executable

2018-03-02 Thread Eric Blake
The majority of our iotests have the executable bit set; fix the few outliers for consistency. Signed-off-by: Eric Blake --- Let's see if patchew groks this one :) tests/qemu-iotests/096 | 0 tests/qemu-iotests/129 | 0 tests/qemu-iotests/132 | 0 tests/qemu-iotests/136 | 0

Re: [Qemu-block] [Qemu-devel] [PATCH] iotests: Mark all tests executable

2018-03-02 Thread Fam Zheng
On Fri, Mar 2, 2018 at 9:49 PM, Eric Blake wrote: > The majority of our iotests have the executable bit set; fix the > few outliers for consistency. > > Signed-off-by: Eric Blake > --- > > Let's see if patchew groks this one :) You've caught it! Patchew

[Qemu-block] ping Re: [PATCH v10 00/12] Dirty bitmaps postcopy migration

2018-03-02 Thread Vladimir Sementsov-Ogievskiy
ping 07.02.2018 18:58, Vladimir Sementsov-Ogievskiy wrote: Hi all! There is a new version of dirty bitmap postcopy migration series. Now it is based on Max's block tree: https://github.com/XanClic/qemu/commits/block, where it needs only one patch: "block: maintain persistent disabled

Re: [Qemu-block] [PATCH] virtio-blk: notify guest directly

2018-03-02 Thread Stefan Hajnoczi
On Tue, Dec 19, 2017 at 1:33 PM, sochin.jiang wrote: > Till now, we've already notify guest as a batch mostly, an > extra BH won't decrease guest interrupts much, but cause a > significant notification loss. Generally, we could have 15% > or so performance lost in

Re: [Qemu-block] [PATCH 1/9] nbd/server: add nbd_opt_invalid helper

2018-03-02 Thread Vladimir Sementsov-Ogievskiy
16.02.2018 01:01, Eric Blake wrote: On 02/15/2018 07:51 AM, Vladimir Sementsov-Ogievskiy wrote: NBD_REP_ERR_INVALID is often parameter to nbd_opt_drop and it would be used more in following patches. So, let's add a helper. Signed-off-by: Vladimir Sementsov-Ogievskiy

[Qemu-block] [PATCH v3] iotests: Tweak 030 in order to trigger a race condition with parallel jobs

2018-03-02 Thread Alberto Garcia
This patch tweaks TestParallelOps in iotest 030 so it allocates data in smaller regions (256KB/512KB instead of 512KB/1MB) and the block-stream job in test_stream_commit() only needs to copy data that is at the very end of the image. This way when the block-stream job is awakened it will finish

Re: [Qemu-block] [PATCH v2 resend] block/mirror: change the semantic of 'force' of block-job-cancel

2018-03-02 Thread Eric Blake
On 02/26/2018 08:05 PM, Liang Li wrote: When doing drive mirror to a low speed shared storage, if there was heavy BLK IO write workload in VM after the 'ready' event, drive mirror block job can't be canceled immediately, it would keep running until the heavy BLK IO workload stopped in the VM.