Re: [Qemu-block] [Qemu-devel] [PATCH 15/17] iotests: add default node-name

11.04.2017 00:49, John Snow wrote: On 02/17/2017 02:51 PM, Dr. David Alan Gilbert wrote: * Fam Zheng (f...@redhat.com) wrote: On Fri, 02/17 16:36, Vladimir Sementsov-Ogievskiy wrote: 17.02.2017 15:21, Fam Zheng wrote: On Fri, 02/17 13:20, Vladimir Sementsov-Ogievskiy wrote: 16.02.2017

Re: [Qemu-block] [Qemu-devel] [PATCH v9 03/13] qemu-io: Switch 'alloc' command to byte-based length

Hi Eric, On 04/10/2017 10:17 PM, Eric Blake wrote: For the 'alloc' command, accepting an offset in bytes but a length in sectors, and reporting output in sectors, is confusing. Do everything in bytes, and adjust the expected output accordingly. Signed-off-by: Eric Blake

Re: [Qemu-block] [PATCH v3] migration/block: use blk_pwrite_zeroes for each zero cluster

On Tue, Apr 11, 2017 at 12:00 AM, Stefan Hajnoczi wrote: > On Sun, Apr 09, 2017 at 08:37:40PM +0800, jemmy858...@gmail.com wrote: >> From: Lidong Chen >> >> BLOCK_SIZE is (1 << 20), qcow2 cluster size is 65536 by default, >> this maybe cause the qcow2

[Qemu-block] [PATCH v9 13/13] tests: Add coverage for recent block geometry fixes

Use blkdebug's new geometry constraints to emulate setups that have caused recent regression fixes: write zeroes asserting when running through a loopback block device with max-transfer smaller than cluster size, and discard rounding away portions of requests not aligned to preferred boundaries.

[Qemu-block] [PATCH v9 10/13] blkdebug: Add pass-through write_zero and discard support

In order to test the effects of artificial geometry constraints on operations like write zero or discard, we first need blkdebug to manage these actions. It also allows us to inject errors on those operations, just like we can for read/write/flush. We can also test the contract promised by the

[Qemu-block] [PATCH v9 12/13] blkdebug: Add ability to override unmap geometries

Make it easier to simulate various unusual hardware setups (for example, recent commits 3482b9b and b8d0a98 affect the Dell Equallogic iSCSI with its 15M preferred and maximum unmap and write zero sizing, or b2f95fe deals with the Linux loopback block device having a max_transfer of 64k), by

[Qemu-block] [PATCH v9 07/13] qcow2: Discard/zero clusters by byte count

Passing a byte offset, but sector count, when we ultimately want to operate on cluster granularity, is madness. Clean up the external interfaces to take both offset and count as bytes, while still keeping the assertion added previously that the caller must align the values to a cluster. Then

[Qemu-block] [PATCH v9 08/13] blkdebug: Sanity check block layer guarantees

Commits 04ed95f4 and 1a62d0ac updated the block layer to auto-fragment any I/O to fit within device boundaries. Additionally, when using a minimum alignment of 4k, we want to ensure the block layer does proper read-modify-write rather than requesting I/O on a slice of a sector. Let's enforce that

[Qemu-block] [PATCH v9 11/13] blkdebug: Simplify override logic

Rather than store into a local variable, then copy to the struct if the value is valid, then reporting errors otherwise, it is simpler to just store into the struct and report errors if the value is invalid. This however requires that the struct store a 64-bit number, rather than a narrower type.

[Qemu-block] [PATCH v9 05/13] qcow2: Optimize write zero of unaligned tail cluster

We've already improved discards to operate efficiently on the tail of an unaligned qcow2 image; it's time to make a similar improvement to write zeroes. The special case is only valid at the tail cluster of a file, where we must recognize that any sectors beyond the image end would implicitly

[Qemu-block] [PATCH v9 09/13] blkdebug: Refactor error injection

Rather than repeat the logic at each caller of checking if a Rule exists that warrants an error injection, fold that logic into inject_error(); and rename it to rule_check() for legibility. This will help the next patch, which adds two more callers that need to check rules for the potential of

[Qemu-block] [PATCH v9 06/13] qcow2: Assert that cluster operations are aligned

We already audited (in commit 0c1bd469) that qcow2_discard_clusters() is only passed cluster-aligned start values; but we can further tighten the assertion that the only unaligned end value is at EOF. Recent commits have taken advantage of an unaligned tail cluster, for both discard and write

[Qemu-block] [PATCH for-2.10 v9 00/13] add blkdebug tests

Available as a tag at: git fetch git://repo.or.cz/qemu/ericb.git nbd-blkdebug-v9 Prerequisite: Max's block-next tree: https://lists.gnu.org/archive/html/qemu-devel/2017-04/msg01298.html v6 was: https://lists.gnu.org/archive/html/qemu-devel/2017-03/msg01562.html v7 and v8 were the first half of

[Qemu-block] [PATCH v9 03/13] qemu-io: Switch 'alloc' command to byte-based length

For the 'alloc' command, accepting an offset in bytes but a length in sectors, and reporting output in sectors, is confusing. Do everything in bytes, and adjust the expected output accordingly. Signed-off-by: Eric Blake --- qemu-io-cmds.c| 30

[Qemu-block] [PATCH v9 02/13] iotests: Add test 179 to cover write zeroes with unmap

No tests were covering write zeroes with unmap. Additionally, I wanted to prove that my patch to optimize write zeroes for compat=0.10 images actually had an impact; for that, run: ./check -qcow2 -o compat=0.10 179 Writing the test to work correctly for both old and new qcow2 images is a bit

[Qemu-block] [PATCH v9 04/13] qemu-io: Switch 'map' output to byte-based reporting

Mixing byte offset and sector allocation counts is a bit confusing. Also, reporting n/m sectors, where m decreases according to the remaining size of the file, isn't really adding any useful information. Update the output to use byte counts, and adjust the affected tests (./check -qcow2 102,

[Qemu-block] [PATCH v9 01/13] qcow2: Unallocate unmapped zero clusters if no backing file

'qemu-img map' already coalesces information about unallocated clusters (those with status 0) and pure zero clusters (those with status BDRV_BLOCK_ZERO and no offset). Furthermore, all qcow2 images with no backing file already report all unallocated clusters (in the preallocation sense of

Re: [Qemu-block] [Qemu-devel] [PATCH 15/17] iotests: add default node-name

On 02/17/2017 02:51 PM, Dr. David Alan Gilbert wrote: > * Fam Zheng (f...@redhat.com) wrote: >> On Fri, 02/17 16:36, Vladimir Sementsov-Ogievskiy wrote: >>> 17.02.2017 15:21, Fam Zheng wrote: On Fri, 02/17 13:20, Vladimir Sementsov-Ogievskiy wrote: > 16.02.2017 16:48, Fam Zheng wrote:

Re: [Qemu-block] [Qemu-devel] [PATCH v2 for-2.10 0/8] RBD reopen, read_only cleanup

On 04/07/2017 04:55 PM, Jeff Cody wrote: > Changes from v1: > > Patch 2: Has v1 patch 8 (do not blindly xset bs->read_only) squashed into it > (thanks Stefan) > COW -> "copy-on-read" (Thanks John) > Drop unneeded call in vvfat, and bypass enable_write_target (Stefan)

Re: [Qemu-block] [PATCH v3 0/2] block: Quiesce old aio context during bdrv_set_aio_context

On Mon, Apr 10, 2017 at 10:26:34AM +0800, Fam Zheng wrote: > v3: Use bdrv_parent_drained_begin/end. [Kevin] > Do it before releasing new_context. [Stefan] > > Fam Zheng (2): > block: Make bdrv_parent_drained_begin/end public > block: Quiesce old aio context during bdrv_set_aio_context >

Re: [Qemu-block] [PATCH v3] migration/block: use blk_pwrite_zeroes for each zero cluster

On Sun, Apr 09, 2017 at 08:37:40PM +0800, jemmy858...@gmail.com wrote: > From: Lidong Chen > > BLOCK_SIZE is (1 << 20), qcow2 cluster size is 65536 by default, > this maybe cause the qcow2 file size is bigger after migration. > This patch check each cluster, use

Re: [Qemu-block] [PATCH v2 for-2.10 0/8] RBD reopen, read_only cleanup

On Fri, Apr 07, 2017 at 04:55:24PM -0400, Jeff Cody wrote: > > Changes from v1: > > Patch 2: Has v1 patch 8 (do not blindly xset bs->read_only) squashed into it > (thanks Stefan) > COW -> "copy-on-read" (Thanks John) > Drop unneeded call in vvfat, and bypass

Re: [Qemu-block] [PATCH for 2.9 v3 09/10] block: Use bdrv_coroutine_enter to start I/O coroutines

On 04/10/2017 10:05 AM, Fam Zheng wrote: > BDRV_POLL_WHILE waits for the started I/O by releasing bs's ctx then polling > the main context, which relies on the yielded the coroutine would continue on > bs->ctx and notify qemu_aio_context with bdrv_wakeup(). Thus, using Reads awkwardly. I'm

Re: [Qemu-block] [RFC] Proposed qcow2 extension: subcluster allocation

On Fri, Apr 07, 2017 at 03:01:29PM +0200, Kevin Wolf wrote: > Am 07.04.2017 um 14:20 hat Stefan Hajnoczi geschrieben: > > On Thu, Apr 06, 2017 at 06:01:48PM +0300, Alberto Garcia wrote: > > > Here are the results (subcluster size in brackets): > > > > > >

Re: [Qemu-block] [PATCH for-2.9] iscsi: Fix iscsi_create

On 10.04.2017 09:54, Fam Zheng wrote: > Since d5895fcb (iscsi: Split URL into individual options), creating > qcow2 image on an iscsi LUN fails: > > qemu-img create -f qcow2 iscsi://$SERVER/$IQN/0 1G > qemu-img: iscsi://$SERVER/$IQN/0: Could not create image: Invalid > argument >

[Qemu-block] [PATCH for 2.9 v3 06/10] block: Introduce bdrv_coroutine_enter and *_if_inactive

Signed-off-by: Fam Zheng --- block.c | 10 ++ include/block/block.h | 11 +++ 2 files changed, 21 insertions(+) diff --git a/block.c b/block.c index a995a8e..e65b906 100644 --- a/block.c +++ b/block.c @@ -4324,6 +4324,16 @@ AioContext

[Qemu-block] [PATCH for 2.9 v3 07/10] blockjob: Use bdrv_coroutine_enter to start coroutine

Resuming and especially starting of the block job coroutine, could be issued in the main thread. However the coroutine's "home" ctx should be set to the same context as job->blk. Use bdrv_coroutine_enter to ensure that. Signed-off-by: Fam Zheng --- blockjob.c | 4 ++-- 1 file

[Qemu-block] [PATCH for 2.9 v3 10/10] block: Fix bdrv_co_flush early return

bdrv_inc_in_flight and bdrv_dec_in_flight are mandatory for BDRV_POLL_WHILE to work, even for the shortcut case where flush is unnecessary. Move the if block to below bdrv_dec_in_flight, and BTW fix the variable declaration position. Signed-off-by: Fam Zheng --- block/io.c | 16

[Qemu-block] [PATCH for 2.9 v3 08/10] qemu-io-cmds: Use bdrv_coroutine_enter

qemu_coroutine_create associates @co to qemu_aio_context but we poll blk's context below. If the coroutine yields, it may never get resumed again. Use bdrv_coroutine_enter to make sure we are starting the I/O on the right context. Signed-off-by: Fam Zheng --- qemu-io-cmds.c |

[Qemu-block] [PATCH for 2.9 v3 09/10] block: Use bdrv_coroutine_enter to start I/O coroutines

BDRV_POLL_WHILE waits for the started I/O by releasing bs's ctx then polling the main context, which relies on the yielded the coroutine would continue on bs->ctx and notify qemu_aio_context with bdrv_wakeup(). Thus, using qemu_coroutine_enter to start I/O is wrong because if the coroutine is

[Qemu-block] [PATCH for 2.9 v3 02/10] block: Quiesce old aio context during bdrv_set_aio_context

The fact that the bs->aio_context is changing can confuse the dataplane iothread, because of the now fine granularity aio context lock. bdrv_drain should rather be a bdrv_drained_begin/end pair, but since bs->aio_context is changing, we can just use aio_disable_external and

[Qemu-block] [PATCH for 2.9 v3 01/10] block: Make bdrv_parent_drained_begin/end public

Signed-off-by: Fam Zheng --- block/io.c| 4 ++-- include/block/block.h | 16 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/block/io.c b/block/io.c index 7321dda..9598646 100644 --- a/block/io.c +++ b/block/io.c @@ -44,7 +44,7 @@

[Qemu-block] [PATCH for 2.9 v3 05/10] async: Introduce aio_co_enter and aio_co_enter_if_inactive

They start the coroutine on the specified context. Signed-off-by: Fam Zheng --- include/block/aio.h | 18 ++ util/async.c| 14 +- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/include/block/aio.h b/include/block/aio.h index

[Qemu-block] [PATCH for 2.9 v3 03/10] tests/block-job-txn: Don't start block job before adding to txn

Previously, before test_block_job_start returns, the job can already complete, as a result, the transactional state of other jobs added to the same txn later cannot be handled correctly. Move the block_job_start() calls to callers after block_job_txn_add_job() calls. Signed-off-by: Fam Zheng

[Qemu-block] [PATCH for 2.9 v3 00/10] block: Fixes regarding dataplane and management operations

v3: Respin the unmerged changes from v2 and include one new fix: (Yes, it is a big series for the last -rc, and I personally prefer the v2 approach for the 4-9 part of the problem, which is much more mechanical.) - 1, 2 are redoing previous patch 4, using

[Qemu-block] [PATCH for 2.9 v3 04/10] coroutine: Extract qemu_aio_coroutine_enter

It's a variant of qemu_coroutine_enter with an explicit AioContext parameter. Signed-off-by: Fam Zheng --- include/qemu/coroutine.h | 5 + util/qemu-coroutine.c| 11 --- util/trace-events| 2 +- 3 files changed, 14 insertions(+), 4 deletions(-) diff

Re: [Qemu-block] [RFC] Proposed qcow2 extension: subcluster allocation

On 10.04.2017 10:42, Kevin Wolf wrote: > Am 07.04.2017 um 19:10 hat Max Reitz geschrieben: >> One case I'd be especially interested in are of course 4 kB subclusters >> for 64 kB clusters (because 4 kB is a usual page size and can be >> configured to be the block size of a guest device; and

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.9] iscsi: Fix iscsi_create

On 04/10/2017 02:54 AM, Fam Zheng wrote: > Since d5895fcb (iscsi: Split URL into individual options), creating > qcow2 image on an iscsi LUN fails: > > qemu-img create -f qcow2 iscsi://$SERVER/$IQN/0 1G > qemu-img: iscsi://$SERVER/$IQN/0: Could not create image: Invalid > argument

Re: [Qemu-block] [PATCH v3] migration/block:limit the time used for block migration

On Sat, Apr 08, 2017 at 09:17:58PM +0800, 858585 jemmy wrote: > On Fri, Apr 7, 2017 at 7:34 PM, Stefan Hajnoczi wrote: > > On Fri, Apr 07, 2017 at 09:30:33AM +0800, 858585 jemmy wrote: > >> On Thu, Apr 6, 2017 at 10:02 PM, Stefan Hajnoczi > >> wrote: >

Re: [Qemu-block] [PATCH v3 1/6] vmdk: Move vmdk_find_offset_in_cluster() to the top

On Sat, Apr 1, 2017 at 8:14 PM, Ashijeet Acharya wrote: > Move the existing vmdk_find_offset_in_cluster() function to the top of > the driver. Also, introduce a new helper function size_to_clusters() > which returns the number of clusters for a given size in bytes.

Re: [Qemu-block] [Qemu-devel] [PATCH RFC 00/16] block: Protect AIO context change with perm API

On Mon, 04/10 10:04, Stefan Hajnoczi wrote: > On Tue, Mar 21, 2017 at 11:16:19AM +0800, Fam Zheng wrote: > > Eject / change of scsi-cd on a virtio-scsi dataplane bus causes abort() > > because > > the new BDS doesn't get proper bdrv_set_aio_context(). > > > > Store the AioContext in BB and do it

Re: [Qemu-block] [PATCH v3] migration/block:limit the time used for block migration

On Sat, Apr 08, 2017 at 06:09:16PM +0800, Paolo Bonzini wrote: > On 07/04/2017 19:33, Stefan Hajnoczi wrote: > > The migration thread is holding the QEMU global mutex, the AioContext, > > and the qcow2 s->lock while the L2 table is read from disk. > > > > The QEMU global mutex is needed for block

Re: [Qemu-block] [PATCH v2 for-2.10 13/16] block/qcow2: qcow2_calc_size_usage() for truncate

On Fri, Apr 07, 2017 at 05:42:16PM +0200, Max Reitz wrote: > On 06.04.2017 15:04, Stefan Hajnoczi wrote: > > On Mon, Apr 03, 2017 at 06:09:33PM +0200, Max Reitz wrote: > >> -/* total size of refcount tables */ > >> -nreftablee = nrefblocke / refblock_size; > >> -nreftablee =

Re: [Qemu-block] [PATCH v2 4/6] block: Quiesce old aio context during bdrv_set_aio_context

Am 10.04.2017 um 10:45 hat Fam Zheng geschrieben: > On Mon, 04/10 10:06, Kevin Wolf wrote: > > Am 08.04.2017 um 05:43 hat Fam Zheng geschrieben: > > > On Fri, 04/07 13:50, Stefan Hajnoczi wrote: > > > > On Fri, Apr 07, 2017 at 02:54:12PM +0800, Fam Zheng wrote: > > > > > @@ -4413,6 +4416,10 @@

Re: [Qemu-block] [PATCH RFC 00/16] block: Protect AIO context change with perm API

On Tue, Mar 21, 2017 at 11:16:19AM +0800, Fam Zheng wrote: > Eject / change of scsi-cd on a virtio-scsi dataplane bus causes abort() > because > the new BDS doesn't get proper bdrv_set_aio_context(). > > Store the AioContext in BB and do it in blk_insert_bs. That is done by > Vladimir's patch. >

Re: [Qemu-block] [PATCH RFC 04/16] block: Propagate BLK_PERM_AIO_CONTEXT_CHANGE down the graph

On Tue, Mar 21, 2017 at 11:16:23AM +0800, Fam Zheng wrote: > @@ -1713,21 +1714,22 @@ void bdrv_format_default_perms(BlockDriverState *bs, > BdrvChild *c, > perm |= BLK_PERM_CONSISTENT_READ; > shared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE); > } else { > -/* We want

Re: [Qemu-block] [PATCH v2 4/6] block: Quiesce old aio context during bdrv_set_aio_context

On Mon, 04/10 10:06, Kevin Wolf wrote: > Am 08.04.2017 um 05:43 hat Fam Zheng geschrieben: > > On Fri, 04/07 13:50, Stefan Hajnoczi wrote: > > > On Fri, Apr 07, 2017 at 02:54:12PM +0800, Fam Zheng wrote: > > > > @@ -4413,6 +4416,10 @@ void bdrv_set_aio_context(BlockDriverState *bs, > > > >

Re: [Qemu-block] [RFC] Proposed qcow2 extension: subcluster allocation

Am 07.04.2017 um 19:10 hat Max Reitz geschrieben: > One case I'd be especially interested in are of course 4 kB subclusters > for 64 kB clusters (because 4 kB is a usual page size and can be > configured to be the block size of a guest device; and because 64 kB > simply is the standard cluster

Re: [Qemu-block] [PATCH v2 4/6] block: Quiesce old aio context during bdrv_set_aio_context

Am 08.04.2017 um 05:43 hat Fam Zheng geschrieben: > On Fri, 04/07 13:50, Stefan Hajnoczi wrote: > > On Fri, Apr 07, 2017 at 02:54:12PM +0800, Fam Zheng wrote: > > > @@ -4413,6 +4416,10 @@ void bdrv_set_aio_context(BlockDriverState *bs, > > > AioContext *new_context) > > >

[Qemu-block] [PATCH for-2.9] iscsi: Fix iscsi_create

Since d5895fcb (iscsi: Split URL into individual options), creating qcow2 image on an iscsi LUN fails: qemu-img create -f qcow2 iscsi://$SERVER/$IQN/0 1G qemu-img: iscsi://$SERVER/$IQN/0: Could not create image: Invalid argument The problem is iscsi_open now expects that