Re: [Qemu-block] [PATCH v2] migration/block: use blk_pwrite_zeroes for each zero cluster

2017-04-07 Thread 858585 jemmy
On Fri, Apr 7, 2017 at 6:10 PM, Fam Zheng wrote: > On Fri, 04/07 16:44, jemmy858...@gmail.com wrote: >> From: Lidong Chen >> >> BLOCK_SIZE is (1 << 20), qcow2 cluster size is 65536 by default, >> this maybe cause the qcow2 file size is bigger after

Re: [Qemu-block] [PATCH v2 4/6] block: Quiesce old aio context during bdrv_set_aio_context

2017-04-07 Thread Fam Zheng
On Fri, 04/07 13:50, Stefan Hajnoczi wrote: > On Fri, Apr 07, 2017 at 02:54:12PM +0800, Fam Zheng wrote: > > @@ -4413,6 +4416,10 @@ void bdrv_set_aio_context(BlockDriverState *bs, > > AioContext *new_context) > > aio_context_acquire(new_context); > > bdrv_attach_aio_context(bs,

Re: [Qemu-block] [PATCH v2 6/6] tests/block-job-txn: Don't start block job before adding to txn

2017-04-07 Thread Fam Zheng
On Fri, 04/07 14:05, John Snow wrote: > > > On 04/07/2017 09:28 AM, Stefan Hajnoczi wrote: > > On Fri, Apr 07, 2017 at 02:54:14PM +0800, Fam Zheng wrote: > >> Previously, before test_block_job_start returns, the job can already > >> complete, as a result, the transactional state of other jobs

Re: [Qemu-block] [PATCH v2 5/6] coroutine: Explicitly specify AioContext when entering coroutine

2017-04-07 Thread Fam Zheng
On Fri, 04/07 14:26, Stefan Hajnoczi wrote: > I wasn't expecting the patch to touch so many places. If a coroutine > can be permanently associated with an AioContext then there's no need to > keep assigning co->ctx on each qemu_coroutine_enter(). Maybe, but bs->job->co->ctx changes when

[Qemu-block] [PATCH v2 for-2.10 4/8] block: code movement

2017-04-07 Thread Jeff Cody
Move bdrv_is_read_only() up with its friends. Reviewed-by: Stefan Hajnoczi Reviewed-by: John Snow Signed-off-by: Jeff Cody --- block.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/block.c b/block.c

[Qemu-block] [PATCH v2 for-2.10 8/8] block/rbd: Add support for reopen()

2017-04-07 Thread Jeff Cody
This adds support for reopen in rbd, for changing between r/w and r/o. Note, that this is only a flag change, but we will block a change from r/o to r/w if we are using an RBD internal snapshot. Reviewed-by: Stefan Hajnoczi Signed-off-by: Jeff Cody ---

[Qemu-block] [PATCH v2 for-2.10 7/8] block/rbd - update variable names to more apt names

2017-04-07 Thread Jeff Cody
Update 'clientname' to be 'user', which tracks better with both the QAPI and rados variable naming. Update 'name' to be 'image_name', as it indicates the rbd image. Naming it 'image' would have been ideal, but we are using that for the rados_image_t value returned by rbd_open(). Reviewed-by:

[Qemu-block] [PATCH v2 for-2.10 6/8] block: use bdrv_can_set_read_only() during reopen

2017-04-07 Thread Jeff Cody
Signed-off-by: Jeff Cody --- block.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/block.c b/block.c index 1514ae9..5d560f5 100644 --- a/block.c +++ b/block.c @@ -2785,6 +2785,7 @@ int bdrv_reopen_prepare(BDRVReopenState *reopen_state,

[Qemu-block] [PATCH v2 for-2.10 5/8] block: introduce bdrv_can_set_read_only()

2017-04-07 Thread Jeff Cody
Introduce check function for setting read_only flags. Will return < 0 on error, with appropriate Error value set. Does not alter any flags. Signed-off-by: Jeff Cody --- block.c | 14 +- include/block/block.h | 1 + 2 files changed, 14

[Qemu-block] [PATCH v2 for-2.10 2/8] block: do not set BDS read_only if copy_on_read enabled

2017-04-07 Thread Jeff Cody
A few block drivers will set the BDS read_only flag from their .bdrv_open() function. This means the bs->read_only flag could be set after we enable copy_on_read, as the BDRV_O_COPY_ON_READ flag check occurs prior to the call to bdrv->bdrv_open(). This adds an error return to

[Qemu-block] [PATCH v2 for-2.10 0/8] RBD reopen, read_only cleanup

2017-04-07 Thread Jeff Cody
Changes from v1: Patch 2: Has v1 patch 8 (do not blindly xset bs->read_only) squashed into it (thanks Stefan) COW -> "copy-on-read" (Thanks John) Drop unneeded call in vvfat, and bypass enable_write_target (Stefan) Patch 5: Rename bdrv_try_... to

[Qemu-block] [PATCH v2 for-2.10 1/8] block: add bdrv_set_read_only() helper function

2017-04-07 Thread Jeff Cody
We have a helper wrapper for checking for the BDS read_only flag, add a helper wrapper to set the read_only flag as well. Reviewed-by: Stefan Hajnoczi Signed-off-by: Jeff Cody --- block.c | 5 + block/bochs.c | 2 +-

Re: [Qemu-block] [PATCH v2 6/6] tests/block-job-txn: Don't start block job before adding to txn

2017-04-07 Thread John Snow
On 04/07/2017 09:28 AM, Stefan Hajnoczi wrote: > On Fri, Apr 07, 2017 at 02:54:14PM +0800, Fam Zheng wrote: >> Previously, before test_block_job_start returns, the job can already >> complete, as a result, the transactional state of other jobs added to >> the same txn later cannot be handled

Re: [Qemu-block] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-07 Thread Max Reitz
On 06.04.2017 17:01, Alberto Garcia wrote: > Hi all, > > over the past couple of months I discussed with some of you the > possibility to extend the qcow2 format in order to improve its > performance and reduce its memory requirements (particularly with very > large images). > > After some

Re: [Qemu-block] [PATCH v2 for-2.10 13/16] block/qcow2: qcow2_calc_size_usage() for truncate

2017-04-07 Thread Max Reitz
On 06.04.2017 15:04, Stefan Hajnoczi wrote: > On Mon, Apr 03, 2017 at 06:09:33PM +0200, Max Reitz wrote: >> +BDRVQcow2State *s = bs->opaque; >> +uint64_t aligned_cur_size = align_offset(current_size, >> cluster_size); >> +uint64_t creftable_length; >> +uint64_t i;

Re: [Qemu-block] [Qemu-devel] [PULL 00/10] Block layer fixes for 2.9.0-rc4

2017-04-07 Thread Peter Maydell
On 7 April 2017 at 14:47, Kevin Wolf wrote: > The following changes since commit 5fe2339e6b09da7d6f48b9bef0f1a7360392b489: > > Merge remote-tracking branch > 'remotes/awilliam/tags/vfio-updates-20170406.0' into staging (2017-04-07 > 10:29:56 +0100) > > are available in the

Re: [Qemu-block] [PATCH v2 5/6] coroutine: Explicitly specify AioContext when entering coroutine

2017-04-07 Thread Kevin Wolf
Am 07.04.2017 um 08:54 hat Fam Zheng geschrieben: > Coroutine in block layer should always be waken up in bs->aio_context > rather than the "current" context where it is entered. They differ when > the main loop is doing QMP tasks. > > Race conditions happen without this patch, because the wrong

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-07 Thread Alberto Garcia
On Fri 07 Apr 2017 02:41:21 PM CEST, Kevin Wolf wrote: >> 6356 5548 4740 3932 3124 2316 15 8 7 0 >> >> **<> <---><-->*

Re: [Qemu-block] [PATCH for-2.9] throttle: Remove block from group on hot-unplug

2017-04-07 Thread Stefan Hajnoczi
On Thu, Apr 06, 2017 at 02:08:47PM -0500, Eric Blake wrote: > When a block device that is part of a throttle group is hot-unplugged, > we forgot to remove it from the throttle group. This leaves stale > memory around, and causes an easily reproducible crash: > > $

Re: [Qemu-block] [PATCH for-2.9?] qcow2: Allow discard of final unaligned cluster

2017-04-07 Thread Max Reitz
On 07.04.2017 03:37, Eric Blake wrote: > As mentioned in commit 0c1bd46, we ignored requests to > discard the trailing cluster of an unaligned image. While > discard is an advisory operation from the guest standpoint, > (and we are therefore free to ignore any request), our > qcow2 implementation

Re: [Qemu-block] [PATCH for-2.9] throttle: Remove block from group on hot-unplug

2017-04-07 Thread Max Reitz
On 06.04.2017 21:08, Eric Blake wrote: > When a block device that is part of a throttle group is hot-unplugged, > we forgot to remove it from the throttle group. This leaves stale > memory around, and causes an easily reproducible crash: > > $ ./x86_64-softmmu/qemu-system-x86_64 -nodefaults

Re: [Qemu-block] [Qemu-devel] [PULL 07/10] block: Don't check permissions for copy on read

2017-04-07 Thread Eric Blake
On 04/07/2017 08:47 AM, Kevin Wolf wrote: > The assertion is currently failing. We can't require callers to have > write permissions when all they are doing is a read, so comment it out. > Add a FIXME comment in the code so that the check is re-enabled when > copy on read is refactored into its

Re: [Qemu-block] [PATCH v2 5/6] coroutine: Explicitly specify AioContext when entering coroutine

2017-04-07 Thread Eric Blake
On 04/07/2017 01:54 AM, Fam Zheng wrote: > Coroutine in block layer should always be waken up in bs->aio_context s/waken up/awakened/ > rather than the "current" context where it is entered. They differ when > the main loop is doing QMP tasks. > > Race conditions happen without this patch,

Re: [Qemu-block] [PATCH for-2.9] block: Don't check permissions for copy on read

2017-04-07 Thread Max Reitz
On 07.04.2017 15:53, Eric Blake wrote: > On 04/07/2017 05:32 AM, Kevin Wolf wrote: >> The assertion is currently failing. We can't require callers to have >> write permissions when all they are doing is a read, so comment it out. >> Add a FIXME comment in the code so that the check is re-enabled

Re: [Qemu-block] [PATCH for-2.9] block: Don't check permissions for copy on read

2017-04-07 Thread Eric Blake
On 04/07/2017 05:32 AM, Kevin Wolf wrote: > The assertion is currently failing. We can't require callers to have > write permissions when all they are doing is a read, so comment it out. > Add a FIXME comment in the code so that the check is re-enabled when > copy on read is refactored into its

Re: [Qemu-block] [Qemu-devel] New iotest repros failures on virtio external snapshot with iothread

2017-04-07 Thread Max Reitz
On 30.03.2017 04:16, Eric Blake wrote: > On 03/29/2017 09:01 PM, Ed Swierk via Qemu-devel wrote: >> Parts of qemu's block code have changed a lot in recent months but are >> not well exercised by current tests. >> >> Subtle bugs have crept in causing assertion failures, hangs and other >> crashes

[Qemu-block] [PULL 08/10] block: Fix unpaired aio_disable_external in external snapshot

2017-04-07 Thread Kevin Wolf
From: Fam Zheng bdrv_replace_child_noperm tries to hand over the quiesce_counter state from old bs to the new one, but if they are not on the same aio context this causes unbalance. Fix this by setting the correct aio context before calling bdrv_append(). Reported-by: Ed

[Qemu-block] [PULL 10/10] mirror: Fix aio context of mirror_top_bs

2017-04-07 Thread Kevin Wolf
From: Fam Zheng It should be moved to the same context as source, before inserting to the graph. Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf Signed-off-by: Fam Zheng Signed-off-by: Kevin Wolf ---

[Qemu-block] [PULL 06/10] qemu-img: img_create does not support image-opts, fix docs

2017-04-07 Thread Kevin Wolf
From: Jeff Cody The documentation and help for qemu-img claims that 'qemu-img create' will take the '--image-opts' argument. This is not true, so this patch removes those claims. Signed-off-by: Jeff Cody Reviewed-by: Eric Blake

[Qemu-block] [PULL 09/10] block: Assert attached child node has right aio context

2017-04-07 Thread Kevin Wolf
From: Fam Zheng Suggested-by: Kevin Wolf Signed-off-by: Fam Zheng Signed-off-by: Kevin Wolf --- block.c | 4 1 file changed, 4 insertions(+) diff --git a/block.c b/block.c index 927ba89..b8a3011 100644 --- a/block.c

[Qemu-block] [PULL 01/10] block: Ignore guest dev permissions during incoming migration

2017-04-07 Thread Kevin Wolf
Usually guest devices don't like other writers to the same image, so they use blk_set_perm() to prevent this from happening. In the migration phase before the VM is actually running, though, they don't have a problem with writes to the image. On the other hand, storage migration needs to be able

[Qemu-block] [PULL 00/10] Block layer fixes for 2.9.0-rc4

2017-04-07 Thread Kevin Wolf
The following changes since commit 5fe2339e6b09da7d6f48b9bef0f1a7360392b489: Merge remote-tracking branch 'remotes/awilliam/tags/vfio-updates-20170406.0' into staging (2017-04-07 10:29:56 +0100) are available in the git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream for

[Qemu-block] [PULL 03/10] commit: Set commit_top_bs->total_sectors

2017-04-07 Thread Kevin Wolf
Like in the mirror filter driver, we also need to set the image size for the commit filter driver. This is less likely to be a problem in practice than for the mirror because we're not at the active layer here, but attaching new parents to a node in the middle of the chain is possible, so the size

[Qemu-block] [PULL 05/10] iotests: Add mirror tests for orphaned source

2017-04-07 Thread Kevin Wolf
From: Max Reitz Signed-off-by: Max Reitz Reviewed-by: John Snow Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf --- tests/qemu-iotests/041| 46 +++

Re: [Qemu-block] [PATCH v2 3/6] mirror: Fix aio context of mirror_top_bs

2017-04-07 Thread Stefan Hajnoczi
On Fri, Apr 07, 2017 at 02:54:11PM +0800, Fam Zheng wrote: > It should be moved to the same context as source, before inserting to the > graph. > > Reviewed-by: Eric Blake > Reviewed-by: Kevin Wolf > Signed-off-by: Fam Zheng > --- >

Re: [Qemu-block] [PATCH v2 2/6] block: Assert attached child node has right aio context

2017-04-07 Thread Stefan Hajnoczi
On Fri, Apr 07, 2017 at 02:54:10PM +0800, Fam Zheng wrote: > Suggested-by: Kevin Wolf > Signed-off-by: Fam Zheng > --- > block.c | 4 > 1 file changed, 4 insertions(+) Reviewed-by: Stefan Hajnoczi signature.asc Description: PGP

Re: [Qemu-block] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-07 Thread Kevin Wolf
Am 07.04.2017 um 14:20 hat Stefan Hajnoczi geschrieben: > On Thu, Apr 06, 2017 at 06:01:48PM +0300, Alberto Garcia wrote: > > Here are the results (subcluster size in brackets): > > > > |-++-+---| > > | cluster size |

Re: [Qemu-block] [PATCH v2 4/6] block: Quiesce old aio context during bdrv_set_aio_context

2017-04-07 Thread Stefan Hajnoczi
On Fri, Apr 07, 2017 at 02:54:12PM +0800, Fam Zheng wrote: > @@ -4413,6 +4416,10 @@ void bdrv_set_aio_context(BlockDriverState *bs, > AioContext *new_context) > aio_context_acquire(new_context); > bdrv_attach_aio_context(bs, new_context); > aio_context_release(new_context); > +

Re: [Qemu-block] [PATCH v2 0/6] block: Fixes regarding dataplane and management operations

2017-04-07 Thread Kevin Wolf
Am 07.04.2017 um 08:54 hat Fam Zheng geschrieben: > v2: - Drop patch 4 in v1. A second thought made me feel neither it nor Kevin's > suggestion to move the BH process to bdrv_drain_recurse/BDRV_POLL_WHILE > is a complete fix. So leave it for a separate patch. > - Add rev-by to

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-07 Thread Kevin Wolf
Am 06.04.2017 um 18:40 hat Eric Blake geschrieben: > On 04/06/2017 10:01 AM, Alberto Garcia wrote: > > I thought of three alternatives for storing the subcluster bitmaps. I > > haven't made my mind completely about which one is the best one, so > > I'd like to present all three for discussion.

Re: [Qemu-block] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-07 Thread Alberto Garcia
On Fri 07 Apr 2017 02:20:21 PM CEST, Stefan Hajnoczi wrote: >> Here are the results when writing to an empty 40GB qcow2 image with no >> backing file. The numbers are of course different but as you can see >> the patterns are similar: >> >>

Re: [Qemu-block] [PATCH v3] migration/block:limit the time used for block migration

2017-04-07 Thread Stefan Hajnoczi
On Fri, Apr 07, 2017 at 09:30:33AM +0800, 858585 jemmy wrote: > On Thu, Apr 6, 2017 at 10:02 PM, Stefan Hajnoczi wrote: > > On Wed, Apr 05, 2017 at 05:27:58PM +0800, jemmy858...@gmail.com wrote: > > > > A proper solution is to refactor the synchronous code to make it > >

[Qemu-block] [PATCH] qemu-img: improve convert_iteration_sectors()

2017-04-07 Thread Vladimir Sementsov-Ogievskiy
Do not do extra call to _get_block_status() Signed-off-by: Vladimir Sementsov-Ogievskiy --- Also, I'm not sure about last line: s->status = s->target_has_backing ? BLK_BACKING_FILE : BLK_DATA; (which is equal to old code) may be, it should be s->status =

Re: [Qemu-block] [PATCH v3] migration/block:limit the time used for block migration

2017-04-07 Thread Stefan Hajnoczi
On Fri, Apr 07, 2017 at 09:30:33AM +0800, 858585 jemmy wrote: > On Thu, Apr 6, 2017 at 10:02 PM, Stefan Hajnoczi wrote: > > On Wed, Apr 05, 2017 at 05:27:58PM +0800, jemmy858...@gmail.com wrote: > >> From: Lidong Chen > >> > >> when migration with

[Qemu-block] [PATCH for-2.9] block: Don't check permissions for copy on read

2017-04-07 Thread Kevin Wolf
The assertion is currently failing. We can't require callers to have write permissions when all they are doing is a read, so comment it out. Add a FIXME comment in the code so that the check is re-enabled when copy on read is refactored into its own filter driver. Reported-by: Richard W.M. Jones

Re: [Qemu-block] [Qemu-devel] [PULL 43/46] block: Assertions for write permissions

2017-04-07 Thread Kevin Wolf
Am 06.04.2017 um 23:29 hat Richard W.M. Jones geschrieben: > On Thu, Apr 06, 2017 at 04:23:19PM -0500, Eric Blake wrote: > > On 04/06/2017 04:15 PM, Richard W.M. Jones wrote: > > > > +++ b/block/io.c > > @@ -945,6 +945,8 @@ static int coroutine_fn > >

Re: [Qemu-block] [PATCH v2] migration/block: use blk_pwrite_zeroes for each zero cluster

2017-04-07 Thread Fam Zheng
On Fri, 04/07 16:44, jemmy858...@gmail.com wrote: > From: Lidong Chen > > BLOCK_SIZE is (1 << 20), qcow2 cluster size is 65536 by default, > this maybe cause the qcow2 file size is bigger after migration. > This patch check each cluster, use blk_pwrite_zeroes for each >

Re: [Qemu-block] [PATCH v2 5/6] coroutine: Explicitly specify AioContext when entering coroutine

2017-04-07 Thread Fam Zheng
On Fri, 04/07 14:54, Fam Zheng wrote: > > main loopiothread > --- > blockdev_snapshot > aio_context_acquire(bs->ctx) > bdrv_flush(bs) > bdrv_co_flush(bs) > ... >

Re: [Qemu-block] [PATCH for-2.10 9/9] block/rbd: Add support for reopen()

2017-04-07 Thread Stefan Hajnoczi
On Wed, Apr 05, 2017 at 02:28:51PM -0400, Jeff Cody wrote: > This adds support for reopen in rbd, for changing between r/w and r/o. > > Note, that this is only a flag change, but we will block a change from > r/o to r/w if we are using an RBD internal snapshot. > > Signed-off-by: Jeff Cody

Re: [Qemu-block] [PATCH for-2.10 7/9] block/rbd - update variable names to more apt names

2017-04-07 Thread Stefan Hajnoczi
On Wed, Apr 05, 2017 at 02:28:49PM -0400, Jeff Cody wrote: > Update 'clientname' to be 'user', which tracks better with both > the QAPI and rados viarable naming. > > Update 'name' to be 'image_name', as it indicates the rbd image. > Naming it 'image' would have been ideal, but we are using that

Re: [Qemu-block] [PATCH for-2.10 2/9] block: do not set BDS read_only if copy_on_read enabled

2017-04-07 Thread Stefan Hajnoczi
On Wed, Apr 05, 2017 at 02:28:44PM -0400, Jeff Cody wrote: > @@ -1173,7 +1171,18 @@ static int vvfat_open(BlockDriverState *bs, QDict > *options, int flags, > if (ret < 0) { > goto fail; > } > -bdrv_set_read_only(bs, false); > +ret =

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.10 5/9] block: introduce bdrv_try_set_read_only()

2017-04-07 Thread Stefan Hajnoczi
On Wed, Apr 05, 2017 at 08:23:12PM -0400, Jeff Cody wrote: > On Wed, Apr 05, 2017 at 04:28:44PM -0400, John Snow wrote: > > On 04/05/2017 02:28 PM, Jeff Cody wrote: > > > diff --git a/include/block/block.h b/include/block/block.h > > > index beb563a..0049b57 100644 > > > ---

Re: [Qemu-block] [PATCH for-2.10 4/9] block: code movement

2017-04-07 Thread Stefan Hajnoczi
On Wed, Apr 05, 2017 at 02:28:46PM -0400, Jeff Cody wrote: > Move bdrv_is_read_only() up with its friends. > > Signed-off-by: Jeff Cody > --- > block.c | 10 +- > 1 file changed, 5 insertions(+), 5 deletions(-) Reviewed-by: Stefan Hajnoczi

Re: [Qemu-block] [PATCH for-2.10 2/9] block: do not set BDS read_only if copy_on_read enabled

2017-04-07 Thread Stefan Hajnoczi
On Wed, Apr 05, 2017 at 02:28:44PM -0400, Jeff Cody wrote: Minor comments but: Reviewed-by: Stefan Hajnoczi > diff --git a/block.c b/block.c > index 7b4c7ef..f60d5ea 100644 > --- a/block.c > +++ b/block.c > @@ -192,9 +192,17 @@ void path_combine(char *dest, int dest_size,

Re: [Qemu-block] [PATCH for-2.10 1/9] block: add bdrv_set_read_only() helper function

2017-04-07 Thread Stefan Hajnoczi
On Wed, Apr 05, 2017 at 02:28:43PM -0400, Jeff Cody wrote: > We have a helper wrapper for checking for the BDS read_only flag, > add a helper wrapper to set the read_only flag as well. > > Signed-off-by: Jeff Cody > --- > block.c | 5 + > block/bochs.c

Re: [Qemu-block] [PATCH v3 05/13] qobject: Drop useless QObject casts

2017-04-07 Thread Stefan Hajnoczi
On Wed, Apr 05, 2017 at 02:47:33PM -0500, Eric Blake wrote: > We have macros in place to make it less verbose to add a subtype > of QObject to both QDict and QList. While we have made cleanups > like this in the past (see commit fcfcd8ffc, for example), having > it be automated by Coccinelle makes

Re: [Qemu-block] [Qemu-devel] [RFC] Proposed qcow2 extension: subcluster allocation

2017-04-07 Thread Alberto Garcia
On Thu 06 Apr 2017 06:40:41 PM CEST, Eric Blake wrote: >> This e-mail is the formal presentation of my proposal to extend the >> on-disk qcow2 format. As you can see this is still an RFC. Due to the >> nature of the changes I would like to get as much feedback as >> possible before going forward.

[Qemu-block] [PATCH v2] migration/block: use blk_pwrite_zeroes for each zero cluster

2017-04-07 Thread jemmy858585
From: Lidong Chen BLOCK_SIZE is (1 << 20), qcow2 cluster size is 65536 by default, this maybe cause the qcow2 file size is bigger after migration. This patch check each cluster, use blk_pwrite_zeroes for each zero cluster. Signed-off-by: Lidong Chen

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.9 5/5] coroutine: Explicitly specify AioContext when creating coroutine

2017-04-07 Thread Fam Zheng
On Thu, 04/06 18:20, Kevin Wolf wrote: > For example, another case where this happens is that block jobs follow > their nodes if the AioContext changes and even implement > .attached_aio_context callbacks when they need to drag additional nodes > into the new context. With your change, the job

Re: [Qemu-block] [PATCH v3] migration/block:limit the time used for block migration

2017-04-07 Thread 858585 jemmy
On Fri, Apr 7, 2017 at 9:30 AM, 858585 jemmy wrote: > On Thu, Apr 6, 2017 at 10:02 PM, Stefan Hajnoczi wrote: >> On Wed, Apr 05, 2017 at 05:27:58PM +0800, jemmy858...@gmail.com wrote: >>> From: Lidong Chen >>> >>> when

Re: [Qemu-block] [PATCH for-2.9 3/5] block: Quiesce old aio context during bdrv_set_aio_context

2017-04-07 Thread Kevin Wolf
Am 07.04.2017 um 01:44 hat Fam Zheng geschrieben: > On Thu, 04/06 17:17, Kevin Wolf wrote: > > Am 06.04.2017 um 16:25 hat Fam Zheng geschrieben: > > > The fact that the bs->aio_context is changing can confuse the dataplane > > > iothread, because of the now fine granularity aio context lock. > > >

Re: [Qemu-block] [PATCH 2/2] migration/block: use blk_pwrite_zeroes for each zero cluster

2017-04-07 Thread 858585 jemmy
On Fri, Apr 7, 2017 at 3:08 PM, Fam Zheng wrote: > On Thu, 04/06 21:15, jemmy858...@gmail.com wrote: >> From: Lidong Chen >> >> BLOCK_SIZE is (1 << 20), qcow2 cluster size is 65536 by default, >> this maybe cause the qcow2 file size is bigger after

Re: [Qemu-block] [PATCH for-2.9 0/2] commit: Fix commit_top_bs problems

2017-04-07 Thread Fam Zheng
On Thu, 04/06 19:14, Kevin Wolf wrote: > The same problem that Fam's patch "mirror: Fix aio context of > mirror_top_bs" fixes for the mirror block job also exists for the > commit block job and the 'commit' HMP command. While comparing the > respective places in mirror.c and commit.c, I also

Re: [Qemu-block] [PATCH 2/2] migration/block: use blk_pwrite_zeroes for each zero cluster

2017-04-07 Thread Fam Zheng
On Thu, 04/06 21:15, jemmy858...@gmail.com wrote: > From: Lidong Chen > > BLOCK_SIZE is (1 << 20), qcow2 cluster size is 65536 by default, > this maybe cause the qcow2 file size is bigger after migration. > This patch check each cluster, use blk_pwrite_zeroes for each >

[Qemu-block] [PATCH v2 5/6] coroutine: Explicitly specify AioContext when entering coroutine

2017-04-07 Thread Fam Zheng
Coroutine in block layer should always be waken up in bs->aio_context rather than the "current" context where it is entered. They differ when the main loop is doing QMP tasks. Race conditions happen without this patch, because the wrong context is acquired in co_schedule_bh_cb, while the entered

Re: [Qemu-block] [PATCH 0/2] use blk_pwrite_zeroes for each zero cluster

2017-04-07 Thread 858585 jemmy
On Fri, Apr 7, 2017 at 1:22 PM, 858585 jemmy wrote: > the test result for this patch: > > the migration command : > virsh migrate --live 165cf436-312f-47e7-90f2-f8aa63f34893 > --copy-storage-all qemu+ssh://10.59.163.38/system > > the qemu-img info on source host: > qemu-img

[Qemu-block] [PATCH v2 6/6] tests/block-job-txn: Don't start block job before adding to txn

2017-04-07 Thread Fam Zheng
Previously, before test_block_job_start returns, the job can already complete, as a result, the transactional state of other jobs added to the same txn later cannot be handled correctly. Move the block_job_start() calls to callers after block_job_txn_add_job() calls. Signed-off-by: Fam Zheng

[Qemu-block] [PATCH v2 2/6] block: Assert attached child node has right aio context

2017-04-07 Thread Fam Zheng
Suggested-by: Kevin Wolf Signed-off-by: Fam Zheng --- block.c | 4 1 file changed, 4 insertions(+) diff --git a/block.c b/block.c index 927ba89..b8a3011 100644 --- a/block.c +++ b/block.c @@ -1752,6 +1752,9 @@ static void

[Qemu-block] [PATCH v2 4/6] block: Quiesce old aio context during bdrv_set_aio_context

2017-04-07 Thread Fam Zheng
The fact that the bs->aio_context is changing can confuse the dataplane iothread, because of the now fine granularity aio context lock. bdrv_drain should rather be a bdrv_drained_begin/end pair, but since bs->aio_context is changing, we can just use aio_disable_external and block_job_pause.

[Qemu-block] [PATCH v2 0/6] block: Fixes regarding dataplane and management operations

2017-04-07 Thread Fam Zheng
v2: - Drop patch 4 in v1. A second thought made me feel neither it nor Kevin's suggestion to move the BH process to bdrv_drain_recurse/BDRV_POLL_WHILE is a complete fix. So leave it for a separate patch. - Add rev-by to patches 1, 3, 4. - Split from patch 1 in v1 and add patch

[Qemu-block] [PATCH v2 1/6] block: Fix unpaired aio_disable_external in external snapshot

2017-04-07 Thread Fam Zheng
bdrv_replace_child_noperm tries to hand over the quiesce_counter state from old bs to the new one, but if they are not on the same aio context this causes unbalance. Fix this by setting the correct aio context before calling bdrv_append(). Reported-by: Ed Swierk