Re: [Qemu-block] [PATCH for-2.7 00/15] block: Lock images when opening

2016-04-13 Thread Denis V. Lunev
On 04/14/2016 09:23 AM, Fam Zheng wrote: On Thu, 04/14 09:14, Denis V. Lunev wrote: On 04/14/2016 08:46 AM, Fam Zheng wrote: On Thu, 04/14 08:04, Denis V. Lunev wrote: unfortunately no. If the lock will be on the image file, we will have get it on the target node on QEMU start and re-acquire i

Re: [Qemu-block] [PATCH for-2.7 00/15] block: Lock images when opening

2016-04-13 Thread Fam Zheng
On Thu, 04/14 09:14, Denis V. Lunev wrote: > On 04/14/2016 08:46 AM, Fam Zheng wrote: > >On Thu, 04/14 08:04, Denis V. Lunev wrote: > >>unfortunately no. If the lock will be on the image file, > >>we will have get it on the target node on QEMU start > >>and re-acquire it in bdrv_invalidate_cache. >

Re: [Qemu-block] [PATCH for-2.7 00/15] block: Lock images when opening

2016-04-13 Thread Fam Zheng
On Thu, 04/14 08:04, Denis V. Lunev wrote: > unfortunately no. If the lock will be on the image file, > we will have get it on the target node on QEMU start > and re-acquire it in bdrv_invalidate_cache. > > From my POW you should not get the lock if > BDRV_O_INACTIVE is set. That is what I meant.

Re: [Qemu-block] [Qemu-devel] [PATCH v2 for-2.6] block: convert iscsi target to a valid ID for -iscsi arg lookup

2016-04-13 Thread John Ferlan
On 04/13/2016 12:17 PM, Daniel P. Berrange wrote: > The iSCSI block driver has a very strange approach whereby it > does not accept options directly as part of the -drive arg, > but instead takes them indirectly from a -iscsi arg. To make > up -driver and -iscsi args, it takes the iSCSI target na

Re: [Qemu-block] [PATCH for-2.7 00/15] block: Lock images when opening

2016-04-13 Thread Denis V. Lunev
On 04/14/2016 05:36 AM, Fam Zheng wrote: On Wed, 04/13 13:18, Denis V. Lunev wrote: On 04/13/2016 12:09 PM, Fam Zheng wrote: Too many troubles have been caused by two processes writing to the same image unexpectedly. This series introduces automatical image locking into QEMU to avoid such trage

Re: [Qemu-block] [PATCH v17 7/8] Implement new driver for block replication

2016-04-13 Thread Changlong Xie
On 04/13/2016 09:02 PM, Stefan Hajnoczi wrote: On Mon, Apr 11, 2016 at 04:22:58PM +0800, Changlong Xie wrote: +static coroutine_fn int replication_co_writev(BlockDriverState *bs, + int64_t sector_num, + int

Re: [Qemu-block] [PATCH for-2.7 00/15] block: Lock images when opening

2016-04-13 Thread Fam Zheng
On Wed, 04/13 13:18, Denis V. Lunev wrote: > On 04/13/2016 12:09 PM, Fam Zheng wrote: > >Too many troubles have been caused by two processes writing to the same image > >unexpectedly. This series introduces automatical image locking into QEMU to > >avoid such tragedy. With this, the user won't be a

Re: [Qemu-block] [PATCH for-2.7 00/15] block: Lock images when opening

2016-04-13 Thread Fam Zheng
On Wed, 04/13 10:19, Daniel P. Berrange wrote: > On Wed, Apr 13, 2016 at 05:09:49PM +0800, Fam Zheng wrote: > > Too many troubles have been caused by two processes writing to the same > > image > > unexpectedly. This series introduces automatical image locking into QEMU to > > avoid such tragedy.

Re: [Qemu-block] [PATCH for-2.7 05/15] raw-posix: Implement .bdrv_lockf

2016-04-13 Thread Fam Zheng
On Wed, 04/13 10:21, Daniel P. Berrange wrote: > On Wed, Apr 13, 2016 at 05:09:54PM +0800, Fam Zheng wrote: > > Because virtlockd in libvirt already uses the fcntl lock on the image file, > > we > > have to workaround this by locking a digest-mapped temporary file. > > > > Signed-off-by: Fam Zhen

Re: [Qemu-block] [PATCH v17 6/8] Introduce new APIs to do replication operation

2016-04-13 Thread Changlong Xie
On 04/13/2016 08:47 PM, Stefan Hajnoczi wrote: On Mon, Apr 11, 2016 at 04:22:57PM +0800, Changlong Xie wrote: +/* + * The caller of the function MUST make sure vm stopped + */ +void replication_start_all(ReplicationMode mode, Error **errp) +{ +ReplicationState *rs, *next; + +QLIST_FOREAC

Re: [Qemu-block] [PATCH for-2.6 0/7] block: VHD format fixes

2016-04-13 Thread Kevin Wolf
Am 23.03.2016 um 04:33 hat Jeff Cody geschrieben: > Fixes for a regression in vpc_create(), as well as a few issues with VHD > format > compatibility. Thanks, applied to the block brannch. Kevin

Re: [Qemu-block] [PATCH for-2.6 7/7] block/vpc: update comments to be compliant w/coding guidelines

2016-04-13 Thread Kevin Wolf
Am 23.03.2016 um 04:33 hat Jeff Cody geschrieben: > Signed-off-by: Jeff Cody > --- > block/vpc.c | 70 > ++--- > 1 file changed, 35 insertions(+), 35 deletions(-) > > diff --git a/block/vpc.c b/block/vpc.c > index 5dd9950..0b48cf4 100644 >

Re: [Qemu-block] [PATCH for-2.6 1/7] block/vpc: fix VPC 'qemu-img create' regression

2016-04-13 Thread Kevin Wolf
Am 13.04.2016 um 17:40 hat Jeff Cody geschrieben: > On Tue, Mar 22, 2016 at 11:33:38PM -0400, Jeff Cody wrote: > > Commit 'b8f45cdf7827e39f9a1e6cc446f5972cc6144237' switched VPC > > over to using blk_pwrite() instead of bdrv_pwrite_sync(). The > > return value of bdrv_pwrite_sync() was always 0 fo

[Qemu-block] [PATCH v2 for-2.6] block: convert iscsi target to a valid ID for -iscsi arg lookup

2016-04-13 Thread Daniel P. Berrange
The iSCSI block driver has a very strange approach whereby it does not accept options directly as part of the -drive arg, but instead takes them indirectly from a -iscsi arg. To make up -driver and -iscsi args, it takes the iSCSI target name and uses that as an ID value for the -iscsi arg lookup.

Re: [Qemu-block] [PATCH for 2.6] block: convert iscsi target to a valid ID for -iscsi arg lookup

2016-04-13 Thread Pino Toscano
On Wednesday 13 April 2016 15:18:20 Daniel P. Berrange wrote: > The iSCSI block driver has a very strange approach whereby it > does not accept options directly as part of the -drive arg, > but instead takes them indirectly from a -iscsi arg. To make > up -driver and -iscsi args, it takes the iSCSI

Re: [Qemu-block] [PATCH for-2.6 1/7] block/vpc: fix VPC 'qemu-img create' regression

2016-04-13 Thread Jeff Cody
On Tue, Mar 22, 2016 at 11:33:38PM -0400, Jeff Cody wrote: > Commit 'b8f45cdf7827e39f9a1e6cc446f5972cc6144237' switched VPC > over to using blk_pwrite() instead of bdrv_pwrite_sync(). The > return value of bdrv_pwrite_sync() was always 0 for success, and > create_dynamic_disk() in one instance che

Re: [Qemu-block] [PULL 00/13] Block patches for 2.6

2016-04-13 Thread Peter Maydell
On 12 April 2016 at 17:18, Kevin Wolf wrote: > The following changes since commit 42bb626f7ebc9197d2943b897a99e127315275ab: > > Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' > into staging (2016-04-12 09:34:52 +0100) > > are available in the git repository at: > > >

Re: [Qemu-block] [Qemu-devel] [PATCH] iSCSI: start moving options also for -drive

2016-04-13 Thread Daniel P. Berrange
On Tue, Apr 12, 2016 at 04:57:42PM +0200, Pino Toscano wrote: > Hi, > > to overcome the limitations of the options handling [1], I'm planning > to move more options for iSCSI also as block options, so it is possible > to specify them with -drive. > > The only patch in this series is for initiator

[Qemu-block] [PATCH for 2.6] block: convert iscsi target to a valid ID for -iscsi arg lookup

2016-04-13 Thread Daniel P. Berrange
The iSCSI block driver has a very strange approach whereby it does not accept options directly as part of the -drive arg, but instead takes them indirectly from a -iscsi arg. To make up -driver and -iscsi args, it takes the iSCSI target name and uses that as an ID value for the -iscsi arg lookup.

[Qemu-block] [PATCH 1/1] qemu-img: check block status of backing file when converting.

2016-04-13 Thread Ren Kimura
Signed-off-by: Ren Kimura --- qemu-img.c | 27 --- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index 06264d9..53471a1 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -1451,6 +1451,21 @@ static void convert_select_part(ImgConvertState

[Qemu-block] [PATCH 0/1] qemu-img: check block status of backing file when converting.

2016-04-13 Thread Ren Kimura
When converting images, check the block status of it's backing file chain to avoid needlessly reading zeros.

Re: [Qemu-block] [PULL 08/12] mirror: Rewrite mirror_iteration

2016-04-13 Thread Kevin Wolf
Am 29.02.2016 um 21:08 hat Jeff Cody geschrieben: > From: Fam Zheng > > The "pnum < nb_sectors" condition in deciding whether to actually copy > data is unnecessarily strict, and the qiov initialization is > unnecessarily for bdrv_aio_write_zeroes and bdrv_aio_discard. > > Rewrite mirror_iterati

Re: [Qemu-block] [PATCH v17 7/8] Implement new driver for block replication

2016-04-13 Thread Stefan Hajnoczi
On Mon, Apr 11, 2016 at 04:22:58PM +0800, Changlong Xie wrote: > +static coroutine_fn int replication_co_writev(BlockDriverState *bs, > + int64_t sector_num, > + int remaining_sectors, > +

Re: [Qemu-block] [PATCH v17 6/8] Introduce new APIs to do replication operation

2016-04-13 Thread Stefan Hajnoczi
On Mon, Apr 11, 2016 at 04:22:57PM +0800, Changlong Xie wrote: > +/* > + * The caller of the function MUST make sure vm stopped > + */ > +void replication_start_all(ReplicationMode mode, Error **errp) > +{ > +ReplicationState *rs, *next; > + > +QLIST_FOREACH_SAFE(rs, &replication_states, no

[Qemu-block] [PATCH for-2.6 2/2] block: Fix blk_aio_write_zeroes()

2016-04-13 Thread Kevin Wolf
Commit 57d6a428 broke blk_aio_write_zeroes() because in some write functions in the call path don't have an explicit length argument but reuse qiov->size instead. Which is great, except that write_zeroes doesn't have a qiov, which this commit interprets as 0 bytes. Consequently, blk_aio_write_zeroe

[Qemu-block] [PATCH for-2.6 0/2] block: Fix blk_aio_write_zeroes()

2016-04-13 Thread Kevin Wolf
Kevin Wolf (2): qemu-io: Support 'aio_write -z' block: Fix blk_aio_write_zeroes() block/block-backend.c | 20 +++ qemu-io-cmds.c | 64 +++- tests/qemu-iotests/033 | 8 +++-- tests/qemu-iotests/033.out | 82 +

[Qemu-block] [PATCH for-2.6 1/2] qemu-io: Support 'aio_write -z'

2016-04-13 Thread Kevin Wolf
This allows testing blk_aio_write_zeroes(). Signed-off-by: Kevin Wolf --- qemu-io-cmds.c | 64 +++--- 1 file changed, 48 insertions(+), 16 deletions(-) diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c index 382faa8..51ae5a5 100644 --- a/qemu-io-cm

Re: [Qemu-block] [PATCH for-2.7 00/15] block: Lock images when opening

2016-04-13 Thread Denis V. Lunev
On 04/13/2016 12:09 PM, Fam Zheng wrote: Too many troubles have been caused by two processes writing to the same image unexpectedly. This series introduces automatical image locking into QEMU to avoid such tragedy. With this, the user won't be able to open the image from two processes (e.g. using

Re: [Qemu-block] [PATCH for-2.7 05/15] raw-posix: Implement .bdrv_lockf

2016-04-13 Thread Daniel P. Berrange
On Wed, Apr 13, 2016 at 05:09:54PM +0800, Fam Zheng wrote: > Because virtlockd in libvirt already uses the fcntl lock on the image file, we > have to workaround this by locking a digest-mapped temporary file. > > Signed-off-by: Fam Zheng > --- > block/raw-posix.c | 97 >

[Qemu-block] [PATCH for-2.7 11/15] qemu-iotests: Disable image lock when checking test image

2016-04-13 Thread Fam Zheng
The VM is running, qemu-io would fail the lock acquisition. Signed-off-by: Fam Zheng --- tests/qemu-iotests/030 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030 index 3ac2443..fa996ef 100755 --- a/tests/qemu-iotests/030 +++ b/tes

Re: [Qemu-block] [PATCH for-2.7 00/15] block: Lock images when opening

2016-04-13 Thread Daniel P. Berrange
On Wed, Apr 13, 2016 at 05:09:49PM +0800, Fam Zheng wrote: > Too many troubles have been caused by two processes writing to the same image > unexpectedly. This series introduces automatical image locking into QEMU to > avoid such tragedy. With this, the user won't be able to open the image from > t

[Qemu-block] [PATCH for-2.7 14/15] ide-test: Specify "lock-image=off" in command lines

2016-04-13 Thread Fam Zheng
If a failure in a previous test case doesn't clean up the running qemu process (it happens), the subsequent ones can fail because of a image locking failure. That is not an authentic failure of the test case itself and could be sometimes confusing. Disable image locking to avoid that. Signed-off-

[Qemu-block] [PATCH for-2.7 09/15] qemu-iotests: 046: Move version detection out from verify_io

2016-04-13 Thread Fam Zheng
So the image lock won't complain. Signed-off-by: Fam Zheng --- tests/qemu-iotests/046 | 22 -- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/tests/qemu-iotests/046 b/tests/qemu-iotests/046 index e0be46c..40c4bc0 100755 --- a/tests/qemu-iotests/046 +++ b/test

[Qemu-block] [PATCH for-2.7 15/15] block: Turn on image locking by default

2016-04-13 Thread Fam Zheng
Now that test cases are covered, we can turn it on. Signed-off-by: Fam Zheng --- blockdev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blockdev.c b/blockdev.c index 93bd43e..87d22c3 100644 --- a/blockdev.c +++ b/blockdev.c @@ -383,7 +383,7 @@ static void extract_common_b

[Qemu-block] [PATCH for-2.7 13/15] ahci-test: Specify "lock-image=off" in CLI

2016-04-13 Thread Fam Zheng
The case is the temporary image is sometimes used by more than one QEMU processes, just use the nop lock to avoid image locking failures. Signed-off-by: Fam Zheng --- tests/ahci-test.c | 16 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/tests/ahci-test.c b/tests

[Qemu-block] [PATCH for-2.7 12/15] qemu-iotests: 051: Disable image lock in the command line

2016-04-13 Thread Fam Zheng
Signed-off-by: Fam Zheng --- tests/qemu-iotests/051| 2 +- tests/qemu-iotests/051.out| 10 +- tests/qemu-iotests/051.pc.out | 10 +- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/tests/qemu-iotests/051 b/tests/qemu-iotests/051 index 88b3d91..2fee7e8

[Qemu-block] [PATCH for-2.7 05/15] raw-posix: Implement .bdrv_lockf

2016-04-13 Thread Fam Zheng
Because virtlockd in libvirt already uses the fcntl lock on the image file, we have to workaround this by locking a digest-mapped temporary file. Signed-off-by: Fam Zheng --- block/raw-posix.c | 97 +++ 1 file changed, 97 insertions(+) diff --

[Qemu-block] [PATCH for-2.7 08/15] qemu-iotests: 140: Disable image lock for qemu-io access

2016-04-13 Thread Fam Zheng
The VM is still on, the image locking check would complain. Signed-off-by: Fam Zheng --- tests/qemu-iotests/140 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/140 b/tests/qemu-iotests/140 index 05e4506..412b26f 100755 --- a/tests/qemu-iotests/140 +++ b/tes

[Qemu-block] [PATCH for-2.7 10/15] qemu-iotests: Fix lock-image for shared disk in test case 091

2016-04-13 Thread Fam Zheng
Because the source and the destination QEMU instances both open the image, we have to disable the lock. Signed-off-by: Fam Zheng --- tests/qemu-iotests/091 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/091 b/tests/qemu-iotests/091 index 32bbd56..2a3c9

[Qemu-block] [PATCH for-2.7 06/15] gluster: Implement .bdrv_lockf

2016-04-13 Thread Fam Zheng
Signed-off-by: Fam Zheng --- block/gluster.c | 34 ++ 1 file changed, 34 insertions(+) diff --git a/block/gluster.c b/block/gluster.c index 51e154c..e76ec87 100644 --- a/block/gluster.c +++ b/block/gluster.c @@ -672,6 +672,36 @@ static void qemu_gluster_close(Bloc

[Qemu-block] [PATCH for-2.7 03/15] blockdev: Add and parse "lock-image" option for block devices

2016-04-13 Thread Fam Zheng
Honor the locking switch specified in CLI or QMP, and set the open flags for the image accordingly. Signed-off-by: Fam Zheng --- blockdev.c | 8 1 file changed, 8 insertions(+) diff --git a/blockdev.c b/blockdev.c index f1f520a..93bd43e 100644 --- a/blockdev.c +++ b/blockdev.c @@ -382,

[Qemu-block] [PATCH for-2.7 02/15] qapi: Add lock-image in blockdev-add options

2016-04-13 Thread Fam Zheng
To allow overriding the default locking behavior when opening the image. Signed-off-by: Fam Zheng --- qapi/block-core.json | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 1d09079..2913f3e 100644 --- a/qapi/block-core.json

[Qemu-block] [PATCH for-2.7 04/15] block: Introduce image file locking interface

2016-04-13 Thread Fam Zheng
Block drivers can implement this new operation .bdrv_lockf to actually lock the image in the protocol specific way. Signed-off-by: Fam Zheng --- block.c | 25 + include/block/block.h | 8 include/block/block_int.h | 5 + 3 files change

[Qemu-block] [PATCH for-2.7 07/15] qemu-io: Add "-L" option for BDRV_O_NO_LOCK

2016-04-13 Thread Fam Zheng
Signed-off-by: Fam Zheng --- qemu-io.c | 22 -- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/qemu-io.c b/qemu-io.c index 288bba8..6bb6232 100644 --- a/qemu-io.c +++ b/qemu-io.c @@ -107,6 +107,7 @@ static void open_help(void) " -r, -- open file read-only\n"

[Qemu-block] [PATCH for-2.7 01/15] block: Add BDRV_O_NO_LOCK

2016-04-13 Thread Fam Zheng
Later the block layer will automatically lock the images to avoid unexpected concurrent accesses to the same image, which will easily corrupt the metadata or user data, unless in some very special cases, like migration. The exceptional cases like shared storage migration and testing should set BDR

[Qemu-block] [PATCH for-2.7 00/15] block: Lock images when opening

2016-04-13 Thread Fam Zheng
Too many troubles have been caused by two processes writing to the same image unexpectedly. This series introduces automatical image locking into QEMU to avoid such tragedy. With this, the user won't be able to open the image from two processes (e.g. using qemu-img when the image is attached to the

[Qemu-block] [PATCH v13 2/3] quorum: implement bdrv_add_child() and bdrv_del_child()

2016-04-13 Thread Changlong Xie
From: Wen Congyang Signed-off-by: Wen Congyang Signed-off-by: zhanghailiang Signed-off-by: Gonglei Signed-off-by: Changlong Xie --- block.c | 8 +++--- block/quorum.c| 78 +-- include/block/block.h | 4 +++ 3 files chang

[Qemu-block] [PATCH v13 3/3] qmp: add monitor command to add/remove a child

2016-04-13 Thread Changlong Xie
From: Wen Congyang The new QMP command name is x-blockdev-change. It's just for adding/removing quorum's child now, and doesn't support all kinds of children, all kinds of operations, nor all block drivers. So it is experimental now. Signed-off-by: Wen Congyang Signed-off-by: zhanghailiang Sig

[Qemu-block] [PATCH v13 0/3] qapi: child add/delete support

2016-04-13 Thread Changlong Xie
ChangLog: v13: 1. Rebase to the newest codes 2. Address commets from Betro and Max p1. Add R-B, fix incorrect syntax p2. Add missing "qemu/cutils.h" since 2.6, and rewrite quorum_add/del_child p3. Remove unnecessary "id", add "since 2.7" v11~v12: 1. Address comments from Max p1. Add R-B p2. Add R-B

[Qemu-block] [PATCH v13 1/3] Add new block driver interface to add/delete a BDS's child

2016-04-13 Thread Changlong Xie
From: Wen Congyang In some cases, we want to take a quorum child offline, and take another child online. Signed-off-by: Wen Congyang Signed-off-by: zhanghailiang Signed-off-by: Gonglei Signed-off-by: Changlong Xie Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia --- block.c