Re: [Qemu-block] [Qemu-devel] [PATCH] block: record new size in bdrv_dirty_bitmap_truncate

2015-06-08 Thread Eric Blake
On 06/08/2015 02:49 PM, John Snow wrote: ce1ffea8 neglected to update the BdrvDirtyBitmap structure itself for internal consistency. It's currently not an issue, but for migration and persistence series this will cause headaches. Signed-off-by: John Snow js...@redhat.com --- block.c | 1 +

[Qemu-block] [PATCH 4/6] block: Add check on mirror target

2015-06-08 Thread Fam Zheng
Signed-off-by: Fam Zheng f...@redhat.com --- blockdev.c| 3 +++ include/block/block.h | 1 + 2 files changed, 4 insertions(+) diff --git a/blockdev.c b/blockdev.c index c32a9a9..44030da 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2929,6 +2929,9 @@ static void

[Qemu-block] [PATCH 3/6] block: Extract blockdev part of qmp_drive_mirror

2015-06-08 Thread Fam Zheng
This is the part that will be reused by blockdev-mirror. Signed-off-by: Fam Zheng f...@redhat.com --- blockdev.c | 158 +++-- 1 file changed, 92 insertions(+), 66 deletions(-) diff --git a/blockdev.c b/blockdev.c index b573e56..c32a9a9

[Qemu-block] [PATCH 2/6] block: Rename BLOCK_OP_TYPE_MIRROR to BLOCK_OP_TYPE_MIRROR_SOURCE

2015-06-08 Thread Fam Zheng
It's necessary to distinguish source and target before we can add blockdev-mirror, because we would want a concrete type of operation to check on target bs before starting. Signed-off-by: Fam Zheng f...@redhat.com --- blockdev.c | 2 +- hw/block/dataplane/virtio-blk.c | 2 +-

[Qemu-block] [PATCH 0/6] qmp: Add blockdev-mirror

2015-06-08 Thread Fam Zheng
This is the counterpart of blockdev-backup. The biggest value of this command is to allow full flexibility on target image open options, via blockdev-add. For example this could help solve the target provisioning issue in: http://lists.gnu.org/archive/html/qemu-devel/2015-06/msg02139.html

[Qemu-block] [PATCH 1/6] block: Add blocker on mirror target

2015-06-08 Thread Fam Zheng
In block/backup.c, we already check and add blocker on the target bs, which is necessary so that it won't be intervened with other operations. In block/mirror.c we should also protect the mirror target bs, because it could have a node-name (drive-mirror ... node-name=XXX), and on top of that it's

Re: [Qemu-block] [Qemu-devel] [PATCH v3 23/38] blockdev: Pull out blockdev option extraction

2015-06-08 Thread Fam Zheng
On Wed, 06/03 21:44, Max Reitz wrote: Extract some of the blockdev option extraction code from blockdev_init() into its own function. This simplifies blockdev_init() and will allow reusing the code in a different function added in a follow-up patch. Signed-off-by: Max Reitz mre...@redhat.com

[Qemu-block] [PATCH 6/6] iotests: Add test cases for blockdev-mirror

2015-06-08 Thread Fam Zheng
Signed-off-by: Fam Zheng f...@redhat.com --- tests/qemu-iotests/041 | 100 +++-- tests/qemu-iotests/041.out | 4 +- 2 files changed, 80 insertions(+), 24 deletions(-) diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041 index 59a8f73..922f53c

[Qemu-block] [PULL v2 6/9] blkdebug: Simplify passing of Error through qemu_opts_foreach()

2015-06-08 Thread Markus Armbruster
Cc: Kevin Wolf kw...@redhat.com Cc: qemu-block@nongnu.org Signed-off-by: Markus Armbruster arm...@redhat.com Reviewed-by: Eric Blake ebl...@redhat.com Acked-by: Kevin Wolf kw...@redhat.com --- block/blkdebug.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions(-) diff --git

[Qemu-block] [PATCH 0/2] mirror: Allow detection of zeroes on source sectors

2015-06-08 Thread Fam Zheng
[These patches go on top of the block: Mirror discarded sectors series] Some protocols don't have an easy way to query sparseness, (e.g. block/nfs.c, block/nbd.c), for which block layer always reports block status as allocated data. This will let mirror job do full provisioning even if data is

[Qemu-block] [PATCH v7 5/8] block: Remove bdrv_reset_dirty

2015-06-08 Thread Fam Zheng
Using this function would always be wrong because a dirty bitmap must have a specific owner that consumes the dirty bits and calls bdrv_reset_dirty_bitmap(). Remove the unused function to avoid future misuse. Reviewed-by: Eric Blake ebl...@redhat.com Signed-off-by: Fam Zheng f...@redhat.com

Re: [Qemu-block] [PATCH 1/2] block-backend: Introduce blk_drain() and replace blk_drain_all()

2015-06-08 Thread Christian Borntraeger
Am 03.06.2015 um 15:46 schrieb Alexander Yarygin: Each call of the virtio_blk_reset() function calls blk_drain_all(), which works for all existing BlockDriverStates, while only one BlockDriverState needs to be drained. This patch introduces the blk_drain() function and replaces

Re: [Qemu-block] [PATCH 0/4] blockdev: Defer creation of implicit PCI devices for IF_VIRTIO drives

2015-06-08 Thread Christian Borntraeger
Am 08.06.2015 um 10:02 schrieb Christian Borntraeger: So I would prefer to not have this workaround and doing index c480f64..7627d57 100644 --- a/blockdev.c +++ b/blockdev.c @@ -976,17 +976,6 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)

[Qemu-block] [PATCH 1/2] qapi: Add detect-zeroes option to drive-mirror

2015-06-08 Thread Fam Zheng
The new optional flag defaults to true, in which case, mirror job would check the read sectors and use sparse write if they are zero. Otherwise data will be fully copied. Signed-off-by: Fam Zheng f...@redhat.com --- block/mirror.c| 21 +++-- blockdev.c

Re: [Qemu-block] [PATCH 0/4] blockdev: Defer creation of implicit PCI devices for IF_VIRTIO drives

2015-06-08 Thread Christian Borntraeger
Am 04.06.2015 um 18:20 schrieb Peter Maydell: blockdev.c will create implicit virtio-blk-* devices for IF_VIRTIO drives. I want to turn this on for the ARM virt board (now it has PCI), so that users can use shorter and more comprehensible command lines. Unfortunately, the code as it stands

[Qemu-block] [PATCH v7 8/8] iotests: Use event_wait in wait_ready

2015-06-08 Thread Fam Zheng
Only poll the specific type of event we are interested in, to avoid stealing events that should be consumed by someone else. Suggested-by: John Snow js...@redhat.com Signed-off-by: Fam Zheng f...@redhat.com Reviewed-by: John Snow js...@redhat.com --- tests/qemu-iotests/iotests.py | 9 ++---

[Qemu-block] [PATCH 2/2] iotests: Add test cases for drive-mirror detect-zeroes option

2015-06-08 Thread Fam Zheng
Signed-off-by: Fam Zheng f...@redhat.com --- tests/qemu-iotests/132| 26 +++--- tests/qemu-iotests/132.out| 4 ++-- tests/qemu-iotests/iotests.py | 7 +++ 3 files changed, 32 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/132

Re: [Qemu-block] [PATCH v2 2/3] qapi: Add detect-zeroes option to drive-mirror

2015-06-08 Thread Paolo Bonzini
On 08/06/2015 12:34, Fam Zheng wrote: - unmap: whether the target sectors should be discarded where source has only zeroes. (json-bool, optional, default true) +- detect-zeroes: if true, the source sectors that are zeroes will be written as + sparse on target. (json-bool, optional,

Re: [Qemu-block] [PATCH 1/2] block-backend: Introduce blk_drain() and replace blk_drain_all()

2015-06-08 Thread Kevin Wolf
Am 03.06.2015 um 15:46 hat Alexander Yarygin geschrieben: Each call of the virtio_blk_reset() function calls blk_drain_all(), which works for all existing BlockDriverStates, while only one BlockDriverState needs to be drained. This patch introduces the blk_drain() function and replaces

[Qemu-block] [PATCH v2 2/3] qapi: Add detect-zeroes option to drive-mirror

2015-06-08 Thread Fam Zheng
The new optional flag defaults to true, in which case, mirror job would check the read sectors and use sparse write if they are zero. Otherwise data will be fully copied. Signed-off-by: Fam Zheng f...@redhat.com --- blockdev.c | 26 +- hmp.c| 2

Re: [Qemu-block] [PATCH 0/2] vmdk: Fix vmdk_co_get_block_status

2015-06-08 Thread Kevin Wolf
Am 04.06.2015 um 08:02 hat Fam Zheng geschrieben: The buggy index_in_cluster was missed in b1649fae49a8. Fix that and dedup the calculation. Thanks, applied to the block branch. Kevin

[Qemu-block] [PATCH v2 3/3] iotests: Add test cases for drive-mirror detect-zeroes option

2015-06-08 Thread Fam Zheng
Signed-off-by: Fam Zheng f...@redhat.com --- tests/qemu-iotests/132| 28 +--- tests/qemu-iotests/132.out| 4 ++-- tests/qemu-iotests/iotests.py | 7 +++ 3 files changed, 34 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/132

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] block-backend: Introduce blk_drain() and replace blk_drain_all()

2015-06-08 Thread Stefan Hajnoczi
On Wed, Jun 03, 2015 at 04:46:14PM +0300, Alexander Yarygin wrote: Each call of the virtio_blk_reset() function calls blk_drain_all(), which works for all existing BlockDriverStates, while only one BlockDriverState needs to be drained. This patch introduces the blk_drain() function and

[Qemu-block] [PATCH v2 0/3] mirror: Allow detection of zeroes on source sectors

2015-06-08 Thread Fam Zheng
[These patches go on top of the block: Mirror discarded sectors series] v2: Rely on block/io.c zero detection. [Paolo] Some protocols don't have an easy way to query sparseness, (e.g. block/nfs.c, block/nbd.c), for which block layer always reports block status as allocated data. This will let

Re: [Qemu-block] [PATCH 0/4] blockdev: Defer creation of implicit PCI devices for IF_VIRTIO drives

2015-06-08 Thread Peter Maydell
On 8 June 2015 at 09:18, Christian Borntraeger borntrae...@de.ibm.com wrote: Am 08.06.2015 um 10:02 schrieb Christian Borntraeger: So I would prefer to not have this workaround and doing index c480f64..7627d57 100644 --- a/blockdev.c +++ b/blockdev.c @@ -976,17 +976,6 @@ DriveInfo

Re: [Qemu-block] [Qemu-devel] [PATCH v2 1/3] block: Extrace bdrv_parse_detect_zeroes_flags

2015-06-08 Thread Eric Blake
On 06/08/2015 04:34 AM, Fam Zheng wrote: The logic will be shared with qmp_drive_mirror. s/Extrace/Extract/ in the subject Signed-off-by: Fam Zheng f...@redhat.com --- block.c | 26 ++ blockdev.c| 14 ++ include/block/block.h

Re: [Qemu-block] [Qemu-devel] [PATCH v2 2/3] qapi: Add detect-zeroes option to drive-mirror

2015-06-08 Thread Eric Blake
On 06/08/2015 04:34 AM, Fam Zheng wrote: The new optional flag defaults to true, in which case, mirror job would check the read sectors and use sparse write if they are zero. Otherwise data will be fully copied. Is that a different default than in qemu 2.3? That's okay, but I need to figure

Re: [Qemu-block] [Qemu-devel] [PATCH 0/4] blockdev: Defer creation of implicit PCI devices for IF_VIRTIO drives

2015-06-08 Thread Markus Armbruster
Peter Maydell peter.mayd...@linaro.org writes: blockdev.c will create implicit virtio-blk-* devices for IF_VIRTIO drives. I want to turn this on for the ARM virt board (now it has PCI), so that users can use shorter and more comprehensible command lines. I had to read further until understood

Re: [Qemu-block] [PATCH v7 0/8] block: Mirror discarded sectors

2015-06-08 Thread Stefan Hajnoczi
On Mon, Jun 08, 2015 at 01:56:06PM +0800, Fam Zheng wrote: v7: Fix the lost assignment of s-unmap. v6: Fix pnum in bdrv_get_block_status_above. [Paolo] v5: Rewrite patch 1. Address Eric's comments on patch 3. Add Eric's rev-by to patches 2 4. Check BDRV_BLOCK_DATA in patch

Re: [Qemu-block] I/O accounting overhaul

2015-06-08 Thread Stefan Hajnoczi
On Wed, Jun 03, 2015 at 03:40:42PM +0200, Alberto Garcia wrote: Please structure the patches so that each statistic or group of statistics has its own patch. That will make it easy to review and possibly merge a subset if some of the statistics prove to be controversial. uint64_t

Re: [Qemu-block] [PATCH v7 2/8] qmp: Add optional bool unmap to drive-mirror

2015-06-08 Thread Paolo Bonzini
On 08/06/2015 16:51, Eric Blake wrote: +# @unmap: #optional Whether to try to unmap target sectors where source has +# only zero. If true, and target unallocated sectors will read as zero, +# target image sectors will be unmapped; otherwise, zeroes will be +#

Re: [Qemu-block] [PATCH v7 2/8] qmp: Add optional bool unmap to drive-mirror

2015-06-08 Thread Eric Blake
On 06/07/2015 11:56 PM, Fam Zheng wrote: If specified as true, it allows discarding on target sectors where source is not allocated. Signed-off-by: Fam Zheng f...@redhat.com --- block/mirror.c| 8 ++-- blockdev.c| 5 + hmp.c | 2 +-

Re: [Qemu-block] [Qemu-devel] [PATCH v2 1/3] block: Extrace bdrv_parse_detect_zeroes_flags

2015-06-08 Thread Paolo Bonzini
On 08/06/2015 16:17, Eric Blake wrote: + +if (detect_zeroes == BLOCKDEV_DETECT_ZEROES_OPTIONS_UNMAP +!(bdrv_flags BDRV_O_UNMAP)) { +error_setg(errp, setting detect-zeroes to unmap is not allowed + without setting discard operation to

[Qemu-block] [PATCH v9 7/8] throttle: Update throttle infrastructure copyright

2015-06-08 Thread Alberto Garcia
Signed-off-by: Alberto Garcia be...@igalia.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- include/qemu/throttle.h | 8 +--- util/throttle.c | 8 +--- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/include/qemu/throttle.h b/include/qemu/throttle.h index

[Qemu-block] [PATCH v9 4/8] throttle: Add throttle group support

2015-06-08 Thread Alberto Garcia
The throttle group support use a cooperative round robin scheduling algorithm. The principles of the algorithm are simple: - Each BDS of the group is used as a token in a circular way. - The active BDS computes if a wait must be done and arms the right timer. - If a wait must be done the token

[Qemu-block] [PATCH v9 1/8] throttle: Extract timers from ThrottleState into a separate structure

2015-06-08 Thread Alberto Garcia
From: BenoƮt Canet benoit.ca...@nodalink.com Group throttling will share ThrottleState between multiple bs. As a consequence the ThrottleState will be accessed by multiple aio context. Timers are tied to their aio context so they must go out of the ThrottleState structure. This commit paves the

[Qemu-block] [PATCH v9 6/8] throttle: add the name of the ThrottleGroup to BlockDeviceInfo

2015-06-08 Thread Alberto Garcia
Signed-off-by: Alberto Garcia be...@igalia.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- block/qapi.c | 3 +++ hmp.c| 6 -- qapi/block-core.json | 4 +++- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/block/qapi.c b/block/qapi.c index

[Qemu-block] [PATCH v9 8/8] qemu-iotests: expand test 093 to support group throttling

2015-06-08 Thread Alberto Garcia
This patch improves the test by attaching a different number of drives to the VM and putting them in the same throttling group. The test verifies that the I/O is evenly distributed among all members of the group, and that the limits are enforced. By default the test is repeated 3 times with 1, 2

[Qemu-block] [PATCH v9 5/8] throttle: acquire the ThrottleGroup lock in bdrv_swap()

2015-06-08 Thread Alberto Garcia
bdrv_swap() touches the fields of a BlockDriverState that are protected by the ThrottleGroup lock. Although those fields end up in their original place, they are temporarily swapped in the process, so there's a chance that an operation on a member of the same group happening on a different thread

[Qemu-block] [PATCH v9 0/8] Block Throttle Group Support

2015-06-08 Thread Alberto Garcia
V9: - Make test-throttle.c call bdrv_init() to make sure that throttle_groups_init() gets called and the mutex is initialized. This was breaking the tests in Mac OS X. V8: https://lists.gnu.org/archive/html/qemu-devel/2015-05/msg03716.html - Rebased against the current master. V7:

[Qemu-block] [PATCH v9 2/8] throttle: Add throttle group infrastructure

2015-06-08 Thread Alberto Garcia
Signed-off-by: Alberto Garcia be...@igalia.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- block/Makefile.objs | 1 + block/throttle-groups.c | 261 include/block/block_int.h | 1 + include/block/throttle-groups.h |