Re: [Qemu-block] [PATCH v4 18/26] tests: add output filter to python I/O tests helper

On Mon, Mar 14, 2016 at 11:57:11AM -0600, Eric Blake wrote: > On 02/29/2016 05:00 AM, Daniel P. Berrange wrote: > > Add a 'log' method to iotests.py which prints messages to > > stdout, with optional filtering of data. Port over some > > standard filters for present in the shell common.filter > > c

[Qemu-block] [PULL 20/40] block: Use writeback in .bdrv_create() implementations

There's no reason to use a writethrough cache mode while creating an image. Signed-off-by: Kevin Wolf --- block/parallels.c | 3 ++- block/qcow.c | 3 ++- block/qcow2.c | 3 ++- block/sheepdog.c | 6 -- block/vdi.c | 3 ++- block/vhdx.c | 3 ++- block/vmdk.c | 9 +++

[Qemu-block] [PULL 37/40] quorum: Fix crash in quorum_aio_cb()

From: Alberto Garcia quorum_aio_cb() emits the QUORUM_REPORT_BAD event if there's an I/O error in a Quorum child. However sacb->aiocb must be correctly initialized for this to happen. read_quorum_children() and read_fifo_child() are not doing this, which results in a QEMU crash. Signed-off-by: A

[Qemu-block] [PULL 40/40] iotests: Add test for QMP event rates

From: Alberto Garcia This test verifies that the rate-limited QMP events are emitted at a maximum rate of 1 per second as defined in monitor_qapi_event_conf in monitor.c It also checks that QUORUM_REPORT_BAD events generated from different nodes are kept in separate queues so they don't mask eac

Re: [Qemu-block] [PATCH v4 18/26] tests: add output filter to python I/O tests helper

On 02/29/2016 05:00 AM, Daniel P. Berrange wrote: > Add a 'log' method to iotests.py which prints messages to > stdout, with optional filtering of data. Port over some > standard filters for present in the shell common.filter > code. I'm not sure what you meant by 'for present'. > > Signed-off-b

[Qemu-block] [PULL 19/40] hmp: Extend drive_del to delete nodes without BB

Now that we can use drive_add to create new nodes without a BB, we also want to be able to delete such nodes again. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- blockdev.c | 9 + 1 file changed, 9 insertions(+) diff --git a/blockdev.c b/blockdev.c index 1297c90..322ca03 100644

[Qemu-block] [PULL 38/40] monitor: Separate QUORUM_REPORT_BAD events according to the node name

From: Alberto Garcia The QUORUM_REPORT_BAD event is emitted whenever there's an I/O error in a child of a Quorum device. This event is emitted at a maximum rate of 1 per second. This means that an error in one of the children will mask errors in the other children if they happen within the same 1

[Qemu-block] [PULL 39/40] monitor: Use QEMU_CLOCK_VIRTUAL for the event queue in qtest mode

From: Alberto Garcia This allows us to perform tests on the monitor queues to verify that the rate limits are enforced. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Message-id: dde511809e954a5c32d5b648bb184c03c89ed5d5.1457610443.git.be...@igalia.com Signed-off-by: Max Reitz --- moni

[Qemu-block] [PULL 29/40] vmdk: Use BB functions in .bdrv_create()

All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/vmdk.c | 77 +--- 1 file changed, 43 insertions(+),

[Qemu-block] [PULL 25/40] qed: Use BB functions in .bdrv_create()

All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/qed.c | 28 +++- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/blo

[Qemu-block] [PULL 31/40] backup: Use Bitmap to replace "s->bitmap"

From: Fam Zheng "s->bitmap" tracks done sectors, we only check bit states without using any iterator which HBitmap is good for. Switch to "Bitmap" which is simpler and more memory efficient. Meanwhile, rename it to done_bitmap, to reflect the intention. Signed-off-by: Fam Zheng Reviewed-by: Jo

[Qemu-block] [PULL 33/40] typedefs: Add BdrvDirtyBitmap

From: Fam Zheng Following patches to refactor and move block dirty bitmap code could use this. Signed-off-by: Fam Zheng Reviewed-by: John Snow Message-id: 1457412306-18940-4-git-send-email-f...@redhat.com Signed-off-by: Max Reitz --- include/block/block.h | 1 - include/qemu/typedefs.h | 1

[Qemu-block] [PULL 34/40] block: Move block dirty bitmap code to separate files

From: Fam Zheng The only code change is making bdrv_dirty_bitmap_truncate public. It is used in block.c. Also two long lines (bdrv_get_dirty) are wrapped. Signed-off-by: Fam Zheng Reviewed-by: John Snow Message-id: 1457412306-18940-5-git-send-email-f...@redhat.com Signed-off-by: Max Reitz --

[Qemu-block] [PULL 30/40] vpc: Use BB functions in .bdrv_create()

All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/vpc.c | 37 + 1 file changed, 21 insertions(+), 16 deletions(-) diff --

[Qemu-block] [PULL 35/40] block: Remove unused typedef of BlockDriverDirtyHandler

From: Fam Zheng Signed-off-by: Fam Zheng Reviewed-by: John Snow Message-id: 1457412306-18940-6-git-send-email-f...@redhat.com Signed-off-by: Max Reitz --- include/block/block.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/include/block/block.h b/include/block/block.h index 9688d7f..ea

[Qemu-block] [PULL 36/40] iotests: Correct 081's reference output

From: Max Reitz The newly added type parameter for the QUORUM_REPORT_BAD event changed the output of iotest 081, so the reference should be amended accordingly. Signed-off-by: Max Reitz Message-id: 1457705687-27122-1-git-send-email-mre...@redhat.com Reviewed-by: Alberto Garcia --- tests/qemu-

[Qemu-block] [PULL 28/40] vhdx: Use BB functions in .bdrv_create()

All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/vhdx.c | 29 - 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/b

[Qemu-block] [PULL 23/40] qcow: Use BB functions in .bdrv_create()

All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/qcow.c | 25 ++--- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/block

[Qemu-block] [PULL 18/40] hmp: 'drive_add -n' for creating a node without BB

This patch adds an option to the drive_add HMP command to create only a BlockDriverState without a BlockBackend on top. The motivation for this is that libvirt needs to specify options to a migration target (specifically, detect-zeroes). drive-mirror doesn't allow specifying options, and the prope

[Qemu-block] [PULL 07/40] block/vpc: give option to force the current_size field in .bdrv_create

From: Jeff Cody When QEMU creates a VHD image, it goes by the original spec, calculating the current_size based on the nearest CHS geometry (with an exception for disks > 127GB). Apparently, Azure will only allow images that are sized to the nearest MB, and the current_size as calculated from CH

[Qemu-block] [PULL 06/40] block/vpc: tests for auto-detecting VPC and Hyper-V VHD images

From: Jeff Cody This tests auto-detection, and overrides, of VHD image sizes created by Virtual PC, Hyper-V, and Disk2vhd. This adds three sample images: hyperv2012r2-dynamic.vhd.bz2 - dynamic VHD image created with Hyper-V virtualpc-dynamic.vhd.bz2- dynamic VHD image created with Virtual P

[Qemu-block] [PULL 13/40] block: Fix snapshot=on cache modes

Since commit 91a097e, we end up with a somewhat weird cache mode configuration with snapshot=on: The commit broke the cache mode inheritance for the snapshot overlay so that it is opened as writethrough instead of unsafe now. The following bdrv_append() call to put it on top of the tree swaps the W

[Qemu-block] [PULL 15/40] vmdk: Switch to heap arrays for vmdk_write_cid

From: Fam Zheng It is only called once for each opened image, so we can do it the easy way. Reviewed-by: Peter Xu Signed-off-by: Fam Zheng Reviewed-by: Paolo Bonzini Signed-off-by: Kevin Wolf --- block/vmdk.c | 25 ++--- 1 file changed, 14 insertions(+), 11 deletions(-)

[Qemu-block] [PULL 14/40] block: Fix cache mode defaults in bds_tree_init()

Without setting explicit defaults in the options, blockdev-add without an ID ended up defaulting to writethrough. It should be writeback as documented. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake --- blockdev.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/blockdev.c b/blockd

[Qemu-block] [PULL 22/40] parallels: Use BB functions in .bdrv_create()

All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/parallels.c | 25 ++--- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/

[Qemu-block] [PULL 24/40] qcow2: Use BB functions in .bdrv_create()

All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/qcow2.c | 62 +++ 1 file changed, 33 insertions(+),

[Qemu-block] [PULL 26/40] sheepdog: Use BB functions in .bdrv_create()

All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/sheepdog.c | 43 --- 1 file changed, 24 insertions(+), 19 deletions(

[Qemu-block] [PULL 27/40] vdi: Use BB functions in .bdrv_create()

All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/vdi.c | 23 ++- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/block/vdi

[Qemu-block] [PULL 32/40] block: Include hbitmap.h in block.h

From: Fam Zheng Signed-off-by: Fam Zheng Reviewed-by: John Snow Message-id: 1457412306-18940-3-git-send-email-f...@redhat.com Signed-off-by: Max Reitz --- include/block/block.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/block/block.h b/include/block/block.h in

[Qemu-block] [PULL 17/40] vmdk: Switch to heap arrays for vmdk_parent_open

From: Fam Zheng Signed-off-by: Fam Zheng Reviewed-by: Paolo Bonzini Signed-off-by: Kevin Wolf --- block/vmdk.c | 17 +++-- 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/block/vmdk.c b/block/vmdk.c index c68f456..03be7f0 100644 --- a/block/vmdk.c +++ b/block/vmdk.c

[Qemu-block] [PULL 00/40] Block patches

The following changes since commit 0dcee62261cb044339b10e4bda1f67ef7dc82803: Merge remote-tracking branch 'remotes/amit-migration/tags/migration-for-2.6-7' into staging (2016-03-14 13:51:21 +) are available in the git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream f

[Qemu-block] [PULL 05/40] block/vpc: choose size calculation method based on creator_app field

From: Jeff Cody The VHD file format is used by both Virtual PC, and Hyper-V. However, how the virtual disk size is calculated varies between the two. Virtual PC uses the CHS drive parameters to determine the drive size. Hyper-V, on the other hand, uses the current_size field in the footer when

[Qemu-block] [PULL 11/40] quorum: modify vote rules for flush operation

From: Changlong Xie Keep flush interface the same logic as quorum read/write, Otherwise in following scenario, we'll encounter unexpected errors. Quorum has two children(A, B). A do flush sucessfully, but B flush failed. This cause the filesystem of guest become read-only with following errors:

[Qemu-block] [PULL 12/40] blockdev: Snapshotting must not open second instance of old top

Calling bdrv_img_create() with a size of -1 means that it determines the size automatically by opening the backing file. However, in the case of live snapshots, the backing file is already opened and we must avoid opening the same image twice at the same time. Apart from that, just getting the size

[Qemu-block] [PULL 21/40] block: Introduce blk_set_allow_write_beyond_eof()

We check that the guest can't write beyond the end of its disk, but for other internal users it can make sense to allow growing a file. Signed-off-by: Kevin Wolf --- block/block-backend.c | 23 --- include/sysemu/block-backend.h | 1 + 2 files changed, 17 insertions

[Qemu-block] [PULL 16/40] vmdk: Switch to heap arrays for vmdk_read_cid

From: Fam Zheng Signed-off-by: Fam Zheng Reviewed-by: Paolo Bonzini Signed-off-by: Kevin Wolf --- block/vmdk.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/block/vmdk.c b/block/vmdk.c index 1ec2452..c68f456 100644 --- a/block/vmdk.c +++ b/block/vmdk.c @@ -242,15 +24

[Qemu-block] [PULL 10/40] qmp event: Refactor QUORUM_REPORT_BAD

From: Changlong Xie Introduce QuorumOpType, and make QUORUM_REPORT_BAD compatible with it. Cc: Dr. David Alan Gilbert Cc: Wen Congyang Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie Reviewed-by: Alberto Garcia Signed-off-by: Kevin Wolf --- block/quorum.c | 17 --

[Qemu-block] [PULL 09/40] docs: fix invalid node name in qmp event

From: Changlong Xie Cc: Dr. David Alan Gilbert Cc: Wen Congyang Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie Reviewed-by: Alberto Garcia Signed-off-by: Kevin Wolf --- docs/qmp-events.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/qmp-events.txt b/d

[Qemu-block] [PULL 08/40] block/vpc: add tests for image creation force_size parameter

From: Jeff Cody Signed-off-by: Jeff Cody Signed-off-by: Kevin Wolf --- tests/qemu-iotests/146 | 51 ++ tests/qemu-iotests/146.out | 32 + 2 files changed, 83 insertions(+) diff --git a/tests/qemu-iotests/146 b/tests/q

[Qemu-block] [PULL 04/40] block/qapi: Include empty drives in query-blockstats

Since commit 5ec18f8c, query-blockstats didn't return the statistics of drives without media any more because such drives have only a BB now, but not a BDS any more. This patch fixes the regression so that query-blockstats iterates over BBs by default and empty drives are displayed again. Signed-

[Qemu-block] [PULL 03/40] block/qapi: Factor out bdrv_query_bds_stats()

The new functions handles the data that is taken from the BlockDriverState. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- block/qapi.c | 31 --- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/block/qapi.c b/block/qapi.c index c04f1d8..31ae879

[Qemu-block] [PULL 01/40] qemu-img: eliminate memory leak

From: Paolo Bonzini Not particularly important since qemu-img exits immediately after calling img_rebase, but easily fixed. Coverity says thanks. Signed-off-by: Paolo Bonzini Reviewed-by: Alberto Garcia Signed-off-by: Kevin Wolf --- qemu-img.c | 9 - 1 file changed, 4 insertions(+),

[Qemu-block] [PULL 02/40] block/qapi: Factor out bdrv_query_blk_stats()

The new functions handles the data that is taken from the BlockBackend. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- block/qapi.c | 131 +++ 1 file changed, 68 insertions(+), 63 deletions(-) diff --git a/block/qapi.c b/block/qapi.

Re: [Qemu-block] [Qemu-devel] [PATCH 00/11] block: Use BB function in .bdrv_create() implementations

Am 14.03.2016 um 17:55 hat Max Reitz geschrieben: > On 14.03.2016 12:31, Kevin Wolf wrote: > > Am 08.03.2016 um 17:34 hat Kevin Wolf geschrieben: > >> All users of the block layers are supposed to go through a BlockBackend. > >> The .bdrv_create() implementations are such users, so this series > >>

Re: [Qemu-block] [Qemu-devel] [PATCH 00/11] block: Use BB function in .bdrv_create() implementations

On 14.03.2016 12:31, Kevin Wolf wrote: > Am 08.03.2016 um 17:34 hat Kevin Wolf geschrieben: >> All users of the block layers are supposed to go through a BlockBackend. >> The .bdrv_create() implementations are such users, so this series >> converts them. >> >> This series (specifically patch 1) wil

Re: [Qemu-block] [PATCH 5/5] block: Reject writethrough mode except at the root

On 03/14/2016 09:44 AM, Kevin Wolf wrote: > Writethrough mode is going to become a BlockBackend feature rather than > a BDS one, so forbid it in places where we won't be able to support it > when the code finally matches the envisioned design. > > We only allowed setting the cache mode of non-root

Re: [Qemu-block] [PATCH 4/5] block: Make backing files always writeback

On 03/14/2016 09:44 AM, Kevin Wolf wrote: > First of all, we're generally not writing to backing files, but when we > do, it's in the context of block jobs which know very well when to flush > the image. > > Signed-off-by: Kevin Wolf > --- > block.c| 5 +++-- > tests/qemu-io

Re: [Qemu-block] [PATCH 3/5] block: Remove cache.writeback from blockdev-add

Am 14.03.2016 um 17:10 hat Eric Blake geschrieben: > On 03/14/2016 09:44 AM, Kevin Wolf wrote: > > The WCE bit is a frontend property and should not be part of the backend > > configuration. This is especially important because the same BDS can be > > used by different users with different WCE requ

Re: [Qemu-block] [PATCH v2 0/4] Separate QUORUM_REPORT_BAD events according to their node name

On 10.03.2016 12:55, Alberto Garcia wrote: > QUORUM_REPORT_BAD events are limited to a maximum rate of 1 per > second. While this is not a problem in itself, this means that an > error in one a Quorum child will mask errors in the other children if > they happen within the same 1 second interval. >

Re: [Qemu-block] [PATCH 3/5] block: Remove cache.writeback from blockdev-add

On 03/14/2016 09:44 AM, Kevin Wolf wrote: > The WCE bit is a frontend property and should not be part of the backend > configuration. This is especially important because the same BDS can be > used by different users with different WCE requirements. > > Signed-off-by: Kevin Wolf > --- > qapi/blo

Re: [Qemu-block] [PATCH 2/5] block: Remove dirty bitmaps from bdrv_move_feature_fields()

On 03/14/2016 09:44 AM, Kevin Wolf wrote: > This patch changes dirty bitmaps from following a BlockBackend in graph > changes to sticking with the node they were created at. For the full > discussion, read the following mailing list thread: > > [Qemu-block] block: Dirty bitmaps and COR in bdrv_m

Re: [Qemu-block] [PATCH 1/5] block: Remove copy-on-read from bdrv_move_feature_fields()

On 03/14/2016 09:44 AM, Kevin Wolf wrote: > Ever since we first introduced bdrv_append() in commit 8802d1fd ('qapi: > Introduce blockdev-group-snapshot-sync command'), the copy-on-read flag > was moved to the new top layer when taking a snapshot. The only problem > is that it doesn't make a whole l

[Qemu-block] [PATCH 5/5] block: Reject writethrough mode except at the root

Writethrough mode is going to become a BlockBackend feature rather than a BDS one, so forbid it in places where we won't be able to support it when the code finally matches the envisioned design. We only allowed setting the cache mode of non-root nodes after the 2.5 release, so we're still free to

[Qemu-block] [PATCH 3/5] block: Remove cache.writeback from blockdev-add

The WCE bit is a frontend property and should not be part of the backend configuration. This is especially important because the same BDS can be used by different users with different WCE requirements. Signed-off-by: Kevin Wolf --- qapi/block-core.json | 4 +--- 1 file changed, 1 insertion(+), 3

[Qemu-block] [PATCH 4/5] block: Make backing files always writeback

First of all, we're generally not writing to backing files, but when we do, it's in the context of block jobs which know very well when to flush the image. Signed-off-by: Kevin Wolf --- block.c| 5 +++-- tests/qemu-iotests/142.out | 10 +- 2 files changed, 8 insertio

[Qemu-block] [PATCH 1/5] block: Remove copy-on-read from bdrv_move_feature_fields()

Ever since we first introduced bdrv_append() in commit 8802d1fd ('qapi: Introduce blockdev-group-snapshot-sync command'), the copy-on-read flag was moved to the new top layer when taking a snapshot. The only problem is that it doesn't make a whole lot of sense. The use case for manually enabled Co

[Qemu-block] [PATCH 2/5] block: Remove dirty bitmaps from bdrv_move_feature_fields()

This patch changes dirty bitmaps from following a BlockBackend in graph changes to sticking with the node they were created at. For the full discussion, read the following mailing list thread: [Qemu-block] block: Dirty bitmaps and COR in bdrv_move_feature_fields() https://lists.nongnu.org/arch

[Qemu-block] [PATCH 0/5] block: API changes for 2.6

This series contains those part of my current work that I want to be in the 2.6 release because they change the semantics of some QMP interfaces. This is especially true for the WCE changes, which concern things that weren't even accessible in 2.5 yet, so we're still completely free to change them

Re: [Qemu-block] [PATCH v4 09/26] crypto: import an implementation of the XTS cipher mode

On Fri, Mar 11, 2016 at 12:51:29PM -0700, Eric Blake wrote: > On 02/29/2016 05:00 AM, Daniel P. Berrange wrote: > > The XTS (XEX with tweaked-codebook and ciphertext stealing) > > cipher mode is commonly used in full disk encryption. There > > is unfortunately no implementation of it in either libg

Re: [Qemu-block] [PATCH v4 13/26] crypto: implement the LUKS block encryption format

On Fri, Mar 11, 2016 at 03:31:59PM -0700, Eric Blake wrote: > On 02/29/2016 05:00 AM, Daniel P. Berrange wrote: > > Provide a block encryption implementation that follows the > > LUKS/dm-crypt specification. > > > > This supports all combinations of hash, cipher algorithm, > > cipher mode and iv g

Re: [Qemu-block] [PATCH v4 12/15] hbitmap: serialization

On 11.03.2016 19:27, Max Reitz wrote: On 08.03.2016 05:45, Fam Zheng wrote: From: Vladimir Sementsov-Ogievskiy Functions to serialize / deserialize(restore) HBitmap. HBitmap should be saved to linear sequence of bits independently of endianness and bitmap array element (unsigned long) size. Th

Re: [Qemu-block] [Qemu-devel] [PATCH] quorum: Fix crash in quorum_aio_cb()

On Mon 14 Mar 2016 02:57:31 AM CET, Changlong Xie wrote: >> And now that we're at it, shouldn't we call quorum_report_bad() in >> FIFO mode as well? Or is there any reason not to do it? > > IMO, no reason not to do it. I'll send a patch to fix that. Thanks, Berto

Re: [Qemu-block] [PATCH 00/11] block: Use BB function in .bdrv_create() implementations

Am 08.03.2016 um 17:34 hat Kevin Wolf geschrieben: > All users of the block layers are supposed to go through a BlockBackend. > The .bdrv_create() implementations are such users, so this series > converts them. > > This series (specifically patch 1) will also help with moving the > writethrough ca