Re: [Qemu-block] [PATCH v4 18/26] tests: add output filter to python I/O tests helper

2016-03-14 Thread Daniel P. Berrange
On Mon, Mar 14, 2016 at 11:57:11AM -0600, Eric Blake wrote: > On 02/29/2016 05:00 AM, Daniel P. Berrange wrote: > > Add a 'log' method to iotests.py which prints messages to > > stdout, with optional filtering of data. Port over some > > standard filters for present in the shell common.filter > >

[Qemu-block] [PULL 20/40] block: Use writeback in .bdrv_create() implementations

2016-03-14 Thread Kevin Wolf
There's no reason to use a writethrough cache mode while creating an image. Signed-off-by: Kevin Wolf --- block/parallels.c | 3 ++- block/qcow.c | 3 ++- block/qcow2.c | 3 ++- block/sheepdog.c | 6 -- block/vdi.c | 3 ++- block/vhdx.c | 3 ++-

[Qemu-block] [PULL 37/40] quorum: Fix crash in quorum_aio_cb()

2016-03-14 Thread Kevin Wolf
From: Alberto Garcia quorum_aio_cb() emits the QUORUM_REPORT_BAD event if there's an I/O error in a Quorum child. However sacb->aiocb must be correctly initialized for this to happen. read_quorum_children() and read_fifo_child() are not doing this, which results in a QEMU

[Qemu-block] [PULL 40/40] iotests: Add test for QMP event rates

2016-03-14 Thread Kevin Wolf
From: Alberto Garcia This test verifies that the rate-limited QMP events are emitted at a maximum rate of 1 per second as defined in monitor_qapi_event_conf in monitor.c It also checks that QUORUM_REPORT_BAD events generated from different nodes are kept in separate queues so

Re: [Qemu-block] [PATCH v4 18/26] tests: add output filter to python I/O tests helper

2016-03-14 Thread Eric Blake
On 02/29/2016 05:00 AM, Daniel P. Berrange wrote: > Add a 'log' method to iotests.py which prints messages to > stdout, with optional filtering of data. Port over some > standard filters for present in the shell common.filter > code. I'm not sure what you meant by 'for present'. > >

[Qemu-block] [PULL 19/40] hmp: Extend drive_del to delete nodes without BB

2016-03-14 Thread Kevin Wolf
Now that we can use drive_add to create new nodes without a BB, we also want to be able to delete such nodes again. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- blockdev.c | 9 + 1 file changed, 9 insertions(+) diff --git a/blockdev.c

[Qemu-block] [PULL 39/40] monitor: Use QEMU_CLOCK_VIRTUAL for the event queue in qtest mode

2016-03-14 Thread Kevin Wolf
From: Alberto Garcia This allows us to perform tests on the monitor queues to verify that the rate limits are enforced. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Message-id:

[Qemu-block] [PULL 29/40] vmdk: Use BB functions in .bdrv_create()

2016-03-14 Thread Kevin Wolf
All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/vmdk.c | 77 +--- 1 file changed,

[Qemu-block] [PULL 25/40] qed: Use BB functions in .bdrv_create()

2016-03-14 Thread Kevin Wolf
All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/qed.c | 28 +++- 1 file changed, 15 insertions(+), 13

[Qemu-block] [PULL 31/40] backup: Use Bitmap to replace "s->bitmap"

2016-03-14 Thread Kevin Wolf
From: Fam Zheng "s->bitmap" tracks done sectors, we only check bit states without using any iterator which HBitmap is good for. Switch to "Bitmap" which is simpler and more memory efficient. Meanwhile, rename it to done_bitmap, to reflect the intention. Signed-off-by: Fam

[Qemu-block] [PULL 33/40] typedefs: Add BdrvDirtyBitmap

2016-03-14 Thread Kevin Wolf
From: Fam Zheng Following patches to refactor and move block dirty bitmap code could use this. Signed-off-by: Fam Zheng Reviewed-by: John Snow Message-id: 1457412306-18940-4-git-send-email-f...@redhat.com Signed-off-by: Max Reitz

[Qemu-block] [PULL 34/40] block: Move block dirty bitmap code to separate files

2016-03-14 Thread Kevin Wolf
From: Fam Zheng The only code change is making bdrv_dirty_bitmap_truncate public. It is used in block.c. Also two long lines (bdrv_get_dirty) are wrapped. Signed-off-by: Fam Zheng Reviewed-by: John Snow Message-id:

[Qemu-block] [PULL 30/40] vpc: Use BB functions in .bdrv_create()

2016-03-14 Thread Kevin Wolf
All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/vpc.c | 37 + 1 file changed, 21 insertions(+), 16

[Qemu-block] [PULL 35/40] block: Remove unused typedef of BlockDriverDirtyHandler

2016-03-14 Thread Kevin Wolf
From: Fam Zheng Signed-off-by: Fam Zheng Reviewed-by: John Snow Message-id: 1457412306-18940-6-git-send-email-f...@redhat.com Signed-off-by: Max Reitz --- include/block/block.h | 2 -- 1 file changed, 2 deletions(-) diff

[Qemu-block] [PULL 36/40] iotests: Correct 081's reference output

2016-03-14 Thread Kevin Wolf
From: Max Reitz The newly added type parameter for the QUORUM_REPORT_BAD event changed the output of iotest 081, so the reference should be amended accordingly. Signed-off-by: Max Reitz Message-id: 1457705687-27122-1-git-send-email-mre...@redhat.com

[Qemu-block] [PULL 28/40] vhdx: Use BB functions in .bdrv_create()

2016-03-14 Thread Kevin Wolf
All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/vhdx.c | 29 - 1 file changed, 16 insertions(+), 13

[Qemu-block] [PULL 23/40] qcow: Use BB functions in .bdrv_create()

2016-03-14 Thread Kevin Wolf
All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/qcow.c | 25 ++--- 1 file changed, 14 insertions(+), 11 deletions(-)

[Qemu-block] [PULL 18/40] hmp: 'drive_add -n' for creating a node without BB

2016-03-14 Thread Kevin Wolf
This patch adds an option to the drive_add HMP command to create only a BlockDriverState without a BlockBackend on top. The motivation for this is that libvirt needs to specify options to a migration target (specifically, detect-zeroes). drive-mirror doesn't allow specifying options, and the

[Qemu-block] [PULL 07/40] block/vpc: give option to force the current_size field in .bdrv_create

2016-03-14 Thread Kevin Wolf
From: Jeff Cody When QEMU creates a VHD image, it goes by the original spec, calculating the current_size based on the nearest CHS geometry (with an exception for disks > 127GB). Apparently, Azure will only allow images that are sized to the nearest MB, and the current_size as

[Qemu-block] [PULL 13/40] block: Fix snapshot=on cache modes

2016-03-14 Thread Kevin Wolf
Since commit 91a097e, we end up with a somewhat weird cache mode configuration with snapshot=on: The commit broke the cache mode inheritance for the snapshot overlay so that it is opened as writethrough instead of unsafe now. The following bdrv_append() call to put it on top of the tree swaps the

[Qemu-block] [PULL 22/40] parallels: Use BB functions in .bdrv_create()

2016-03-14 Thread Kevin Wolf
All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/parallels.c | 25 ++--- 1 file changed, 14 insertions(+), 11

[Qemu-block] [PULL 24/40] qcow2: Use BB functions in .bdrv_create()

2016-03-14 Thread Kevin Wolf
All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/qcow2.c | 62 +++ 1 file changed,

[Qemu-block] [PULL 27/40] vdi: Use BB functions in .bdrv_create()

2016-03-14 Thread Kevin Wolf
All users of the block layers are supposed to go through a BlockBackend. The .bdrv_create() implementation is one such user, so this patch converts it. Signed-off-by: Kevin Wolf --- block/vdi.c | 23 ++- 1 file changed, 14 insertions(+), 9 deletions(-)

[Qemu-block] [PULL 32/40] block: Include hbitmap.h in block.h

2016-03-14 Thread Kevin Wolf
From: Fam Zheng Signed-off-by: Fam Zheng Reviewed-by: John Snow Message-id: 1457412306-18940-3-git-send-email-f...@redhat.com Signed-off-by: Max Reitz --- include/block/block.h | 2 +- 1 file changed, 1 insertion(+), 1

[Qemu-block] [PULL 17/40] vmdk: Switch to heap arrays for vmdk_parent_open

2016-03-14 Thread Kevin Wolf
From: Fam Zheng Signed-off-by: Fam Zheng Reviewed-by: Paolo Bonzini Signed-off-by: Kevin Wolf --- block/vmdk.c | 17 +++-- 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/block/vmdk.c

[Qemu-block] [PULL 00/40] Block patches

2016-03-14 Thread Kevin Wolf
The following changes since commit 0dcee62261cb044339b10e4bda1f67ef7dc82803: Merge remote-tracking branch 'remotes/amit-migration/tags/migration-for-2.6-7' into staging (2016-03-14 13:51:21 +) are available in the git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream

[Qemu-block] [PULL 05/40] block/vpc: choose size calculation method based on creator_app field

2016-03-14 Thread Kevin Wolf
From: Jeff Cody The VHD file format is used by both Virtual PC, and Hyper-V. However, how the virtual disk size is calculated varies between the two. Virtual PC uses the CHS drive parameters to determine the drive size. Hyper-V, on the other hand, uses the current_size field

[Qemu-block] [PULL 11/40] quorum: modify vote rules for flush operation

2016-03-14 Thread Kevin Wolf
From: Changlong Xie Keep flush interface the same logic as quorum read/write, Otherwise in following scenario, we'll encounter unexpected errors. Quorum has two children(A, B). A do flush sucessfully, but B flush failed. This cause the filesystem of guest become

[Qemu-block] [PULL 12/40] blockdev: Snapshotting must not open second instance of old top

2016-03-14 Thread Kevin Wolf
Calling bdrv_img_create() with a size of -1 means that it determines the size automatically by opening the backing file. However, in the case of live snapshots, the backing file is already opened and we must avoid opening the same image twice at the same time. Apart from that, just getting the

[Qemu-block] [PULL 16/40] vmdk: Switch to heap arrays for vmdk_read_cid

2016-03-14 Thread Kevin Wolf
From: Fam Zheng Signed-off-by: Fam Zheng Reviewed-by: Paolo Bonzini Signed-off-by: Kevin Wolf --- block/vmdk.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/block/vmdk.c b/block/vmdk.c index

[Qemu-block] [PULL 21/40] block: Introduce blk_set_allow_write_beyond_eof()

2016-03-14 Thread Kevin Wolf
We check that the guest can't write beyond the end of its disk, but for other internal users it can make sense to allow growing a file. Signed-off-by: Kevin Wolf --- block/block-backend.c | 23 --- include/sysemu/block-backend.h | 1 + 2 files

[Qemu-block] [PULL 10/40] qmp event: Refactor QUORUM_REPORT_BAD

2016-03-14 Thread Kevin Wolf
From: Changlong Xie Introduce QuorumOpType, and make QUORUM_REPORT_BAD compatible with it. Cc: Dr. David Alan Gilbert Cc: Wen Congyang Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie

[Qemu-block] [PULL 08/40] block/vpc: add tests for image creation force_size parameter

2016-03-14 Thread Kevin Wolf
From: Jeff Cody Signed-off-by: Jeff Cody Signed-off-by: Kevin Wolf --- tests/qemu-iotests/146 | 51 ++ tests/qemu-iotests/146.out | 32 + 2 files changed, 83

[Qemu-block] [PULL 01/40] qemu-img: eliminate memory leak

2016-03-14 Thread Kevin Wolf
From: Paolo Bonzini Not particularly important since qemu-img exits immediately after calling img_rebase, but easily fixed. Coverity says thanks. Signed-off-by: Paolo Bonzini Reviewed-by: Alberto Garcia Signed-off-by: Kevin Wolf

[Qemu-block] [PULL 04/40] block/qapi: Include empty drives in query-blockstats

2016-03-14 Thread Kevin Wolf
Since commit 5ec18f8c, query-blockstats didn't return the statistics of drives without media any more because such drives have only a BB now, but not a BDS any more. This patch fixes the regression so that query-blockstats iterates over BBs by default and empty drives are displayed again.

[Qemu-block] [PULL 02/40] block/qapi: Factor out bdrv_query_blk_stats()

2016-03-14 Thread Kevin Wolf
The new functions handles the data that is taken from the BlockBackend. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- block/qapi.c | 131 +++ 1 file changed, 68 insertions(+), 63 deletions(-)

Re: [Qemu-block] [Qemu-devel] [PATCH 00/11] block: Use BB function in .bdrv_create() implementations

2016-03-14 Thread Kevin Wolf
Am 14.03.2016 um 17:55 hat Max Reitz geschrieben: > On 14.03.2016 12:31, Kevin Wolf wrote: > > Am 08.03.2016 um 17:34 hat Kevin Wolf geschrieben: > >> All users of the block layers are supposed to go through a BlockBackend. > >> The .bdrv_create() implementations are such users, so this series >

Re: [Qemu-block] [Qemu-devel] [PATCH 00/11] block: Use BB function in .bdrv_create() implementations

2016-03-14 Thread Max Reitz
On 14.03.2016 12:31, Kevin Wolf wrote: > Am 08.03.2016 um 17:34 hat Kevin Wolf geschrieben: >> All users of the block layers are supposed to go through a BlockBackend. >> The .bdrv_create() implementations are such users, so this series >> converts them. >> >> This series (specifically patch 1)

Re: [Qemu-block] [PATCH 5/5] block: Reject writethrough mode except at the root

2016-03-14 Thread Eric Blake
On 03/14/2016 09:44 AM, Kevin Wolf wrote: > Writethrough mode is going to become a BlockBackend feature rather than > a BDS one, so forbid it in places where we won't be able to support it > when the code finally matches the envisioned design. > > We only allowed setting the cache mode of

Re: [Qemu-block] [PATCH 4/5] block: Make backing files always writeback

2016-03-14 Thread Eric Blake
On 03/14/2016 09:44 AM, Kevin Wolf wrote: > First of all, we're generally not writing to backing files, but when we > do, it's in the context of block jobs which know very well when to flush > the image. > > Signed-off-by: Kevin Wolf > --- > block.c| 5

Re: [Qemu-block] [PATCH 3/5] block: Remove cache.writeback from blockdev-add

2016-03-14 Thread Kevin Wolf
Am 14.03.2016 um 17:10 hat Eric Blake geschrieben: > On 03/14/2016 09:44 AM, Kevin Wolf wrote: > > The WCE bit is a frontend property and should not be part of the backend > > configuration. This is especially important because the same BDS can be > > used by different users with different WCE

Re: [Qemu-block] [PATCH 3/5] block: Remove cache.writeback from blockdev-add

2016-03-14 Thread Eric Blake
On 03/14/2016 09:44 AM, Kevin Wolf wrote: > The WCE bit is a frontend property and should not be part of the backend > configuration. This is especially important because the same BDS can be > used by different users with different WCE requirements. > > Signed-off-by: Kevin Wolf

Re: [Qemu-block] [PATCH 2/5] block: Remove dirty bitmaps from bdrv_move_feature_fields()

2016-03-14 Thread Eric Blake
On 03/14/2016 09:44 AM, Kevin Wolf wrote: > This patch changes dirty bitmaps from following a BlockBackend in graph > changes to sticking with the node they were created at. For the full > discussion, read the following mailing list thread: > > [Qemu-block] block: Dirty bitmaps and COR in

Re: [Qemu-block] [PATCH 1/5] block: Remove copy-on-read from bdrv_move_feature_fields()

2016-03-14 Thread Eric Blake
On 03/14/2016 09:44 AM, Kevin Wolf wrote: > Ever since we first introduced bdrv_append() in commit 8802d1fd ('qapi: > Introduce blockdev-group-snapshot-sync command'), the copy-on-read flag > was moved to the new top layer when taking a snapshot. The only problem > is that it doesn't make a whole

[Qemu-block] [PATCH 5/5] block: Reject writethrough mode except at the root

2016-03-14 Thread Kevin Wolf
Writethrough mode is going to become a BlockBackend feature rather than a BDS one, so forbid it in places where we won't be able to support it when the code finally matches the envisioned design. We only allowed setting the cache mode of non-root nodes after the 2.5 release, so we're still free

[Qemu-block] [PATCH 3/5] block: Remove cache.writeback from blockdev-add

2016-03-14 Thread Kevin Wolf
The WCE bit is a frontend property and should not be part of the backend configuration. This is especially important because the same BDS can be used by different users with different WCE requirements. Signed-off-by: Kevin Wolf --- qapi/block-core.json | 4 +--- 1 file

[Qemu-block] [PATCH 4/5] block: Make backing files always writeback

2016-03-14 Thread Kevin Wolf
First of all, we're generally not writing to backing files, but when we do, it's in the context of block jobs which know very well when to flush the image. Signed-off-by: Kevin Wolf --- block.c| 5 +++-- tests/qemu-iotests/142.out | 10 +- 2 files

[Qemu-block] [PATCH 1/5] block: Remove copy-on-read from bdrv_move_feature_fields()

2016-03-14 Thread Kevin Wolf
Ever since we first introduced bdrv_append() in commit 8802d1fd ('qapi: Introduce blockdev-group-snapshot-sync command'), the copy-on-read flag was moved to the new top layer when taking a snapshot. The only problem is that it doesn't make a whole lot of sense. The use case for manually enabled

[Qemu-block] [PATCH 2/5] block: Remove dirty bitmaps from bdrv_move_feature_fields()

2016-03-14 Thread Kevin Wolf
This patch changes dirty bitmaps from following a BlockBackend in graph changes to sticking with the node they were created at. For the full discussion, read the following mailing list thread: [Qemu-block] block: Dirty bitmaps and COR in bdrv_move_feature_fields()

[Qemu-block] [PATCH 0/5] block: API changes for 2.6

2016-03-14 Thread Kevin Wolf
This series contains those part of my current work that I want to be in the 2.6 release because they change the semantics of some QMP interfaces. This is especially true for the WCE changes, which concern things that weren't even accessible in 2.5 yet, so we're still completely free to change

Re: [Qemu-block] [PATCH v4 09/26] crypto: import an implementation of the XTS cipher mode

2016-03-14 Thread Daniel P. Berrange
On Fri, Mar 11, 2016 at 12:51:29PM -0700, Eric Blake wrote: > On 02/29/2016 05:00 AM, Daniel P. Berrange wrote: > > The XTS (XEX with tweaked-codebook and ciphertext stealing) > > cipher mode is commonly used in full disk encryption. There > > is unfortunately no implementation of it in either

Re: [Qemu-block] [PATCH v4 13/26] crypto: implement the LUKS block encryption format

2016-03-14 Thread Daniel P. Berrange
On Fri, Mar 11, 2016 at 03:31:59PM -0700, Eric Blake wrote: > On 02/29/2016 05:00 AM, Daniel P. Berrange wrote: > > Provide a block encryption implementation that follows the > > LUKS/dm-crypt specification. > > > > This supports all combinations of hash, cipher algorithm, > > cipher mode and iv

Re: [Qemu-block] [PATCH v4 12/15] hbitmap: serialization

2016-03-14 Thread Vladimir Sementsov-Ogievskiy
On 11.03.2016 19:27, Max Reitz wrote: On 08.03.2016 05:45, Fam Zheng wrote: From: Vladimir Sementsov-Ogievskiy Functions to serialize / deserialize(restore) HBitmap. HBitmap should be saved to linear sequence of bits independently of endianness and bitmap array

Re: [Qemu-block] [Qemu-devel] [PATCH] quorum: Fix crash in quorum_aio_cb()

2016-03-14 Thread Alberto Garcia
On Mon 14 Mar 2016 02:57:31 AM CET, Changlong Xie wrote: >> And now that we're at it, shouldn't we call quorum_report_bad() in >> FIFO mode as well? Or is there any reason not to do it? > > IMO, no reason not to do it. I'll send a patch to fix that. Thanks, Berto

Re: [Qemu-block] [PATCH 00/11] block: Use BB function in .bdrv_create() implementations

2016-03-14 Thread Kevin Wolf
Am 08.03.2016 um 17:34 hat Kevin Wolf geschrieben: > All users of the block layers are supposed to go through a BlockBackend. > The .bdrv_create() implementations are such users, so this series > converts them. > > This series (specifically patch 1) will also help with moving the > writethrough

Re: [Qemu-block] [PATCH v12 2/3] quorum: implement bdrv_add_child() and bdrv_del_child()

2016-03-14 Thread Changlong Xie
On 03/11/2016 08:21 PM, Alberto Garcia wrote: On Thu 10 Mar 2016 03:49:40 AM CET, Changlong Xie wrote: @@ -81,6 +82,8 @@ typedef struct BDRVQuorumState { bool rewrite_corrupted;/* true if the driver must rewrite-on-read corrupted * block if Quorum is