[Qemu-block] [PATCH] blockjob: cancel blockjobs before stopping all iothreads

From: "sochin.jiang" commit 88b062c and commit c9d1a56 introduce BDRV_POLL_WHILE, checking bs->in_flight in the main thread and exit till bs->in_flight reaches 0. This leaves a chance that BDRV_POLL_WHILE will hang until iothread complete the relative block job when

Re: [Qemu-block] [PATCH 0/2] commit: Fix use after free in completion

On 06/02/2017 05:12 PM, Kevin Wolf wrote: > Kevin Wolf (2): > commit: Fix use after free in completion > qemu-iotests: Test automatic commit job cancel on hot unplug > > block/commit.c | 7 +++ > tests/qemu-iotests/040 | 35 +-- >

Re: [Qemu-block] [Qemu-devel] [PATCH v20 14/30] qcow2: support .bdrv_reopen_bitmaps_rw

On 06/02/2017 07:21 AM, Vladimir Sementsov-Ogievskiy wrote: > Realize bdrv_reopen_bitmaps_rw interface. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow > --- > block/qcow2-bitmap.c | 61 >

Re: [Qemu-block] [Qemu-devel] [PATCH v20 13/30] block: new bdrv_reopen_bitmaps_rw interface

On 06/02/2017 07:21 AM, Vladimir Sementsov-Ogievskiy wrote: > Add format driver handler, which should mark loaded read-only > bitmaps as 'IN_USE' in the image and unset read_only field in > corresponding BdrvDirtyBitmap's. > > Signed-off-by: Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [Qemu-devel] [PATCH v20 12/30] block: refactor bdrv_reopen_commit

On 06/02/2017 07:21 AM, Vladimir Sementsov-Ogievskiy wrote: > Add bs local variable to simplify code. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block.c | 14 -- > 1 file changed, 8 insertions(+), 6 deletions(-) > > diff --git a/block.c

Re: [Qemu-block] [Qemu-devel] [PATCH v20 11/30] qcow2: autoloading dirty bitmaps

On 06/02/2017 07:21 AM, Vladimir Sementsov-Ogievskiy wrote: > Auto loading bitmaps are bitmaps in Qcow2, with the AUTO flag set. They > are loaded when the image is opened and become BdrvDirtyBitmaps for the > corresponding drive. > > Extra data in bitmaps is not supported for now. > >

[Qemu-block] [PATCH 2/2] qemu-iotests: Test automatic commit job cancel on hot unplug

Signed-off-by: Kevin Wolf --- tests/qemu-iotests/040 | 35 +-- tests/qemu-iotests/040.out | 4 ++-- 2 files changed, 35 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 index 5bdaf3d..9d381d9 100755

[Qemu-block] [PATCH 0/2] commit: Fix use after free in completion

Kevin Wolf (2): commit: Fix use after free in completion qemu-iotests: Test automatic commit job cancel on hot unplug block/commit.c | 7 +++ tests/qemu-iotests/040 | 35 +-- tests/qemu-iotests/040.out | 4 ++-- 3 files changed, 42

[Qemu-block] [PATCH 1/2] commit: Fix use after free in completion

The final bdrv_set_backing_hd() could be working on already freed nodes because the commit job drops its references (through BlockBackends) to both overlay_bs and top already a bit earlier. One way to trigger the bug is hot unplugging a disk for which blockdev_mark_auto_del() cancels the block

Re: [Qemu-block] [Qemu-devel] [PATCH v20 10/30] block/dirty-bitmap: add readonly field to BdrvDirtyBitmap

On 06/02/2017 07:21 AM, Vladimir Sementsov-Ogievskiy wrote: > It will be needed in following commits for persistent bitmaps. > If bitmap is loaded from read-only storage (and we can't mark it > "in use" in this storage) corresponding BdrvDirtyBitmap should be > read-only. > > Signed-off-by:

Re: [Qemu-block] [Qemu-devel] [PATCH v20 09/30] block/dirty-bitmap: fix comment for BlockDirtyBitmap.disabled field

On 06/02/2017 07:21 AM, Vladimir Sementsov-Ogievskiy wrote: > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/dirty-bitmap.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c > index

Re: [Qemu-block] [Qemu-devel] [PATCH 09/25] block/dirty-bitmap: add readonly field to BdrvDirtyBitmap

On 06/02/2017 05:45 AM, Vladimir Sementsov-Ogievskiy wrote: > 02.06.2017 12:01, Vladimir Sementsov-Ogievskiy wrote: >> 02.06.2017 11:56, Vladimir Sementsov-Ogievskiy wrote: >>> 02.06.2017 02:25, John Snow wrote: On 06/01/2017 03:30 AM, Sementsov-Ogievskiy Vladimir wrote: > Hi John!

Re: [Qemu-block] [PULL 0/1] Block patches

On 2 June 2017 at 16:33, Jeff Cody wrote: > The following changes since commit d47a851caeda96d5979bf48d4bae6a87784ad91d: > > Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20170601' > into staging (2017-06-02 14:07:53 +0100) > > are available in the git

Re: [Qemu-block] [PATCH 10/29] qed: Remove callback from qed_write_header()

On Thu, Jun 01, 2017 at 05:59:53PM +0200, Kevin Wolf wrote: > Am 31.05.2017 um 14:32 hat Stefan Hajnoczi geschrieben: > > On Fri, May 26, 2017 at 10:21:51PM +0200, Kevin Wolf wrote: > > > static void qed_clear_need_check(void *opaque, int ret) > > > { > > > BDRVQEDState *s = opaque; > > >

Re: [Qemu-block] [PATCH 10/29] qed: Remove callback from qed_write_header()

On Thu, Jun 01, 2017 at 06:04:25PM +0200, Paolo Bonzini wrote: > > > On 01/06/2017 17:59, Kevin Wolf wrote: > > Am 31.05.2017 um 14:32 hat Stefan Hajnoczi geschrieben: > >> On Fri, May 26, 2017 at 10:21:51PM +0200, Kevin Wolf wrote: > >>> static void qed_clear_need_check(void *opaque, int ret)

[Qemu-block] [PULL 1/1] gluster: add support for PREALLOC_MODE_FALLOC

From: Niels de Vos Add missing support for "preallocation=falloc" to the Gluster block driver. This change bases its logic on that of block/file-posix.c and removed the gluster_supports_zerofill() and qemu_gluster_zerofill() functions in favour of #ifdef checks in an easy to

[Qemu-block] [PULL 0/1] Block patches

The following changes since commit d47a851caeda96d5979bf48d4bae6a87784ad91d: Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20170601' into staging (2017-06-02 14:07:53 +0100) are available in the git repository at: git://github.com/codyprime/qemu-kvm-jtc.git

Re: [Qemu-block] [PATCH 1/4] block: add bdrv_get_format_alloc_stat format interface

30.05.2017 17:53, Eric Blake wrote: On 05/30/2017 05:48 AM, Vladimir Sementsov-Ogievskiy wrote: The function should collect statistics, about allocted/unallocated by top-level format driver space (in its .file) and allocation status (allocated/hole/after eof) of corresponding areas in this

Re: [Qemu-block] [Qemu-devel] [PATCH v2 02/15] file-posix: support BDRV_REQ_ALLOCATE

On 06/01/2017 10:54 PM, Eric Blake wrote: On 06/01/2017 02:49 PM, Eric Blake wrote: On 06/01/2017 10:14 AM, Anton Nefedov wrote: Current write_zeroes implementation is good enough to satisfy this flag too Signed-off-by: Anton Nefedov --- block/file-posix.c | 9

Re: [Qemu-block] NVME: is there any plan to support SGL data transfer?

Am 02.06.2017 um 03:47 hat Qu Wenruo geschrieben: > When going through NVMe specification and hw/block/nvme.c, > I found that it seems that NVMe qemu implementation only support PRP > for sq entry. > And NvmeRwCmd doesn't even use union to define DPTR, but just prp1 and prp2. > > Although I am

[Qemu-block] NVME: is there any plan to support SGL data transfer?

Hi, When going through NVMe specification and hw/block/nvme.c, I found that it seems that NVMe qemu implementation only support PRP for sq entry. And NvmeRwCmd doesn't even use union to define DPTR, but just prp1 and prp2. Although I am just a newbie, but I'm quite interested in NVMe and want

Re: [Qemu-block] [PATCH 1/2] qcow2: add reduce image support

Am 02.06.2017 um 11:53 hat Pavel Butsykin geschrieben: > On 01.06.2017 17:41, Kevin Wolf wrote: > >Am 31.05.2017 um 16:43 hat Pavel Butsykin geschrieben: > >>This patch adds the reduction of the image file for qcow2. As a result, this > >>allows us to reduce the virtual image size and free up

Re: [Qemu-block] [PATCH v2 03/15] blkdebug: support BDRV_REQ_ALLOCATE

On 06/01/2017 10:50 PM, Eric Blake wrote: On 06/01/2017 10:14 AM, Anton Nefedov wrote: Support the flag if the underlying BDS supports it Signed-off-by: Anton Nefedov --- block/blkdebug.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) Shouldn't

[Qemu-block] [PATCH v20 15/30] block/dirty-bitmap: add autoload field to BdrvDirtyBitmap

Mirror AUTO flag from Qcow2 bitmap in BdrvDirtyBitmap. This will be needed in future, to save this flag back to Qcow2 for persistent bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow

[Qemu-block] [PATCH v20 09/30] block/dirty-bitmap: fix comment for BlockDirtyBitmap.disabled field

Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 90af37287f..f25428868c 100644 --- a/block/dirty-bitmap.c +++

[Qemu-block] [PATCH v20 17/30] block: introduce persistent dirty bitmaps

New field BdrvDirtyBitmap.persistent means, that bitmap should be saved by format driver in .bdrv_close and .bdrv_inactivate. No format driver supports it for now. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz ---

[Qemu-block] [PATCH v20 18/30] block/dirty-bitmap: add bdrv_dirty_bitmap_next()

Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/dirty-bitmap.c | 7 +++ include/block/dirty-bitmap.h | 3 +++ 2 files changed, 10 insertions(+) diff --git

[Qemu-block] [PATCH v20 20/30] qcow2: store bitmaps on reopening image as read-only

Store bitmaps and mark them read-only on reopening image as read-only. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 22 ++ block/qcow2.c| 5 + block/qcow2.h| 1 + 3 files changed, 28 insertions(+)

[Qemu-block] [PATCH v20 10/30] block/dirty-bitmap: add readonly field to BdrvDirtyBitmap

It will be needed in following commits for persistent bitmaps. If bitmap is loaded from read-only storage (and we can't mark it "in use" in this storage) corresponding BdrvDirtyBitmap should be read-only. Signed-off-by: Vladimir Sementsov-Ogievskiy ---

[Qemu-block] [PATCH v20 28/30] qcow2: add .bdrv_remove_persistent_dirty_bitmap

Realize .bdrv_remove_persistent_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/qcow2-bitmap.c | 41 +

[Qemu-block] [PATCH v20 01/30] specs/qcow2: fix bitmap granularity qemu-specific note

Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- docs/specs/qcow2.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt index 80cdfd0e91..dda53dd2a3 100644 ---

[Qemu-block] [PATCH v20 05/30] block: fix bdrv_dirty_bitmap_granularity signature

Make getter signature const-correct. This allows other functions with const dirty bitmap parameter use bdrv_dirty_bitmap_granularity(). Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: John Snow

[Qemu-block] [PATCH v20 30/30] block: release persistent bitmaps on inactivate

We should release them here to reload on invalidate cache. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 4 block/dirty-bitmap.c | 29 +++-- include/block/dirty-bitmap.h | 1 + 3 files changed, 28

[Qemu-block] [PATCH v20 26/30] iotests: test qcow2 persistent dirty bitmap

Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/165 | 105 + tests/qemu-iotests/165.out | 5 +++ tests/qemu-iotests/group | 1 + 3 files changed, 111

[Qemu-block] [PATCH v20 11/30] qcow2: autoloading dirty bitmaps

Auto loading bitmaps are bitmaps in Qcow2, with the AUTO flag set. They are loaded when the image is opened and become BdrvDirtyBitmaps for the corresponding drive. Extra data in bitmaps is not supported for now. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by:

[Qemu-block] [PATCH v20 19/30] qcow2: add persistent dirty bitmaps support

Store persistent dirty bitmaps in qcow2 image. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-bitmap.c | 475 +++ block/qcow2.c| 9 + block/qcow2.h

[Qemu-block] [PATCH v20 08/30] qcow2: add bitmaps extension

Add bitmap extension as specified in docs/specs/qcow2.txt. For now, just mirror extension header into Qcow2 state and check constraints. Also, calculate refcounts for qcow2 bitmaps, to not break qemu-img check. For now, disable image resize if it has bitmaps. It will be fixed later.

[Qemu-block] [PATCH v20 27/30] block/dirty-bitmap: add bdrv_remove_persistent_dirty_bitmap

Interface for removing persistent bitmap from its storage. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/dirty-bitmap.c | 18 ++

[Qemu-block] [PATCH v20 24/30] qmp: add autoload parameter to block-dirty-bitmap-add

Optional. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz Reviewed-by: John Snow --- blockdev.c | 18 --

[Qemu-block] [PATCH v20 29/30] qmp: block-dirty-bitmap-remove: remove persistent

Remove persistent bitmap from the storage on block-dirty-bitmap-remove. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- blockdev.c | 10 ++ qapi/block-core.json | 3

[Qemu-block] [PATCH v20 23/30] qmp: add persistent flag to block-dirty-bitmap-add

Add optional 'persistent' flag to qmp command block-dirty-bitmap-add. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz Reviewed-by: John Snow ---

[Qemu-block] [PATCH v20 13/30] block: new bdrv_reopen_bitmaps_rw interface

Add format driver handler, which should mark loaded read-only bitmaps as 'IN_USE' in the image and unset read_only field in corresponding BdrvDirtyBitmap's. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 17 +

[Qemu-block] [PATCH v20 21/30] block: add bdrv_can_store_new_dirty_bitmap

This will be needed to check some restrictions before making bitmap persistent in qmp-block-dirty-bitmap-add (this functionality will be added by future patch) Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow

[Qemu-block] [PATCH v20 16/30] block: bdrv_close: release bitmaps after drv->bdrv_close

Release bitmaps after 'if (bs->drv) { ... }' block. This will allow format driver to save persistent bitmaps, which will appear in following commits. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block.c | 6 +++--- 1 file

[Qemu-block] [PATCH v20 00/30] qcow2: persistent dirty bitmaps

Hi all! There is a new update of qcow2-bitmap series - v20. web: https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=qcow2-bitmap-v20 git: https://src.openvz.org/scm/~vsementsov/qemu.git (tag qcow2-bitmap-v20) v20: handle reopening images ro and rw. On reopening ro: store bitmaps

[Qemu-block] [PATCH v20 25/30] qmp: add x-debug-block-dirty-bitmap-sha256

Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/dirty-bitmap.c | 5 + blockdev.c | 29 + include/block/dirty-bitmap.h |

[Qemu-block] [PATCH v20 02/30] specs/qcow2: do not use wording 'bitmap header'

A bitmap directory entry is sometimes called a 'bitmap header'. This patch leaves only one name - 'bitmap directory entry'. The name 'bitmap header' creates misunderstandings with 'qcow2 header' and 'qcow2 bitmap header extension' (which is extension of qcow2 header) Signed-off-by: Vladimir

[Qemu-block] [PATCH v20 22/30] qcow2: add .bdrv_can_store_new_dirty_bitmap

Realize .bdrv_can_store_new_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Max Reitz --- block/qcow2-bitmap.c | 51 +++

[Qemu-block] [PATCH v20 07/30] qcow2-refcount: rename inc_refcounts() and make it public

This is needed for the following patch, which will introduce refcounts checking for qcow2 bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/qcow2-refcount.c | 53

[Qemu-block] [PATCH v20 03/30] hbitmap: improve dirty iter

Make dirty iter resistant to resetting bits in corresponding HBitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- include/qemu/hbitmap.h | 26 --

[Qemu-block] [PATCH v20 04/30] tests: add hbitmap iter test

Test that hbitmap iter is resistant to bitmap resetting. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz Reviewed-by: John Snow --- tests/test-hbitmap.c | 19

[Qemu-block] [PATCH v20 12/30] block: refactor bdrv_reopen_commit

Add bs local variable to simplify code. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/block.c b/block.c index 50ba264143..04af7697dc 100644 --- a/block.c +++ b/block.c @@

[Qemu-block] [PATCH v20 06/30] block/dirty-bitmap: add deserialize_ones func

Add bdrv_dirty_bitmap_deserialize_ones() function, which is needed for qcow2 bitmap loading, to handle unallocated bitmap parts, marked as all-ones. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Kevin Wolf Reviewed-by: John Snow

[Qemu-block] [PATCH v20 14/30] qcow2: support .bdrv_reopen_bitmaps_rw

Realize bdrv_reopen_bitmaps_rw interface. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 61 block/qcow2.c| 2 ++ block/qcow2.h| 1 + 3 files changed, 64 insertions(+) diff

Re: [Qemu-block] [PATCH 1/2] qcow2: add reduce image support

On 01.06.2017 17:41, Kevin Wolf wrote: Am 31.05.2017 um 16:43 hat Pavel Butsykin geschrieben: This patch adds the reduction of the image file for qcow2. As a result, this allows us to reduce the virtual image size and free up space on the disk without copying the image. Image can be

Re: [Qemu-block] [Qemu-devel] [PATCH 09/25] block/dirty-bitmap: add readonly field to BdrvDirtyBitmap

02.06.2017 12:01, Vladimir Sementsov-Ogievskiy wrote: 02.06.2017 11:56, Vladimir Sementsov-Ogievskiy wrote: 02.06.2017 02:25, John Snow wrote: On 06/01/2017 03:30 AM, Sementsov-Ogievskiy Vladimir wrote: Hi John! Look at our discussion about this in v18 thread. Shortly: readonly is not the

Re: [Qemu-block] [Qemu-devel] [PATCH 09/25] block/dirty-bitmap: add readonly field to BdrvDirtyBitmap

02.06.2017 11:56, Vladimir Sementsov-Ogievskiy wrote: 02.06.2017 02:25, John Snow wrote: On 06/01/2017 03:30 AM, Sementsov-Ogievskiy Vladimir wrote: Hi John! Look at our discussion about this in v18 thread. Shortly: readonly is not the same as disabled. disabled= bitmap just ignores all

Re: [Qemu-block] [Qemu-devel] [PATCH 09/25] block/dirty-bitmap: add readonly field to BdrvDirtyBitmap

02.06.2017 02:25, John Snow wrote: On 06/01/2017 03:30 AM, Sementsov-Ogievskiy Vladimir wrote: Hi John! Look at our discussion about this in v18 thread. Shortly: readonly is not the same as disabled. disabled= bitmap just ignores all writes. readonly= writes are not allowed at all. And I

Re: [Qemu-block] [PATCH v8 19/20] qcow2: report encryption specific image information

On Thu 01 Jun 2017 07:27:33 PM CEST, Daniel P. Berrange wrote: > Currently 'qemu-img info' reports a simple "encrypted: yes" > field. This is not very useful now that qcow2 can support > multiple encryption formats. Users want to know which format > is in use and some data related to it.

Re: [Qemu-block] [PATCH 00/29] qed: Convert to coroutines

On 01/06/2017 19:08, Kevin Wolf wrote: > Am 01.06.2017 um 18:40 hat Paolo Bonzini geschrieben: >> On 01/06/2017 18:28, Kevin Wolf wrote: - qed_acquire/qed_release can be removed as you inline stuff, but this should not cause bugs so you can either do it as a final patch or let me

Re: [Qemu-block] virtio-scsi and request splitting

Am 01.06.2017 um 18:54 schrieb Paolo Bonzini: > > On 01/06/2017 16:25, Peter Lieven wrote: >>> DEFINE_PROP_UINT32("max_sectors", VirtIOSCSI, >>> parent_obj.conf.max_sectors, >>> 0x), >>> >>> We could increase it now that we have support for