[Qemu-block] [PATCH v22 18/30] block/dirty-bitmap: add bdrv_dirty_bitmap_next()

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- block/dirty-bitmap.c | 7 +++ include/block/dirty-bitmap.h | 2 ++ 2 files changed, 9 insertions(+) diff -

[Qemu-block] [PATCH v22 19/30] qcow2: add persistent dirty bitmaps support

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Store persistent dirty bitmaps in qcow2 image. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> --- block/qcow2-bitmap.c | 475 +++ block/qcow2.c| 9 + b

[Qemu-block] [PATCH v22 27/30] block/dirty-bitmap: add bdrv_remove_persistent_dirty_bitmap

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Interface for removing persistent bitmap from its storage. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- block/dirty-bitmap.c | 18 +++

[Qemu-block] [PATCH v22 11/30] qcow2: autoloading dirty bitmaps

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Auto loading bitmaps are bitmaps in Qcow2, with the AUTO flag set. They are loaded when the image is opened and become BdrvDirtyBitmaps for the corresponding drive. Extra data in bitmaps is not supported for now. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Re

[Qemu-block] [PATCH v22 06/30] block/dirty-bitmap: add deserialize_ones func

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Add bdrv_dirty_bitmap_deserialize_ones() function, which is needed for qcow2 bitmap loading, to handle unallocated bitmap parts, marked as all-ones. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Kevin Wolf <kw...@redhat.com> Reviewed-by: J

[Qemu-block] [PATCH v22 12/30] block: refactor bdrv_reopen_commit

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Add bs local variable to simplify code. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: John Snow <js...@redhat.com> --- block.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/block.c b/block.c index 694396281b

[Qemu-block] [PATCH v22 28/30] qcow2: add .bdrv_remove_persistent_dirty_bitmap

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Realize .bdrv_remove_persistent_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- block/qcow2-bitmap.c | 41

[Qemu-block] [PATCH v22 29/30] qmp: block-dirty-bitmap-remove: remove persistent

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Remove persistent bitmap from the storage on block-dirty-bitmap-remove. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- blockdev.c | 10 ++ qapi/bl

[Qemu-block] [PATCH v22 24/30] qmp: add autoload parameter to block-dirty-bitmap-add

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Optional. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Signed-off-by: Denis V. Lunev <d...@openvz.org> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- blockdev.c | 18 ++

[Qemu-block] [PATCH v22 02/30] specs/qcow2: do not use wording 'bitmap header'

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Eric Blake <ebl...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- docs/interop/qcow2.txt | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.tx

[Qemu-block] [PATCH v22 21/30] block: add bdrv_can_store_new_dirty_bitmap

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
This will be needed to check some restrictions before making bitmap persistent in qmp-block-dirty-bitmap-add (this functionality will be added by future patch) Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-b

[Qemu-block] [PATCH v22 08/30] qcow2: add bitmaps extension

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- block/Makefile.objs| 2 +- block/qcow2-bitmap.c | 439 + block/qcow2-refcoun

[Qemu-block] [PATCH v22 17/30] block: introduce persistent dirty bitmaps

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
New field BdrvDirtyBitmap.persistent means, that bitmap should be saved by format driver in .bdrv_close and .bdrv_inactivate. No format driver supports it for now. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/dirty-bitmap.c

[Qemu-block] [PATCH v22 09/30] block/dirty-bitmap: fix comment for BlockDirtyBitmap.disabled field

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: John Snow <js...@redhat.com> Reviewed-by: Max Reitz <mre...@redhat.com> --- block/dirty-bitmap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/dirty-bitmap.c b/bl

[Qemu-block] [PATCH v22 20/30] qcow2: store bitmaps on reopening image as read-only

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Store bitmaps and mark them read-only on reopening image as read-only. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> --- block/qcow2-bitmap.c | 22 ++ block/qcow2.c| 5 + b

[Qemu-block] [PATCH v22 15/30] block/dirty-bitmap: add autoload field to BdrvDirtyBitmap

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Mirror AUTO flag from Qcow2 bitmap in BdrvDirtyBitmap. This will be needed in future, to save this flag back to Qcow2 for persistent bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/dirty-bitmap.c | 18 ++ block/qcow2-bi

[Qemu-block] [PATCH v22 22/30] qcow2: add .bdrv_can_store_new_dirty_bitmap

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Realize .bdrv_can_store_new_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: John Snow <js...@redhat.com> Reviewed-by: Max Reitz <mre...@redhat.com> --- block/qcow2-bitmap.c | 51

[Qemu-block] [PATCH v22 13/30] block: new bdrv_reopen_bitmaps_rw interface

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Add format driver handler, which should mark loaded read-only bitmaps as 'IN_USE' in the image and unset read_only field in corresponding BdrvDirtyBitmap's. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: John Snow <js...@redhat.com>

[Qemu-block] [PATCH v22 14/30] qcow2: support .bdrv_reopen_bitmaps_rw

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Realize bdrv_reopen_bitmaps_rw interface. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: John Snow <js...@redhat.com> Reviewed-by: Max Reitz <mre...@redhat.com> --- block/qcow2-bitmap.c | 61 +

[Qemu-block] [PATCH v22 00/30] qcow2: persistent dirty bitmaps

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
tps://src.openvz.org/users/vsementsov/repos/qemu/browse?at=refs%2Ftags%2Fqcow2-bitmap-v6 based on block-next (https://github.com/XanClic/qemu/commits/block-next) There are a lot of changes, reorderings and additions in comparement with v5. One principal thing: now bitmaps are removed from image

[Qemu-block] [PATCH v22 10/30] block/dirty-bitmap: add readonly field to BdrvDirtyBitmap

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
It will be needed in following commits for persistent bitmaps. If bitmap is loaded from read-only storage (and we can't mark it "in use" in this storage) corresponding BdrvDirtyBitmap should be read-only. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com>

[Qemu-block] [PATCH v22 23/30] qmp: add persistent flag to block-dirty-bitmap-add

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Add optional 'persistent' flag to qmp command block-dirty-bitmap-add. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Signed-off-by: Denis V. Lunev <d...@openvz.org> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow

Re: [Qemu-block] [PATCH v22 13/30] block: new bdrv_reopen_bitmaps_rw interface

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
It is interesting, but I see this problem only in your answers, in my letters I see this white-space on its place. 28.06.2017 15:36, Eric Blake wrote: [meta-comment] On 06/28/2017 07:10 AM, Vladimir Sementsov-Ogievskiy wrote: 28.06.2017 15:05, Vladimir Sementsov-Ogievskiy wrote: Add format

Re: [Qemu-block] [PATCH v22 13/30] block: new bdrv_reopen_bitmaps_rw interface

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
28.06.2017 16:02, Vladimir Sementsov-Ogievskiy wrote: It is interesting, but I see this problem only in your answers, in my letters I see this white-space on its place. In outgoing letter I see this white-space, but in letter from mailing-list it is absent. 28.06.2017 15:36, Eric Blake

Re: [Qemu-block] [PATCH v22 13/30] block: new bdrv_reopen_bitmaps_rw interface

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Finally, it looks like thunderbird bug https://bugzilla.mozilla.org/show_bug.cgi?id=1160880 [sorry for so much offtopic] 28.06.2017 16:13, Vladimir Sementsov-Ogievskiy wrote: 28.06.2017 16:02, Vladimir Sementsov-Ogievskiy wrote: It is interesting, but I see this problem only in your answers

[Qemu-block] [PATCH v22 30/30] block: release persistent bitmaps on inactivate

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
We should release them here to reload on invalidate cache. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> --- block.c | 4 block/dirty-bitmap.c | 29 +++-- i

[Qemu-block] [PATCH v22 04/30] tests: add hbitmap iter test

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Test that hbitmap iter is resistant to bitmap resetting. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Signed-off-by: Denis V. Lunev <d...@openvz.org> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- t

[Qemu-block] [PATCH v22 16/30] block: bdrv_close: release bitmaps after drv->bdrv_close

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Release bitmaps after 'if (bs->drv) { ... }' block. This will allow format driver to save persistent bitmaps, which will appear in following commits. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> --- block.c | 6

[Qemu-block] [PATCH v22 03/30] hbitmap: improve dirty iter

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Make dirty iter resistant to resetting bits in corresponding HBitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- include/qemu/hbitmap.h | 26 --

[Qemu-block] [PATCH v22 25/30] qmp: add x-debug-block-dirty-bitmap-sha256

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/dirty-bitmap.c | 5 + blockdev.c | 25 + include/block/dirty-bitmap.h | 1 + include/qemu/hbitmap.h | 8 qapi/block-core.json

[Qemu-block] [PATCH v22 07/30] qcow2-refcount: rename inc_refcounts() and make it public

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
This is needed for the following patch, which will introduce refcounts checking for qcow2 bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- block/qco

[Qemu-block] [PATCH v22 01/30] specs/qcow2: fix bitmap granularity qemu-specific note

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: John Snow <js...@redhat.com> --- docs/interop/qcow2.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index 80cdfd0e91..dda53

Re: [Qemu-block] [PATCH v22 13/30] block: new bdrv_reopen_bitmaps_rw interface

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
28.06.2017 15:05, Vladimir Sementsov-Ogievskiy wrote: Add format driver handler, which should mark loaded read-only bitmaps as 'IN_USE' in the image and unset read_only field in corresponding BdrvDirtyBitmap's. Signed-off-by: Vladimir Sementsov-Ogievskiy<vsement...@virtuozzo.com> Re

Re: [Qemu-block] [PATCH v3 1/3] block: add bdrv_get_format_alloc_stat format interface

2017-06-29 Thread Vladimir Sementsov-Ogievskiy
29.06.2017 03:15, John Snow wrote: On 06/28/2017 11:59 AM, Vladimir Sementsov-Ogievskiy wrote: 27.06.2017 02:19, John Snow wrote: On 06/06/2017 12:26 PM, Vladimir Sementsov-Ogievskiy wrote: The function should collect statistics, about used/unused by top-level format driver space (in its

Re: [Qemu-block] [PATCH v22 00/30] qcow2: persistent dirty bitmaps

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
28.06.2017 16:01, Paolo Bonzini wrote: On 28/06/2017 14:05, Vladimir Sementsov-Ogievskiy wrote: Rebase on master, so changes, mostly related to new dirty bitmaps mutex: 10: - asserts now in bdrv_{re,}set_dirty_bitmap_locked functions. - also add assert into bdrv_undo_clear_dirty_bitmap

Re: [Qemu-block] [PATCH v22 13/30] block: new bdrv_reopen_bitmaps_rw interface

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
28.06.2017 16:31, Paolo Bonzini wrote: On 28/06/2017 15:02, Vladimir Sementsov-Ogievskiy wrote: It is interesting, but I see this problem only in your answers, in my letters I see this white-space on its place. That's the good old Thunderbird "format=flowed" bug. Vladimir, dow

Re: [Qemu-block] [PATCH v3 1/3] block: add bdrv_get_format_alloc_stat format interface

2017-06-28 Thread Vladimir Sementsov-Ogievskiy
27.06.2017 02:19, John Snow wrote: On 06/06/2017 12:26 PM, Vladimir Sementsov-Ogievskiy wrote: The function should collect statistics, about used/unused by top-level format driver space (in its .file) and allocation status (data/zero/discarded/after-eof) of corresponding areas in this .file

Re: [Qemu-block] [Qemu-devel] [PATCH v22 23/30] qmp: add persistent flag to block-dirty-bitmap-add

2017-07-07 Thread Vladimir Sementsov-Ogievskiy
07.07.2017 10:54, Markus Armbruster wrote: QAPI schema review only... I apologize for its lateness. Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> writes: Add optional 'persistent' flag to qmp command block-dirty-bitmap-add. Default is false. Signed-off-by: Vladimir Sem

[Qemu-block] [PATCH 15/24] block: add bdrv_can_store_new_dirty_bitmap

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
This will be needed to check some restrictions before making bitmap persistent in qmp-block-dirty-bitmap-add (this functionality will be added by future patch) Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-b

[Qemu-block] [PATCH v17 00/24] qcow2: persistent dirty bitmaps

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
-next) There are a lot of changes, reorderings and additions in comparement with v5. One principal thing: now bitmaps are removed from image after loading instead of marking them in_use. It is simpler and we do not need to store superfluous data. Also, we are no more interested in command line i

[Qemu-block] [PATCH 07/24] qcow2-refcount: rename inc_refcounts() and make it public

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
This is needed for the following patch, which will introduce refcounts checking for qcow2 bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- block/qco

[Qemu-block] [PATCH 22/24] qcow2: add .bdrv_remove_persistent_dirty_bitmap

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Realize .bdrv_remove_persistent_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- block/qcow2-bitmap.c | 41

[Qemu-block] [PATCH 05/24] block: fix bdrv_dirty_bitmap_granularity signature

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
d-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/dirty-bitmap.c | 2 +- include/block/dirty-bitmap.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 519737c8d3..186941cfc3 100644 --- a/block/di

[Qemu-block] [PATCH 23/24] qmp: block-dirty-bitmap-remove: remove persistent

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Remove persistent bitmap from the storage on block-dirty-bitmap-remove. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- blockdev.c | 10 ++ 1 file chan

[Qemu-block] [PATCH 16/24] qcow2: add .bdrv_can_store_new_dirty_bitmap

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Realize .bdrv_can_store_new_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: John Snow <js...@redhat.com> Reviewed-by: Max Reitz <mre...@redhat.com> --- block/qcow2-bitmap.c | 51

[Qemu-block] [PATCH 09/24] qcow2: autoloading dirty bitmaps

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Auto loading bitmaps are bitmaps in Qcow2, with the AUTO flag set. They are loaded when the image is opened and become BdrvDirtyBitmaps for the corresponding drive. Extra data in bitmaps is not supported for now. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> ---

[Qemu-block] [PATCH 19/24] qmp: add x-debug-block-dirty-bitmap-sha256

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- block/dirty-bitmap.c | 5 + blockdev.c | 29 + include/bl

[Qemu-block] [PATCH 01/24] specs/qcow2: fix bitmap granularity qemu-specific note

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: John Snow <js...@redhat.com> --- docs/specs/qcow2.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt index 80cdfd0e91..dda53

[Qemu-block] [PATCH 13/24] block/dirty-bitmap: add bdrv_dirty_bitmap_next()

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- block/dirty-bitmap.c | 7 +++ include/block/dirty-bitmap.h | 3 +++ 2 files changed, 10 insertions(+)

[Qemu-block] [PATCH 14/24] qcow2: add persistent dirty bitmaps support

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Store persistent dirty bitmaps in qcow2 image. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/qcow2-bitmap.c | 473 +++ block/qcow2.c| 9 + block/qcow2.h| 1 + 3 files changed, 483 inse

[Qemu-block] [PATCH 17/24] qmp: add persistent flag to block-dirty-bitmap-add

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Add optional 'persistent' flag to qmp command block-dirty-bitmap-add. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Signed-off-by: Denis V. Lunev <d...@openvz.org> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow

[Qemu-block] [PATCH 08/24] qcow2: add bitmaps extension

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- block/Makefile.objs| 2 +- block/qcow2-bitmap.c | 439 + block/qcow2-refcoun

[Qemu-block] [PATCH 11/24] block: bdrv_close: release bitmaps after drv->bdrv_close

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Release bitmaps after 'if (bs->drv) { ... }' block. This will allow format driver to save persistent bitmaps, which will appear in following commits. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block.c | 6 +++--- 1 file changed, 3 insertions(+), 3

[Qemu-block] [PATCH 18/24] qmp: add autoload parameter to block-dirty-bitmap-add

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Optional. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Signed-off-by: Denis V. Lunev <d...@openvz.org> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- blockdev.c | 18 ++

[Qemu-block] [PATCH 20/24] iotests: test qcow2 persistent dirty bitmap

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- tests/qemu-iotests/165 | 89 ++ tests/qemu-iotests/165.out | 5 +++

[Qemu-block] [PATCH 06/24] block/dirty-bitmap: add deserialize_ones func

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Add bdrv_dirty_bitmap_deserialize_ones() function, which is needed for qcow2 bitmap loading, to handle unallocated bitmap parts, marked as all-ones. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Kevin Wolf <kw...@redhat.com> Reviewed-by: J

[Qemu-block] [PATCH 24/24] block: release persistent bitmaps on inactivate

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
We should release them here to reload on invalidate cache. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- Alternative is to release bitmaps from format driver, immediately after storing them - I'm not sure what is better. If this is OK, I can merge this to some

[Qemu-block] [PATCH 04/24] tests: add hbitmap iter test

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Test that hbitmap iter is resistant to bitmap resetting. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Signed-off-by: Denis V. Lunev <d...@openvz.org> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- t

[Qemu-block] [PATCH 03/24] hbitmap: improve dirty iter

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Make dirty iter resistant to resetting bits in corresponding HBitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- include/qemu/hbitmap.h | 26 --

[Qemu-block] [PATCH 21/24] block/dirty-bitmap: add bdrv_remove_persistent_dirty_bitmap

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Interface for removing persistent bitmap from its storage. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- block/dirty-bitmap.c | 18 +++

[Qemu-block] [PATCH 02/24] specs/qcow2: do not use wording 'bitmap header'

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Eric Blake <ebl...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> --- docs/specs/qcow2.txt | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt index dda5

[Qemu-block] [PATCH 10/24] block/dirty-bitmap: add autoload field to BdrvDirtyBitmap

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
Mirror AUTO flag from Qcow2 bitmap in BdrvDirtyBitmap. This will be needed in future, to save this flag back to Qcow2 for persistent bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Reviewed-by: Max Reitz <mre...@redhat.com> Reviewed-by: John Snow <j

Re: [Qemu-block] [PATCH 12/17] migration: add postcopy migration of dirty bitmaps

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
25.02.2017 20:56, Vladimir Sementsov-Ogievskiy wrote: 16.02.2017 16:04, Fam Zheng wrote: On Mon, 02/13 12:54, Vladimir Sementsov-Ogievskiy wrote: Postcopy migration of dirty bitmaps. Only named dirty bitmaps, associated with root nodes and non-root named nodes are migrated. If destination

Re: [Qemu-block] [Qemu-devel] [PATCH 15/17] iotests: add default node-name

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
11.04.2017 06:37, Vladimir Sementsov-Ogievskiy wrote: 11.04.2017 00:49, John Snow wrote: On 02/17/2017 02:51 PM, Dr. David Alan Gilbert wrote: * Fam Zheng (f...@redhat.com) wrote: On Fri, 02/17 16:36, Vladimir Sementsov-Ogievskiy wrote: 17.02.2017 15:21, Fam Zheng wrote: On Fri, 02/17 13

Re: [Qemu-block] [PATCH 2/4] qmp-cont: invalidate on RUN_STATE_PRELAUNCH

2017-04-26 Thread Vladimir Sementsov-Ogievskiy
26.04.2017 15:22, Kashyap Chamarthy wrote: On Tue, Mar 07, 2017 at 01:11:23PM +0300, Vladimir Sementsov-Ogievskiy wrote: 07.03.2017 13:02, Kevin Wolf wrote: Am 25.02.2017 um 20:31 hat Vladimir Sementsov-Ogievskiy geschrieben: We must invalidate on RUN_STATE_PRELAUNCH too, as it is available

Re: [Qemu-block] [Qemu-devel] [PATCH v17 00/24] qcow2: persistent dirty bitmaps

2017-04-28 Thread Vladimir Sementsov-Ogievskiy
27.04.2017 19:43, John Snow wrote: On 04/26/2017 07:30 AM, Vladimir Sementsov-Ogievskiy wrote: Hi all! There is a new update of qcow2-bitmap series - v17. web: https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=qcow2-bitmap-v17 git: https://src.openvz.org/scm/~vsementsov/qemu.git

[Qemu-block] ping Re: [PATCH 3/4] savevm: fix savevm after migration

2017-04-25 Thread Vladimir Sementsov-Ogievskiy
29.03.2017 18:53, Paolo Bonzini wrote: On 29/03/2017 17:29, Dr. David Alan Gilbert wrote: 'abort' is not very good too I think. migration is completed, nothing to abort.. (may be successful migration to file for suspend, some kind of vm cloning, etc) There is already migrate_cancel. Does it

Re: [Qemu-block] [PATCH] iotests: fix 185

2017-08-08 Thread Vladimir Sementsov-Ogievskiy
08.08.2017 11:53, Kevin Wolf wrote: Am 08.08.2017 um 10:42 hat Vladimir Sementsov-Ogievskiy geschrieben: 07.08.2017 18:57, Kevin Wolf wrote: Am 07.08.2017 um 16:16 hat Vladimir Sementsov-Ogievskiy geschrieben: 185 iotest is broken. How to test: i=0; while ./check -qcow2 -nocache 185; do ((i

Re: [Qemu-block] [PATCH] iotests: fix 185

2017-08-08 Thread Vladimir Sementsov-Ogievskiy
08.08.2017 12:04, Vladimir Sementsov-Ogievskiy wrote: 08.08.2017 11:53, Kevin Wolf wrote: Am 08.08.2017 um 10:42 hat Vladimir Sementsov-Ogievskiy geschrieben: 07.08.2017 18:57, Kevin Wolf wrote: Am 07.08.2017 um 16:16 hat Vladimir Sementsov-Ogievskiy geschrieben: 185 iotest is broken. How

Re: [Qemu-block] [PATCH] iotests: fix 185

2017-08-08 Thread Vladimir Sementsov-Ogievskiy
07.08.2017 18:57, Kevin Wolf wrote: Am 07.08.2017 um 16:16 hat Vladimir Sementsov-Ogievskiy geschrieben: 185 iotest is broken. How to test: i=0; while ./check -qcow2 -nocache 185; do ((i+=1)); echo N = $i; \ done; echo N = $i finished for me like this: 185 2s ... - output mismatch (see

Re: [Qemu-block] [PATCH 07/18] nbd: Minimal structured read for client

2017-08-01 Thread Vladimir Sementsov-Ogievskiy
01.08.2017 18:41, Vladimir Sementsov-Ogievskiy wrote: 07.02.2017 23:14, Eric Blake wrote: On 02/03/2017 09:47 AM, Vladimir Sementsov-Ogievskiy wrote: Minimal implementation: always send DF flag, to not deal with fragmented replies. This works well with your minimal server implementation

Re: [Qemu-block] [PATCH 07/18] nbd: Minimal structured read for client

2017-08-01 Thread Vladimir Sementsov-Ogievskiy
07.02.2017 23:14, Eric Blake wrote: On 02/03/2017 09:47 AM, Vladimir Sementsov-Ogievskiy wrote: Minimal implementation: always send DF flag, to not deal with fragmented replies. This works well with your minimal server implementation, but I worry that it will cause us to fall over when talking

Re: [Qemu-block] [PATCH] iotests: fix 185

2017-08-08 Thread Vladimir Sementsov-Ogievskiy
08.08.2017 12:04, Vladimir Sementsov-Ogievskiy wrote: 08.08.2017 12:04, Vladimir Sementsov-Ogievskiy wrote: 08.08.2017 11:53, Kevin Wolf wrote: Am 08.08.2017 um 10:42 hat Vladimir Sementsov-Ogievskiy geschrieben: 07.08.2017 18:57, Kevin Wolf wrote: Am 07.08.2017 um 16:16 hat Vladimir

Re: [Qemu-block] [PATCH v2 for 2.10] iotests: fix 185

2017-08-09 Thread Vladimir Sementsov-Ogievskiy
09.08.2017 18:41, Eric Blake wrote: On 08/09/2017 10:19 AM, Vladimir Sementsov-Ogievskiy wrote: 09.08.2017 18:17, Vladimir Sementsov-Ogievskiy wrote: 185 can sometimes produce wrong output like this: This is because quite happens before first mirror request is done s/quite/quit

[Qemu-block] [PATCH v2 for 2.10] block/nbd-client: always return EIO on and after the first io channel error

2017-08-08 Thread Vladimir Sementsov-Ogievskiy
Do not communicate after the first error to avoid communicating throught broken channel. The only exclusion is try to send NBD_CMD_DISC anyway on in nbd_client_close. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- Hi all. Here is a patch, fixing a problem

Re: [Qemu-block] [Qemu-devel] [PATCH] iotests: fix 185

2017-08-08 Thread Vladimir Sementsov-Ogievskiy
08.08.2017 18:07, Eric Blake wrote: On 08/08/2017 04:04 AM, Vladimir Sementsov-Ogievskiy wrote: Throttling "guaranties" that there will not be more than one request. But what prevent less than one, i.e. zero, like in my reproduction? Yes, I understand. Can we somehow make sure tha

[Qemu-block] [PATCH for 2.10] block/nbd-client: always return EIO on and after the first io channel error

2017-08-08 Thread Vladimir Sementsov-Ogievskiy
Do not communicate after the first error to avoid communicating throught broken channel. The only exclusion is try to send NBD_CMD_DISC anyway on in nbd_client_close. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- Hi all. Here is a patch, fixing a problem

Re: [Qemu-block] [PATCH for 2.10] block/nbd-client: always return EIO on and after the first io channel error

2017-08-08 Thread Vladimir Sementsov-Ogievskiy
08.08.2017 17:29, Vladimir Sementsov-Ogievskiy wrote: Do not communicate after the first error to avoid communicating throught broken channel. The only exclusion is try to send NBD_CMD_DISC anyway on in nbd_client_close. worth add: To simplify things, return -EIO in case of disconnect too

Re: [Qemu-block] [PATCH for 2.10] block/nbd-client: always return EIO on and after the first io channel error

2017-08-08 Thread Vladimir Sementsov-Ogievskiy
08.08.2017 17:44, Eric Blake wrote: On 08/08/2017 09:29 AM, Vladimir Sementsov-Ogievskiy wrote: Do not communicate after the first error to avoid communicating throught broken channel. The only exclusion is try to send NBD_CMD_DISC anyway on in nbd_client_close. Signed-off-by: Vladimir

Re: [Qemu-block] [PATCH v2 for 2.10] iotests: fix 185

2017-08-09 Thread Vladimir Sementsov-Ogievskiy
09.08.2017 18:17, Vladimir Sementsov-Ogievskiy wrote: 185 can sometimes produce wrong output like this: = 185 2s ... - output mismatch (see 185.out.bad) --- /work/src/qemu/master/tests/qemu-iotests/185.out2017-07-14 \ 15:14:29.520343805 +0300

[Qemu-block] [PATCH v2 for 2.10] iotests: fix 185

2017-08-09 Thread Vladimir Sementsov-Ogievskiy
65536, "type": "mirror"}} === Start backup job and exit qemu === Failures: 185 Failed 1 of 1 tests = This is because quite happens before first mirror request is done (and, in specified case, even before block-job len field is set

Re: [Qemu-block] [PATCH 02/17] nbd/client: refactor nbd_read_eof

2017-08-07 Thread Vladimir Sementsov-Ogievskiy
07.08.2017 14:42, Eric Blake wrote: On 08/04/2017 10:14 AM, Vladimir Sementsov-Ogievskiy wrote: Refactor nbd_read_eof to return 1 on success, 0 on eof, when no data was read and <0 for other cases, because returned size of read data is not actually used. Signed-off-by: Vladimir Sement

[Qemu-block] [PATCH] iotests: fix 185

2017-08-07 Thread Vladimir Sementsov-Ogievskiy
ot;speed": 65536, "type": \ "mirror"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, \ "event": "BLOCK_JOB_CANCELLED", "data": {"device": "disk", \

Re: [Qemu-block] [PATCH 11/17] block/nbd-client: fix nbd_co_request: set s->reply.handle to 0 on error

2017-08-07 Thread Vladimir Sementsov-Ogievskiy
07.08.2017 14:55, Eric Blake wrote: On 08/04/2017 10:14 AM, Vladimir Sementsov-Ogievskiy wrote: We set s->reply.handle to 0 on one error path and don't set on another. For consistancy and to avoid assert in nbd_read_reply_entry let's set s->reply.handle to 0 in case of wrong handle too.

Re: [Qemu-block] [PATCH 06/17] block/nbd-client: fix nbd_read_reply_entry

2017-08-07 Thread Vladimir Sementsov-Ogievskiy
07.08.2017 14:52, Eric Blake wrote: On 08/04/2017 10:14 AM, Vladimir Sementsov-Ogievskiy wrote: Set reply.handle to 0 on error path to prevent normal path of nbd_co_receive_reply. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/nbd-client.c | 1 +

[Qemu-block] [PATCH 11/17] block/nbd-client: fix nbd_co_request: set s->reply.handle to 0 on error

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
We set s->reply.handle to 0 on one error path and don't set on another. For consistancy and to avoid assert in nbd_read_reply_entry let's set s->reply.handle to 0 in case of wrong handle too. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/nbd-

[Qemu-block] [PATCH 02/17] nbd/client: refactor nbd_read_eof

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
Refactor nbd_read_eof to return 1 on success, 0 on eof, when no data was read and <0 for other cases, because returned size of read data is not actually used. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- nbd/nbd-internal.h

[Qemu-block] [PATCH 13/17] block/nbd-client: refactor NBDClientSession.recv_coroutine

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
Move from recv_coroutine[i] to requests[i].co. This is needed for further refactoring, new fields will be added to created structure. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/nbd-client.h | 4 +++- block/nbd-client.c | 20 ++-- 2

[Qemu-block] [PATCH 06/17] block/nbd-client: fix nbd_read_reply_entry

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
Set reply.handle to 0 on error path to prevent normal path of nbd_co_receive_reply. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/nbd-client.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/nbd-client.c b/block/nbd-client.c index dc19

[Qemu-block] [PATCH 00/17] nbd client refactoring and fixing

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
A bit more refactoring and fixing before BLOCK_STATUS series. I've tried to make individual patches simple enough, so there are a lot of them. Vladimir Sementsov-Ogievskiy (17): nbd/client: fix nbd_opt_go nbd/client: refactor nbd_read_eof nbd/client: refactor nbd_receive_reply nbd/client

[Qemu-block] [PATCH 08/17] block/nbd-client: rename nbd_recv_coroutines_enter_all

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
Rename nbd_recv_coroutines_enter_all to nbd_recv_coroutines_wake_all, as it most probably just add all recv coroutines into co_queue_wakeup, not directly enter them. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/nbd-client.c | 4 ++-- 1 file chan

[Qemu-block] [PATCH 15/17] block/nbd-client: refactor reading reply

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
Read the whole reply in one place - in nbd_read_reply_entry. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/nbd-client.h | 1 + block/nbd-client.c | 27 +-- 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/blo

[Qemu-block] [PATCH 04/17] nbd/client: fix nbd_send_request to return int

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
Fix nbd_send_request to return int, as it returns a return value of nbd_write (which is int), and the only user of nbd_send_request's return value (nbd_co_send_request) consider it as int too. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- include/block/nbd

[Qemu-block] [PATCH 07/17] block/nbd-client: refactor request send/receive

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
Move nbd_co_receive_reply and nbd_coroutine_end calls into nbd_co_send_request and rename the latter to just nbd_co_request. This removes code duplications in nbd_client_co_{pwrite,pread,...} functions. Also this is needed for further refactoring. Signed-off-by: Vladimir Sementsov-Ogievskiy

[Qemu-block] [PATCH 16/17] block/nbd-client: drop reply field from NBDClientSession

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
that nbd_read_reply_entry is out for some reason (error or disconnect) To simplify this scheme: - drop NBDClientSession.reply - introduce NBDClientSession.requests[...].ret for (1) - introduce NBDClientSession.eio_to_all for (2) Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- blo

[Qemu-block] [PATCH 05/17] block/nbd-client: get rid of ssize_t

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
Use int variable for nbd_co_send_request return value (as nbd_co_send_request returns int). Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/nbd-client.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/block/nbd-client.c b/blo

[Qemu-block] [PATCH 14/17] block/nbd-client: exit reply-reading coroutine on incorrect handle

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
ply-handle. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/nbd-client.h | 1 + block/nbd-client.c | 9 +++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/block/nbd-client.h b/block/nbd-client.h index 48e2559df6..aa36be8950 100644 --- a/

[Qemu-block] [PATCH 03/17] nbd/client: refactor nbd_receive_reply

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
Refactor nbd_receive_reply to return 1 on success, 0 on eof, when no data was read and <0 for other cases, because returned size of read data is not actually used. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- include/block/nbd.h | 2 +- nbd/client.c

[Qemu-block] [PATCH 12/17] block/nbd-client: refactor nbd_co_request

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
Reduce nesting, get rid of extra variable. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/nbd-client.c | 25 - 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/block/nbd-client.c b/block/nbd-client.c index b84c

[Qemu-block] [PATCH 01/17] nbd/client: fix nbd_opt_go

2017-08-04 Thread Vladimir Sementsov-Ogievskiy
Do not send NBD_OPT_ABORT to the broken server. After sending NBD_REP_ACK on NBD_OPT_GO server is most probably in transmission phase, when option sending is finished. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- nbd/client.c | 2 -- 1 file changed, 2 del

<    3   4   5   6   7   8   9   10   11   12   >