Re: [Qemu-block] [PATCH v2 6/6] qemu-img: copy *key-secret opts when opening newly created files

2017-02-03 Thread Max Reitz
On 03.02.2017 13:02, Daniel P. Berrange wrote: > The qemu-img dd/convert commands will create a image file and > then try to open it. Historically it has been possible to open > new files without passing any options. With encrypted files > though, the *key-secret options are mandatory, so we need

Re: [Qemu-block] [PATCH v2 5/6] qemu-img: introduce --target-image-opts for 'convert' command

2017-02-03 Thread Max Reitz
On 03.02.2017 13:02, Daniel P. Berrange wrote: > The '--image-opts' flags indicates whether the source filename > includes options. The target filename has to remain in the > plain filename format though, since it needs to be passed to > bdrv_create(). When using --skip-create though, it would be

Re: [Qemu-block] [PATCH v2 4/6] qemu-img: add support for -o arg to dd command

2017-02-03 Thread Max Reitz
On 03.02.2017 13:02, Daniel P. Berrange wrote: > The -o arg to the convert command allows specification of format/protocol > options for the newly created image. This adds a -o arg to the dd command > to get feature parity. > > Signed-off-by: Daniel P. Berrange > --- >

Re: [Qemu-block] [PATCH v2 3/6] qemu-img: add support for conv=nocreat, notrunc args to dd command

2017-02-03 Thread Max Reitz
On 03.02.2017 13:02, Daniel P. Berrange wrote: > The -n arg to the convert command allows use of a pre-existing image, > rather than creating a new image. This adds equivalent functionality > to the dd command using the 'conv' arg. If 'conv=nocreat' is used, > then it will assume the image already

Re: [Qemu-block] [PATCH v2 2/6] qemu-img: fix --image-opts usage with dd command

2017-02-03 Thread Max Reitz
On 03.02.2017 13:02, Daniel P. Berrange wrote: > The --image-opts flag can only be used to affect the parsing > of the source image. The target image has to be specified in > the traditional style regardless, since it needs to be passed > to the bdrv_create() API which does not support the new

Re: [Qemu-block] [PATCH v2 1/6] qemu-img: add support for --object with 'dd' command

2017-02-03 Thread Max Reitz
On 03.02.2017 13:02, Daniel P. Berrange wrote: > The qemu-img dd command added --image-opts support, but missed > the corresponding --object support. This prevented passing > secrets (eg auth passwords) needed by certain disk images. > > Reviewed-by: Eric Blake >

Re: [Qemu-block] Non-flat command line option argument syntax

2017-02-03 Thread Max Reitz
I like both JSON and dot syntax. But I like them differently in different places. I love JSON when it's in some file where I can turn out syntax highlighting and let my $EDITOR match brackets and braces. I hate JSON when it's on the command line. You have to escape, you get strings in strings,

Re: [Qemu-block] [Qemu-devel] Non-flat command line option argument syntax

2017-02-03 Thread Max Reitz
On 03.02.2017 08:50, Markus Armbruster wrote: > "Dr. David Alan Gilbert" writes: > >> * Markus Armbruster (arm...@redhat.com) wrote: >>> = Introduction = >>> >> >> >> >>> = Structured option argument syntax = >>> >>> == JSON == >>> >>> The obvious way to provide the

[Qemu-block] [PATCH] qemu-img: Do not truncate before preallocation

2017-02-03 Thread Nir Soffer
When using file system that does not support fallocate() (e.g. NFS < 4.2), truncating the file only when preallocation=OFF speeds up creating raw file. Here is example run, tested on Fedora 24 machine, creating raw file on NFS version 3 server. $ time ./qemu-img-master create -f raw -o

Re: [Qemu-block] [PATCH 16/17] iotests: add dirty bitmap migration test

2017-02-03 Thread Max Reitz
On 03.02.2017 12:08, Vladimir Sementsov-Ogievskiy wrote: > 03.02.2017 13:10, Vladimir Sementsov-Ogievskiy wrote: >> 02.02.2017 02:12, Max Reitz wrote: >>> On 22.11.2016 18:54, Vladimir Sementsov-Ogievskiy wrote: The test starts two vms (vm_a, vm_b), create dirty bitmap in the first one,

Re: [Qemu-block] [PATCH 16/17] iotests: add dirty bitmap migration test

2017-02-03 Thread Max Reitz
On 03.02.2017 11:10, Vladimir Sementsov-Ogievskiy wrote: > 02.02.2017 02:12, Max Reitz wrote: >> On 22.11.2016 18:54, Vladimir Sementsov-Ogievskiy wrote: >>> The test starts two vms (vm_a, vm_b), create dirty bitmap in >>> the first one, do several writes to corresponding device and >>> then

Re: [Qemu-block] [PATCH 1/2] qemu-io: don't allow I/O operations larger than INT_MAX

2017-02-03 Thread Max Reitz
On 02.02.2017 09:52, Alberto Garcia wrote: > On Wed 01 Feb 2017 11:16:38 PM CET, Max Reitz wrote: > >> Thanks, applied to my block tree, with >> %s/INT_MAX/BDRV_REQUEST_MAX_BYTES/g: > > I think you can use %d to print BDRV_REQUEST_MAX_BYTES, after all the > definition

Re: [Qemu-block] [PATCH 23/24] qcow2: add .bdrv_remove_persistent_dirty_bitmap

2017-02-03 Thread Max Reitz
On 02.02.2017 11:41, Vladimir Sementsov-Ogievskiy wrote: > 01.02.2017 02:20, Max Reitz wrote: >> On 23.01.2017 13:10, Vladimir Sementsov-Ogievskiy wrote: >>> Realize .bdrv_remove_persistent_dirty_bitmap interface. >>> >>> Signed-off-by: Vladimir Sementsov-Ogievskiy >>>

Re: [Qemu-block] [Qemu-devel] [PATCH v1 3/6] qemu-img: add support for -n arg to dd command

2017-02-03 Thread Max Reitz
On 02.02.2017 08:32, Markus Armbruster wrote: > Max Reitz writes: > >> On 01.02.2017 13:28, Daniel P. Berrange wrote: >>> On Wed, Feb 01, 2017 at 01:23:54PM +0100, Max Reitz wrote: On 01.02.2017 13:16, Daniel P. Berrange wrote: > On Wed, Feb 01, 2017 at 01:13:39PM

Re: [Qemu-block] [Qemu-devel] Non-flat command line option argument syntax

2017-02-03 Thread Richard W.M. Jones
On Thu, Feb 02, 2017 at 08:42:33PM +0100, Markus Armbruster wrote: > There's also the -drive file=json:... syntax. It's a bad fit for > QemuOpts, because QemuOpts and JSON fight for the comma. I'd show you > if I could get it to work. I think this refers to the json: syntax for qemu URIs?

Re: [Qemu-block] [Qemu-devel] Non-flat command line option argument syntax

2017-02-03 Thread Dr. David Alan Gilbert
* Markus Armbruster (arm...@redhat.com) wrote: > "Dr. David Alan Gilbert" writes: > > > * Markus Armbruster (arm...@redhat.com) wrote: > >> = Introduction = > >> > > > > > > > >> = Structured option argument syntax = > >> > >> == JSON == > >> > >> The obvious way to

[Qemu-block] [PATCH 17/18] nbd: BLOCK_STATUS for standard get_block_status function: server part

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Minimal realization: only one extent in server answer is supported. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/nbd.h | 3 ++ nbd/nbd-internal.h | 1 + nbd/server.c| 80 - 3 files changed,

[Qemu-block] [PATCH 16/18] iotests: add test for nbd dirty bitmap export

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/180 | 133 + tests/qemu-iotests/180.out | 5 ++ tests/qemu-iotests/group | 1 + 3 files changed, 139 insertions(+) create mode 100755

[Qemu-block] [PATCH 11/18] nbd: BLOCK_STATUS for bitmap export: server part

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Only one meta context type is defined: qemu-bitmap:. Maximum one query is allowed for NBD_OPT_{SET,LIST}_META_CONTEXT, NBD_REP_ERR_TOO_BIG is returned otherwise. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/nbd.h | 15 ++ nbd/nbd-internal.h | 6 +

[Qemu-block] [PATCH 06/18] nbd/client: refactor drop_sync

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Return 0 on success to simplify success checking. Signed-off-by: Vladimir Sementsov-Ogievskiy --- nbd/client.c | 35 +++ 1 file changed, 19 insertions(+), 16 deletions(-) diff --git a/nbd/client.c b/nbd/client.c index

[Qemu-block] [PATCH 12/18] nbd: BLOCK_STATUS for bitmap export: client part

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/nbd-client.c | 146 ++- block/nbd-client.h | 6 +++ block/nbd.c | 9 +++- include/block/nbd.h | 6 ++- nbd/client.c | 103

[Qemu-block] [PATCH 02/18] nbd-server: refactor simple reply sending

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Rename functions appropriately and also make a separate copy of NBDReply - NBDSimpleReply, to replace NBDReply for the server. NBDReply itself will be upgraded in future patches to handle both simple and structured replies in the client. Signed-off-by: Vladimir Sementsov-Ogievskiy

[Qemu-block] [PATCH 15/18] qmp: add block-dirty-bitmap-load

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
For loading dirty bitmap from nbd server. Or for underlying storages for other formats. Signed-off-by: Vladimir Sementsov-Ogievskiy --- blockdev.c | 28 qapi/block-core.json | 14 ++ 2 files changed, 42 insertions(+)

[Qemu-block] [PATCH 07/18] nbd: Minimal structured read for client

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Minimal implementation: always send DF flag, to not deal with fragmented replies. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/nbd-client.c | 47 +++ block/nbd-client.h | 2 + include/block/nbd.h | 15 +++-- nbd/client.c| 170

[Qemu-block] [PATCH 04/18] nbd/client: refactor nbd_receive_starttls

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Split out nbd_receive_simple_option to be reused for structured reply option. Signed-off-by: Vladimir Sementsov-Ogievskiy --- nbd/client.c | 54 +++--- nbd/nbd-internal.h | 14 ++ 2 files changed, 49

[Qemu-block] [PATCH 03/18] nbd: Minimal structured read for server

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Minimal implementation of structured read: one data chunk + finishing none chunk. No segmentation. Minimal structured error implementation: no text message. Support DF flag, but just ignore it, as there is no segmentation any way. Signed-off-by: Vladimir Sementsov-Ogievskiy

[Qemu-block] [PATCH 13/18] nbd: add nbd_dirty_bitmap_load

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Realize bdrv_dirty_bitmap_load interface. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/nbd.c | 32 1 file changed, 32 insertions(+) diff --git a/block/nbd.c b/block/nbd.c index 63bc3f04d0..b2b6fd1cf9 100644 --- a/block/nbd.c

[Qemu-block] [PATCH 08/18] hbitmap: add next_zero function

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
The function searches for next zero bit. Also add interface for BdrvDirtyBitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 5 + include/block/dirty-bitmap.h | 2 ++ include/qemu/hbitmap.h | 8 util/hbitmap.c

[Qemu-block] [PATCH 05/18] nbd/client: fix drop_sync

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Comparison symbol is misused. It may lead to memory corruption. Signed-off-by: Vladimir Sementsov-Ogievskiy --- nbd/client.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nbd/client.c b/nbd/client.c index 6caf6bda6d..351731bc63 100644 ---

[Qemu-block] [PATCH 18/18] nbd: BLOCK_STATUS for standard get_block_status function: client part

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Minimal realization: only first extent from the answer is used. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/nbd-client.c | 41 - block/nbd-client.h | 5 + block/nbd.c | 3 +++ include/block/nbd.h | 2 +-

[Qemu-block] [PATCH 09/18] block/dirty-bitmap: add bdrv_dirty_bitmap_next()

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 7 +++ include/block/dirty-bitmap.h | 3 +++ 2 files changed, 10 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index

[Qemu-block] [PATCH 10/18] block/dirty-bitmap: add bdrv_load_dirty_bitmap

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 53 include/block/block_int.h| 4 include/block/dirty-bitmap.h | 3 +++ 3 files changed, 60 insertions(+) diff --git

[Qemu-block] [PATCH 00/18] nbd: BLOCK_STATUS

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Hi all! We really need exporting dirty bitmaps feature as well as remote get_block_status for nbd devices. So, here is minimalistic and restricted realization of 'structured reply' and 'block status' nbd protocol extension (as second is developed over the first the combined spec may be found

[Qemu-block] [PATCH 01/18] nbd: rename NBD_REPLY_MAGIC to NBD_SIMPLE_REPLY_MAGIC

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
To be consistent when NBD_STRUCTURED_REPLY_MAGIC will be introduced. Signed-off-by: Vladimir Sementsov-Ogievskiy --- nbd/client.c | 4 ++-- nbd/nbd-internal.h | 2 +- nbd/server.c | 4 ++--

[Qemu-block] [PATCH 14/18] qmp: add x-debug-block-dirty-bitmap-sha256

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 5 + blockdev.c | 29 + include/block/dirty-bitmap.h | 2 ++ include/qemu/hbitmap.h | 8 qapi/block-core.json | 27

Re: [Qemu-block] [Qemu-devel] [PATCH v2 8/8] hw: Drop superfluous special checks for orphaned -drive

2017-02-03 Thread Markus Armbruster
Fam Zheng writes: > On Fri, 02/03 14:35, Markus Armbruster wrote: >> > I guess the painpoint is "okay, what the heck does the machine support >> > then?" - >> >> At the place where we can reliably detect orphaned drives regardless of >> what the machine initialization code

Re: [Qemu-block] [Qemu-devel] [PATCH v2 8/8] hw: Drop superfluous special checks for orphaned -drive

2017-02-03 Thread Fam Zheng
On Fri, 02/03 14:35, Markus Armbruster wrote: > > I guess the painpoint is "okay, what the heck does the machine support > > then?" - > > At the place where we can reliably detect orphaned drives regardless of > what the machine initialization code does, we have no idea. > > > that "3 > 2" is

Re: [Qemu-block] [Qemu-devel] Non-flat command line option argument syntax

2017-02-03 Thread Markus Armbruster
Peter Krempa writes: > On Thu, Feb 02, 2017 at 20:42:33 +0100, Markus Armbruster wrote: > > [...] > >> === Comparison === >> >> In my opinion, dotted keys are weird and ugly, but at least they don't >> add to the quoting mess. Structured values look better, except when >>

Re: [Qemu-block] [Qemu-devel] [PATCH v2 8/8] hw: Drop superfluous special checks for orphaned -drive

2017-02-03 Thread Markus Armbruster
Fam Zheng writes: > On Mon, 01/30 09:10, Markus Armbruster wrote: >> John Snow writes: >> >> > On 01/27/2017 11:04 AM, Markus Armbruster wrote: >> >> John Snow writes: >> >> >> >>> On 01/27/2017 06:51 AM, Markus Armbruster wrote: >>

Re: [Qemu-block] Non-flat command line option argument syntax

2017-02-03 Thread Peter Krempa
On Thu, Feb 02, 2017 at 20:42:33 +0100, Markus Armbruster wrote: [...] > === Comparison === > > In my opinion, dotted keys are weird and ugly, but at least they don't > add to the quoting mess. Structured values look better, except when > they do add to the quoting mess. From libvirt's point

[Qemu-block] [PATCH v2 5/6] qemu-img: introduce --target-image-opts for 'convert' command

2017-02-03 Thread Daniel P. Berrange
The '--image-opts' flags indicates whether the source filename includes options. The target filename has to remain in the plain filename format though, since it needs to be passed to bdrv_create(). When using --skip-create though, it would be possible to use image-opts syntax. This adds

[Qemu-block] [PATCH v2 1/6] qemu-img: add support for --object with 'dd' command

2017-02-03 Thread Daniel P. Berrange
The qemu-img dd command added --image-opts support, but missed the corresponding --object support. This prevented passing secrets (eg auth passwords) needed by certain disk images. Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrange --- qemu-img.c

[Qemu-block] [PATCH v2 4/6] qemu-img: add support for -o arg to dd command

2017-02-03 Thread Daniel P. Berrange
The -o arg to the convert command allows specification of format/protocol options for the newly created image. This adds a -o arg to the dd command to get feature parity. Signed-off-by: Daniel P. Berrange --- qemu-img-cmds.hx | 2 +- qemu-img.c | 32

[Qemu-block] [PATCH v2 6/6] qemu-img: copy *key-secret opts when opening newly created files

2017-02-03 Thread Daniel P. Berrange
The qemu-img dd/convert commands will create a image file and then try to open it. Historically it has been possible to open new files without passing any options. With encrypted files though, the *key-secret options are mandatory, so we need to provide those options when opening the newly created

[Qemu-block] [PATCH v2 3/6] qemu-img: add support for conv=nocreat, notrunc args to dd command

2017-02-03 Thread Daniel P. Berrange
The -n arg to the convert command allows use of a pre-existing image, rather than creating a new image. This adds equivalent functionality to the dd command using the 'conv' arg. If 'conv=nocreat' is used, then it will assume the image already exists. The existing image will be truncated to match

[Qemu-block] [PATCH v2 2/6] qemu-img: fix --image-opts usage with dd command

2017-02-03 Thread Daniel P. Berrange
The --image-opts flag can only be used to affect the parsing of the source image. The target image has to be specified in the traditional style regardless, since it needs to be passed to the bdrv_create() API which does not support the new style opts. Signed-off-by: Daniel P. Berrange

Re: [Qemu-block] [Qemu-devel] Non-flat command line option argument syntax

2017-02-03 Thread Markus Armbruster
"Daniel P. Berrange" writes: > On Thu, Feb 02, 2017 at 08:42:33PM +0100, Markus Armbruster wrote: >> === Dotted keys === >> >> One sufficiently powerful syntax extension already exists: the dotted >> key convention. It's syntactically unambiguous only when none of the >>

Re: [Qemu-block] [PATCH 03/17] migration: split common postcopy out of ram postcopy

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
01.02.2017 14:06, Vladimir Sementsov-Ogievskiy wrote: 24.01.2017 22:53, Dr. David Alan Gilbert wrote: * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: 24.01.2017 12:24, Juan Quintela wrote: Vladimir Sementsov-Ogievskiy wrote: Split common postcopy

Re: [Qemu-block] [PATCH 16/17] iotests: add dirty bitmap migration test

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
03.02.2017 13:10, Vladimir Sementsov-Ogievskiy wrote: 02.02.2017 02:12, Max Reitz wrote: On 22.11.2016 18:54, Vladimir Sementsov-Ogievskiy wrote: The test starts two vms (vm_a, vm_b), create dirty bitmap in the first one, do several writes to corresponding device and then migrate vm_a to vm_b

Re: [Qemu-block] [Qemu-devel] [PATCH v2 8/8] hw: Drop superfluous special checks for orphaned -drive

2017-02-03 Thread Fam Zheng
On Mon, 01/30 09:10, Markus Armbruster wrote: > John Snow writes: > > > On 01/27/2017 11:04 AM, Markus Armbruster wrote: > >> John Snow writes: > >> > >>> On 01/27/2017 06:51 AM, Markus Armbruster wrote: > John Snow writes: > >

[Qemu-block] [PATCH v13 00/24] qcow2: persistent dirty bitmaps

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Hi all! There is a new update of qcow2-bitmap series - v13. web: https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=qcow2-bitmap-v13 git: https://src.openvz.org/scm/~vsementsov/qemu.git (tag qcow2-bitmap-v13) v13: Just a fix for style checker. 13: line over 80 14: line over 80 22:

Re: [Qemu-block] [PATCH 16/17] iotests: add dirty bitmap migration test

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
02.02.2017 02:12, Max Reitz wrote: On 22.11.2016 18:54, Vladimir Sementsov-Ogievskiy wrote: The test starts two vms (vm_a, vm_b), create dirty bitmap in the first one, do several writes to corresponding device and then migrate vm_a to vm_b with dirty bitmaps. Signed-off-by: Vladimir

[Qemu-block] [PATCH 10/24] block/dirty-bitmap: add autoload field to BdrvDirtyBitmap

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Mirror AUTO flag from Qcow2 bitmap in BdrvDirtyBitmap. This will be needed in future, to save this flag back to Qcow2 for persistent bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 16

Re: [Qemu-block] Non-flat command line option argument syntax

2017-02-03 Thread Daniel P. Berrange
On Thu, Feb 02, 2017 at 08:42:33PM +0100, Markus Armbruster wrote: > === Dotted keys === > > One sufficiently powerful syntax extension already exists: the dotted > key convention. It's syntactically unambiguous only when none of the > KEYs involved contains '.' To adopt it across the board,

[Qemu-block] [PATCH 17/24] qmp: add autoload parameter to block-dirty-bitmap-add

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Optional. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz --- blockdev.c | 18 -- qapi/block-core.json | 6 +- 2 files changed, 21

[Qemu-block] [PATCH 07/24] qcow2: add bitmaps extension

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Add bitmap extension as specified in docs/specs/qcow2.txt. For now, just mirror extension header into Qcow2 state and check constraints. For now, disable image resize if it has bitmaps. It will be fixed later. Signed-off-by: Vladimir Sementsov-Ogievskiy ---

[Qemu-block] [PATCH 20/24] qcow2-refcount: rename inc_refcounts() and make it public

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
This is needed for the following patch, which will introduce refcounts checking for qcow2 bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-refcount.c | 53 ++

[Qemu-block] [PATCH 02/24] specs/qcow2: do not use wording 'bitmap header'

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
A bitmap directory entry is sometimes called a 'bitmap header'. This patch leaves only one name - 'bitmap directory entry'. The name 'bitmap header' creates misunderstandings with 'qcow2 header' and 'qcow2 bitmap header extension' (which is extension of qcow2 header) Signed-off-by: Vladimir

[Qemu-block] [PATCH 22/24] block/dirty-bitmap: add bdrv_remove_persistent_dirty_bitmap

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Interface for removing persistent bitmap from its storage. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 18 ++ include/block/block_int.h| 3 +++ include/block/dirty-bitmap.h | 3 +++ 3 files changed, 24

[Qemu-block] [PATCH 03/24] hbitmap: improve dirty iter

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Make dirty iter resistant to resetting bits in corresponding HBitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- include/qemu/hbitmap.h | 26 --

[Qemu-block] [PATCH 24/24] qmp: block-dirty-bitmap-remove: remove persistent

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Remove persistent bitmap from the storage on block-dirty-bitmap-remove. Signed-off-by: Vladimir Sementsov-Ogievskiy --- blockdev.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/blockdev.c b/blockdev.c index c41b791289..a365cdf3ed 100644 ---

[Qemu-block] [PATCH 13/24] qcow2: add .bdrv_store_persistent_dirty_bitmaps()

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Realize block bitmap storing interface, to allow qcow2 images store persistent bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-bitmap.c | 489 +--

[Qemu-block] [PATCH 16/24] qmp: add persistent flag to block-dirty-bitmap-add

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Add optional 'persistent' flag to qmp command block-dirty-bitmap-add. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz --- blockdev.c | 18

[Qemu-block] [PATCH 19/24] iotests: test qcow2 persistent dirty bitmap

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/165 | 89 ++ tests/qemu-iotests/165.out | 5 +++ tests/qemu-iotests/group | 1 + 3 files changed, 95

[Qemu-block] [PATCH 21/24] qcow2-bitmap: refcounts

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Calculate refcounts for qcow2 bitmaps. It is needed for qcow2's qemu-img check implementation. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-bitmap.c | 76 ++

[Qemu-block] [PATCH 12/24] block/dirty-bitmap: add bdrv_dirty_bitmap_next()

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 7 +++ include/block/dirty-bitmap.h | 3 +++ 2 files changed, 10 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index

[Qemu-block] [PATCH 23/24] qcow2: add .bdrv_remove_persistent_dirty_bitmap

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Realize .bdrv_remove_persistent_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 41 + block/qcow2.c| 1 + block/qcow2.h| 3 +++ 3 files changed, 45 insertions(+)

[Qemu-block] [PATCH 15/24] qcow2: add .bdrv_can_store_new_dirty_bitmap

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Realize .bdrv_can_store_new_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-bitmap.c | 40 block/qcow2.c| 1 + block/qcow2.h| 4

[Qemu-block] [PATCH 01/24] specs/qcow2: fix bitmap granularity qemu-specific note

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- docs/specs/qcow2.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt index 80cdfd0e91..dda53dd2a3 100644 ---

[Qemu-block] [PATCH 09/24] qcow2: add .bdrv_load_autoloading_dirty_bitmaps

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Auto loading bitmaps are bitmaps in Qcow2, with the AUTO flag set. They are loaded when the image is opened and become BdrvDirtyBitmaps for the corresponding drive. Extra data in bitmaps is not supported for now. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by:

[Qemu-block] [PATCH 06/24] block/dirty-bitmap: add deserialize_ones func

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Add bdrv_dirty_bitmap_deserialize_ones() function, which is needed for qcow2 bitmap loading, to handle unallocated bitmap parts, marked as all-ones. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Kevin Wolf Reviewed-by: John Snow

[Qemu-block] [PATCH 18/24] qmp: add x-debug-block-dirty-bitmap-sha256

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 5 + blockdev.c | 29 + include/block/dirty-bitmap.h | 2 ++ include/qemu/hbitmap.h | 8

[Qemu-block] [PATCH 14/24] block: add bdrv_can_store_new_dirty_bitmap

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
This will be needed to check some restrictions before making bitmap persistent in qmp-block-dirty-bitmap-add (this functionality will be added by future patch) Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block.c

[Qemu-block] [PATCH 04/24] tests: add hbitmap iter test

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Test that hbitmap iter is resistant to bitmap resetting. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz Reviewed-by: John Snow --- tests/test-hbitmap.c | 19

[Qemu-block] [PATCH 08/24] block: introduce auto-loading bitmaps

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Auto loading bitmaps are bitmaps stored in the disk image, which should be loaded when the image is opened and become BdrvDirtyBitmaps for the corresponding drive. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Max

[Qemu-block] [PATCH 05/24] block: fix bdrv_dirty_bitmap_granularity signature

2017-02-03 Thread Vladimir Sementsov-Ogievskiy
Make getter signature const-correct. This allows other functions with const dirty bitmap parameter use bdrv_dirty_bitmap_granularity(). Reviewed-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Signed-off-by: Vladimir