Re: [Qemu-block] [Qemu-devel] Non-flat command line option argument syntax

2017-02-02 Thread Markus Armbruster
Eric Blake writes: > On 02/02/2017 01:42 PM, Markus Armbruster wrote: > >> >> === Structured values === >> >> The dotted key convention messes with KEY syntax to permit structured >> values. Works, but the more conventional way to support structured >> values is a syntax for structured values.

Re: [Qemu-block] [Qemu-devel] Non-flat command line option argument syntax

2017-02-02 Thread Markus Armbruster
"Dr. David Alan Gilbert" writes: > * Markus Armbruster (arm...@redhat.com) wrote: >> = Introduction = >> > > > >> = Structured option argument syntax = >> >> == JSON == >> >> The obvious way to provide the expressiveness of JSON on the command >> line is JSON. Easy enough[2]. However, besid

Re: [Qemu-block] [Qemu-devel] Non-flat command line option argument syntax

2017-02-02 Thread Dr. David Alan Gilbert
* Markus Armbruster (arm...@redhat.com) wrote: > = Introduction = > > = Structured option argument syntax = > > == JSON == > > The obvious way to provide the expressiveness of JSON on the command > line is JSON. Easy enough[2]. However, besides not being compatible, > it's rather heavy on s

Re: [Qemu-block] Non-flat command line option argument syntax

2017-02-02 Thread Eric Blake
On 02/02/2017 01:42 PM, Markus Armbruster wrote: > > === Structured values === > > The dotted key convention messes with KEY syntax to permit structured > values. Works, but the more conventional way to support structured > values is a syntax for structured values. > > An obvious one is to u

Re: [Qemu-block] [Qemu-devel] [PATCH 02/10] qemu-thread: introduce QemuLockCnt

2017-02-02 Thread Emilio G. Cota
On Tue, Nov 29, 2016 at 12:46:59 +0100, Paolo Bonzini wrote: > A QemuLockCnt comprises a counter and a mutex, with primitives > to increment and decrement the counter, and to take and release the > mutex. It can be used to do lock-free visits to a data structure > whenever mutexes would be too hea

Re: [Qemu-block] [Qemu-devel] [PATCH 02/10] qemu-thread: introduce QemuLockCnt

2017-02-02 Thread Emilio G. Cota
Just noticed the message above mistakenly sat in my outbox for nearly 2 months. Just flushed it, so do not be surprised by its original date. Sorry for the noise, Emilio

Re: [Qemu-block] Non-flat command line option argument syntax

2017-02-02 Thread Eric Blake
On 02/02/2017 01:42 PM, Markus Armbruster wrote: quick comment before I reply to the overall message... > Arbitrary nesting with dotted key convention: > > -drive driver=qcow2,file.driver=gluster, >file.volume=testvol,file.path=/path/a.qcow2,file.debug=9, >file.serve

[Qemu-block] Non-flat command line option argument syntax

2017-02-02 Thread Markus Armbruster
= Introduction = If you're familiar with prior discussion of non-flat arguments for -object and -blockdev, you can probably skip ahead to "= Structured option argument syntax =". Structured option arguments use KEY=VALUE,... syntax. Goes back many years (at least to commit 7c9d8e0, Nov 2005). S

Re: [Qemu-block] [Qemu-devel] [RFC][PATCH] qemu-img: make convert async

2017-02-02 Thread no-reply
Hi, Your series failed automatic build test. Please find the testing commands and their output below. If you have docker installed, you can probably reproduce it locally. Type: series Subject: [Qemu-devel] [RFC][PATCH] qemu-img: make convert async Message-id: 1486051604-32310-1-git-send-email...@

Re: [Qemu-block] [Qemu-devel] [RFC][PATCH] qemu-img: make convert async

2017-02-02 Thread no-reply
Hi, Your series seems to have some coding style problems. See output below for more information: Type: series Subject: [Qemu-devel] [RFC][PATCH] qemu-img: make convert async Message-id: 1486051604-32310-1-git-send-email...@kamp.de === TEST SCRIPT BEGIN === #!/bin/bash BASE=base n=1 total=$(git

[Qemu-block] [RFC][PATCH] qemu-img: make convert async

2017-02-02 Thread Peter Lieven
this is something I have been thinking about for almost 2 years now. we heavily have the following two use cases when using qemu-img convert. a) reading from NFS and writing to iSCSI for deploying templates b) reading from iSCSI and writing to NFS for backups In both processes we use libiscsi and

Re: [Qemu-block] [PULL 0/5] Block patches

2017-02-02 Thread Peter Maydell
On 1 February 2017 at 05:34, Jeff Cody wrote: > The following changes since commit a0def594286d9110a6035e02eef558cf3cf5d847: > > Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into > staging (2017-01-30 10:23:20 +) > > are available in the git repository at: > > https://

Re: [Qemu-block] [Qemu-devel] [PATCH v12 00/24] qcow2: persistent dirty bitmaps

2017-02-02 Thread no-reply
Hi, Your series seems to have some coding style problems. See output below for more information: Type: series Subject: [Qemu-devel] [PATCH v12 00/24] qcow2: persistent dirty bitmaps Message-id: 20170202132145.168790-1-vsement...@virtuozzo.com === TEST SCRIPT BEGIN === #!/bin/bash BASE=base n=1

[Qemu-block] [PATCH 21/24] qcow2-bitmap: refcounts

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Calculate refcounts for qcow2 bitmaps. It is needed for qcow2's qemu-img check implementation. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-bitmap.c | 76 ++ block/qcow2-refcount.c | 6 block/qcow2.h

[Qemu-block] [PATCH 24/24] qmp: block-dirty-bitmap-remove: remove persistent

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Remove persistent bitmap from the storage on block-dirty-bitmap-remove. Signed-off-by: Vladimir Sementsov-Ogievskiy --- blockdev.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/blockdev.c b/blockdev.c index c41b791289..a365cdf3ed 100644 --- a/blockdev.c +++ b/blockdev.c @@ -276

[Qemu-block] [PATCH 18/24] qmp: add x-debug-block-dirty-bitmap-sha256

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 5 + blockdev.c | 29 + include/block/dirty-bitmap.h | 2 ++ include/qemu/hbitmap.h | 8 qapi/block-core.json | 27 +++

[Qemu-block] [PATCH 16/24] qmp: add persistent flag to block-dirty-bitmap-add

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Add optional 'persistent' flag to qmp command block-dirty-bitmap-add. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz --- blockdev.c | 18 +- qapi/block-core.json | 8 +++- 2 files changed, 24 ins

[Qemu-block] [PATCH 08/24] block: introduce auto-loading bitmaps

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Auto loading bitmaps are bitmaps stored in the disk image, which should be loaded when the image is opened and become BdrvDirtyBitmaps for the corresponding drive. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Max Reitz --- block.c | 14 +

[Qemu-block] [PATCH 01/24] specs/qcow2: fix bitmap granularity qemu-specific note

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- docs/specs/qcow2.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt index 80cdfd0e91..dda53dd2a3 100644 --- a/docs/specs/qcow2.txt +++ b/docs/specs/qcow2.txt

[Qemu-block] [PATCH 15/24] qcow2: add .bdrv_can_store_new_dirty_bitmap

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Realize .bdrv_can_store_new_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-bitmap.c | 40 block/qcow2.c| 1 + block/qcow2.h| 4 3 files changed, 45 insertions(+) diff --g

[Qemu-block] [PATCH 19/24] iotests: test qcow2 persistent dirty bitmap

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/165 | 89 ++ tests/qemu-iotests/165.out | 5 +++ tests/qemu-iotests/group | 1 + 3 files changed, 95 insertions(+) create mode 100755 tests/qemu-iotests/

[Qemu-block] [PATCH 05/24] block: fix bdrv_dirty_bitmap_granularity signature

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Make getter signature const-correct. This allows other functions with const dirty bitmap parameter use bdrv_dirty_bitmap_granularity(). Reviewed-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 2 +- i

[Qemu-block] [PATCH 11/24] block: introduce persistent dirty bitmaps

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
New field BdrvDirtyBitmap.persistent means, that bitmap should be saved on bdrv_close, using format driver. Format driver should maintain bitmap storing. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block.c | 32 bl

[Qemu-block] [PATCH 17/24] qmp: add autoload parameter to block-dirty-bitmap-add

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Optional. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz --- blockdev.c | 18 -- qapi/block-core.json | 6 +- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/blockdev.c b/blockdev

[Qemu-block] [PATCH 23/24] qcow2: add .bdrv_remove_persistent_dirty_bitmap

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Realize .bdrv_remove_persistent_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 41 + block/qcow2.c| 1 + block/qcow2.h| 3 +++ 3 files changed, 45 insertions(+) diff --git a/block/qcow2-bit

[Qemu-block] [PATCH 02/24] specs/qcow2: do not use wording 'bitmap header'

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
A bitmap directory entry is sometimes called a 'bitmap header'. This patch leaves only one name - 'bitmap directory entry'. The name 'bitmap header' creates misunderstandings with 'qcow2 header' and 'qcow2 bitmap header extension' (which is extension of qcow2 header) Signed-off-by: Vladimir Sement

[Qemu-block] [PATCH 03/24] hbitmap: improve dirty iter

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Make dirty iter resistant to resetting bits in corresponding HBitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- include/qemu/hbitmap.h | 26 -- util/hbitmap.c | 23 ++- 2 files changed, 26 i

[Qemu-block] [PATCH 09/24] qcow2: add .bdrv_load_autoloading_dirty_bitmaps

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Auto loading bitmaps are bitmaps in Qcow2, with the AUTO flag set. They are loaded when the image is opened and become BdrvDirtyBitmaps for the corresponding drive. Extra data in bitmaps is not supported for now. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/Make

[Qemu-block] [PATCH v12 00/24] qcow2: persistent dirty bitmaps

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Hi all! There is a new update of qcow2-bitmap series - v12. web: https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=qcow2-bitmap-v12 git: https://src.openvz.org/scm/~vsementsov/qemu.git (tag qcow2-bitmap-v12) v12: 07: do not update header in qcow2_read_extensions, instead do it in the

[Qemu-block] [PATCH 12/24] block/dirty-bitmap: add bdrv_dirty_bitmap_next()

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 7 +++ include/block/dirty-bitmap.h | 3 +++ 2 files changed, 10 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index 4d026df1bd..fe34d488aa 100644 --- a/block/dirty

[Qemu-block] [PATCH 04/24] tests: add hbitmap iter test

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Test that hbitmap iter is resistant to bitmap resetting. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz Reviewed-by: John Snow --- tests/test-hbitmap.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/tests/test-hbitmap

[Qemu-block] [PATCH 22/24] block/dirty-bitmap: add bdrv_remove_persistent_dirty_bitmap

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Interface for removing persistent bitmap from its storage. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 19 +++ include/block/block_int.h| 3 +++ include/block/dirty-bitmap.h | 3 +++ 3 files changed, 25 insertions(+) diff --git a/block/di

[Qemu-block] [PATCH 13/24] qcow2: add .bdrv_store_persistent_dirty_bitmaps()

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Realize block bitmap storing interface, to allow qcow2 images store persistent bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-bitmap.c | 488 +-- block/qcow2.c| 1 + block/qcow2.h| 1

[Qemu-block] [PATCH 20/24] qcow2-refcount: rename inc_refcounts() and make it public

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
This is needed for the following patch, which will introduce refcounts checking for qcow2 bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-refcount.c | 53 ++ block/qcow2.h | 4 2 files cha

[Qemu-block] [PATCH 07/24] qcow2: add bitmaps extension

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Add bitmap extension as specified in docs/specs/qcow2.txt. For now, just mirror extension header into Qcow2 state and check constraints. For now, disable image resize if it has bitmaps. It will be fixed later. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 127 +

[Qemu-block] [PATCH 10/24] block/dirty-bitmap: add autoload field to BdrvDirtyBitmap

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Mirror AUTO flag from Qcow2 bitmap in BdrvDirtyBitmap. This will be needed in future, to save this flag back to Qcow2 for persistent bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 16 block/qcow2-bitmap.c |

[Qemu-block] [PATCH 06/24] block/dirty-bitmap: add deserialize_ones func

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
Add bdrv_dirty_bitmap_deserialize_ones() function, which is needed for qcow2 bitmap loading, to handle unallocated bitmap parts, marked as all-ones. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Kevin Wolf Reviewed-by: John Snow --- block/dirty-bitmap.c | 7 +++ include

[Qemu-block] [PATCH 14/24] block: add bdrv_can_store_new_dirty_bitmap

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
This will be needed to check some restrictions before making bitmap persistent in qmp-block-dirty-bitmap-add (this functionality will be added by future patch) Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block.c | 22 ++ include/b

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] nvme: implement the DSM command

2017-02-02 Thread Paolo Bonzini
> On Wed, Feb 01, 2017 at 12:29:22PM -0800, Paolo Bonzini wrote: > > On 01/02/2017 08:40, Stefan Hajnoczi wrote: > > >> +goto out_free_range; > > >> +} > > >> + > > >> +if (dma_buf_write((uint8_t *)range, sizeof(range), &qsg)) { > > > > > > Did you mean sizeof(*range) * nr? > > >

Re: [Qemu-block] [PATCH 23/24] qcow2: add .bdrv_remove_persistent_dirty_bitmap

2017-02-02 Thread Vladimir Sementsov-Ogievskiy
01.02.2017 02:20, Max Reitz wrote: On 23.01.2017 13:10, Vladimir Sementsov-Ogievskiy wrote: Realize .bdrv_remove_persistent_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 40 block/qcow2.c| 1 +

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] nvme: implement the DSM command

2017-02-02 Thread Stefan Hajnoczi
On Wed, Feb 01, 2017 at 12:29:22PM -0800, Paolo Bonzini wrote: > On 01/02/2017 08:40, Stefan Hajnoczi wrote: > >> +goto out_free_range; > >> +} > >> + > >> +if (dma_buf_write((uint8_t *)range, sizeof(range), &qsg)) { > > > > Did you mean sizeof(*range) * nr? > > Did you also mean

Re: [Qemu-block] [PATCH 1/2] qemu-io: don't allow I/O operations larger than INT_MAX

2017-02-02 Thread Alberto Garcia
On Wed 01 Feb 2017 11:16:38 PM CET, Max Reitz wrote: > Thanks, applied to my block tree, with > %s/INT_MAX/BDRV_REQUEST_MAX_BYTES/g: I think you can use %d to print BDRV_REQUEST_MAX_BYTES, after all the definition guarantees that it won't be larger than MIN(SIZE_MAX,INT_MAX) But it's fine with