[PATCH v3 2/3] qemu-img: sort key options alphabetically

2020-01-14 Thread Andrey Shinkevich
Put qemu-img key options in the alphabetical order in the code for a faster finding. Suggested-by: Eric Blake Signed-off-by: Andrey Shinkevich --- qemu-img.c | 22 +++--- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index 6233b8c..c86f7

[PATCH v3 0/3] Dump QCOW2 metadata

2020-01-14 Thread Andrey Shinkevich
The information about QCOW2 metadata allocations in an image ELF-file is helpful for finding issues with the image data integrity. v3: Descriptions of the new key option were added. The names of identifiers were amended. The QEMU-IMG key options were put in the sorted order in the code. Al

[PATCH v3 1/3] qcow2: introduce Qcow2Metadata structure

2020-01-14 Thread Andrey Shinkevich
The preliminary patch to provide an extendable structure for dumping QCOW2 metadata allocations in image. The command line optional key is introduced in the patch that follows. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich --- qapi/block-core.json | 209 +++

[PATCH v3 3/3] qcow2: dump QCOW2 metadata

2020-01-14 Thread Andrey Shinkevich
Let QEMU-IMG CHECK command show QCOW2 structure to inform a user about metadata allocations on disk. Introduce '-M'('--dump-meta') key option. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich --- block/qcow2-bitmap.c | 54 +--- block/qcow

Re: Priority of -accel

2020-01-14 Thread Markus Armbruster
Paolo Bonzini writes: > On 13/01/20 17:17, Markus Armbruster wrote: >> Perfect opportunity to change the default to something more useful. > > I am not sure acutally if it's that more useful, now that we have > sanctioned qemu-kvm as the fast alternative. If there is a fast alternative, why ship

[PULL 0/1] Block patches

2020-01-14 Thread Stefan Hajnoczi
The following changes since commit dc65a5bdc9fa543690a775b50d4ffbeb22c56d6d: Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-5.0-20200108' into staging (2020-01-10 16:15:04 +) are available in the Git repository at: https://github.com/stefanha/qemu.git tags/block-pull-request

[PULL 1/1] linux-aio: increasing MAX_EVENTS to a larger hardcoded value

2020-01-14 Thread Stefan Hajnoczi
From: Wangyong Since commit 6040aedddb5f474a9c2304b6a432a652d82b3d3c "virtio-blk: make queue size configurable",if the user set the queue size to more than 128 ,it will not take effect. That's because linux aio's maximum outstanding requests at a time is always less than or equal to 128. This pa

Re: [PATCH v3 04/15] block/io_uring: implements interfaces for io_uring

2020-01-14 Thread Stefan Hajnoczi
On Mon, Jan 13, 2020 at 12:24:07PM +0100, Stefano Garzarella wrote: > On Wed, Dec 18, 2019 at 04:32:17PM +, Stefan Hajnoczi wrote: > > From: Aarushi Mehta > > > > Aborts when sqe fails to be set as sqes cannot be returned to the > > ring. Adds slow path for short reads for older kernels > >

Re: [PATCH v3 08/15] block/file-posix.c: extend to use io_uring

2020-01-14 Thread Stefan Hajnoczi
On Mon, Jan 13, 2020 at 12:49:27PM +0100, Stefano Garzarella wrote: > On Wed, Dec 18, 2019 at 04:32:21PM +, Stefan Hajnoczi wrote: > > @@ -503,9 +504,11 @@ static int raw_open_common(BlockDriverState *bs, QDict > > *options, > > goto fail; > > } > > > > -aio_default = (bdrv

Re: Priority of -accel

2020-01-14 Thread Paolo Bonzini
On 14/01/20 09:59, Markus Armbruster wrote: > Paolo Bonzini writes: > >> On 13/01/20 17:17, Markus Armbruster wrote: >>> Perfect opportunity to change the default to something more useful. >> >> I am not sure acutally if it's that more useful, now that we have >> sanctioned qemu-kvm as the fast a

[PATCH v4 01/15] configure: permit use of io_uring

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Signed-off-by: Aarushi Mehta Reviewed-by: Maxim Levitsky Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- configure | 27 +++ 1 file changed, 27 insertions(+) diff --git a/configure b/configure index 08c3a1c1f0..ca0af11db0 100755 --

[PATCH v4 00/15] io_uring: add Linux io_uring AIO engine

2020-01-14 Thread Stefan Hajnoczi
v13: * Drop unnecessary changes in Patch 8 [Stefano] v12: * Reword BlockdevAioOptions QAPI schema commit description [Markus] * Increase QAPI "Since: 4.2" to "Since: 5.0" * Explain rationale for io_uring stubs in commit description [Kevin] * Tried to use file.aio=io_uring instead of BDRV_O_IO

[PATCH v4 04/15] block/io_uring: implements interfaces for io_uring

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Aborts when sqe fails to be set as sqes cannot be returned to the ring. Adds slow path for short reads for older kernels Signed-off-by: Aarushi Mehta Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- v11: * Fix git bisect compilation breakage: move trace_lur

[PATCH v4 03/15] block/block: add BDRV flag for io_uring

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Signed-off-by: Aarushi Mehta Reviewed-by: Maxim Levitsky Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- include/block/block.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/block/block.h b/include/block/block.h index e9dcfef7fa..ea155da45a 10

[PATCH v4 02/15] qapi/block-core: add option for io_uring

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Since io_uring is the actual name of the Linux API, we use it as enum value even though the QAPI schema conventions would prefer io-uring. Signed-off-by: Aarushi Mehta Acked-by: Markus Armbruster Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- v12: * Rewo

[PATCH v4 07/15] blockdev: adds bdrv_parse_aio to use io_uring

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Signed-off-by: Aarushi Mehta Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- block.c | 22 ++ blockdev.c| 12 include/block/block.h | 1 + 3 files changed, 27 insertions(+), 8 deletions(-) diff --

[PATCH v4 08/15] block/file-posix.c: extend to use io_uring

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Signed-off-by: Aarushi Mehta Reviewed-by: Maxim Levitsky Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- block/file-posix.c | 85 +- 1 file changed, 69 insertions(+), 16 deletions(-) diff --git a/block/file-posi

[PATCH v4 05/15] stubs: add stubs for io_uring interface

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Follow linux-aio.o and stub out the block/io_uring.o APIs that will be missing when a binary is linked with obj-util-y but without block-util-y (e.g. vhost-user-gpu). For example, the stubs are necessary so that a binary using util/async.o from obj-util-y for qemu_bh_new() li

[PATCH v4 14/15] tests/qemu-iotests: enable testing with aio options

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Signed-off-by: Aarushi Mehta Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- tests/qemu-iotests/check | 15 ++- tests/qemu-iotests/common.rc | 14 ++ tests/qemu-iotests/iotests.py | 12 ++-- 3 files changed, 38 insertio

[PATCH v4 09/15] block: add trace events for io_uring

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Signed-off-by: Aarushi Mehta Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- block/io_uring.c | 23 --- block/trace-events | 12 2 files changed, 32 insertions(+), 3 deletions(-) diff --git a/block/io_uring.c b/block/io_u

[PATCH v4 13/15] qemu-nbd: adds option for aio engines

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Signed-off-by: Aarushi Mehta Acked-by: Eric Blake Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- qemu-nbd.c| 12 qemu-nbd.texi | 4 ++-- 2 files changed, 6 insertions(+), 10 deletions(-) diff --git a/qemu-nbd.c b/qemu-nbd.c index 108a51

[PATCH v4 11/15] qemu-io: adds option to use aio engine

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Signed-off-by: Aarushi Mehta Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- qemu-io.c | 25 + 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/qemu-io.c b/qemu-io.c index 91e3276592..3adc5a7d0d 100644 --- a/qemu-io.c +

[PATCH v4 06/15] util/async: add aio interfaces for io_uring

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Signed-off-by: Aarushi Mehta Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- util/async.c | 36 1 file changed, 36 insertions(+) diff --git a/util/async.c b/util/async.c index b1fa5319e5..c192a24a61 100644 --- a/util/as

[PATCH v4 10/15] block/io_uring: adds userspace completion polling

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Signed-off-by: Aarushi Mehta Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- block/io_uring.c | 17 - 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/block/io_uring.c b/block/io_uring.c index a5c0d16220..56892fd1ab 100644 --- a

[PATCH v4 12/15] qemu-img: adds option to use aio engine for benchmarking

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Signed-off-by: Aarushi Mehta Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- v10: * Add missing space to qemu-img command-line documentation --- qemu-img-cmds.hx | 4 ++-- qemu-img.c | 11 ++- qemu-img.texi| 5 - 3 files changed, 16

[PATCH v4 15/15] tests/qemu-iotests: use AIOMODE with various tests

2020-01-14 Thread Stefan Hajnoczi
From: Aarushi Mehta Signed-off-by: Aarushi Mehta Acked-by: Stefano Garzarella Signed-off-by: Stefan Hajnoczi --- v11: * Drop line-wrapping changes that accidentally broke qemu-iotests --- tests/qemu-iotests/028 | 2 +- tests/qemu-iotests/058 | 2 +- tests/qemu-iotests/089 | 4 ++-- tests/

Re: [PATCH v2] qcow2: Assert that host cluster offsets fit in L2 table entries

2020-01-14 Thread Max Reitz
On 13.01.20 17:11, Alberto Garcia wrote: > The standard cluster descriptor in L2 table entries has a field to > store the host cluster offset. When we need to get that offset from an > entry we use L2E_OFFSET_MASK to ensure that we only use the bits that > belong to that field. > > But while that

Re: [PATCH] block: Use a GString in bdrv_perm_names()

2020-01-14 Thread Max Reitz
On 10.01.20 18:15, Alberto Garcia wrote: > This is a bit more efficient than having to allocate and free memory > for each new permission. > > The default size (30) is enough for "consistent read, write, resize". > > Signed-off-by: Alberto Garcia > --- > block.c | 11 ++- > 1 file chang

Re: [PATCH] block: Use a GString in bdrv_perm_names()

2020-01-14 Thread Philippe Mathieu-Daudé
On 1/10/20 6:15 PM, Alberto Garcia wrote: This is a bit more efficient than having to allocate and free memory for each new permission. The default size (30) is enough for "consistent read, write, resize". Signed-off-by: Alberto Garcia --- block.c | 11 ++- 1 file changed, 6 inserti

Re: [PATCH v2 1/4] qcow2: Require that the virtual size is a multiple of the sector size

2020-01-14 Thread Max Reitz
On 09.01.20 20:12, Alberto Garcia wrote: > The qcow2 header specifies the virtual size of the image in bytes, but > BlockDriverState stores it as a number of 512-byte sectors. > > If the user tries to create an image with a size that is not a > multiple of the sector size then this is fixed on cre

Re: [PATCH v2 1/4] qcow2: Require that the virtual size is a multiple of the sector size

2020-01-14 Thread Alberto Garcia
On Tue 14 Jan 2020 02:01:03 PM CET, Max Reitz wrote: >> However when an image is opened the virtual size is rounded down, >> which means that trying to access the last few advertised bytes will >> result in an error. As seen above QEMU cannot create such images and >> there's no good use case that

Re: [PATCH v2] iotests: Add more "skip_if_unsupported" statements to the python tests

2020-01-14 Thread Thomas Huth
On 19/12/2019 13.19, Kevin Wolf wrote: > Am 18.12.2019 um 15:43 hat Thomas Huth geschrieben: >> The python code already contains a possibility to skip tests if the >> corresponding driver is not available in the qemu binary - use it >> in more spots to avoid that the tests are failing if the driver

Re: [PATCH v2 2/4] qcow2: Don't round the L1 table allocation up to the sector size

2020-01-14 Thread Max Reitz
On 09.01.20 20:13, Alberto Garcia wrote: > The L1 table is read from disk using the byte-based bdrv_pread() and > is never accessed beyond its last element, so there's no need to > allocate more memory than that. > > Signed-off-by: Alberto Garcia > --- > block/qcow2-cluster.c | 5 ++--- > block

Re: [PATCH v2 1/4] qcow2: Require that the virtual size is a multiple of the sector size

2020-01-14 Thread Max Reitz
On 14.01.20 14:20, Alberto Garcia wrote: > On Tue 14 Jan 2020 02:01:03 PM CET, Max Reitz wrote: >>> However when an image is opened the virtual size is rounded down, >>> which means that trying to access the last few advertised bytes will >>> result in an error. As seen above QEMU cannot create suc

Re: [PATCH v2 3/4] qcow2: Tighten cluster_offset alignment assertions

2020-01-14 Thread Max Reitz
On 09.01.20 20:13, Alberto Garcia wrote: > qcow2_alloc_cluster_offset() and qcow2_get_cluster_offset() always > return offsets that are cluster-aligned so don't just check that they > are sector-aligned. > > The check in qcow2_co_preadv_task() is also replaced by an assertion > for the same reason

Re: [PATCH v2 3/4] qcow2: Tighten cluster_offset alignment assertions

2020-01-14 Thread Max Reitz
On 10.01.20 13:14, Alberto Garcia wrote: > On Thu 09 Jan 2020 08:13:01 PM CET, Alberto Garcia wrote: >> -assert((cluster_offset & 511) == 0); >> +assert(QEMU_IS_ALIGNED(cluster_offset, s->cluster_size)); > > On second thoughts this patch could also use the (probably more > efficien

Re: [PATCH v2 1/4] qcow2: Require that the virtual size is a multiple of the sector size

2020-01-14 Thread Alberto Garcia
On Tue 14 Jan 2020 02:47:53 PM CET, Max Reitz wrote: >> Yes, but does it make sense to try to support images with unaligned >> sizes if no one is going to create them ever and QEMU cannot even >> generate them? > > If nobody uses such images ever, isn’t the current code fine as-is? I'll rephrase:

[PATCH v3] iotests: Add more "skip_if_unsupported" statements to the python tests

2020-01-14 Thread Thomas Huth
The python code already contains a possibility to skip tests if the corresponding driver is not available in the qemu binary - use it in more spots to avoid that the tests are failing if the driver has been disabled. While we're at it, we can now also remove some of the old checks that were using

Re: [PATCH v2 1/4] qcow2: Require that the virtual size is a multiple of the sector size

2020-01-14 Thread Max Reitz
On 14.01.20 14:58, Alberto Garcia wrote: > On Tue 14 Jan 2020 02:47:53 PM CET, Max Reitz wrote: > >>> Yes, but does it make sense to try to support images with unaligned >>> sizes if no one is going to create them ever and QEMU cannot even >>> generate them? >> >> If nobody uses such images ever,

Re: [PATCH v2 4/4] qcow2: Use BDRV_SECTOR_SIZE instead of the hardcoded value

2020-01-14 Thread Max Reitz
On 09.01.20 20:13, Alberto Garcia wrote: > This replaces all remaining instances in the qcow2 code. > > Signed-off-by: Alberto Garcia > --- > block/qcow2-cluster.c | 2 +- > block/qcow2.c | 10 ++ > 2 files changed, 7 insertions(+), 5 deletions(-) The question of course is why

Re: [PATCH v3] iotests: Add more "skip_if_unsupported" statements to the python tests

2020-01-14 Thread Kevin Wolf
Am 14.01.2020 um 15:02 hat Thomas Huth geschrieben: > The python code already contains a possibility to skip tests if the > corresponding driver is not available in the qemu binary - use it > in more spots to avoid that the tests are failing if the driver has > been disabled. > > While we're at it

Re: [Qemu-devel] [PATCH] qemu-doc: Do not hard-code the name of the QEMU binary

2020-01-14 Thread Peter Maydell
On Wed, 28 Aug 2019 at 10:37, Thomas Huth wrote: > > In our documentation, we use a mix of "$QEMU", "qemu-system-i386" and > "qemu-system-x86_64" when we give examples to the users how to run > QEMU. Some more consistency would be good here. Also some distributions > use different names for the QE

Re: [Qemu-devel] [PATCH] qemu-doc: Do not hard-code the name of the QEMU binary

2020-01-14 Thread Miroslav Rezanina
- Original Message - > From: "Peter Maydell" > To: "Thomas Huth" > Cc: "QEMU Developers" , "Paolo Bonzini" > , "QEMU Trivial" > , "Miroslav Rezanina" , > "Eduardo Habkost" , > "Qemu-block" , "Richard Henderson" > Sent: Tuesday, January 14, 2020 3:34:22 PM > Subject: Re: [Qemu-devel]

[PATCH] qcow2: Use a GString in report_unsupported_feature()

2020-01-14 Thread Alberto Garcia
This is a bit more efficient than having to allocate and free memory for each item. The default size (60) is enough for all the existing incompatible features. Suggested-by: Philippe Mathieu-Daudé Signed-off-by: Alberto Garcia --- block/qcow2.c | 24 1 file changed, 12

Re: [PATCH v2 1/4] luks: extract block_crypto_calculate_payload_offset()

2020-01-14 Thread Max Reitz
On 09.01.20 12:10, Stefan Hajnoczi wrote: > The qcow2 .bdrv_measure() code calculates the crypto payload offset. > This logic really belongs in block/crypto.c where it can be reused by > other image formats. > > The "luks" block driver will need this same logic in order to implement > .bdrv_measur

Re: [PATCH v2 2/4] luks: implement .bdrv_measure()

2020-01-14 Thread Max Reitz
On 09.01.20 12:10, Stefan Hajnoczi wrote: > Add qemu-img measure support in the "luks" block driver. > > Signed-off-by: Stefan Hajnoczi > --- > block/crypto.c | 82 ++ > 1 file changed, 82 insertions(+) > > diff --git a/block/crypto.c b/block/cryp

Re: [PATCH v2 3/4] qemu-img: allow qemu-img measure --object without a filename

2020-01-14 Thread Max Reitz
On 09.01.20 12:10, Stefan Hajnoczi wrote: > In most qemu-img sub-commands the --object option only makes sense when > there is a filename. qemu-img measure is an exception because objects > may be referenced from the image creation options instead of an existing > image file. Allow --object witho

Re: [PATCH v2 4/4] iotests: add 282 luks qemu-img measure test

2020-01-14 Thread Max Reitz
On 09.01.20 12:10, Stefan Hajnoczi wrote: > This test exercises the block/crypto.c "luks" block driver > .bdrv_measure() code. > > Signed-off-by: Stefan Hajnoczi > --- > tests/qemu-iotests/282 | 93 ++ > tests/qemu-iotests/282.out | 30 > test

Re: [PULL 0/1] Block patches

2020-01-14 Thread Peter Maydell
On Tue, 14 Jan 2020 at 09:31, Stefan Hajnoczi wrote: > > The following changes since commit dc65a5bdc9fa543690a775b50d4ffbeb22c56d6d: > > Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-5.0-20200108' > into staging (2020-01-10 16:15:04 +) > > are available in the Git repository a

Re: [PATCH v2 0/3] block/io: serialising request clean up and locking fix

2020-01-14 Thread Stefan Hajnoczi
On Wed, Jan 08, 2020 at 03:55:53PM +0100, Paolo Bonzini wrote: > Peter Lieven noticed that reqs->overlap_offset and reqs->overlap_bytes > are written outside bs->reqs_lock. Patch 3 fixes it, while patches 1 > and 2 are preparatory cleanups. > > v1->v2: fix comment in patch 2, commit message in pa

Re: [PATCH 2/4] block: Mark 'block_resize' as coroutine

2020-01-14 Thread Stefan Hajnoczi
On Mon, Jan 13, 2020 at 06:10:09PM +0100, Kevin Wolf wrote: > Am 13.01.2020 um 17:56 hat Stefan Hajnoczi geschrieben: > > On Thu, Jan 09, 2020 at 07:35:43PM +0100, Kevin Wolf wrote: > > > block_resize is safe to run in a coroutine, so use it as an example for > > > the new 'coroutine': true annotat

Re: Priority of -accel

2020-01-14 Thread Christophe de Dinechin
> On 13 Jan 2020, at 17:25, Paolo Bonzini wrote: > > On 13/01/20 17:17, Markus Armbruster wrote: >> Perfect opportunity to change the default to something more useful. > > I am not sure acutally if it's that more useful, now that we have > sanctioned qemu-kvm as the fast alternative. OK, hal

Re: Priority of -accel

2020-01-14 Thread Daniel P . Berrangé
On Tue, Jan 14, 2020 at 06:49:45PM +0100, Christophe de Dinechin wrote: > > > > On 13 Jan 2020, at 17:25, Paolo Bonzini wrote: > > > > On 13/01/20 17:17, Markus Armbruster wrote: > >> Perfect opportunity to change the default to something more useful. > > > > I am not sure acutally if it's tha

Re: [PATCH] qcow2: Use a GString in report_unsupported_feature()

2020-01-14 Thread Alex Bennée
Alberto Garcia writes: > This is a bit more efficient than having to allocate and free memory > for each item. > > The default size (60) is enough for all the existing incompatible > features. > > Suggested-by: Philippe Mathieu-Daudé > Signed-off-by: Alberto Garcia > --- > block/qcow2.c | 24

[PATCH v2 2/4] vl: Initialise main loop earlier

2020-01-14 Thread Kevin Wolf
We want to be able to use qemu_aio_context in the monitor initialisation. Signed-off-by: Kevin Wolf Reviewed-by: Marc-André Lureau --- vl.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/vl.c b/vl.c index 86474a55c9..4c79a00857 100644 --- a/vl.c +++ b/vl.c @@ -29

[PATCH v2 1/4] qapi: Add a 'coroutine' flag for commands

2020-01-14 Thread Kevin Wolf
This patch adds a new 'coroutine' flag to QMP command definitions that tells the QMP dispatcher that the command handler is safe to be run in a coroutine. Signed-off-by: Kevin Wolf Reviewed-by: Marc-André Lureau --- tests/qapi-schema/qapi-schema-test.json | 1 + docs/devel/qapi-code-gen.txt

[PATCH v2 0/4] qmp: Optionally run handlers in coroutines

2020-01-14 Thread Kevin Wolf
Some QMP command handlers can block the main loop for a relatively long time, for example because they perform some I/O. This is quite nasty. Allowing such handlers to run in a coroutine where they can yield (and therefore release the BQL) while waiting for an event such as I/O completion solves th

[PATCH v2 3/4] qmp: Move dispatcher to a coroutine

2020-01-14 Thread Kevin Wolf
This moves the QMP dispatcher to a coroutine and runs all QMP command handlers that declare 'coroutine': true in coroutine context so they can avoid blocking the main loop while doing I/O or waiting for other events. For commands that are not declared safe to run in a coroutine, the dispatcher dro

[PATCH v2 4/4] block: Mark 'block_resize' as coroutine

2020-01-14 Thread Kevin Wolf
block_resize is safe to run in a coroutine, so use it as an example for the new 'coroutine': true annotation in the QAPI schema. Signed-off-by: Kevin Wolf Reviewed-by: Marc-André Lureau --- qapi/block-core.json | 3 ++- blockdev.c | 6 +++--- 2 files changed, 5 insertions(+), 4 deleti

Re: [PATCH] qcow2: Use a GString in report_unsupported_feature()

2020-01-14 Thread Philippe Mathieu-Daudé
On 1/14/20 7:08 PM, Alex Bennée wrote: Alberto Garcia writes: This is a bit more efficient than having to allocate and free memory for each item. The default size (60) is enough for all the existing incompatible features. Suggested-by: Philippe Mathieu-Daudé Signed-off-by: Alberto Garcia -

Re: [PATCH v2 0/4] qmp: Optionally run handlers in coroutines

2020-01-14 Thread Daniel P . Berrangé
On Tue, Jan 14, 2020 at 07:27:31PM +0100, Kevin Wolf wrote: > Some QMP command handlers can block the main loop for a relatively long > time, for example because they perform some I/O. This is quite nasty. > Allowing such handlers to run in a coroutine where they can yield (and > therefore release

Re: [PATCH v2 0/4] qmp: Optionally run handlers in coroutines

2020-01-14 Thread Kevin Wolf
Am 14.01.2020 um 19:45 hat Daniel P. Berrangé geschrieben: > On Tue, Jan 14, 2020 at 07:27:31PM +0100, Kevin Wolf wrote: > > Some QMP command handlers can block the main loop for a relatively long > > time, for example because they perform some I/O. This is quite nasty. > > Allowing such handlers t

[PATCH 00/13] LUKS: encryption slot management using amend interface

2020-01-14 Thread Maxim Levitsky
Hi! Here is the updated series of my patches, incorporating all the feedback I received. Patches are strictly divided by topic to 3 groups, and each group depends on former groups. * Patches 1,2 implement qcrypto generic amend interface, including definition of structs used in crypto.json an

[PATCH 05/13] block/crypto: rename two functions

2020-01-14 Thread Maxim Levitsky
rename the write_func to create_write_func, and init_func to create_init_func. This is preparation for other write_func that will be used to update the encryption keys. No functional changes Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé --- block/crypto.c | 25

[PATCH 01/13] qcrypto: add generic infrastructure for crypto options amendment

2020-01-14 Thread Maxim Levitsky
This will be used first to implement luks keyslot management. block_crypto_amend_opts_init will be used to convert qemu-img cmdline to QCryptoBlockAmendOptions Signed-off-by: Maxim Levitsky --- block/crypto.c | 17 + block/crypto.h | 3 +++ crypto/block.c

[PATCH 03/13] block: amend: add 'force' option

2020-01-14 Thread Maxim Levitsky
'force' option will be used for some unsafe amend operations. This includes things like erasing last keyslot in luks based formats (which destroys the data, unless the master key is backed up by external means), but that _might_ be desired result. Signed-off-by: Maxim Levitsky Reviewed-by: Danie

[PATCH 04/13] block: amend: separate amend and create options for qemu-img

2020-01-14 Thread Maxim Levitsky
Some options are only useful for creation (or hard to be amended, like cluster size for qcow2), while some other options are only useful for amend, like upcoming keyslot management options for luks Since currently only qcow2 supports amend, move all its options to a common macro and then include i

[PATCH 07/13] qcow2: extend qemu-img amend interface with crypto options

2020-01-14 Thread Maxim Levitsky
Now that we have all the infrastructure in place, wire it in the qcow2 driver and expose this to the user. Signed-off-by: Maxim Levitsky --- block/qcow2.c | 101 +++--- 1 file changed, 79 insertions(+), 22 deletions(-) diff --git a/block/qcow2.c b/blo

[PATCH 08/13] iotests: filter few more luks specific create options

2020-01-14 Thread Maxim Levitsky
This allows more tests to be able to have same output on both qcow2 luks encrypted images and raw luks images Signed-off-by: Maxim Levitsky --- tests/qemu-iotests/087.out | 6 +++--- tests/qemu-iotests/134.out | 2 +- tests/qemu-iotests/158.out | 4 ++-- tests/qemu-iotests/188

[PATCH 10/13] block: add generic infrastructure for x-blockdev-amend qmp command

2020-01-14 Thread Maxim Levitsky
blockdev-amend will be used similiar to blockdev-create to allow on the fly changes of the structure of the format based block devices. Current plan is to first support encryption keyslot management for luks based formats (raw and embedded in qcow2) Signed-off-by: Maxim Levitsky --- block/Makef

[PATCH 12/13] block/qcow2: implement blockdev-amend

2020-01-14 Thread Maxim Levitsky
Currently the implementation only supports amending the encryption options, unlike the qemu-img version Signed-off-by: Maxim Levitsky --- block/qcow2.c| 39 +++ qapi/block-core.json | 16 +++- 2 files changed, 54 insertions(+), 1 deletion(-

[PATCH 13/13] iotests: add tests for blockdev-amend

2020-01-14 Thread Maxim Levitsky
This commit adds two tests that cover the new blockdev-amend functionality of luks and qcow2 driver Signed-off-by: Maxim Levitsky --- tests/qemu-iotests/302 | 284 + tests/qemu-iotests/302.out | 40 ++ tests/qemu-iotests/303 | 235

[PATCH 06/13] block/crypto: implement the encryption key management

2020-01-14 Thread Maxim Levitsky
This implements the encryption key management using the generic code in qcrypto layer and exposes it to the user via qemu-img This code adds another 'write_func' because the initialization write_func works directly on the underlying file, and amend works on instance of luks device. This commit al

[PATCH 09/13] qemu-iotests: qemu-img tests for luks key management

2020-01-14 Thread Maxim Levitsky
This commit adds two tests, which test the new amend interface of both luks raw images and qcow2 luks encrypted images. Signed-off-by: Maxim Levitsky --- tests/qemu-iotests/300 | 207 + tests/qemu-iotests/300.out | 99 ++ tests/qemu-iotest

[PATCH 02/13] qcrypto-luks: implement encryption key management

2020-01-14 Thread Maxim Levitsky
Next few patches will expose that functionality to the user. Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 374 +++- qapi/crypto.json| 50 +- 2 files changed, 421 insertions(+), 3 deletions(-) diff --git a/crypto/block-luks.c b/crypto/b

Re: [PATCH v2 0/3] block/io: serialising request clean up and locking fix

2020-01-14 Thread Paolo Bonzini
On 14/01/20 17:28, Stefan Hajnoczi wrote: > On Wed, Jan 08, 2020 at 03:55:53PM +0100, Paolo Bonzini wrote: >> Peter Lieven noticed that reqs->overlap_offset and reqs->overlap_bytes >> are written outside bs->reqs_lock. Patch 3 fixes it, while patches 1 >> and 2 are preparatory cleanups. >> >> v1->

[PATCH 11/13] block/crypto: implement blockdev-amend

2020-01-14 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- block/crypto.c | 70 qapi/block-core.json | 14 - 2 files changed, 64 insertions(+), 20 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index 081880bced..6836337863 100644 --- a/block/crypto.c

Re: [PATCH v2 0/4] qmp: Optionally run handlers in coroutines

2020-01-14 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200114182735.5553-1-kw...@redhat.com/ Hi, This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN === #!/

Re: [PATCH 00/13] LUKS: encryption slot management using amend interface

2020-01-14 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200114193350.10830-1-mlevi...@redhat.com/ Hi, This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN ==

Re: [PATCH 00/13] LUKS: encryption slot management using amend interface

2020-01-14 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200114193350.10830-1-mlevi...@redhat.com/ Hi, This series seems to have some coding style problems. See output below for more information: Subject: [PATCH 00/13] LUKS: encryption slot management using amend interface Type: series Message-id: 20200114193

Re: [PATCH] qcow2: Use a GString in report_unsupported_feature()

2020-01-14 Thread Alex Bennée
Philippe Mathieu-Daudé writes: > On 1/14/20 7:08 PM, Alex Bennée wrote: >> Alberto Garcia writes: >> >>> This is a bit more efficient than having to allocate and free memory >>> for each item. >>> >>> The default size (60) is enough for all the existing incompatible >>> features. >>> >>> Sugg

Re: [PATCH v6 01/11] qapi/error: add (Error **errp) cleaning APIs

2020-01-14 Thread Greg Kurz
On Fri, 10 Jan 2020 22:41:48 +0300 Vladimir Sementsov-Ogievskiy wrote: > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > Reviewed-by: Greg Kurz > CC: Cornelia Huck > CC: Eric Blake > CC: Kevin Wolf > CC: Max Reitz > CC: Greg Kurz > CC: Stefan Hajnoczi > CC: Stefano Stabellini > CC