Re: [Qemu-block] [Qemu-devel] [PATCH v3 09/12] tests/libqos/pci: Clean up string interpolation into QMP input

Eric Blake writes: > On 07/31/2017 02:29 AM, Markus Armbruster wrote: >> Eric Blake writes: >> >>> On 07/28/2017 11:35 AM, Eric Blake wrote: >> +QObject *extra_args = qobject_from_jsonf("{ 'shm': '%s', 'size': >> '1M' }", >> +

Re: [Qemu-block] [PATCH] block: check BlockDriverState object before dereference

On 07/23/2017 10:36 AM, Paolo Bonzini wrote: > On 21/07/2017 17:47, Stefan Hajnoczi wrote: >> Hmm...BlockDriverState still has bdrv_is_inserted() even though >> BlockBackend->root can be NULL? CCing Markus in case he has thoughts on >> the BB/BDS split. >> >> I find it weird that

Re: [Qemu-block] [PATCH v3 09/13] qcow2: move is_zero_sectors() up

On 07/31/2017 11:22 AM, Anton Nefedov wrote: > To be used in the following commit without a forward declaration. > > Signed-off-by: Anton Nefedov > --- > block/qcow2.c | 39 +++ > 1 file changed, 19 insertions(+), 20 deletions(-)

Re: [Qemu-block] [PATCH v3 04/13] block: support BDRV_REQ_ALLOCATE in passthrough drivers

On 07/31/2017 11:21 AM, Anton Nefedov wrote: > Support the flag if the underlying BDS supports it > > Signed-off-by: Anton Nefedov > --- > block/blkdebug.c | 3 ++- > block/raw-format.c | 3 ++- > 2 files changed, 4 insertions(+), 2 deletions(-) What about

Re: [Qemu-block] [Qemu-devel] [PATCH v3] qemu-iotests: add a "how to" to ./README

On 07/31/2017 12:26 PM, Stefan Hajnoczi wrote: > There is not much getting started documentation for qemu-iotests. This > patch explains how to create a new test and covers the overall testing > approach. > > Cc: Ishani Chugh > Reviewed-by: Eric Blake

Re: [Qemu-block] [Qemu-devel] [PATCH v3 09/12] tests/libqos/pci: Clean up string interpolation into QMP input

On 07/31/2017 02:29 AM, Markus Armbruster wrote: > Eric Blake writes: > >> On 07/28/2017 11:35 AM, Eric Blake wrote: > +QObject *extra_args = qobject_from_jsonf("{ 'shm': '%s', 'size': > '1M' }", > + tmpshm); >>

Re: [Qemu-block] [RFC] block-insert-node and block-job-delete

On Fri, Jul 28, 2017 at 02:08:43PM +0200, Kevin Wolf wrote: Am 27.07.2017 um 12:07 hat Stefan Hajnoczi geschrieben: On Wed, Jul 26, 2017 at 09:23:20PM +0300, Manos Pitsidianakis wrote: > On Wed, Jul 26, 2017 at 04:12:21PM +0100, Stefan Hajnoczi wrote: > > On Wed, Jul 26, 2017 at 05:19:24PM

Re: [Qemu-block] [PATCH v3] qemu-iotests: add a "how to" to ./README

On 07/31/2017 11:26 AM, Stefan Hajnoczi wrote: > There is not much getting started documentation for qemu-iotests. This > patch explains how to create a new test and covers the overall testing > approach. > > Cc: Ishani Chugh > Reviewed-by: Eric Blake

Re: [Qemu-block] [PATCH v3] qemu-iotests: add a "how to" to ./README

On Mon, Jul 31, 2017 at 05:26:41PM +0100, Stefan Hajnoczi wrote: > There is not much getting started documentation for qemu-iotests. This > patch explains how to create a new test and covers the overall testing > approach. > > Cc: Ishani Chugh > Reviewed-by:

Re: [Qemu-block] [Qemu-devel] [PATCH v3 00/13] qcow2: space preallocation and COW improvements

Hi, This series failed build test on FreeBSD host. Please find the details below. Message-id: 1501518125-29851-1-git-send-email-anton.nefe...@virtuozzo.com Subject: [Qemu-devel] [PATCH v3 00/13] qcow2: space preallocation and COW improvements Type: series === TEST SCRIPT BEGIN === #!/bin/sh #

[Qemu-block] [PATCH v3 08/13] qcow2: check space leak at the end of the image

From: Pavel Butsykin Preallocated space in the image may remain unused; the patch adds the functionality to identify and fix it in the qcow2_check to avoid wasting storage space on the host. Signed-off-by: Pavel Butsykin Signed-off-by: Denis V.

[Qemu-block] [PATCH v3 12/13] iotest 190: test BDRV_REQ_ALLOCATE

Signed-off-by: Anton Nefedov --- tests/qemu-iotests/190 | 146 + tests/qemu-iotests/190.out | 50 tests/qemu-iotests/group | 1 + 3 files changed, 197 insertions(+) create mode 100755

Re: [Qemu-block] [PATCH v2] qemu-iotests: add a "how to" to ./README

On Thu, Jul 27, 2017 at 02:28:02PM +0200, Kevin Wolf wrote: > Am 25.07.2017 um 17:36 hat Stefan Hajnoczi geschrieben: > > There is not much getting started documentation for qemu-iotests. This > > patch explains how to create a new test and covers the overall testing > > approach. > > > > Cc:

[Qemu-block] [PATCH v3] qemu-iotests: add a "how to" to ./README

There is not much getting started documentation for qemu-iotests. This patch explains how to create a new test and covers the overall testing approach. Cc: Ishani Chugh Reviewed-by: Eric Blake Reviewed-by: Philippe Mathieu-Daudé

[Qemu-block] [PATCH v3 10/13] qcow2: skip writing zero buffers to empty COW areas

It can be detected that 1. COW alignment of a write request is zeroes 2. Respective areas on the underlying BDS already read as zeroes after being preallocated previously If both of these true, COW may be skipped Signed-off-by: Anton Nefedov ---

[Qemu-block] [PATCH v3 11/13] qcow2: allocate image space by-cluster

If COW areas of the newly allocated clusters are zeroes on the backing image: (even if preallocation feature is not used or it cannot detect if the image already reads as zeroes, e.g. writing to a hole / preallocated zero cluster) efficient bdrv_write_zeroes(flags=BDRV_REQ_ALLOCATE) can be used on

[Qemu-block] [PATCH v3 09/13] qcow2: move is_zero_sectors() up

To be used in the following commit without a forward declaration. Signed-off-by: Anton Nefedov --- block/qcow2.c | 39 +++ 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index

[Qemu-block] [PATCH v3 13/13] iotest 134: test cluster-misaligned encrypted write

COW (even empty/zero) areas require encryption too Signed-off-by: Anton Nefedov --- tests/qemu-iotests/134 | 9 + tests/qemu-iotests/134.out | 10 ++ 2 files changed, 19 insertions(+) diff --git a/tests/qemu-iotests/134 b/tests/qemu-iotests/134

[Qemu-block] [PATCH v3 07/13] qcow2: truncate preallocated space

From: "Denis V. Lunev" This could be done after calculation of the end of data and metadata in the qcow2 image. Signed-off-by: Denis V. Lunev Signed-off-by: Anton Nefedov --- block/qcow2.h | 3 +++ block/qcow2-cluster.c

[Qemu-block] [PATCH v3 00/13] qcow2: space preallocation and COW improvements

Here goes a revisited series on qcow2 preallocation. It's probably a bit better integrated this time and the amount of code is reduced significantly. Changes in v3: - requests intersection detection from the previous versions is removed from qcow2 driver. Instead, tracked request

[Qemu-block] [PATCH v3 03/13] file-posix: support BDRV_REQ_ALLOCATE

Current write_zeroes implementation is good enough to satisfy this flag too Signed-off-by: Anton Nefedov --- block/file-posix.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/block/file-posix.c b/block/file-posix.c index cfbb236..bd52b31

[Qemu-block] [PATCH v3 01/13] block: introduce BDRV_REQ_ALLOCATE flag

The flag is supposed to indicate that the region of the disk image has to be sufficiently allocated so it reads as zeroes. The call with the flag set has to return -ENOTSUP if allocation cannot be done efficiently (i.e. without falling back to writing actual buffers) Signed-off-by: Anton Nefedov

[Qemu-block] [PATCH v3 04/13] block: support BDRV_REQ_ALLOCATE in passthrough drivers

Support the flag if the underlying BDS supports it Signed-off-by: Anton Nefedov --- block/blkdebug.c | 3 ++- block/raw-format.c | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/block/blkdebug.c b/block/blkdebug.c index c19ab28..ef2d764

[Qemu-block] [PATCH v3 06/13] qcow2: set inactive flag

Qcow2State and BlockDriverState flags have to be in sync Signed-off-by: Anton Nefedov --- block/qcow2.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/qcow2.c b/block/qcow2.c index 1939d9b..b11dc48 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2135,6

[Qemu-block] [PATCH v3 02/13] block: treat BDRV_REQ_ALLOCATE as serialising

The idea is that ALLOCATE requests may overlap with other requests. Reuse the existing block layer infrastructure for serialising requests. Use the following approach: - mark ALLOCATE serialising, so subsequent requests to the area wait - ALLOCATE request itself must never wait if another

[Qemu-block] [PATCH v3 05/13] qcow2: preallocation at image expand

From: "Denis V. Lunev" This patch adds image preallocation at expand to provide better locality of QCOW2 image file and optimize this procedure for some distributed storage where this procedure is slow. Preallocation is not issued upon writing metadata clusters. Possible

Re: [Qemu-block] [PATCH v3] docs: add qemu-block-drivers(7) man page

Am 31.07.2017 um 17:56 hat Eric Blake geschrieben: > On 07/26/2017 08:03 AM, Stefan Hajnoczi wrote: > > Block driver documentation is available in qemu-doc.html. It would be > > convenient to have documentation for formats, protocols, and filter > > drivers in a man page. > > > > Extract the

Re: [Qemu-block] [PATCH v3] docs: add qemu-block-drivers(7) man page

On 07/26/2017 08:03 AM, Stefan Hajnoczi wrote: > Block driver documentation is available in qemu-doc.html. It would be > convenient to have documentation for formats, protocols, and filter > drivers in a man page. > > Extract the relevant part of qemu-doc.html into a new file called >

Re: [Qemu-block] [PATCH v4 0/3] qemu-img check: format allocation info

31.07.2017 18:14, Eric Blake wrote: On 07/29/2017 11:41 AM, Vladimir Sementsov-Ogievskiy wrote: Hi all. See 01 patch for the doc. Question to discuss. If I understand correctly get_block_status flags allocated, zero, and data actually provide 5 possible combinations, which I combine into

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.10] block/qapi: Remove redundat NULL check to silence Coverity

On 07/31/2017 12:17 PM, Jeff Cody wrote: On Mon, Jul 31, 2017 at 11:54:57AM -0300, Philippe Mathieu-Daudé wrote: On 07/31/2017 11:38 AM, Jeff Cody wrote: On Mon, Jul 31, 2017 at 02:51:11PM +0200, Kevin Wolf wrote: When skipping implicit nodes in bdrv_block_device_info(), we know that bs0 is

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.10] block/qapi: Remove redundat NULL check to silence Coverity

On Mon, Jul 31, 2017 at 11:54:57AM -0300, Philippe Mathieu-Daudé wrote: > On 07/31/2017 11:38 AM, Jeff Cody wrote: > >On Mon, Jul 31, 2017 at 02:51:11PM +0200, Kevin Wolf wrote: > >>When skipping implicit nodes in bdrv_block_device_info(), we know that > >>bs0 is always non-NULL; initially,

Re: [Qemu-block] [PATCH v4 0/3] qemu-img check: format allocation info

On 07/29/2017 11:41 AM, Vladimir Sementsov-Ogievskiy wrote: > Hi all. > > See 01 patch for the doc. > > Question to discuss. > If I understand correctly get_block_status flags allocated, zero, and data > actually provide 5 possible combinations, which I combine into three. There are actually 8

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.10] block/qapi: Remove redundat NULL check to silence Coverity

Am 31.07.2017 um 16:54 hat Philippe Mathieu-Daudé geschrieben: > On 07/31/2017 11:38 AM, Jeff Cody wrote: > > On Mon, Jul 31, 2017 at 02:51:11PM +0200, Kevin Wolf wrote: > > > When skipping implicit nodes in bdrv_block_device_info(), we know that > > > bs0 is always non-NULL; initially, because

Re: [Qemu-block] [PATCH v2 for-2.11 1/3] qemu-iotests: set TEST_DIR to a unique dir for each test

On Mon, Jul 31, 2017 at 09:44:13AM -0500, Eric Blake wrote: > On 07/31/2017 08:47 AM, Jeff Cody wrote: > > Right now, all qemu-iotests output data into the same scratch directory, > > and so each tests needs to be responsible for cleanup up its own files. > > s/cleanup up/cleaning up/ > Thanks -

Re: [Qemu-block] [PATCH v2 for-2.11 2/3] qemu-iotests: remove file cleanup from bash tests

On 07/31/2017 08:47 AM, Jeff Cody wrote: > All files for a given test are now self-contained in a subdirectory, > and therefore the "./check" script can do all file-related cleanup > without any help. > > This removes file cleanups from the bash tests. The only cleanup left > is whatever is

Re: [Qemu-block] [PATCH v2 for-2.11 1/3] qemu-iotests: set TEST_DIR to a unique dir for each test

On 07/31/2017 08:47 AM, Jeff Cody wrote: > Right now, all qemu-iotests output data into the same scratch directory, > and so each tests needs to be responsible for cleanup up its own files. s/cleanup up/cleaning up/ > > Have each test use 'scratch/$seq' as its temp directory, so the check >

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.10] block/qapi: Remove redundat NULL check to silence Coverity

On 07/31/2017 11:38 AM, Jeff Cody wrote: On Mon, Jul 31, 2017 at 02:51:11PM +0200, Kevin Wolf wrote: When skipping implicit nodes in bdrv_block_device_info(), we know that bs0 is always non-NULL; initially, because it's taken from a BdrvChild Not to mention, we deference bs0 in the chunk of

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.10] block/qapi: Remove redundat NULL check to silence Coverity

On Mon, Jul 31, 2017 at 02:51:11PM +0200, Kevin Wolf wrote: > When skipping implicit nodes in bdrv_block_device_info(), we know that > bs0 is always non-NULL; initially, because it's taken from a BdrvChild Not to mention, we deference bs0 in the chunk of code right above this, so we'd segfault

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.10] block/qapi: Remove redundat NULL check to silence Coverity

Am 31.07.2017 um 16:00 hat Eric Blake geschrieben: > On 07/31/2017 08:06 AM, Eric Blake wrote: > > On 07/31/2017 07:51 AM, Kevin Wolf wrote: > > In the subject line, s/redundat/redundant/ Thanks, I'll fix this. Kevin signature.asc Description: PGP signature

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.10] block/qapi: Remove redundat NULL check to silence Coverity

On 07/31/2017 08:06 AM, Eric Blake wrote: > On 07/31/2017 07:51 AM, Kevin Wolf wrote: In the subject line, s/redundat/redundant/ >> When skipping implicit nodes in bdrv_block_device_info(), we know that >> bs0 is always non-NULL; initially, because it's taken from a BdrvChild >> and a BdrvChild

[Qemu-block] [PATCH v2 for-2.11 3/3] qemu-iotests: add option to save temp files on error

Now that ./check takes care of cleaning up after each tests, it can also selectively not clean up. Add option to leave all output from tests intact if that test encountered an error. Note: this currently only works for bash tests, as the python tests still clean up after themselves manually.

[Qemu-block] [PATCH v2 for-2.11 2/3] qemu-iotests: remove file cleanup from bash tests

All files for a given test are now self-contained in a subdirectory, and therefore the "./check" script can do all file-related cleanup without any help. This removes file cleanups from the bash tests. The only cleanup left is whatever is needed to kill any spawned processes; e.g. _cleanup_qemu.

[Qemu-block] [PATCH v2 for-2.11 1/3] qemu-iotests: set TEST_DIR to a unique dir for each test

Right now, all qemu-iotests output data into the same scratch directory, and so each tests needs to be responsible for cleanup up its own files. Have each test use 'scratch/$seq' as its temp directory, so the check script can do simple cleanup of removing the whole temporary directory.

[Qemu-block] [PATCH v2 for-2.11 0/3] qemu-iotests: place output in unique dir

Differences v1 -> v2: Rebase on Kevin's block branch [Suggested-by Eric] Patch 1: 'check': Added && to the directory change so that the test cannot run if the directory change fails [Thanks Eric] Patch 2: [Thanks Eric] 058: Removed the _cleanup(), and just trap _cleanup_nbd() 091:

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.10] block/qapi: Remove redundat NULL check to silence Coverity

On 07/31/2017 07:51 AM, Kevin Wolf wrote: > When skipping implicit nodes in bdrv_block_device_info(), we know that > bs0 is always non-NULL; initially, because it's taken from a BdrvChild > and a BdrvChild never has a NULL bs, and after the first iteration > because implicit nodes always have a

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.11 3/3] qemu-iotests: add option to save temp files on error

On 07/31/2017 12:04 AM, Jeff Cody wrote: > Now that ./check takes care of cleaning up after each tests, it > can also selectively not clean up. Add option to leave all output from > tests intact if that test encountered an error. > > Note: this currently only works for bash tests, as the python

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.11 2/3] qemu-iotests: remove file cleanup from bash tests

On Mon, Jul 31, 2017 at 07:20:03AM -0500, Eric Blake wrote: > On 07/31/2017 12:04 AM, Jeff Cody wrote: > > All files for a given test are now self-contained in a subdirectory, > > and therefore the "./check" script can do all file-related cleanup > > without any help. > > > > This removes file

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.11 2/3] qemu-iotests: remove file cleanup from bash tests

On 07/31/2017 12:04 AM, Jeff Cody wrote: > All files for a given test are now self-contained in a subdirectory, > and therefore the "./check" script can do all file-related cleanup > without any help. > > This removes file cleanups from the bash tests. The only cleanup left > is whatever is

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.11 1/3] qemu-iotests: set TEST_DIR to a unique dir for each test

On 07/31/2017 12:04 AM, Jeff Cody wrote: > Right now, all qemu-iotests output data into the same scratch directory, > and so each tests needs to be responsible for cleanup up its own files. And, tests must either use unique names or else cannot be run in parallel. > > Have each test use

[Qemu-block] [PATCH v3 7/7] block: add throttle block filter driver interface tests

Signed-off-by: Manos Pitsidianakis --- tests/qemu-iotests/184 | 237 + tests/qemu-iotests/184.out | 319 + tests/qemu-iotests/group | 1 + 3 files changed, 557 insertions(+) create mode

[Qemu-block] [PATCH v3 6/7] block: add BlockDevOptionsThrottle to QAPI

This is needed to configure throttle filter driver nodes with QAPI. Signed-off-by: Manos Pitsidianakis --- qapi/block-core.json | 20 +++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index

[Qemu-block] [PATCH v3 1/7] block: move ThrottleGroup membership to ThrottleGroupMember

This commit eliminates the 1:1 relationship between BlockBackend and throttle group state. Users will be able to create multiple throttle nodes, each with its own throttle group state, in the future. The throttle group state cannot be per-BlockBackend anymore, it must be per-throttle node. This

[Qemu-block] [PATCH v3 4/7] block: convert ThrottleGroup to object with QOM

ThrottleGroup is converted to an object. This will allow the future throttle block filter drive easy creation and configuration of throttle groups in QMP and cli. A new QAPI struct, ThrottleLimits, is introduced to provide a shared struct for all throttle configuration needs in QMP.

[Qemu-block] [PATCH v3 2/7] block: add aio_context field in ThrottleGroupMember

timer_cb() needs to know about the current Aio context of the throttle request that is woken up. In order to make ThrottleGroupMember backend agnostic, this information is stored in an aio_context field instead of accessing it from BlockBackend. Reviewed-by: Stefan Hajnoczi

[Qemu-block] [PATCH v3 0/7] add throttle block driver filter

This series adds a throttle block driver filter. Currently throttling is done at the BlockBackend level. Using block driver interfaces we can move the throttling to any point in the BDS graph using a throttle node which uses the existing throttling code. This allows for potentially more complex

[Qemu-block] [PATCH v3 5/7] block: add throttle block filter driver

block/throttle.c uses existing I/O throttle infrastructure inside a block filter driver. I/O operations are intercepted in the filter's read/write coroutines, and referred to block/throttle-groups.c The driver can be used with the syntax -drive driver=throttle,file.filename=foo.qcow2, \

[Qemu-block] [PATCH v3 3/7] block: tidy ThrottleGroupMember initializations

Move the CoMutex and CoQueue inits inside throttle_group_register_tgm() which is called whenever a ThrottleGroupMember is initialized. There's no need for them to be separate. Reviewed-by: Stefan Hajnoczi Signed-off-by: Manos Pitsidianakis ---

Re: [Qemu-block] [Qemu-devel] [PATCH v3 09/12] tests/libqos/pci: Clean up string interpolation into QMP input

Eric Blake writes: > On 07/28/2017 11:35 AM, Eric Blake wrote: +QObject *extra_args = qobject_from_jsonf("{ 'shm': '%s', 'size': '1M' }", + tmpshm); >>> > >> Passing '%s' through qobject_from_jsonf() is generally

Re: [Qemu-block] [Qemu-devel] [PATCH v3 08/12] qtests: convert tests to use qmp_cmd

Eric Blake writes: > On 07/28/2017 01:53 PM, Markus Armbruster wrote: >> Eric Blake writes: >> >>> Now that we have the qmp_cmd() helper, we can further simplify >>> some of the tests by using it. >>> >>> Signed-off-by: Eric Blake >>>