[Qemu-block] [PATCH v3 2/4] block/qcow2: refactor qcow2_co_preadv_part

2019-08-15 Thread Vladimir Sementsov-Ogievskiy
Further patch will run partial requests of iterations of qcow2_co_preadv in parallel for performance reasons. To prepare for this, separate part which may be parallelized into separate function (qcow2_co_preadv_task). While being here, also separate encrypted clusters reading to own function,

[Qemu-block] [PATCH v3 4/4] block/qcow2: introduce parallel subrequest handling in read and write

2019-08-15 Thread Vladimir Sementsov-Ogievskiy
It improves performance for fragmented qcow2 images. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 3 ++ block/qcow2.c | 125 - block/trace-events | 1 + 3 files changed, 117 insertions(+), 12 deletions(-) diff --git

[Qemu-block] [PATCH v3 0/4] qcow2: async handling of fragmented io

2019-08-15 Thread Vladimir Sementsov-Ogievskiy
Hi all! Here is an asynchronous scheme for handling fragmented qcow2 reads and writes. Both qcow2 read and write functions loops through sequential portions of data. The series aim it to parallelize these loops iterations. It improves performance for fragmented qcow2 images, I've tested it as

[Qemu-block] [PATCH v3 3/4] block/qcow2: refactor qcow2_co_pwritev_part

2019-08-15 Thread Vladimir Sementsov-Ogievskiy
Similarly to previous commit, prepare for parallelizing write-loop iterations. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 153 +- 1 file changed, 89 insertions(+), 64 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c

Re: [Qemu-block] [PATCH v3 0/4] qcow2: async handling of fragmented io

2019-08-15 Thread Max Reitz
On 15.08.19 14:10, Vladimir Sementsov-Ogievskiy wrote: > 01: - use coroutine_fn where appropriate !!! :-) signature.asc Description: OpenPGP digital signature

Re: [Qemu-block] [PATCH v3 1/4] block: introduce aio task pool

2019-08-15 Thread Max Reitz
On 15.08.19 14:10, Vladimir Sementsov-Ogievskiy wrote: > Common interface for aio task loops. To be used for improving > performance of synchronous io loops in qcow2, block-stream, > copy-on-read, and may be other places. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- >

Re: [Qemu-block] [PATCH v3 4/4] block/qcow2: introduce parallel subrequest handling in read and write

2019-08-15 Thread Max Reitz
On 15.08.19 14:10, Vladimir Sementsov-Ogievskiy wrote: > It improves performance for fragmented qcow2 images. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/qcow2.h | 3 ++ > block/qcow2.c | 125 - > block/trace-events | 1

Re: [Qemu-block] [Qemu-devel] [PATCH 00/13] RFC: luks/encrypted qcow2 key management

2019-08-15 Thread Markus Armbruster
Kevin Wolf writes: > Am 14.08.2019 um 23:08 hat Eric Blake geschrieben: >> On 8/14/19 3:22 PM, Maxim Levitsky wrote: >> >> > This is an issue that was raised today on IRC with Kevin Wolf. Really >> > thanks >> > for the idea! >> > >> > We agreed that this new qmp interface should take the

Re: [Qemu-block] [Qemu-devel] [PATCH v0] Implement new cache mode "target"

2019-08-15 Thread Kevin Wolf
Am 15.08.2019 um 15:53 hat Stefan Hajnoczi geschrieben: > On Wed, Aug 07, 2019 at 04:09:54PM +0300, Artemy Kapitula wrote: > > Hi, > Please use "scripts/get_maintainer.pl -f block.c" to find out which > maintainers to email. qemu-de...@nongnu.org is a high-traffic list and > patches not CCed to

[Qemu-block] [PATCH v3 1/4] block: introduce aio task pool

2019-08-15 Thread Vladimir Sementsov-Ogievskiy
Common interface for aio task loops. To be used for improving performance of synchronous io loops in qcow2, block-stream, copy-on-read, and may be other places. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/aio_task.h | 54 + block/aio_task.c | 124

Re: [Qemu-block] [PATCH v4] blockjob: drain all job nodes in block_job_drain

2019-08-15 Thread Max Reitz
On 02.08.19 11:52, Vladimir Sementsov-Ogievskiy wrote: > Instead of draining additional nodes in each job code, let's do it in > common block_job_drain, draining just all job's children. > BlockJobDriver.drain becomes unused, so, drop it at all. > > It's also a first step to finally get rid of

Re: [Qemu-block] [PATCH v3 0/4] qcow2: async handling of fragmented io

2019-08-15 Thread Max Reitz
On 15.08.19 14:10, Vladimir Sementsov-Ogievskiy wrote: > Hi all! > > Here is an asynchronous scheme for handling fragmented qcow2 > reads and writes. Both qcow2 read and write functions loops through > sequential portions of data. The series aim it to parallelize these > loops iterations. > It

Re: [Qemu-block] [PATCH v3 2/4] block/qcow2: refactor qcow2_co_preadv_part

2019-08-15 Thread Max Reitz
On 15.08.19 14:10, Vladimir Sementsov-Ogievskiy wrote: > Further patch will run partial requests of iterations of > qcow2_co_preadv in parallel for performance reasons. To prepare for > this, separate part which may be parallelized into separate function > (qcow2_co_preadv_task). > > While being

Re: [Qemu-block] [PATCH v3 3/4] block/qcow2: refactor qcow2_co_pwritev_part

2019-08-15 Thread Max Reitz
On 15.08.19 14:10, Vladimir Sementsov-Ogievskiy wrote: > Similarly to previous commit, prepare for parallelizing write-loop > iterations. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/qcow2.c | 153 +- > 1 file changed, 89

Re: [Qemu-block] [Qemu-devel] [PATCH 00/13] RFC: luks/encrypted qcow2 key management

2019-08-15 Thread Maxim Levitsky
On Thu, 2019-08-15 at 16:18 +0200, Markus Armbruster wrote: > Kevin Wolf writes: > > > Am 14.08.2019 um 23:08 hat Eric Blake geschrieben: > > > On 8/14/19 3:22 PM, Maxim Levitsky wrote: > > > > > > > This is an issue that was raised today on IRC with Kevin Wolf. Really > > > > thanks > > > >

Re: [Qemu-block] [PATCH v2] qemu-img convert: Deprecate using -n and -o together

2019-08-15 Thread Eric Blake
On 8/15/19 6:06 AM, Kevin Wolf wrote: > bdrv_create options specified with -o have no effect when skipping image > creation with -n, so this doesn't make sense. Warn against the misuse > and deprecate the combination so we can make it a hard error later. > > Signed-off-by: Kevin Wolf > ---

Re: [Qemu-block] [PATCH v2] qemu-img convert: Deprecate using -n and -o together

2019-08-15 Thread Max Reitz
On 15.08.19 13:06, Kevin Wolf wrote: > bdrv_create options specified with -o have no effect when skipping image > creation with -n, so this doesn't make sense. Warn against the misuse > and deprecate the combination so we can make it a hard error later. > > Signed-off-by: Kevin Wolf > --- > > -

[Qemu-block] Exposing feature deprecation to machine clients (was: [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters)

2019-08-15 Thread Markus Armbruster
John Snow writes: > On 8/14/19 6:07 AM, Vladimir Sementsov-Ogievskiy wrote: >> To get rid of implicit filters related workarounds in future let's >> deprecate them now. >> >> Signed-off-by: Vladimir Sementsov-Ogievskiy >> --- [...] >> diff --git a/blockdev.c b/blockdev.c >> index

Re: [Qemu-block] [Qemu-devel] [PATCH 00/13] RFC: luks/encrypted qcow2 key management

2019-08-15 Thread Eric Blake
On 8/15/19 9:44 AM, Maxim Levitsky wrote: Does the idea of a union type with a default value for the discriminator help? Maybe we have a discriminator which defaults to 'auto', and add a union branch 'auto':'any'. During creation, if the "driver":"auto" branch is selected

Re: [Qemu-block] [libvirt] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters

2019-08-15 Thread Peter Krempa
On Thu, Aug 15, 2019 at 12:49:28 +0200, Kevin Wolf wrote: > Am 14.08.2019 um 21:27 hat John Snow geschrieben: [...] > > example: > > > > { "return": {}, > > "deprecated": True, > > "warning": "Omitting filter-node-name parameter is deprecated, it will > > be required in the future" > > } >

Re: [Qemu-block] [Qemu-devel] [PATCH v0] Implement new cache mode "target"

2019-08-15 Thread Stefan Hajnoczi
On Wed, Aug 07, 2019 at 04:09:54PM +0300, Artemy Kapitula wrote: Hi, Please use "scripts/get_maintainer.pl -f block.c" to find out which maintainers to email. qemu-de...@nongnu.org is a high-traffic list and patches not CCed to the right maintainer may not get quick review. > There is an issue

Re: [Qemu-block] [Qemu-devel] [libvirt] [PATCH 2/2] qapi: deprecate implicit filters

2019-08-15 Thread Markus Armbruster
Peter Krempa writes: > On Thu, Aug 15, 2019 at 12:49:28 +0200, Kevin Wolf wrote: >> Am 14.08.2019 um 21:27 hat John Snow geschrieben: > > [...] > >> > example: >> > >> > { "return": {}, >> > "deprecated": True, >> > "warning": "Omitting filter-node-name parameter is deprecated, it will >> >

Re: [Qemu-block] [PATCH v2 00/12] block: qiov_offset parameter for io

2019-08-15 Thread Vladimir Sementsov-Ogievskiy
29.07.2019 18:24, Stefan Hajnoczi wrote: > On Tue, Jun 04, 2019 at 07:15:02PM +0300, Vladimir Sementsov-Ogievskiy wrote: >> Hi all! >> >> Here is new parameter qiov_offset for io path, to avoid >> a lot of places with same pattern of creating local_qiov or hd_qiov >> variables. >> >> These series

Re: [Qemu-block] [PATCH 2/4] configure: Avoid using libssh deprecated API

2019-08-15 Thread Andrea Bolognani
On Wed, 2019-08-14 at 17:14 +0200, Philippe Mathieu-Daudé wrote: > On 8/14/19 4:51 PM, Andrea Bolognani wrote: > > On Wed, 2019-08-14 at 16:15 +0200, Philippe Mathieu-Daudé wrote: > > > On 8/14/19 3:27 PM, Andrea Bolognani wrote: > > > > On Wed, 2019-08-14 at 14:15 +0200, Philippe Mathieu-Daudé

Re: [Qemu-block] [Qemu-devel] [PULL 5/7] file-posix: Support BDRV_REQ_NO_FALLBACK for zero writes

2019-08-15 Thread Kevin Wolf
Am 15.08.2019 um 04:44 hat Eric Blake geschrieben: > On 3/26/19 10:51 AM, Kevin Wolf wrote: > > We know that the kernel implements a slow fallback code path for > > BLKZEROOUT, so if BDRV_REQ_NO_FALLBACK is given, we shouldn't call it. > > The other operations we call in the context of

Re: [Qemu-block] [libvirt] [Qemu-devel] [PATCH 1/2] qapi: deprecate drive-mirror and drive-backup

2019-08-15 Thread Peter Krempa
On Wed, Aug 14, 2019 at 15:22:15 -0400, John Snow wrote: > > > On 8/14/19 6:07 AM, Vladimir Sementsov-Ogievskiy wrote: > > It's hard and not necessary to maintain outdated versions of these > > commands. > > > > Signed-off-by: Vladimir Sementsov-Ogievskiy > > --- > > qemu-deprecated.texi |

Re: [Qemu-block] [PATCH 00/13] RFC: luks/encrypted qcow2 key management

2019-08-15 Thread Maxim Levitsky
On Wed, 2019-08-14 at 16:08 -0500, Eric Blake wrote: > On 8/14/19 3:22 PM, Maxim Levitsky wrote: > > > This is an issue that was raised today on IRC with Kevin Wolf. Really thanks > > for the idea! > > > > We agreed that this new qmp interface should take the same options as > > blockdev-create

Re: [Qemu-block] [PATCH 00/13] RFC: luks/encrypted qcow2 key management

2019-08-15 Thread Kevin Wolf
Am 14.08.2019 um 23:08 hat Eric Blake geschrieben: > On 8/14/19 3:22 PM, Maxim Levitsky wrote: > > > This is an issue that was raised today on IRC with Kevin Wolf. Really thanks > > for the idea! > > > > We agreed that this new qmp interface should take the same options as > > blockdev-create

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters

2019-08-15 Thread Kevin Wolf
Am 14.08.2019 um 21:27 hat John Snow geschrieben: > > > On 8/14/19 6:07 AM, Vladimir Sementsov-Ogievskiy wrote: > > To get rid of implicit filters related workarounds in future let's > > deprecate them now. > > > > Signed-off-by: Vladimir Sementsov-Ogievskiy > > --- > > qemu-deprecated.texi

[Qemu-block] [PATCH v2] qemu-img convert: Deprecate using -n and -o together

2019-08-15 Thread Kevin Wolf
bdrv_create options specified with -o have no effect when skipping image creation with -n, so this doesn't make sense. Warn against the misuse and deprecate the combination so we can make it a hard error later. Signed-off-by: Kevin Wolf --- - Hopefully removed the "finger-wagging" that John

[Qemu-block] [PATCH v2 1/7] iotests: Fix _filter_img_create()

2019-08-15 Thread Max Reitz
fe646693acc changed qemu-img create's output so that it no longer prints single quotes around parameter values. The subformat and adapter_type filters in _filter_img_create() have never been adapted to that change. Fixes: fe646693acc13ac48b98435d14149ab04dc597bc Signed-off-by: Max Reitz

[Qemu-block] [PATCH v2 2/7] vmdk: Use bdrv_dirname() for relative extent paths

2019-08-15 Thread Max Reitz
This makes iotest 033 pass with e.g. subformat=monolithicFlat. It also turns a former error in 059 into success. Signed-off-by: Max Reitz --- block/vmdk.c | 54 -- tests/qemu-iotests/059 | 7 +++-- tests/qemu-iotests/059.out | 4 ++- 3

[Qemu-block] [PATCH v2 0/7] vmdk: Misc fixes

2019-08-15 Thread Max Reitz
I made the mistake of trying to run the iotests with all non-default subformats our vmdk driver has to offer: - monolithicFlat - twoGbMaxExtentSparse - twoGbMaxExtentFlat - streamOptimized Many things broke, so this series fixes what I found. It’s mostly just iotest fixes, but there are actually

[Qemu-block] [PATCH v2 5/7] iotests: Disable broken streamOptimized tests

2019-08-15 Thread Max Reitz
streamOptimized does not support writes that do not span exactly one cluster. Furthermore, it cannot rewrite already allocated clusters. As such, many iotests do not work with it. Disable them. Signed-off-by: Max Reitz --- tests/qemu-iotests/002 | 1 + tests/qemu-iotests/003 | 1 +

[Qemu-block] [PATCH v2 7/7] iotests: Disable 126 for flat vmdk subformats

2019-08-15 Thread Max Reitz
iotest 126 requires backing file support, which flat vmdks cannot offer. Skip this test for such subformats. Signed-off-by: Max Reitz --- tests/qemu-iotests/126 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/126 b/tests/qemu-iotests/126 index 9b0dcf9255..b7fce1e59d

[Qemu-block] [PATCH v2 3/7] iotests: Keep testing broken relative extent paths

2019-08-15 Thread Max Reitz
We had a test for a case where relative extent paths did not work, but unfortunately we just fixed the underlying problem, so it works now. This patch adds a new test case that still fails. Signed-off-by: Max Reitz Reviewed-by: John Snow --- tests/qemu-iotests/059 | 27

Re: [Qemu-block] [PATCH v3 0/4] qcow2: async handling of fragmented io

2019-08-15 Thread Vladimir Sementsov-Ogievskiy
15.08.2019 16:21, Max Reitz wrote: > On 15.08.19 14:10, Vladimir Sementsov-Ogievskiy wrote: >> 01: - use coroutine_fn where appropriate !!! > > :-) > Ahahaha, I'll explain: When comparing v2 vs v3 and writing this difference script I noticed that I added coroutine_fn marks

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters

2019-08-15 Thread John Snow
On 8/15/19 6:49 AM, Kevin Wolf wrote: > Am 14.08.2019 um 21:27 hat John Snow geschrieben: >> >> >> On 8/14/19 6:07 AM, Vladimir Sementsov-Ogievskiy wrote: >>> To get rid of implicit filters related workarounds in future let's >>> deprecate them now. >>> >>> Signed-off-by: Vladimir

[Qemu-block] [PATCH v2 6/7] iotests: Disable 110 for vmdk.twoGbMaxExtentSparse

2019-08-15 Thread Max Reitz
The error message for the test case where we have a quorum node for which no directory name can be generated is different: For twoGbMaxExtentSparse, it complains that it cannot open the extent file. For other (sub)formats, it just notes that it cannot determine the backing file path. Both are

Re: [Qemu-block] [PATCH v3 0/4] qcow2: async handling of fragmented io

2019-08-15 Thread Vladimir Sementsov-Ogievskiy
15.08.2019 17:09, Max Reitz wrote: > On 15.08.19 14:10, Vladimir Sementsov-Ogievskiy wrote: >> Hi all! >> >> Here is an asynchronous scheme for handling fragmented qcow2 >> reads and writes. Both qcow2 read and write functions loops through >> sequential portions of data. The series aim it to

Re: [Qemu-block] [PATCH v2] qemu-img convert: Deprecate using -n and -o together

2019-08-15 Thread John Snow
On 8/15/19 7:06 AM, Kevin Wolf wrote: > bdrv_create options specified with -o have no effect when skipping image > creation with -n, so this doesn't make sense. Warn against the misuse > and deprecate the combination so we can make it a hard error later. > > Signed-off-by: Kevin Wolf > --- >

Re: [Qemu-block] [PATCH 0/3] block: Make various formats' block_status recurse again

2019-08-15 Thread Max Reitz
On 25.07.19 17:55, Max Reitz wrote: > Hi, > > 69f47505ee66afaa513305de0c1895a224e52c45 changed block_status so that it > would only go down to the protocol layer if the format layer returned > BDRV_BLOCK_RECURSE, thus indicating that it has no sufficient > information whether a given range in the

Re: [Qemu-block] [PATCH] iotests: Fix 141 when run with qed

2019-08-15 Thread Max Reitz
On 09.08.19 20:52, Max Reitz wrote: > 69f47505ee has changed qcow2 in such a way that the commit job run in > test 141 (and 144[1]) returns before it emits the READY event. However, > 141 also runs with qed, where the order is still the other way around. > Just filter out the {"return": {}} so

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters

2019-08-15 Thread John Snow
On 8/15/19 12:48 PM, Kevin Wolf wrote: > Am 15.08.2019 um 18:07 hat John Snow geschrieben: >> On 8/15/19 6:49 AM, Kevin Wolf wrote: >>> Am 14.08.2019 um 21:27 hat John Snow geschrieben: This might be OK to do right away, though. I asked Markus this not too long ago; do we want

Re: [Qemu-block] [PATCH v6 35/42] block: Fix check_to_replace_node()

2019-08-15 Thread Vladimir Sementsov-Ogievskiy
09.08.2019 19:14, Max Reitz wrote: > Currently, check_to_replace_node() only allows mirror to replace a node > in the chain of the source node, and only if it is the first non-filter > node below the source. Well, technically, the idea is that you can > exactly replace a quorum child by mirroring

Re: [Qemu-block] [PATCH] doc: Preallocation does not require writing zeroes

2019-08-15 Thread Max Reitz
On 11.07.19 15:29, Max Reitz wrote: > When preallocating an encrypted qcow2 image, it just lets the protocol > driver write data and then does not mark the clusters as zero. > Therefore, reading this image will yield effectively random data. > > As such, we have not fulfilled the promise of

Re: [Qemu-block] Exposing feature deprecation to machine clients (was: [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters)

2019-08-15 Thread John Snow
On 8/15/19 10:16 AM, Markus Armbruster wrote: > John Snow writes: > >> On 8/14/19 6:07 AM, Vladimir Sementsov-Ogievskiy wrote: >>> To get rid of implicit filters related workarounds in future let's >>> deprecate them now. >>> >>> Signed-off-by: Vladimir Sementsov-Ogievskiy >>> --- > [...]

[Qemu-block] [PATCH v2 4/7] vmdk: Reject invalid compressed writes

2019-08-15 Thread Max Reitz
Compressed writes generally have to write full clusters, not just in theory but also in practice when it comes to vmdk's streamOptimized subformat. It currently is just silently broken for writes with non-zero in-cluster offsets: $ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters

2019-08-15 Thread Kevin Wolf
Am 15.08.2019 um 18:07 hat John Snow geschrieben: > > > On 8/15/19 6:49 AM, Kevin Wolf wrote: > > Am 14.08.2019 um 21:27 hat John Snow geschrieben: > >> > >> > >> On 8/14/19 6:07 AM, Vladimir Sementsov-Ogievskiy wrote: > >>> To get rid of implicit filters related workarounds in future let's >

Re: [Qemu-block] [PATCH v6 35/42] block: Fix check_to_replace_node()

2019-08-15 Thread Max Reitz
On 15.08.19 17:21, Vladimir Sementsov-Ogievskiy wrote: > 09.08.2019 19:14, Max Reitz wrote: >> Currently, check_to_replace_node() only allows mirror to replace a node >> in the chain of the source node, and only if it is the first non-filter >> node below the source. Well, technically, the idea

[Qemu-block] [PULL 9/9] block/nbd: refactor nbd connection parameters

2019-08-15 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy We'll need some connection parameters to be available all the time to implement nbd reconnect. So, let's refactor them: define additional parameters in BDRVNBDState, drop them from function parameters, drop nbd_client_init and separate options parsing instead

[Qemu-block] [PULL 5/9] block/nbd: split connection_co start out of nbd_client_connect

2019-08-15 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy nbd_client_connect is going to be used from connection_co, so, let's refactor nbd_client_connect in advance, leaving io channel configuration all in nbd_client_connect. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Message-Id:

Re: [Qemu-block] [PATCH 02/13] qcrypto-luks: misc refactoring

2019-08-15 Thread John Snow
On 8/14/19 4:22 PM, Maxim Levitsky wrote: > This is also a preparation for key read/write/erase functions > This is a matter of taste and I am not usually reviewing LUKS patches (So don't take me too seriously), but I would prefer not to have "misc" patches and instead split things out by

Re: [Qemu-block] [libvirt] [Qemu-devel] [PATCH 1/2] qapi: deprecate drive-mirror and drive-backup

2019-08-15 Thread John Snow
On 8/15/19 3:44 AM, Peter Krempa wrote: > On Wed, Aug 14, 2019 at 15:22:15 -0400, John Snow wrote: >> >> >> On 8/14/19 6:07 AM, Vladimir Sementsov-Ogievskiy wrote: >>> It's hard and not necessary to maintain outdated versions of these >>> commands. >>> >>> Signed-off-by: Vladimir

Re: [Qemu-block] [PATCH] nbd: Advertise multi-conn for shared read-only connections

2019-08-15 Thread John Snow
On 8/15/19 2:50 PM, Eric Blake wrote: > The NBD specification defines NBD_FLAG_CAN_MULTI_CONN, which can be > advertised when the server promises cache consistency between > simultaneous clients (basically, rules that determine what FUA and > flush from one client are able to guarantee for

Re: [Qemu-block] [PATCH] nbd: Advertise multi-conn for shared read-only connections

2019-08-15 Thread Eric Blake
On 8/15/19 4:45 PM, John Snow wrote: > > > On 8/15/19 2:50 PM, Eric Blake wrote: >> The NBD specification defines NBD_FLAG_CAN_MULTI_CONN, which can be >> advertised when the server promises cache consistency between >> simultaneous clients (basically, rules that determine what FUA and >> flush

Re: [Qemu-block] [PATCH] nbd: Advertise multi-conn for shared read-only connections

2019-08-15 Thread John Snow
On 8/15/19 5:54 PM, Eric Blake wrote: > On 8/15/19 4:45 PM, John Snow wrote: >> >> >> On 8/15/19 2:50 PM, Eric Blake wrote: >>> The NBD specification defines NBD_FLAG_CAN_MULTI_CONN, which can be >>> advertised when the server promises cache consistency between >>> simultaneous clients

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/7] vmdk: Misc fixes

2019-08-15 Thread John Snow
On 8/15/19 11:36 AM, Max Reitz wrote: > I made the mistake of trying to run the iotests with all non-default > subformats our vmdk driver has to offer: > - monolithicFlat > - twoGbMaxExtentSparse > - twoGbMaxExtentFlat > - streamOptimized > > Many things broke, so this series fixes what I

Re: [Qemu-block] [Qemu-devel] [PATCH] nbd: Advertise multi-conn for shared read-only connections

2019-08-15 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20190815185024.7010-1-ebl...@redhat.com/ Hi, This series failed build test on s390x host. Please find the details below. === TEST SCRIPT BEGIN === #!/bin/bash # Testing script will be invoked under the git checkout with # HEAD pointing to a commit that

Re: [Qemu-block] [PATCH v5 1/6] iotests: allow Valgrind checking all QEMU processes

2019-08-15 Thread John Snow
On 7/19/19 12:30 PM, Andrey Shinkevich wrote: > With the '-valgrind' option, let all the QEMU processes be run under > the Valgrind tool. The Valgrind own parameters may be set with its > environment variable VALGRIND_OPTS, e.g. > VALGRIND_OPTS="--leak-check=yes" ./check -qcow2 -valgrind > or

[Qemu-block] [PULL 8/9] block/nbd: add cmdline and qapi parameter reconnect-delay

2019-08-15 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Reconnect will be implemented in the following commit, so for now, in semantics below, disconnect itself is a "serious error". Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Message-Id: <20190618114328.55249-5-vsement...@virtuozzo.com>

[Qemu-block] [PATCH] nbd: Advertise multi-conn for shared read-only connections

2019-08-15 Thread Eric Blake
The NBD specification defines NBD_FLAG_CAN_MULTI_CONN, which can be advertised when the server promises cache consistency between simultaneous clients (basically, rules that determine what FUA and flush from one client are able to guarantee for reads from another client). When we don't permit

Re: [Qemu-block] [PATCH] nbd: Advertise multi-conn for shared read-only connections

2019-08-15 Thread Richard W.M. Jones
On Thu, Aug 15, 2019 at 01:50:24PM -0500, Eric Blake wrote: > The NBD specification defines NBD_FLAG_CAN_MULTI_CONN, which can be > advertised when the server promises cache consistency between > simultaneous clients (basically, rules that determine what FUA and > flush from one client are able to

[Qemu-block] [PULL 6/9] block/nbd: use non-blocking io channel for nbd negotiation

2019-08-15 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy No reason to use blocking channel for negotiation and we'll benefit in further reconnect feature, as qio_channel reads and writes will do qemu_coroutine_yield while waiting for io completion. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake

[Qemu-block] [PULL 3/9] block/stream: use BDRV_REQ_PREFETCH

2019-08-15 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy This helps to avoid extra io, allocations and memory copying. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20190725100550.33801-3-vsement...@virtuozzo.com> Reviewed-by: Stefan Hajnoczi [eblake: fix comment grammar] Signed-off-by: Eric Blake ---

[Qemu-block] [PULL 7/9] block/nbd: move from quit to state

2019-08-15 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy To implement reconnect we need several states for the client: CONNECTED, QUIT and two different CONNECTING states. CONNECTING states will be added in the following patches. This patch implements CONNECTED and QUIT. QUIT means, that we should close the

[Qemu-block] [PULL 4/9] nbd: improve CMD_CACHE: use BDRV_REQ_PREFETCH

2019-08-15 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy This helps to avoid extra io, allocations and memory copying. We assume here that CMD_CACHE is always used with copy-on-read, as otherwise it's a noop. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20190725100550.33801-4-vsement...@virtuozzo.com>

[Qemu-block] [PULL 2/9] block: implement BDRV_REQ_PREFETCH

2019-08-15 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Do effective copy-on-read request when we don't need data actually. It will be used for block-stream and NBD_CMD_CACHE. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20190725100550.33801-2-vsement...@virtuozzo.com> Reviewed-by: Stefan Hajnoczi

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters

2019-08-15 Thread Markus Armbruster
Kevin Wolf writes: > Am 15.08.2019 um 18:07 hat John Snow geschrieben: >> >> >> On 8/15/19 6:49 AM, Kevin Wolf wrote: >> > Am 14.08.2019 um 21:27 hat John Snow geschrieben: >> >> >> >> >> >> On 8/14/19 6:07 AM, Vladimir Sementsov-Ogievskiy wrote: >> >>> To get rid of implicit filters related

Re: [Qemu-block] [PATCH for-4.2 02/13] qcow2: Keep unknown extra snapshot data

2019-08-15 Thread Max Reitz
On 31.07.19 10:54, Max Reitz wrote: > On 30.07.19 19:56, Eric Blake wrote: >> On 7/30/19 12:24 PM, Max Reitz wrote: [...] >>> +if (sn->extra_data_size > sizeof(extra)) { >>> +/* Store unknown extra data */ >>> +size_t unknown_extra_data_size = >>> +

Re: [Qemu-block] [PATCH v5 4/6] iotests: Valgrind fails with nonexistent directory

2019-08-15 Thread John Snow
On 7/19/19 12:30 PM, Andrey Shinkevich wrote: > The Valgrind uses the exported variable TMPDIR and fails if the > directory does not exist. Let us exclude such a test case from > being run under the Valgrind and notify the user of it. > > Suggested-by: Kevin Wolf > Signed-off-by: Andrey

Re: [Qemu-block] [PATCH v5 3/6] iotests: Add casenotrun report to bash tests

2019-08-15 Thread John Snow
On 7/19/19 12:30 PM, Andrey Shinkevich wrote: > The new function _casenotrun() is to be invoked if a test case cannot > be run for some reason. The user will be notified by a message passed > to the function. > Oh, I assume this is a sub-test granularity; if we need to skip individual items.

Re: [Qemu-block] [PATCH v5 2/6] iotests: exclude killed processes from running under Valgrind

2019-08-15 Thread John Snow
On 7/19/19 12:30 PM, Andrey Shinkevich wrote: > The Valgrind tool fails to manage its termination when QEMU raises the > signal SIGKILL in the multi-threaded process. The bug has been > reported to the Valgrind maintainers and was registered as Bug 409141. > Let's exclude such test cases

Re: [Qemu-block] [PATCH v5 6/6] iotests: extend sleeping time under Valgrind

2019-08-15 Thread John Snow
On 7/19/19 12:30 PM, Andrey Shinkevich wrote: > To synchronize the time when QEMU is running longer under the Valgrind, > increase the sleeping time in the test 247. > > Signed-off-by: Andrey Shinkevich > Reviewed-by: Vladimir Sementsov-Ogievskiy > --- > tests/qemu-iotests/247 | 6 +- >

Re: [Qemu-block] [PATCH v5 5/6] iotests: extended timeout under Valgrind

2019-08-15 Thread John Snow
On 7/19/19 12:30 PM, Andrey Shinkevich wrote: > As the iotests run longer under the Valgrind, the QEMU_COMM_TIMEOUT is > to be increased in the test cases 028, 183 and 192 when running under > the Valgrind. > > Suggested-by: Roman Kagan > Signed-off-by: Andrey Shinkevich > Reviewed-by: