Re: [Qemu-block] [PATCH v6 2/2] qapi: add block latency histogram interface

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
11.03.2018 06:20, Eric Blake wrote: On 03/09/2018 10:52 AM, Vladimir Sementsov-Ogievskiy wrote: Set (and clear) histogram through new command block-latency-histogram-set and show new statistics in query-blockstats results. Signed-off-by: Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [PATCH 6/9] nbd: BLOCK_STATUS for standard get_block_status function: client part

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
16.02.2018 23:40, Eric Blake wrote: On 02/15/2018 07:51 AM, Vladimir Sementsov-Ogievskiy wrote: Minimal realization: only one extent in server answer is supported. Flag NBD_CMD_FLAG_REQ_ONE is used to force this behavior. Tests 140, 147 and 205 are fixed due to now server failed on searching

Re: [Qemu-block] [PATCH 4/5] block/blkreplay: Remove protocol-related fields

2018-03-12 Thread Pavel Dovgalyuk
> From: Fabiano Rosas [mailto:faro...@linux.vnet.ibm.com] > The blkreplay driver is not a protocol so it should implement bdrv_open > instead of bdrv_file_open and not provide a protocol_name. > > Attempts to invoke this driver using protocol syntax > (i.e. blkreplay:) will now fail gracefully: >

Re: [Qemu-block] [PATCH 6/9] nbd: BLOCK_STATUS for standard get_block_status function: client part

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
16.02.2018 23:40, Eric Blake wrote: On 02/15/2018 07:51 AM, Vladimir Sementsov-Ogievskiy wrote: Minimal realization: only one extent in server answer is supported. Flag NBD_CMD_FLAG_REQ_ONE is used to force this behavior. Tests 140, 147 and 205 are fixed due to now server failed on searching

Re: [Qemu-block] [PATCH 6/9] nbd: BLOCK_STATUS for standard get_block_status function: client part

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
16.02.2018 23:40, Eric Blake wrote: On 02/15/2018 07:51 AM, Vladimir Sementsov-Ogievskiy wrote: Minimal realization: only one extent in server answer is supported. Flag NBD_CMD_FLAG_REQ_ONE is used to force this behavior. Tests 140, 147 and 205 are fixed due to now server failed on searching

Re: [Qemu-block] [Qemu-devel] [PATCH 8/9] iotests: add file_path helper

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
20.02.2018 08:42, Jeff Cody wrote: On Fri, Feb 16, 2018 at 02:46:35PM -0600, Eric Blake wrote: On 02/15/2018 07:51 AM, Vladimir Sementsov-Ogievskiy wrote: Simple way to have auto generated filenames with auto clenup. Like s/clenup/cleanup/ FilePath but without using 'with' statement and

[Qemu-block] [PATCH v8 9/9] iotest 134: test cluster-misaligned encrypted write

2018-03-12 Thread Anton Nefedov
COW (even empty/zero) areas require encryption too Signed-off-by: Anton Nefedov Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia --- tests/qemu-iotests/134 | 9 +

[Qemu-block] [PATCH v8 8/9] qcow2: skip writing zero buffers to empty COW areas

2018-03-12 Thread Anton Nefedov
If COW areas of the newly allocated clusters are zeroes on the backing image, efficient bdrv_write_zeroes(flags=BDRV_REQ_ALLOCATE) can be used on the whole cluster instead of writing explicit zero buffers later in perform_cow(). iotest 060: write to the discarded cluster does not trigger COW

Re: [Qemu-block] [PATCH 1/6] luks: Separate image file creation from formatting

2018-03-12 Thread Daniel P . Berrangé
On Fri, Mar 09, 2018 at 06:27:08PM +0100, Kevin Wolf wrote: > The crypto driver used to create the image file in a callback from the > crypto subsystem. If we want to implement .bdrv_co_create, this needs to > go away because that callback will get a reference to an already > existing block node.

Re: [Qemu-block] [PATCH 3/6] luks: Support .bdrv_co_create

2018-03-12 Thread Kevin Wolf
Am 12.03.2018 um 12:50 hat Daniel P. Berrangé geschrieben: > On Mon, Mar 12, 2018 at 12:47:21PM +0100, Kevin Wolf wrote: > > Am 12.03.2018 um 12:40 hat Daniel P. Berrangé geschrieben: > > > On Fri, Mar 09, 2018 at 06:27:10PM +0100, Kevin Wolf wrote: > > > > This adds the .bdrv_co_create driver

Re: [Qemu-block] [Qemu-devel] [PATCH v6 0/2] block latency histogram

2018-03-12 Thread Stefan Hajnoczi
On Fri, Mar 09, 2018 at 07:52:10PM +0300, Vladimir Sementsov-Ogievskiy wrote: > v6: > > Use correct header qapi/qapi-builtin-types.h, to fix build again. > Sorry for spam =( > > > v5: > > Revert to v3 and just add qapi-types.h header. > > > v4: > > Move block_latency_histogram_set from

[Qemu-block] [PATCH v8 2/9] blkverify: set supported write/zero flags

2018-03-12 Thread Anton Nefedov
Signed-off-by: Anton Nefedov Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/blkverify.c | 9 + 1 file changed, 9 insertions(+) diff --git a/block/blkverify.c b/block/blkverify.c index 331365b..de2fdc1

Re: [Qemu-block] [PATCH 6/6] qemu-iotests: Test luks QMP image creation

2018-03-12 Thread Daniel P . Berrangé
On Fri, Mar 09, 2018 at 06:27:13PM +0100, Kevin Wolf wrote: > Signed-off-by: Kevin Wolf > --- > tests/qemu-iotests/208 | 211 > +++ > tests/qemu-iotests/208.out | 136 > tests/qemu-iotests/common.rc |

Re: [Qemu-block] [PATCH 1/6] luks: Separate image file creation from formatting

2018-03-12 Thread Daniel P . Berrangé
On Mon, Mar 12, 2018 at 12:43:47PM +0100, Kevin Wolf wrote: > Am 12.03.2018 um 12:35 hat Daniel P. Berrangé geschrieben: > > On Fri, Mar 09, 2018 at 06:27:08PM +0100, Kevin Wolf wrote: > > > The crypto driver used to create the image file in a callback from the > > > crypto subsystem. If we want

Re: [Qemu-block] [PATCH 5/6] luks: Catch integer overflow for huge sizes

2018-03-12 Thread Daniel P . Berrangé
On Fri, Mar 09, 2018 at 06:27:12PM +0100, Kevin Wolf wrote: > When you request an image size close to UINT64_MAX, the addition of the > crypto header may cause an integer overflow. Catch it instead of > silently truncating the image size. > > Signed-off-by: Kevin Wolf > --- >

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] scsi: add block job opblockers for scsi-block

2018-03-12 Thread Kevin Wolf
Am 12.03.2018 um 12:10 hat Paolo Bonzini geschrieben: > On 12/02/2018 15:50, Paolo Bonzini wrote: > > On 12/02/2018 15:48, Kevin Wolf wrote: > >> Am 12.02.2018 um 15:32 hat Paolo Bonzini geschrieben: > >>> Okay, we are in agreement about this and you expressed very well why I > >>> (at the gut

[Qemu-block] [PATCH v8 0/9] qcow2: cluster space preallocation

2018-03-12 Thread Anton Nefedov
v8: - patch 1: flags setup moved to a proper place - new patch 3 for quorum driver - patch 4: expanded the flag description and minor relocations of sanity checks - patch 5: function renamed - patch 6: minor rebase-related changes - patch 7: quorum added -

Re: [Qemu-block] [PATCH 1/6] luks: Separate image file creation from formatting

2018-03-12 Thread Kevin Wolf
Am 12.03.2018 um 12:35 hat Daniel P. Berrangé geschrieben: > On Fri, Mar 09, 2018 at 06:27:08PM +0100, Kevin Wolf wrote: > > The crypto driver used to create the image file in a callback from the > > crypto subsystem. If we want to implement .bdrv_co_create, this needs to > > go away because that

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] block: let blk_add/remove_aio_context_notifier() tolerate BDS changes

2018-03-12 Thread Eric Blake
On 03/12/2018 06:27 AM, Stefan Hajnoczi wrote: On Fri, Mar 09, 2018 at 09:56:44AM -0600, Eric Blake wrote: On 03/06/2018 02:48 PM, Stefan Hajnoczi wrote: Commit 2019ba0a0197 ("block: Add AioContextNotifier functions to BB") added blk_add/remove_aio_context_notifier() and implemented them by

Re: [Qemu-block] [PATCH 3/6] luks: Support .bdrv_co_create

2018-03-12 Thread Eric Blake
On 03/12/2018 06:47 AM, Kevin Wolf wrote: +## +{ 'struct': 'BlockdevCreateOptionsLUKS', + 'data': { 'file': 'BlockdevRef', +'qcrypto': 'QCryptoBlockCreateOptionsLUKS', +'size': 'size' } } s/qcrypto/crypto/ in this field. I do wonder

[Qemu-block] [PATCH v8 5/9] block: treat BDRV_REQ_ALLOCATE as serialising

2018-03-12 Thread Anton Nefedov
The idea is that ALLOCATE requests may overlap with other requests. Reuse the existing block layer infrastructure for serialising requests. Use the following approach: - mark ALLOCATE serialising, so subsequent requests to the area wait - ALLOCATE request itself must never wait if another

Re: [Qemu-block] [PATCH v2] block: make BDRV_POLL_WHILE() re-entrancy safe

2018-03-12 Thread Stefan Hajnoczi
On Wed, Mar 07, 2018 at 12:46:19PM +, Stefan Hajnoczi wrote: > Nested BDRV_POLL_WHILE() calls can occur. Currently > assert(!wait_->wakeup) fails in AIO_WAIT_WHILE() when this happens. > > This patch converts the bool wait_->need_kick flag to an unsigned > wait_->num_waiters counter. > >

Re: [Qemu-block] [PATCH 3/6] luks: Support .bdrv_co_create

2018-03-12 Thread Daniel P . Berrangé
On Fri, Mar 09, 2018 at 06:27:10PM +0100, Kevin Wolf wrote: > This adds the .bdrv_co_create driver callback to luks, which enables > image creation over QMP. > > Signed-off-by: Kevin Wolf > --- > qapi/block-core.json | 17 - > block/crypto.c | 34

Re: [Qemu-block] [PATCH 3/6] luks: Support .bdrv_co_create

2018-03-12 Thread Daniel P . Berrangé
On Mon, Mar 12, 2018 at 12:47:21PM +0100, Kevin Wolf wrote: > Am 12.03.2018 um 12:40 hat Daniel P. Berrangé geschrieben: > > On Fri, Mar 09, 2018 at 06:27:10PM +0100, Kevin Wolf wrote: > > > This adds the .bdrv_co_create driver callback to luks, which enables > > > image creation over QMP. > > >

Re: [Qemu-block] [Qemu-devel] [PULL 00/56] Block layer patches

2018-03-12 Thread Peter Maydell
On 9 March 2018 at 16:18, Kevin Wolf wrote: > The following changes since commit d9bbfea646e86426d549bd612cd9f91e49aa50c2: > > Merge remote-tracking branch 'remotes/riscv/tags/riscv-qemu-upstream-v8.2' > into staging (2018-03-09 10:58:57 +) > > are available in the git

Re: [Qemu-block] [PATCH 3/7] qcow: Support .bdrv_co_create

2018-03-12 Thread Kevin Wolf
Am 09.03.2018 um 22:58 hat Eric Blake geschrieben: > On 03/09/2018 03:46 PM, Kevin Wolf wrote: > > This adds the .bdrv_co_create driver callback to qcow, which > > enables image creation over QMP. > > > > Signed-off-by: Kevin Wolf > > --- > > qapi/block-core.json | 21 +-

Re: [Qemu-block] [PATCH v3 0/7] block: Handle null backing link

2018-03-12 Thread Max Reitz
On 2018-03-10 23:34, Eric Blake wrote: > On 02/24/2018 09:40 AM, Max Reitz wrote: >> Currently, we try to rewrite every occurrence of "backing": null into >> "backing": "" in qmp_blockdev_add().  However, that breaks using the >> same "backing": null construction in json:{} file names (which do

[Qemu-block] [PATCH v2 6/8] iotests.py: tiny refactor: move system imports up

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- v2: add Eric's r-b tests/qemu-iotests/iotests.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/iotests.py

[Qemu-block] [PATCH v2 1/8] nbd/server: add nbd_opt_invalid helper

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
NBD_REP_ERR_INVALID is often parameter to nbd_opt_drop and it would be used more in following patches. So, let's add a helper. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- v2: add Eric's r-b nbd/server.c | 50

[Qemu-block] [PATCH v2 3/8] nbd: BLOCK_STATUS for standard get_block_status function: server part

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
Minimal realization: only one extent in server answer is supported. Signed-off-by: Vladimir Sementsov-Ogievskiy --- v2: - constants and type defs were splitted out by Eric, except for NBD_META_ID_BASE_ALLOCATION - add nbd_opt_skip, to skip meta query

[Qemu-block] [PATCH v2 7/8] iotests: add file_path helper

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
Simple way to have auto generated filenames with auto cleanup. Like FilePath but without using 'with' statement and without additional indentation of the whole test. Signed-off-by: Vladimir Sementsov-Ogievskiy --- v2: fix type in commit message

Re: [Qemu-block] [Qemu-devel] [PATCH v5 00/21] blockjobs: add explicit job management

2018-03-12 Thread no-reply
Hi, This series seems to have some coding style problems. See output below for more information: Type: series Message-id: 20180310082746.24198-1-js...@redhat.com Subject: [Qemu-devel] [PATCH v5 00/21] blockjobs: add explicit job management === TEST SCRIPT BEGIN === #!/bin/bash BASE=base n=1

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/6] luks: Implement .bdrv_co_create

2018-03-12 Thread no-reply
Hi, This series seems to have some coding style problems. See output below for more information: Type: series Message-id: 20180312150218.1314-1-kw...@redhat.com Subject: [Qemu-devel] [PATCH v2 0/6] luks: Implement .bdrv_co_create === TEST SCRIPT BEGIN === #!/bin/bash BASE=base n=1 total=$(git

Re: [Qemu-block] [PATCH v5 01/21] blockjobs: fix set-speed kick

2018-03-12 Thread Jeff Cody
On Sat, Mar 10, 2018 at 03:27:26AM -0500, John Snow wrote: > If speed is '0' it's not actually "less than" the previous speed. > Kick the job in this case too. > > Signed-off-by: John Snow > Reviewed-by: Eric Blake > Reviewed-by: Kevin Wolf

Re: [Qemu-block] [PATCH 1/5] block/replication: Remove protocol_name field

2018-03-12 Thread Max Reitz
On 2018-03-09 19:21, Fabiano Rosas wrote: > The replication driver is only selected explicitly (via > driver=replication,mode=primary,...) so it is not a protocol driver. That's not really the point. It isn't a protocol driver because it has a "file" child (opened in replication_open). All in

[Qemu-block] [PATCH v2 5/6] luks: Catch integer overflow for huge sizes

2018-03-12 Thread Kevin Wolf
When you request an image size close to UINT64_MAX, the addition of the crypto header may cause an integer overflow. Catch it instead of silently truncating the image size. Signed-off-by: Kevin Wolf Reviewed-by: Daniel P. Berrangé --- block/crypto.c | 5

Re: [Qemu-block] [PATCH 4/5] block/blkreplay: Remove protocol-related fields

2018-03-12 Thread Max Reitz
On 2018-03-09 19:22, Fabiano Rosas wrote: > The blkreplay driver is not a protocol so it should implement bdrv_open > instead of bdrv_file_open and not provide a protocol_name. > > Attempts to invoke this driver using protocol syntax > (i.e. blkreplay:) will now fail gracefully: > > $ qemu-img

Re: [Qemu-block] [PATCH 6/9] nbd: BLOCK_STATUS for standard get_block_status function: client part

2018-03-12 Thread Eric Blake
On 03/12/2018 08:13 AM, Vladimir Sementsov-Ogievskiy wrote: We should probably validate that the length field is a multiple of min_block (if a server tells us that all operations must be 512-byte aligned, then reports an extent that is smaller than 512 bytes, we have no way to ask for the

Re: [Qemu-block] [Qemu-devel] [PATCH] MAINTAINERS: add include/block/aio-wait.h

2018-03-12 Thread Eric Blake
On 03/12/2018 08:22 AM, Stefan Hajnoczi wrote: The include/block/aio-wait.h header file was added by commit 7719f3c968c59e1bcda7e177679dc765b59e578f ("block: extract AIO_WAIT_WHILE() from BlockDriverState") without updating MAINTAINERS. A shame checkpatch.pl and/or patchew still doesn't flag

Re: [Qemu-block] [PATCH 6/9] nbd: BLOCK_STATUS for standard get_block_status function: client part

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
16.02.2018 23:40, Eric Blake wrote: On 02/15/2018 07:51 AM, Vladimir Sementsov-Ogievskiy wrote: Minimal realization: only one extent in server answer is supported. Flag NBD_CMD_FLAG_REQ_ONE is used to force this behavior. Tests 140, 147 and 205 are fixed due to now server failed on searching

Re: [Qemu-block] [PATCH 2/5] block/quorum: Remove protocol-related fields

2018-03-12 Thread Max Reitz
On 2018-03-09 19:21, Fabiano Rosas wrote: > The quorum driver is not a protocol so it should implement bdrv_open > instead of bdrv_file_open and not provide a protocol_name. > > Attempts to invoke this driver using protocol syntax > (i.e. quorum:) will now fail gracefully: > > $ qemu-img info

Re: [Qemu-block] [PATCH 5/5] include/block/block_int: Document protocol related functions

2018-03-12 Thread Max Reitz
On 2018-03-09 19:22, Fabiano Rosas wrote: > Clarify that for protocols the brdv_file_open function is used instead > of bdrv_open and that protocol_name is expected to be set. > > Signed-off-by: Fabiano Rosas > --- > include/block/block_int.h | 6 ++ > 1 file

[Qemu-block] [PATCH v2 4/6] luks: Turn invalid assertion into check

2018-03-12 Thread Kevin Wolf
The .bdrv_getlength implementation of the crypto block driver asserted that the payload offset isn't after EOF. This is an invalid assertion to make as the image file could be corrupted. Instead, check it and return -EIO if the file is too small for the payload offset. Zero length images are

[Qemu-block] [PATCH v2 6/6] qemu-iotests: Test luks QMP image creation

2018-03-12 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- tests/qemu-iotests/209 | 210 +++ tests/qemu-iotests/209.out | 136 tests/qemu-iotests/common.rc | 2 +- tests/qemu-iotests/group | 1 + 4 files changed, 348

[Qemu-block] [PATCH v2 0/6] luks: Implement .bdrv_co_create

2018-03-12 Thread Kevin Wolf
This series implements the .bdrv_co_create callback for luks, adds an image creation test for it and contains some bonus fixes for bugs that the test triggered. v2: - Use QCryptoBlockCreateOptionsLUKS as the base type for BlockdevCreateOptionsLUKS [Dan] - Use INT64_MAX instead of UINT64_MAX as

[Qemu-block] [PATCH] MAINTAINERS: add include/block/aio-wait.h

2018-03-12 Thread Stefan Hajnoczi
The include/block/aio-wait.h header file was added by commit 7719f3c968c59e1bcda7e177679dc765b59e578f ("block: extract AIO_WAIT_WHILE() from BlockDriverState") without updating MAINTAINERS. Signed-off-by: Stefan Hajnoczi --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+)

Re: [Qemu-block] [PATCH 3/5] block/throttle: Remove protocol-related fields

2018-03-12 Thread Max Reitz
On 2018-03-09 19:21, Fabiano Rosas wrote: > The throttle driver is not a protocol so it should implement bdrv_open > instead of bdrv_file_open and not provide a protocol_name. > > Attempts to invoke this driver using protocol syntax > (i.e. throttle:) will now fail gracefully: > > $ qemu-img

[Qemu-block] [PATCH v2 2/6] luks: Create block_crypto_co_create_generic()

2018-03-12 Thread Kevin Wolf
Everything that refers to the protocol layer or QemuOpts is moved out of block_crypto_create_generic(), so that the remaining function is suitable to be called by a .bdrv_co_create implementation. LUKS is the only driver that actually implements the old interface, and we don't intend to use it in

[Qemu-block] [PATCH v2 1/6] luks: Separate image file creation from formatting

2018-03-12 Thread Kevin Wolf
The crypto driver used to create the image file in a callback from the crypto subsystem. If we want to implement .bdrv_co_create, this needs to go away because that callback will get a reference to an already existing block node. Move the image file creation to block_crypto_create_generic().

[Qemu-block] [PATCH v2 3/6] luks: Support .bdrv_co_create

2018-03-12 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to luks, which enables image creation over QMP. Signed-off-by: Kevin Wolf --- qapi/block-core.json | 17 - block/crypto.c | 34 ++ 2 files changed, 50 insertions(+), 1

[Qemu-block] [PULL 35/36] block/accounting: introduce latency histogram

2018-03-12 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Introduce latency histogram statics for block devices. For each accounted operation type, the latency region [0, +inf) is divided into subregions by several points. Then, calculate hits for each subregion. Signed-off-by: Vladimir

[Qemu-block] [PULL 36/36] qapi: add block latency histogram interface

2018-03-12 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Set (and clear) histograms through new command block-latency-histogram-set and show new statistics in query-blockstats results. For now, the command is marked experimental with prefix 'x-', to gain experience with the interface

Re: [Qemu-block] [PATCH 6/7] vhdx: Support .bdrv_co_create

2018-03-12 Thread Jeff Cody
On Fri, Mar 09, 2018 at 10:46:10PM +0100, Kevin Wolf wrote: > This adds the .bdrv_co_create driver callback to vhdx, which > enables image creation over QMP. > > Signed-off-by: Kevin Wolf > --- > qapi/block-core.json | 37 ++- > block/vhdx.c | 174 >

[Qemu-block] [PULL 11/36] block: Deprecate "backing": ""

2018-03-12 Thread Eric Blake
From: Max Reitz We have a clear replacement, so let's deprecate it. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia Message-Id: <20180224154033.29559-8-mre...@redhat.com> Signed-off-by:

[Qemu-block] [PULL 07/36] qapi: Replace qobject_to_X(o) by qobject_to(X, o)

2018-03-12 Thread Eric Blake
From: Max Reitz This patch was generated using the following Coccinelle script: @@ expression Obj; @@ ( - qobject_to_qnum(Obj) + qobject_to(QNum, Obj) | - qobject_to_qstring(Obj) + qobject_to(QString, Obj) | - qobject_to_qdict(Obj) + qobject_to(QDict, Obj) | -

Re: [Qemu-block] [Qemu-devel] [PATCH v3 0/2] block latency histogram

2018-03-12 Thread no-reply
Hi, This series failed docker-build@min-glib build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 20180309144918.44975-1-vsement...@virtuozzo.com Subject: [Qemu-devel] [PATCH v3 0/2]

Re: [Qemu-block] [PATCH v5 10/21] blockjobs: add NULL state

2018-03-12 Thread Kevin Wolf
Am 10.03.2018 um 09:27 hat John Snow geschrieben: > Add a new state that specifically demarcates when we begin to permanently > demolish a job after it has performed all work. This makes the transition > explicit in the STM table and highlights conditions under which a job may > be demolished. >

Re: [Qemu-block] [PATCH v2 0/8] nbd block status base:allocation

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
12.03.2018 18:21, Vladimir Sementsov-Ogievskiy wrote: Hi all. Here is minimal realization of base:allocation context of NBD block-status extension, which allows to get block status through NBD. v2 changes are in each patch after "---" line. Forget to mention the main thing: It is based on:

[Qemu-block] [PATCH v2 8/8] iotests: new test 209 for NBD BLOCK_STATUS

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- v2: mv 206 209, make test file executable tests/qemu-iotests/209 | 34 ++ tests/qemu-iotests/209.out | 2 ++ tests/qemu-iotests/group | 1 + 3 files changed, 37 insertions(+)

Re: [Qemu-block] [PATCH 1/2] block: let blk_add/remove_aio_context_notifier() tolerate BDS changes

2018-03-12 Thread Max Reitz
On 2018-03-06 21:48, Stefan Hajnoczi wrote: > Commit 2019ba0a0197 ("block: Add AioContextNotifier functions to BB") > added blk_add/remove_aio_context_notifier() and implemented them by > passing through the bdrv_*() equivalent. > > This doesn't work across bdrv_append(), which detaches child->bs

[Qemu-block] [PATCH v2 5/8] nbd: BLOCK_STATUS for standard get_block_status function: client part

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
Minimal realization: only one extent in server answer is supported. Flag NBD_CMD_FLAG_REQ_ONE is used to force this behavior. Signed-off-by: Vladimir Sementsov-Ogievskiy --- v2: - drop iotests changes, as server is fixed in 03 - rebase to byte-based block status

[Qemu-block] [PATCH v2 2/8] nbd/server: add nbd_read_opt_name helper

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
Add helper to read name in format: uint32 len (<= NBD_MAX_NAME_SIZE) len bytes string (not 0-terminated) The helper would be reused in following patch. Signed-off-by: Vladimir Sementsov-Ogievskiy --- v2: splitted and changed a lot helper from larger patch

Re: [Qemu-block] [PATCH v10 05/12] migration: introduce postcopy-only pending

2018-03-12 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > There would be savevm states (dirty-bitmap) which can migrate only in > postcopy stage. The corresponding pending is introduced here. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- >

Re: [Qemu-block] [PATCH v5 10/21] blockjobs: add NULL state

2018-03-12 Thread John Snow
On 03/12/2018 11:28 AM, Kevin Wolf wrote: > Am 10.03.2018 um 09:27 hat John Snow geschrieben: >> Add a new state that specifically demarcates when we begin to permanently >> demolish a job after it has performed all work. This makes the transition >> explicit in the STM table and highlights

[Qemu-block] [PULL 1/1] block: make BDRV_POLL_WHILE() re-entrancy safe

2018-03-12 Thread Stefan Hajnoczi
Nested BDRV_POLL_WHILE() calls can occur. Currently assert(!wait_->wakeup) fails in AIO_WAIT_WHILE() when this happens. This patch converts the bool wait_->need_kick flag to an unsigned wait_->num_waiters counter. Nesting works correctly because outer AIO_WAIT_WHILE() callers evaluate the

Re: [Qemu-block] [PATCH v10 10/12] migration: add postcopy migration of dirty bitmaps

2018-03-12 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > Postcopy migration of dirty bitmaps. Only named dirty bitmaps are migrated. > > If destination qemu is already containing a dirty bitmap with the same name > as a migrated bitmap (for the same node), then, if their granularities

Re: [Qemu-block] [PATCH v2 3/6] luks: Support .bdrv_co_create

2018-03-12 Thread Daniel P . Berrangé
On Mon, Mar 12, 2018 at 04:02:15PM +0100, Kevin Wolf wrote: > This adds the .bdrv_co_create driver callback to luks, which enables > image creation over QMP. > > Signed-off-by: Kevin Wolf > --- > qapi/block-core.json | 17 - > block/crypto.c | 34

Re: [Qemu-block] [PATCH v2 4/6] luks: Turn invalid assertion into check

2018-03-12 Thread Daniel P . Berrangé
On Mon, Mar 12, 2018 at 04:02:16PM +0100, Kevin Wolf wrote: > The .bdrv_getlength implementation of the crypto block driver asserted > that the payload offset isn't after EOF. This is an invalid assertion to > make as the image file could be corrupted. Instead, check it and return > -EIO if the

Re: [Qemu-block] [PATCH v5 10/21] blockjobs: add NULL state

2018-03-12 Thread John Snow
On 03/12/2018 12:07 PM, Kevin Wolf wrote: > Am 12.03.2018 um 16:41 hat John Snow geschrieben: >> On 03/12/2018 11:28 AM, Kevin Wolf wrote: >>> Am 10.03.2018 um 09:27 hat John Snow geschrieben: Add a new state that specifically demarcates when we begin to permanently demolish a job

[Qemu-block] [PATCH v2 0/8] nbd block status base:allocation

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
Hi all. Here is minimal realization of base:allocation context of NBD block-status extension, which allows to get block status through NBD. v2 changes are in each patch after "---" line. Vladimir Sementsov-Ogievskiy (8): nbd/server: add nbd_opt_invalid helper nbd/server: add

[Qemu-block] [PATCH v2 4/8] block/nbd-client: save first fatal error in nbd_iter_error

2018-03-12 Thread Vladimir Sementsov-Ogievskiy
It is ok, that fatal error hides previous not fatal, but hiding first fatal error is a bad feature. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- v2: add Eric's r-b block/nbd-client.c | 4 +++- 1 file changed, 3

Re: [Qemu-block] [Qemu-devel] [PATCH 0/2] block/ssh: Implement .bdrv_refresh_filename()

2018-03-12 Thread John Snow
On 03/07/2018 09:12 PM, Fam Zheng wrote: > On Wed, 03/07 12:50, John Snow wrote: >> It's something I'd like to see patchew do, actually: >> >> "Here's a list of what's on the list that has no reviews or NACKs, and >> needs some love" > > It's not hard to define a search condition for that: > >

Re: [Qemu-block] [PATCH v2 6/6] qemu-iotests: Test luks QMP image creation

2018-03-12 Thread Daniel P . Berrangé
On Mon, Mar 12, 2018 at 04:02:18PM +0100, Kevin Wolf wrote: > Signed-off-by: Kevin Wolf > --- > tests/qemu-iotests/209 | 210 > +++ > tests/qemu-iotests/209.out | 136 > tests/qemu-iotests/common.rc |

[Qemu-block] [PULL 0/1] Block patches

2018-03-12 Thread Stefan Hajnoczi
The following changes since commit e4ae62b802cec437f877f2cadc4ef059cc0eca76: Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging (2018-03-09 17:28:16 +) are available in the Git repository at: git://github.com/stefanha/qemu.git tags/block-pull-request

Re: [Qemu-block] [PATCH v5 10/21] blockjobs: add NULL state

2018-03-12 Thread Kevin Wolf
Am 12.03.2018 um 16:41 hat John Snow geschrieben: > On 03/12/2018 11:28 AM, Kevin Wolf wrote: > > Am 10.03.2018 um 09:27 hat John Snow geschrieben: > >> Add a new state that specifically demarcates when we begin to permanently > >> demolish a job after it has performed all work. This makes the

Re: [Qemu-block] [PATCH v10 11/12] iotests: add dirty bitmap migration test

2018-03-12 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > The test starts two vms (vm_a, vm_b), create dirty bitmap in > the first one, do several writes to corresponding device and > then migrate vm_a to vm_b. > Signed-off-by: Vladimir Sementsov-Ogievskiy >

Re: [Qemu-block] [PATCH v5 00/21] blockjobs: add explicit job management

2018-03-12 Thread Kevin Wolf
Am 10.03.2018 um 09:27 hat John Snow geschrieben: > This series seeks to address two distinct but closely related issues > concerning the job management API. > > (1) For jobs that complete when a monitor is not attached and receiving > events or notifications, there's no way to discern the

Re: [Qemu-block] [PATCH 2/2] iotests: add 208 nbd-server + blockdev-snapshot-sync test case

2018-03-12 Thread Max Reitz
On 2018-03-06 21:48, Stefan Hajnoczi wrote: > This test case adds an NBD server export and then invokes > blockdev-snapshot-sync, which changes the BlockDriverState node that the > NBD server's BlockBackend points to. This is an interesting scenario to > test and exercises the code path fixed by

[Qemu-block] [PATCH v8 1/9] mirror: inherit supported write/zero flags

2018-03-12 Thread Anton Nefedov
Signed-off-by: Anton Nefedov --- block/mirror.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/block/mirror.c b/block/mirror.c index f5bf620..2fb786f 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1100,6 +1100,15 @@ static BlockDriver

[Qemu-block] [PATCH v8 7/9] block: support BDRV_REQ_ALLOCATE in passthrough drivers

2018-03-12 Thread Anton Nefedov
Support the flag if the underlying BDS supports it Signed-off-by: Anton Nefedov Reviewed-by: Alberto Garcia --- block/blkdebug.c | 3 ++- block/blkverify.c | 2 +- block/mirror.c | 2 +- block/quorum.c | 3 ++- block/raw-format.c | 3

[Qemu-block] [PATCH v8 6/9] file-posix: support BDRV_REQ_ALLOCATE

2018-03-12 Thread Anton Nefedov
Current write_zeroes implementation is good enough to satisfy this flag too Signed-off-by: Anton Nefedov Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/file-posix.c | 10 +- 1 file changed, 9

Re: [Qemu-block] [PATCH 4/6] luks: Turn invalid assertion into check

2018-03-12 Thread Daniel P . Berrangé
On Fri, Mar 09, 2018 at 06:27:11PM +0100, Kevin Wolf wrote: > The .bdrv_getlength implementation of the crypto block driver asserted > that the payload offset isn't after EOF. This is an invalid assertion to > make as the image file could be corrupted. Instead, check it and return > -EIO if the

Re: [Qemu-block] [PATCH 2/6] luks: Create block_crypto_co_create_generic()

2018-03-12 Thread Daniel P . Berrangé
On Fri, Mar 09, 2018 at 06:27:09PM +0100, Kevin Wolf wrote: > Everything that refers to the protocol layer or QemuOpts is moved out of > block_crypto_create_generic(), so that the remaining function is > suitable to be called by a .bdrv_co_create implementation. > > LUKS is the only driver that

Re: [Qemu-block] [PATCH 3/6] luks: Support .bdrv_co_create

2018-03-12 Thread Kevin Wolf
Am 12.03.2018 um 12:40 hat Daniel P. Berrangé geschrieben: > On Fri, Mar 09, 2018 at 06:27:10PM +0100, Kevin Wolf wrote: > > This adds the .bdrv_co_create driver callback to luks, which enables > > image creation over QMP. > > > > Signed-off-by: Kevin Wolf > > --- > >

Re: [Qemu-block] [Qemu-devel] [PULL 7/7] vl: introduce vm_shutdown()

2018-03-12 Thread John Snow
On 03/09/2018 08:19 AM, Stefan Hajnoczi wrote: > Commit 00d09fdbbae5f7864ce754913efc84c12fdf9f1a ("vl: pause vcpus before > stopping iothreads") and commit dce8921b2baaf95974af8176406881872067adfa > ("iothread: Stop threads before main() quits") tried to work around the > fact that emulation was

[Qemu-block] [PULL 10/36] block: Handle null backing link

2018-03-12 Thread Eric Blake
From: Max Reitz Instead of converting all "backing": null instances into "backing": "", handle a null value directly in bdrv_open_inherit(). This enables explicitly null backing links for json:{} filenames. Signed-off-by: Max Reitz Reviewed-by: Alberto

Re: [Qemu-block] [PATCH v10 12/12] iotests: add dirty bitmap postcopy test

2018-03-12 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > Test > - start two vms (vm_a, vm_b) > > - in a > - do writes from set A > - do writes from set B > - fix bitmap sha256 > - clear bitmap > - do writes from set A > - start migration > - than, in b > -

Re: [Qemu-block] [PATCH 1/7] parallels: Support .bdrv_co_create

2018-03-12 Thread Max Reitz
On 2018-03-09 22:46, Kevin Wolf wrote: > This adds the .bdrv_co_create driver callback to parallels, which > enables image creation over QMP. > > Signed-off-by: Kevin Wolf > --- > qapi/block-core.json | 18 - > block/parallels.c| 199 >

Re: [Qemu-block] [PATCH 2/7] qemu-iotests: Enable write tests for parallels

2018-03-12 Thread Max Reitz
On 2018-03-09 22:46, Kevin Wolf wrote: > Originally we added parallels as a read-only format to qemu-iotests > where we did just some tests with a binary image. Since then, write and > image creation support has been added to the driver, so we can now > enable it in _supported_fmt generic. > >

Re: [Qemu-block] [PATCH 3/7] qcow: Support .bdrv_co_create

2018-03-12 Thread Max Reitz
On 2018-03-09 22:46, Kevin Wolf wrote: > This adds the .bdrv_co_create driver callback to qcow, which > enables image creation over QMP. > > Signed-off-by: Kevin Wolf > --- > qapi/block-core.json | 21 +- > block/qcow.c | 196 >

[Qemu-block] [PATCH 2/3] vdi: Move file creation to vdi_co_create_opts

2018-03-12 Thread Max Reitz
Signed-off-by: Max Reitz --- block/vdi.c | 46 -- 1 file changed, 28 insertions(+), 18 deletions(-) diff --git a/block/vdi.c b/block/vdi.c index 0c8f8204ce..2a39b0ac98 100644 --- a/block/vdi.c +++ b/block/vdi.c @@ -721,9 +721,7 @@

[Qemu-block] [PATCH 0/3] vdi: Implement .bdrv_co_create

2018-03-12 Thread Max Reitz
I think cluster-size should not be exposed in QAPI (yet), but Kevin's patch does that, while I had these patches lying around, so here is my proposal. Max Reitz (3): vdi: Pull option parsing from vdi_co_create vdi: Move file creation to vdi_co_create_opts vdi: Implement .bdrv_co_create

[Qemu-block] [PATCH 1/3] vdi: Pull option parsing from vdi_co_create

2018-03-12 Thread Max Reitz
In preparation of QAPI-fying VDI image creation, we have to create a BlockdevCreateOptionsVdi type which is received by a (future) vdi_co_create(). vdi_co_create_opts() now converts the QemuOpts object into such a BlockdevCreateOptionsVdi object. The protocol-layer file is still created in

[Qemu-block] [PATCH 3/3] vdi: Implement .bdrv_co_create

2018-03-12 Thread Max Reitz
Signed-off-by: Max Reitz --- qapi/block-core.json | 2 +- block/vdi.c | 24 +++- 2 files changed, 20 insertions(+), 6 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index e7110c9a47..5c3186fe4f 100644 ---

Re: [Qemu-block] [PATCH v2 0/6] luks: Implement .bdrv_co_create

2018-03-12 Thread Kevin Wolf
Am 12.03.2018 um 16:02 hat Kevin Wolf geschrieben: > This series implements the .bdrv_co_create callback for luks, adds an > image creation test for it and contains some bonus fixes for bugs that > the test triggered. > > v2: > - Use QCryptoBlockCreateOptionsLUKS as the base type for >

Re: [Qemu-block] [Qemu-devel] [PATCH] MAINTAINERS: add include/block/aio-wait.h

2018-03-12 Thread Fam Zheng
On Mon, 03/12 09:05, Eric Blake wrote: > On 03/12/2018 08:22 AM, Stefan Hajnoczi wrote: > > The include/block/aio-wait.h header file was added by commit > > 7719f3c968c59e1bcda7e177679dc765b59e578f ("block: extract > > AIO_WAIT_WHILE() from BlockDriverState") without updating MAINTAINERS. > > A

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/8] nbd block status base:allocation

2018-03-12 Thread Eric Blake
On 03/12/2018 09:06 PM, no-re...@patchew.org wrote: Hi, This series failed build test on s390x host. Please find the details below. Type: series Message-id: 20180312152126.286890-1-vsement...@virtuozzo.com Subject: [Qemu-devel] [PATCH v2 0/8] nbd block status base:allocation CC

Re: [Qemu-block] [PATCH 2/7] qemu-iotests: Enable write tests for parallels

2018-03-12 Thread Jeff Cody
On Fri, Mar 09, 2018 at 10:46:06PM +0100, Kevin Wolf wrote: > Originally we added parallels as a read-only format to qemu-iotests > where we did just some tests with a binary image. Since then, write and > image creation support has been added to the driver, so we can now > enable it in

Re: [Qemu-block] [Qemu-devel] [PATCH v10 05/12] migration: introduce postcopy-only pending

2018-03-12 Thread John Snow
On 03/12/2018 11:30 AM, Dr. David Alan Gilbert wrote: > * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: >> There would be savevm states (dirty-bitmap) which can migrate only in >> postcopy stage. The corresponding pending is introduced here. >> >> Signed-off-by: Vladimir

Re: [Qemu-block] [PATCH 1/7] parallels: Support .bdrv_co_create

2018-03-12 Thread Jeff Cody
On Fri, Mar 09, 2018 at 10:46:05PM +0100, Kevin Wolf wrote: > This adds the .bdrv_co_create driver callback to parallels, which > enables image creation over QMP. > > Signed-off-by: Kevin Wolf > --- > qapi/block-core.json | 18 - > block/parallels.c| 199 >

  1   2   >