Re: [Qemu-block] [PATCH v3 18/19] nbd/client: Work around 3.0 bug for listing meta contexts

2019-01-16 Thread Eric Blake
On 1/16/19 9:43 AM, Vladimir Sementsov-Ogievskiy wrote: >> @@ -839,9 +842,25 @@ static int nbd_list_meta_contexts(QIOChannel *ioc, >> >> ret = nbd_receive_one_meta_context(ioc, NBD_OPT_LIST_META_CONTEXT, >> , NULL, errp); >> +if (ret

Re: [Qemu-block] [PATCH v3 15/19] nbd/client: Add nbd_receive_export_list()

2019-01-16 Thread Eric Blake
On 1/16/19 4:15 AM, Vladimir Sementsov-Ogievskiy wrote: >> This patch adds the low-level client code for grabbing the list >> of exports. It benefits from the recent refactoring patches, as >> well as a minor tweak of changing nbd_opt_go() to nbd_opt_info_or_go(), >> -/* Returns -1 if

Re: [Qemu-block] [PATCH v3 14/19] nbd/client: Pull out oldstyle size determination

2019-01-16 Thread Eric Blake
On 1/15/19 9:35 AM, Vladimir Sementsov-Ogievskiy wrote: > 12.01.2019 20:58, Eric Blake wrote: >> Another refactoring creating nbd_negotiate_finish_oldstyle() >> for further reuse during 'qemu-nbd --list'. >> >> Signed-off-by: Eric Blake >> Message-Id: <20181215135324.152629-18-ebl...@redhat.com>

Re: [Qemu-block] [Xen-devel] [PATCH] hw/block/xen: use proper format string for printing sectors

2019-01-16 Thread Andrew Cooper
On 16/01/2019 12:13, Alex Bennée wrote: > The %lu format string is different depending on the host architecture > which causes builds like the debian-armhf-cross build to fail. Use the > correct PRi64 format string. > > Signed-off-by: Alex Bennée > --- > hw/block/xen-block.c | 2 +- > 1 file

Re: [Qemu-block] [Qemu-devel] [PATCH v3 04/19] nbd/server: Hoist length check to qemp_nbd_server_add

2019-01-16 Thread Eric Blake
On 1/12/19 11:57 AM, Eric Blake wrote: s/qemp/qmp/ in the subject line > We only had two callers to nbd_export_new; qemu-nbd.c always > passed a valid offset/length pair (because it already checked > the file length, to ensure that offset was in bounds), while > blockdev-nbd always passed 0/-1.

Re: [Qemu-block] [PATCH v3 04/19] nbd/server: Hoist length check to qemp_nbd_server_add

2019-01-16 Thread Eric Blake
On 1/15/19 10:58 AM, Eric Blake wrote: > On 1/15/19 10:26 AM, Vladimir Sementsov-Ogievskiy wrote: > @size is not size of the image, but size of the export, so it may be less than dev_offset (qemu-nbd.c do "fd_size -= dev_offset" before "nbd_export_new(bs, dev_offset,

Re: [Qemu-block] [PATCH v4 10/10] iotests: test nbd reconnect

2019-01-16 Thread Eric Blake
On 7/31/18 12:30 PM, Vladimir Sementsov-Ogievskiy wrote: > Add test, which starts backup to nbd target and restarts nbd server > during backup. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > tests/qemu-iotests/220| 67 > +++ >

Re: [Qemu-block] [PATCH v4 09/10] block/nbd-client: nbd reconnect

2019-01-16 Thread Eric Blake
On 7/31/18 12:30 PM, Vladimir Sementsov-Ogievskiy wrote: > Implement reconnect. To achieve this: > > 1. add new modes: >connecting-wait: means, that reconnecting is in progress, and there > were small number of reconnect attempts, so all requests are > waiting for the connection. >

Re: [Qemu-block] [PATCH v4 06/10] block/nbd-client: move from quit to state

2019-01-16 Thread Daniel P . Berrangé
On Wed, Jan 16, 2019 at 10:25:03AM -0600, Eric Blake wrote: > [adding Dan] > > On 7/31/18 12:30 PM, Vladimir Sementsov-Ogievskiy wrote: > > To implement reconnect we need several states for the client: > > CONNECTED, QUIT and two CONNECTING states. CONNECTING states will > > be realized in the

Re: [Qemu-block] [PATCH v5 09/11] block: add lock/unlock range functions

2019-01-16 Thread Max Reitz
On 29.12.18 13:20, Vladimir Sementsov-Ogievskiy wrote: > From: Vladimir Sementsov-Ogievskiy > > Introduce lock/unlock range functionality, based on serialized > requests. This is needed to refactor backup, dropping local > tracked-request-like synchronization. > > Signed-off-by: Vladimir

Re: [Qemu-block] [PATCH v4 07/10] block/nbd-client: rename read_reply_co to connection_co

2019-01-16 Thread Eric Blake
On 7/31/18 12:30 PM, Vladimir Sementsov-Ogievskiy wrote: > This coroutine will serve nbd reconnects, so, rename it to be something > more generic. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/nbd-client.h | 4 ++-- > block/nbd-client.c | 24 > 2 files

Re: [Qemu-block] [PATCH v4 06/10] block/nbd-client: move from quit to state

2019-01-16 Thread Eric Blake
[adding Dan] On 7/31/18 12:30 PM, Vladimir Sementsov-Ogievskiy wrote: > To implement reconnect we need several states for the client: > CONNECTED, QUIT and two CONNECTING states. CONNECTING states will > be realized in the following patches. This patch implements CONNECTED > and QUIT. > > QUIT

Re: [Qemu-block] [PATCH v5 08/11] block/io: refactor wait_serialising_requests

2019-01-16 Thread Max Reitz
On 29.12.18 13:20, Vladimir Sementsov-Ogievskiy wrote: > Split out do_wait_serialising_requests with additional possibility to > not actually wait but just check, that there is something to wait for. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/io.c | 24

Re: [Qemu-block] [PATCH v3 7/9] qcow2: qcow2_co_preadv: skip using hd_qiov when possible

2019-01-16 Thread Alberto Garcia
On Tue 08 Jan 2019 06:06:53 PM CET, Vladimir Sementsov-Ogievskiy wrote: > qemu_iovec_memset has @offset parameter, so using hd_qiov for it is not > needed. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [PATCH v4 05/10] block/nbd-client: don't check ioc

2019-01-16 Thread Eric Blake
On 7/31/18 12:30 PM, Vladimir Sementsov-Ogievskiy wrote: > We have several paranoiac checks for ioc != NULL. But ioc may become s/paranoiac/paranoid/ sounds nicer, even if both forms are valid words. > NULL only on close, which should not happen during requests handling. > Also, we check ioc

Re: [Qemu-block] [PATCH v5 07/11] block: introduce backup-top filter driver

2019-01-16 Thread Max Reitz
On 29.12.18 13:20, Vladimir Sementsov-Ogievskiy wrote: > Backup-top filter does copy-before-write operation. It should be > inserted above active disk and has a target node for CBW, like the > following: > > +---+ > | Guest | > +---+---+ > |r,w > v >

Re: [Qemu-block] [PATCH v4 04/10] block/nbd-client: fix nbd_reply_chunk_iter_receive

2019-01-16 Thread Eric Blake
On 7/31/18 12:30 PM, Vladimir Sementsov-Ogievskiy wrote: > Use exported report, not the variable to be reused (should not really > matter). > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/nbd-client.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Eric Blake

Re: [Qemu-block] [PATCH v4 02/10] block/nbd: move connection code from block/nbd to block/nbd-client

2019-01-16 Thread Eric Blake
On 7/31/18 12:30 PM, Vladimir Sementsov-Ogievskiy wrote: > Keep all connection code in one file, to be able to implement reconnect > in further patches. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/nbd-client.h | 2 +- > block/nbd-client.c | 37

Re: [Qemu-block] [PATCH v4 03/10] block/nbd-client: split connection from initialization

2019-01-16 Thread Eric Blake
On 7/31/18 12:30 PM, Vladimir Sementsov-Ogievskiy wrote: > Split connection code to reuse it for reconnect. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/nbd-client.c | 33 - > 1 file changed, 24 insertions(+), 9 deletions(-) Reviewed-by: Eric

Re: [Qemu-block] [PATCH v3 18/19] nbd/client: Work around 3.0 bug for listing meta contexts

2019-01-16 Thread Vladimir Sementsov-Ogievskiy
12.01.2019 20:58, Eric Blake wrote: > Commit 3d068aff forgot to advertise available qemu: contexts > when the client requests a list with 0 queries. Furthermore, > 3.0 shipped with a qemu-img hack of x-dirty-bitmap (commit > 216ee365) that _silently_ acts as though the entire image is > clean if a

Re: [Qemu-block] [Qemu-devel] [PATCH 03/15] hw/ssi: Remove SSIBus from "qemu/typedefs.h"

2019-01-16 Thread Michael S. Tsirkin
On Wed, Jan 16, 2019 at 12:49:07PM +0100, Paolo Bonzini wrote: > On 16/01/19 12:34, Gerd Hoffmann wrote: > > Hi, > > > >> typedefs.h is useful to avoid rebuilding the world too often if a type > >> is used many times as a pointer, but rarely as a struct and rarely has > >> functions called on

Re: [Qemu-block] [PATCH v3 15/19] nbd/client: Add nbd_receive_export_list()

2019-01-16 Thread Eric Blake
On 1/16/19 4:15 AM, Vladimir Sementsov-Ogievskiy wrote: >> @@ -347,7 +350,8 @@ static int nbd_opt_go(QIOChannel *ioc, NBDExportInfo >> *info, Error **errp) >>* flags still 0 is a witness of a broken server. */ >> info->flags = 0; >> >> -trace_nbd_opt_go_start(info->name); >> +

[Qemu-block] [RFC] nbd: generalize usage of nbd_read

2019-01-16 Thread Vladimir Sementsov-Ogievskiy
We generally do very similar things around nbd_read: error_prepend, specifying, what we have tried to read and be_to_cpu conversion of integers. So, it seems reasonable to move common things to helper functions, which: 1. simplify code a bit 2. generalize nbd_read error descriptions, all starting

Re: [Qemu-block] [PATCH v3 05/19] nbd/server: Favor [u]int64_t over off_t

2019-01-16 Thread Eric Blake
On 1/16/19 2:23 AM, Vladimir Sementsov-Ogievskiy wrote: > > Interesting, decided to search a bit, about don't we have an overflow, > when adding request.offset to exp.dev_offset and found in > nbd_co_receive_request: > > if (request->from > client->exp->size || > request->from +

Re: [Qemu-block] [PATCH v2 0/2] throttle-groups: fix restart coroutine iothread race

2019-01-16 Thread Stefan Hajnoczi
On Mon, Jan 14, 2019 at 01:32:55PM +, Stefan Hajnoczi wrote: > v2: > * Added qemu-iotests test case [Kevin, Alberto] > * Use atomic operations instead of AioContext lock [Paolo] > > This patch series fixes a segfault upon device_del when virtio-scsi is used > with iothreads. > > Stefan

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/6] Acquire the AioContext during _realize()

2019-01-16 Thread Stefan Hajnoczi
On Mon, Jan 14, 2019 at 04:23:58PM +0200, Alberto Garcia wrote: > This series acquires the AioContext in the _realize() functions of > several devices before making use of their block backends. This fixes > at least a couple of crashes (in virtio-blk and scsi). The other > devices don't currently

Re: [Qemu-block] [PATCH v3 6/9] qcow2: qcow2_co_preadv: improve locking

2019-01-16 Thread Alberto Garcia
On Tue 08 Jan 2019 06:06:52 PM CET, Vladimir Sementsov-Ogievskiy wrote: > @@ -1891,39 +1892,38 @@ static coroutine_fn int > qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, > qemu_iovec_reset(_qiov); > qemu_iovec_concat(_qiov, qiov, bytes_done, cur_bytes); > > +

Re: [Qemu-block] [PATCH v5 06/11] iotests: prepare 055 to graph changes during backup job

2019-01-16 Thread Max Reitz
On 29.12.18 13:20, Vladimir Sementsov-Ogievskiy wrote: > Backup will append fleecing-hook node above source node, so, we can't > resume by device name (because resume don't search recursively through > backing chain). > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- >

Re: [Qemu-block] [PATCH v5 03/11] block: improve should_update_child

2019-01-16 Thread Max Reitz
On 14.01.19 17:13, Vladimir Sementsov-Ogievskiy wrote: > 14.01.2019 17:32, Max Reitz wrote: >> On 29.12.18 13:20, Vladimir Sementsov-Ogievskiy wrote: >>> As it already said in the comment, we don't want to create loops in >>> parent->child relations. So, when we try to append @to to @c, we should

Re: [Qemu-block] [PATCH v3 5/9] qcow2-threads: split out generic path

2019-01-16 Thread Alberto Garcia
On Tue 08 Jan 2019 06:06:51 PM CET, Vladimir Sementsov-Ogievskiy wrote: > Move generic part out of qcow2_co_do_compress, to reuse it for > encryption and rename things that would be shared with encryption path. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [PATCH v3 4/9] qcow2-threads: qcow2_co_do_compress: protect queuing by mutex

2019-01-16 Thread Alberto Garcia
On Tue 08 Jan 2019 06:06:50 PM CET, Vladimir Sementsov-Ogievskiy wrote: > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [PATCH v5 05/11] iotests: allow resume_drive by node name

2019-01-16 Thread Max Reitz
On 14.01.19 17:06, Vladimir Sementsov-Ogievskiy wrote: > 14.01.2019 17:46, Max Reitz wrote: >> On 29.12.18 13:20, Vladimir Sementsov-Ogievskiy wrote: >>> After node graph changes, we may not be able to resume_drive by device >>> name (backing files are not recursively searched). So, lets allow to

Re: [Qemu-block] [PATCH v5 01/11] block/backup: simplify backup_incremental_init_copy_bitmap

2019-01-16 Thread Max Reitz
On 14.01.19 15:48, Vladimir Sementsov-Ogievskiy wrote: > 14.01.2019 17:13, Max Reitz wrote: >> On 14.01.19 15:01, Vladimir Sementsov-Ogievskiy wrote: >>> 14.01.2019 16:10, Max Reitz wrote: On 29.12.18 13:20, Vladimir Sementsov-Ogievskiy wrote: > Simplify

Re: [Qemu-block] [PATCH] hw/block/xen: use proper format string for printing sectors

2019-01-16 Thread Philippe Mathieu-Daudé
On 1/16/19 1:19 PM, Paul Durrant wrote: >> -Original Message- >> From: Alex Bennée [mailto:alex.ben...@linaro.org] >> Sent: 16 January 2019 12:14 >> To: peter.mayd...@linaro.org >> Cc: qemu-de...@nongnu.org; Alex Bennée ; Stefano >> Stabellini ; Anthony Perard >> ; Paul Durrant ; Kevin >>

Re: [Qemu-block] [PATCH 1/1] block: Eliminate the S_1KiB, S_2KiB, ... macros

2019-01-16 Thread Alberto Garcia
On Fri 11 Jan 2019 08:14:01 PM CET, Markus Armbruster wrote: > We define 54 macros for the powers of two >= 1024. We use six, in six > macro definitions. Four of them could just as well use the common MiB > macro, so do that. The remaining two can't, because they get passed > to stringify.

Re: [Qemu-block] [PATCH] hw/block/xen: use proper format string for printing sectors

2019-01-16 Thread Paul Durrant
> -Original Message- > From: Alex Bennée [mailto:alex.ben...@linaro.org] > Sent: 16 January 2019 12:14 > To: peter.mayd...@linaro.org > Cc: qemu-de...@nongnu.org; Alex Bennée ; Stefano > Stabellini ; Anthony Perard > ; Paul Durrant ; Kevin > Wolf ; Max Reitz ; open list:X86 > ; open

Re: [Qemu-block] [PATCH] xen: Fix format string

2019-01-16 Thread Paul Durrant
> -Original Message- > From: Xen-devel [mailto:xen-devel-boun...@lists.xenproject.org] On Behalf > Of Paul Durrant > Sent: 16 January 2019 12:04 > To: 'Philippe Mathieu-Daudé' ; qemu-de...@nongnu.org > Cc: Kevin Wolf ; Stefano Stabellini > ; qemu-block@nongnu.org; Max Reitz > ; Anthony

[Qemu-block] [PATCH] hw/block/xen: use proper format string for printing sectors

2019-01-16 Thread Alex Bennée
The %lu format string is different depending on the host architecture which causes builds like the debian-armhf-cross build to fail. Use the correct PRi64 format string. Signed-off-by: Alex Bennée --- hw/block/xen-block.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

Re: [Qemu-block] [PATCH] xen: Fix format string

2019-01-16 Thread Paul Durrant
> -Original Message- > From: Philippe Mathieu-Daudé [mailto:phi...@redhat.com] > Sent: 16 January 2019 12:01 > To: qemu-de...@nongnu.org > Cc: Paul Durrant ; Anthony Perard > ; Max Reitz ; qemu- > bl...@nongnu.org; Stefano Stabellini ; xen- > de...@lists.xenproject.org; Kevin Wolf ;

[Qemu-block] [PATCH] xen: Fix format string

2019-01-16 Thread Philippe Mathieu-Daudé
Trivial format string fix to solve: hw/block/xen-block.c: In function 'xen_block_realize': hw/block/xen-block.c:218:53: error: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'int64_t {aka long long int}' [-Werror=format=]

Re: [Qemu-block] [Qemu-devel] [PATCH 03/15] hw/ssi: Remove SSIBus from "qemu/typedefs.h"

2019-01-16 Thread Paolo Bonzini
On 16/01/19 12:34, Gerd Hoffmann wrote: > Hi, > >> typedefs.h is useful to avoid rebuilding the world too often if a type >> is used many times as a pointer, but rarely as a struct and rarely has >> functions called on its instances. > > Related: Can also be used to keep struct content

Re: [Qemu-block] [Qemu-devel] [PATCH 03/15] hw/ssi: Remove SSIBus from "qemu/typedefs.h"

2019-01-16 Thread Gerd Hoffmann
Hi, > typedefs.h is useful to avoid rebuilding the world too often if a type > is used many times as a pointer, but rarely as a struct and rarely has > functions called on its instances. Related: Can also be used to keep struct content private. struct QemuConsole for example is private to

Re: [Qemu-block] [PATCH v3 16/19] nbd/client: Add meta contexts to nbd_receive_export_list()

2019-01-16 Thread Vladimir Sementsov-Ogievskiy
12.01.2019 20:58, Eric Blake wrote: > We want to be able to detect whether a given qemu NBD server is > exposing the right export(s) and dirty bitmaps, at least for > regression testing. We could use 'nbd-client -l' from the upstream > NBD project to list exports, but it's annoying to rely on >

Re: [Qemu-block] [PATCH v3 10/19] nbd/client: Split out nbd_send_one_meta_context()

2019-01-16 Thread Vladimir Sementsov-Ogievskiy
15.01.2019 18:44, Eric Blake wrote: > On 1/15/19 7:18 AM, Vladimir Sementsov-Ogievskiy wrote: >> 12.01.2019 20:58, Eric Blake wrote: >>> Refactor nbd_negotiate_simple_meta_context() to pull out the >>> code that can be reused to send a LIST request for 0 or 1 query. >>> No semantic change. The

Re: [Qemu-block] [PATCH v3 15/19] nbd/client: Add nbd_receive_export_list()

2019-01-16 Thread Vladimir Sementsov-Ogievskiy
12.01.2019 20:58, Eric Blake wrote: > We want to be able to detect whether a given qemu NBD server is > exposing the right export(s) and dirty bitmaps, at least for > regression testing. We could use 'nbd-client -l' from the upstream > NBD project to list exports, but it's annoying to rely on >

Re: [Qemu-block] [Qemu-devel] [PATCH 03/15] hw/ssi: Remove SSIBus from "qemu/typedefs.h"

2019-01-16 Thread Paolo Bonzini
On 16/01/19 09:33, Markus Armbruster wrote: > What problem exactly are we trying to solve here? > To the best of my knowledge, typedefs.h has been working just fine for > us, and I can't see why it shouldn't continue to work just fine. Patches don't have to solve problems. They can just help

Re: [Qemu-block] [PATCH v12 09/10] qcow2: skip writing zero buffers to empty COW areas

2019-01-16 Thread Alberto Garcia
On Wed 16 Jan 2019 10:32:33 AM CET, Anton Nefedov wrote: >>> +ret = handle_alloc_space(bs, l2meta); >> >> I insist that it would be nice to have a short comment explaining >> what this does. > > Right sorry forgot your comment. > I'd go with: > > +/* Try to efficiently initialize

Re: [Qemu-block] [PATCH v12 09/10] qcow2: skip writing zero buffers to empty COW areas

2019-01-16 Thread Anton Nefedov
On 15/1/2019 6:27 PM, Alberto Garcia wrote: > On Mon 14 Jan 2019 12:18:30 PM CET, Anton Nefedov wrote: >> If COW areas of the newly allocated clusters are zeroes on the backing image, >> efficient bdrv_write_zeroes(flags=BDRV_REQ_ALLOCATE) can be used on the whole >> cluster instead of writing

Re: [Qemu-block] [Qemu-devel] [PATCH 03/15] hw/ssi: Remove SSIBus from "qemu/typedefs.h"

2019-01-16 Thread Markus Armbruster
Paolo Bonzini writes: > On 15/01/19 13:28, Markus Armbruster wrote: >>> Signed-off-by: Philippe Mathieu-Daudé >>> --- >>> hw/arm/strongarm.h | 1 + >>> include/hw/arm/pxa.h| 1 + >>> include/hw/ssi/pl022.h | 1 + >>> include/hw/ssi/ssi.h| 1 + >>> include/qemu/typedefs.h | 1 -

Re: [Qemu-block] [Qemu-devel] [PATCH 03/15] hw/ssi: Remove SSIBus from "qemu/typedefs.h"

2019-01-16 Thread Markus Armbruster
Paolo Bonzini writes: > On 15/01/19 13:56, Thomas Huth wrote: >> Yes, agreed, removing things from typedefs.h just to add lots of >> #includes in other files is not really the best idea. I also dropped >> this patch in v2 of my current PULL request because of this reason. >> Phil, I suggest to

Re: [Qemu-block] [PATCH v3 05/19] nbd/server: Favor [u]int64_t over off_t

2019-01-16 Thread Vladimir Sementsov-Ogievskiy
15.01.2019 18:33, Eric Blake wrote: > On 1/15/19 4:19 AM, Vladimir Sementsov-Ogievskiy wrote: >> 12.01.2019 20:57, Eric Blake wrote: >>> Although our compile-time environment is set up so that we always >>> support long files with 64-bit off_t, we have no guarantee whether >>> off_t is the same