Re: [Qemu-block] [Qemu-devel] [PATCH v2] Add a gitlab-ci file for Continuous Integration testing on Gitlab

2019-02-18 Thread Kevin Wolf
Am 19.02.2019 um 07:44 hat Thomas Huth geschrieben: > On 18/02/2019 19.22, Cleber Rosa wrote: > > > > > > On 2/13/19 6:54 AM, Thomas Huth wrote: > >> This is very convenient for people like me who store their QEMU git trees > >> on gitlab.com: Automatic CI pipelines are now run for each branch

Re: [Qemu-block] [Qemu-devel] Guest unresponsive after Virtqueue size exceeded error

2019-02-18 Thread Fernando Casas Schössow
Problem reproduced with virtio-scsi as well on the same guest, this time it took less than a day. Information from the log file: vdev 0x55823f119f90 ("virtio-scsi") vq 0x55823f122e80 (idx 2) inuse 128 vring.num 128 old_shadow_avail_idx 58367 last_avail_idx 58113 avail_idx 58367 avail 0x3de8a800

Re: [Qemu-block] [Qemu-devel] Configuring pflash devices for OVMF firmware

2019-02-18 Thread Markus Armbruster
Paolo Bonzini writes: > On 30/01/19 15:13, Markus Armbruster wrote: >> -global driver=cfi.pflash01,property=secure,value=on >> >> Affects *all* such devices, but fortunately we have at most two, and the >> one we don't want to affect happens to ignore the property value. > > Is this true?

[Qemu-block] Failing qemu-iotest 233

2019-02-18 Thread Thomas Huth
Hi Eric, hi Daniel, QEMU iotest 233 is failing for me on RHEL7: 233[07:29:30] [07:29:30] [failed, exit status 1] - output mismatch (see 233.out.bad) --- /home/thuth/devel/qemu/tests/qemu-iotests/233.out 2019-02-19 07:14:45.0 +0100 +++

Re: [Qemu-block] [Qemu-devel] [PATCH 09/10] pflash: Clean up after commit 368a354f02b part 2

2019-02-18 Thread Markus Armbruster
Laszlo Ersek writes: > On 02/18/19 18:01, Laszlo Ersek wrote: >> On 02/18/19 13:56, Markus Armbruster wrote: >>> QOMification left parameter @size unused in pflash_cfi01_register() >>> and pflash_cfi02_register(). register(). Obviously, @size should > > I meant to point out the typo above, but

Re: [Qemu-block] [Qemu-devel] [PATCH 00/10] pflash: Fixes and cleanups

2019-02-18 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20190218125615.18970-1-arm...@redhat.com/ Hi, This series seems to have some coding style problems. See output below for more information: Message-id: 20190218125615.18970-1-arm...@redhat.com Subject: [Qemu-devel] [PATCH 00/10] pflash: Fixes and cleanups

Re: [Qemu-block] [Qemu-devel] [PATCH 00/10] pflash: Fixes and cleanups

2019-02-18 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20190218125615.18970-1-arm...@redhat.com/ Hi, This series seems to have some coding style problems. See output below for more information: Message-id: 20190218125615.18970-1-arm...@redhat.com Subject: [Qemu-devel] [PATCH 00/10] pflash: Fixes and cleanups

Re: [Qemu-block] [Qemu-devel] Please help corrupt filesystem image

2019-02-18 Thread Alexander Marx
Thank you for replying. Well i am using latest PROXMOX in a cluster of 4 physical servers. during the weekend i had to stop all hosts because electricians had to work on the fuse box. i shutted down all vm's then powered off all physical hosts. One of them took very long. this host had a raid5 of

Re: [Qemu-block] [RFC PATCH 11/11] qcow2: Add data file to ImageInfoSpecificQCow2

2019-02-18 Thread Max Reitz
On 31.01.19 18:55, Kevin Wolf wrote: > Signed-off-by: Kevin Wolf > --- > qapi/block-core.json | 1 + > block/qcow2.c| 6 +- > 2 files changed, 6 insertions(+), 1 deletion(-) [...] > diff --git a/block/qcow2.c b/block/qcow2.c > index 4959bf16a4..e3427f9fcd 100644 > ---

Re: [Qemu-block] [Qemu-devel] [RFC PATCH 1/2] block/dirty-bitmaps: add inconsistent bit

2019-02-18 Thread John Snow
On 2/18/19 12:46 PM, Vladimir Sementsov-Ogievskiy wrote: > 14.02.2019 2:36, John Snow wrote: >> Add an inconsistent bit to dirty-bitmaps that allows us to report a bitmap as >> persistent but potentially inconsistent, i.e. if we find bitmaps on a qcow2 >> that have been marked as "in use". >>

Re: [Qemu-block] [RFC PATCH 10/11] qcow2: Store data file name in the image

2019-02-18 Thread Max Reitz
On 31.01.19 18:55, Kevin Wolf wrote: > Rather than requiring that the external data file node is passed > explicitly when creating the qcow2 node, store the filename in the > designated header extension during .bdrv_create and read it from there > as a default during .bdrv_open. > >

Re: [Qemu-block] [RFC PATCH 08/11] qcow2: Add basic data-file infrastructure

2019-02-18 Thread Max Reitz
On 31.01.19 18:55, Kevin Wolf wrote: > This adds a .bdrv_open option to specify the external data file node. > > Signed-off-by: Kevin Wolf > --- > qapi/block-core.json | 3 ++- > block/qcow2.h| 4 +++- > block/qcow2.c| 25 +++-- > 3 files changed, 28

Re: [Qemu-block] [RFC PATCH 2/2] block/dirty-bitmap: implement inconsistent bit

2019-02-18 Thread John Snow
On 2/18/19 3:37 PM, Eric Blake wrote: > On 2/18/19 12:13 PM, Vladimir Sementsov-Ogievskiy wrote: >> 14.02.2019 2:36, John Snow wrote: >>> Signed-off-by: John Snow >>> --- >>> block/dirty-bitmap.c | 15 + >>> block/qcow2-bitmap.c | 42

Re: [Qemu-block] [RFC PATCH 07/11] qcow2: External file I/O

2019-02-18 Thread Max Reitz
On 31.01.19 18:55, Kevin Wolf wrote: > This changes the qcow2 implementation to direct all guest data I/O to > s->data_file rather than bs->file, while metadata I/O still uses > bs->file. At the moment, this is still always the same, but soon we'll > add options to set s->data_file to an external

[Qemu-block] [PATCH v5] blockdev: acquire aio_context for bitmap add/remove

2019-02-18 Thread John Snow
When bitmaps are persistent, they may incur a disk read or write when bitmaps are added or removed. For configurations like virtio-dataplane, failing to acquire this lock will abort QEMU when disk IO occurs. We used to acquire aio_context as part of the bitmap lookup, so re-introduce the lock for

Re: [Qemu-block] [PATCH v2 3/6] block/dirty-bitmap: change semantics of enabled predicate

2019-02-18 Thread John Snow
On 2/18/19 11:39 AM, Vladimir Sementsov-Ogievskiy wrote: > 14.02.2019 2:23, John Snow wrote: >> Currently, enabled means something like "the status of the bitmap >> is ACTIVE." After this patch, it should mean exclusively: "This >> bitmap is recording guest writes, and is allowed to do so." >>

Re: [Qemu-block] [RFC PATCH 06/11] qcow2: Don't assume 0 is an invalid cluster offset

2019-02-18 Thread Max Reitz
On 31.01.19 18:55, Kevin Wolf wrote: > The cluster allocation code uses 0 as an invalid offset that is used in > case of errors or as "offset not yet determined". With external data > files, a host cluster offset of 0 becomes valid, though. > > Define a constant INV_OFFSET (which is not cluster

Re: [Qemu-block] [PATCH v2 5/6] block/dirty-bitmaps: unify qmp_locked and user_locked calls

2019-02-18 Thread John Snow
On 2/18/19 12:27 PM, Vladimir Sementsov-Ogievskiy wrote: > 14.02.2019 2:23, John Snow wrote: >> These mean the same thing now. Unify them and rename the merged call >> bdrv_dirty_bitmap_busy to indicate semantically what we are describing, >> as well as help disambiguate from the various

Re: [Qemu-block] [PATCH v2 2/6] block/dirty-bitmaps: rename frozen predicate helper

2019-02-18 Thread John Snow
On 2/18/19 8:57 AM, Vladimir Sementsov-Ogievskiy wrote: > 14.02.2019 2:23, John Snow wrote: >> "Frozen" was a good description a long time ago, but it isn't adequate now. >> Rename the frozen predicate to has_successor to make the semantics of the >> predicate more clear to outside callers. >>

Re: [Qemu-block] [PATCH v4 1/1] blockdev: acquire aio_context for bitmap add/remove

2019-02-18 Thread John Snow
On 2/18/19 4:55 PM, Eric Blake wrote: > On 2/18/19 3:42 PM, John Snow wrote: >> When bitmaps are persistent, they may incur a disk read or write when bitmaps >> are added or removed. For configurations like virtio-dataplane, failing to >> acquire this lock will abort QEMU when disk IO occurs.

Re: [Qemu-block] [PATCH v2 6/6] block/dirty-bitmaps: move comment block

2019-02-18 Thread John Snow
On 2/18/19 12:39 PM, Vladimir Sementsov-Ogievskiy wrote: > 14.02.2019 2:23, John Snow wrote: >> Simply move the big status enum comment block to above the status >> function, and document it as being deprecated. The whole confusing >> block can get deleted in three releases time. >> >>

Re: [Qemu-block] [PATCH v2 4/6] block/dirty-bitmap: explicitly lock bitmaps with successors

2019-02-18 Thread John Snow
On 2/18/19 11:52 AM, Vladimir Sementsov-Ogievskiy wrote: > 14.02.2019 2:23, John Snow wrote: >> Instead of implying a locked status, make it explicit. > > locked interferes with bitmap mutex, so may be better "qmp_locked state", but > not sure. > I agree that "locked" has too many meanings,

Re: [Qemu-block] [PATCH v4 1/1] blockdev: acquire aio_context for bitmap add/remove

2019-02-18 Thread Eric Blake
On 2/18/19 3:42 PM, John Snow wrote: > When bitmaps are persistent, they may incur a disk read or write when bitmaps > are added or removed. For configurations like virtio-dataplane, failing to > acquire this lock will abort QEMU when disk IO occurs. > > We used to acquire aio_context as part of

Re: [Qemu-block] [Qemu-devel] [PATCH] iotests: handle TypeError for Python3 in test 242

2019-02-18 Thread Philippe Mathieu-Daudé
On 2/18/19 9:05 PM, Eric Blake wrote: > [adding Eduardo for some python 2-vs-3 advice] And Cleber. > > On 2/18/19 1:59 PM, Andrey Shinkevich wrote: >> To write one byte to disk, Python2 may use 'chr' type. >> In Python3, conversion to 'byte' type is required. >> >> Signed-off-by: Andrey

Re: [Qemu-block] [Qemu-devel] [PATCH] iotests: Skip 211 on insufficient memory

2019-02-18 Thread Philippe Mathieu-Daudé
On 2/18/19 7:06 PM, Max Reitz wrote: > VDI keeps the whole bitmap in memory, and the maximum size (which is > tested here) is 2 GB. This may not be available on all machines, and it > rarely is available when running a 32 bit build. > > Fix this by making VM.run_job() return the error string if

Re: [Qemu-block] [Qemu-devel] [PATCH] iotests: Skip 211 on insufficient memory

2019-02-18 Thread Eric Blake
On 2/18/19 12:06 PM, Max Reitz wrote: > VDI keeps the whole bitmap in memory, and the maximum size (which is > tested here) is 2 GB. This may not be available on all machines, and it > rarely is available when running a 32 bit build. > > Fix this by making VM.run_job() return the error string if

Re: [Qemu-block] [PATCH 12/12] aio-posix: Assert that aio_poll() is always called in home thread

2019-02-18 Thread Eric Blake
On 2/18/19 10:18 AM, Kevin Wolf wrote: > aio_poll() has an existing assertion that the function is only called > from the AioContext's home thread if blocking is allowed. > > This is not enough, some handlers make assumptions about the thread they > run in. Extend the assertion to non-blocking

Re: [Qemu-block] [PATCH 11/12] block: Use normal drain for bdrv_set_aio_context()

2019-02-18 Thread Eric Blake
On 2/18/19 10:18 AM, Kevin Wolf wrote: > Now that bdrv_set_aio_context() works inside drained sections, it can > also use the real drain function instead of open coding something > similar. > > Signed-off-by: Kevin Wolf > --- > block.c | 14 +- > 1 file changed, 5 insertions(+), 9

Re: [Qemu-block] [PATCH 10/12] test-bdrv-drain: AioContext switch in drained section

2019-02-18 Thread Eric Blake
On 2/18/19 10:18 AM, Kevin Wolf wrote: > Signed-off-by: Kevin Wolf > --- > tests/test-bdrv-drain.c | 32 > 1 file changed, 32 insertions(+) Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226

Re: [Qemu-block] [PATCH 08/12] block: Don't poll in bdrv_set_aio_context()

2019-02-18 Thread Eric Blake
On 2/18/19 10:18 AM, Kevin Wolf wrote: > The explicit aio_poll() call in bdrv_set_aio_context() was added in > commit c2b6428d388 as a workaround for bdrv_drain() failing to achieve > to actually quiesce everything (specifically the NBD client code to > switch AioContext). > > Now that the NBD

Re: [Qemu-block] [PATCH 06/12] nbd: Use low-level QIOChannel API in nbd_read_eof()

2019-02-18 Thread Eric Blake
On 2/18/19 10:18 AM, Kevin Wolf wrote: > Instead of using the convenience wrapper qio_channel_read_all_eof(), use > the lower level QIOChannel API. This means duplicating some code, but > we'll need this because this coroutine yield is special: We want it to > be interruptible so that

Re: [Qemu-block] [PATCH 05/12] nbd: Move nbd_read_eof() to nbd/client.c

2019-02-18 Thread Eric Blake
On 2/18/19 10:18 AM, Kevin Wolf wrote: > The only caller of nbd_read_eof() is nbd_receive_reply(), so it doesn't > have to live in the header file, but can move next to its caller. > > Also add the missing coroutine_fn to the function and its caller. > > Signed-off-by: Kevin Wolf > --- >

Re: [Qemu-block] [RFC PATCH 2/2] block/dirty-bitmap: implement inconsistent bit

2019-02-18 Thread Eric Blake
On 2/18/19 12:13 PM, Vladimir Sementsov-Ogievskiy wrote: > 14.02.2019 2:36, John Snow wrote: >> Signed-off-by: John Snow >> --- >> block/dirty-bitmap.c | 15 + >> block/qcow2-bitmap.c | 42 ++- >> blockdev.c | 43

Re: [Qemu-block] [PATCH 03/12] nbd: Restrict connection_co reentrance

2019-02-18 Thread Eric Blake
On 2/18/19 10:18 AM, Kevin Wolf wrote: > nbd_client_attach_aio_context() schedules connection_co in the new > AioContext and this way reenters it in any arbitrary place that has > yielded. We can restrict this a bit to the function call where the > coroutine actually sits waiting when it's idle. >

Re: [Qemu-block] [Qemu-devel] [PATCH v4 01/17] block: enhance QEMUIOVector structure

2019-02-18 Thread Eric Blake
On 2/18/19 8:09 AM, Vladimir Sementsov-Ogievskiy wrote: > Add a possibility of embedded iovec, for cases when we need only one > local iov. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > include/qemu/iov.h | 64 -- > 1 file changed, 62

Re: [Qemu-block] [PATCH] iotests: handle TypeError for Python3 in test 242

2019-02-18 Thread Eric Blake
[adding Eduardo for some python 2-vs-3 advice] On 2/18/19 1:59 PM, Andrey Shinkevich wrote: > To write one byte to disk, Python2 may use 'chr' type. > In Python3, conversion to 'byte' type is required. > > Signed-off-by: Andrey Shinkevich > --- > tests/qemu-iotests/242 | 9 +++-- > 1 file

[Qemu-block] [PATCH] iotests: handle TypeError for Python3 in test 242

2019-02-18 Thread Andrey Shinkevich
To write one byte to disk, Python2 may use 'chr' type. In Python3, conversion to 'byte' type is required. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/242 | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/242 b/tests/qemu-iotests/242

Re: [Qemu-block] [PATCH v14 3/3] qcow2: list of bitmaps new test 242

2019-02-18 Thread Andrey Shinkevich
On 18/02/2019 19:12, Kevin Wolf wrote: > Am 08.02.2019 um 16:06 hat Andrey Shinkevich geschrieben: >> A new test file 242 added to the qemu-iotests set. It checks >> the format of qcow2 specific information for the new added >> section that lists details of bitmaps. >> >> Signed-off-by: Andrey

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] Introduce a Python module structure

2019-02-18 Thread Caio Carrara
On Wed, Feb 06, 2019 at 11:29:01AM -0500, Cleber Rosa wrote: > This is a simple move of Python code that wraps common QEMU > functionality, and are used by a number of different tests > and scripts. > > By treating that code as a real Python module, we can more easily: > * reuse code > * have a

Re: [Qemu-block] [RFC PATCH 2/2] block/dirty-bitmap: implement inconsistent bit

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
14.02.2019 2:36, John Snow wrote: > Signed-off-by: John Snow > --- > block/dirty-bitmap.c | 15 + > block/qcow2-bitmap.c | 42 ++- > blockdev.c | 43 >

Re: [Qemu-block] [RFC PATCH 0/2] bitmaps: add inconsistent bit

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
14.02.2019 2:36, John Snow wrote: > Allow QEMU to read in bitmaps that have the in-use bit set, for the > purposes of allowing users to clear or reset these bitmaps. > > This is chosen in preference to a hard error on load to minimize > impact for a non-critical error, but to force the user or

Re: [Qemu-block] [PATCH 0/2] qemu-iotests: add a Travis job

2019-02-18 Thread Cleber Rosa
On 2/8/19 11:01 PM, Cleber Rosa wrote: > Besides the obvious reasons of testing more, and somewhat for free, > running the qemu-iotests along the other tests on Travis also makes > sure that changes to shared code such as "scripts/qemu.py" and the > like won't break other users of the same

Re: [Qemu-block] [PATCH 09/10] pflash: Clean up after commit 368a354f02b part 2

2019-02-18 Thread Laszlo Ersek
On 02/18/19 18:01, Laszlo Ersek wrote: > On 02/18/19 13:56, Markus Armbruster wrote: >> QOMification left parameter @size unused in pflash_cfi01_register() >> and pflash_cfi02_register(). register(). Obviously, @size should I meant to point out the typo above, but I got distracted mid-review.

Re: [Qemu-block] [RFC PATCH 1/2] block/dirty-bitmaps: add inconsistent bit

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
14.02.2019 2:36, John Snow wrote: > Add an inconsistent bit to dirty-bitmaps that allows us to report a bitmap as > persistent but potentially inconsistent, i.e. if we find bitmaps on a qcow2 > that have been marked as "in use". > > Signed-off-by: John Snow > --- > block/dirty-bitmap.c

Re: [Qemu-block] [PATCH v2 6/6] block/dirty-bitmaps: move comment block

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
14.02.2019 2:23, John Snow wrote: > Simply move the big status enum comment block to above the status > function, and document it as being deprecated. The whole confusing > block can get deleted in three releases time. > > Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [PATCH 10/10] hw/arm hw/xtensa: De-duplicate pflash creation code some

2019-02-18 Thread Laszlo Ersek
On 02/18/19 13:56, Markus Armbruster wrote: > pflash_cfi01_register() creates a TYPE_CFI_PFLASH01 device, sets > properties, realizes, and wires up. > > We have three modified copies of it, because their users need to set > additional properties, or have the wiring done differently. > > Factor

Re: [Qemu-block] [PATCH 04/12] io: Make qio_channel_yield() interruptible

2019-02-18 Thread Paolo Bonzini
On 18/02/19 17:18, Kevin Wolf wrote: > Similar to how qemu_co_sleep_ns() allows to be preempted by an external > coroutine entry, allow reentering qio_channel_yield() early. > > Signed-off-by: Kevin Wolf > --- > include/io/channel.h | 9 ++--- > io/channel.c | 10 ++ > 2

Re: [Qemu-block] [PATCH v2 2/6] block/dirty-bitmaps: rename frozen predicate helper

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
18.02.2019 16:57, Vladimir Sementsov-Ogievskiy wrote: > 14.02.2019 2:23, John Snow wrote: >> "Frozen" was a good description a long time ago, but it isn't adequate now. >> Rename the frozen predicate to has_successor to make the semantics of the >> predicate more clear to outside callers. >> >> In

Re: [Qemu-block] [PATCH 09/10] pflash: Clean up after commit 368a354f02b part 2

2019-02-18 Thread Laszlo Ersek
On 02/18/19 13:56, Markus Armbruster wrote: > QOMification left parameter @size unused in pflash_cfi01_register() > and pflash_cfi02_register(). register(). Obviously, @size should > match @sector_len and @nb_blocs, i.e. size == sector_len * nb_blocs. > All callers satisfy this. > > Remove

Re: [Qemu-block] [PATCH v2 4/6] block/dirty-bitmap: explicitly lock bitmaps with successors

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
14.02.2019 2:23, John Snow wrote: > Instead of implying a locked status, make it explicit. locked interferes with bitmap mutex, so may be better "qmp_locked state", but not sure. > Now, bitmaps in use by migration, NBD or backup operations > are all treated the same way with the same code

Re: [Qemu-block] [PATCH 02/10] pflash: Macro PFLASH_BUG() is used just once, expand

2019-02-18 Thread Laszlo Ersek
On 02/18/19 13:56, Markus Armbruster wrote: > PFLASH_BUG()'s lone use has a suspicious smell: it prints "Possible > BUG", which sounds like a warning, then calls exit(1), followed by > unreachable goto reset_flash. All this commit does is expanding the > macro, so the smell becomes more poignant,

Re: [Qemu-block] [PATCH 01/10] pflash: Rename pflash_t to PFlashCFI01, PFlashCFI02

2019-02-18 Thread Laszlo Ersek
On 02/18/19 13:56, Markus Armbruster wrote: > flash.h's incomplete struct pflash_t is completed both in > pflash_cfi01.c and in pflash_cfi02.c. The complete types are > incompatible. O_o > This can hide type errors, such as passing a pflash_t > created with pflash_cfi02_register() to

Re: [Qemu-block] [PATCH v2 3/6] block/dirty-bitmap: change semantics of enabled predicate

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
14.02.2019 2:23, John Snow wrote: > Currently, enabled means something like "the status of the bitmap > is ACTIVE." After this patch, it should mean exclusively: "This > bitmap is recording guest writes, and is allowed to do so." > > In many places, this is how this predicate was already used. >

[Qemu-block] [PATCH 11/12] block: Use normal drain for bdrv_set_aio_context()

2019-02-18 Thread Kevin Wolf
Now that bdrv_set_aio_context() works inside drained sections, it can also use the real drain function instead of open coding something similar. Signed-off-by: Kevin Wolf --- block.c | 14 +- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/block.c b/block.c index

[Qemu-block] [PATCH 10/12] test-bdrv-drain: AioContext switch in drained section

2019-02-18 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- tests/test-bdrv-drain.c | 32 1 file changed, 32 insertions(+) diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c index ee1740ff06..1b1f6c17a5 100644 --- a/tests/test-bdrv-drain.c +++ b/tests/test-bdrv-drain.c @@ -1522,6

[Qemu-block] [PATCH 07/12] nbd: Increase bs->in_flight during AioContext switch

2019-02-18 Thread Kevin Wolf
bdrv_drain() must not leave connection_co scheduled, so bs->in_flight needs to be increased while the coroutine is waiting to be scheduled in the new AioContext after nbd_client_attach_aio_context(). Signed-off-by: Kevin Wolf --- block/nbd-client.h | 1 + include/block/nbd.h | 5 +++--

[Qemu-block] [PATCH 12/12] aio-posix: Assert that aio_poll() is always called in home thread

2019-02-18 Thread Kevin Wolf
aio_poll() has an existing assertion that the function is only called from the AioContext's home thread if blocking is allowed. This is not enough, some handlers make assumptions about the thread they run in. Extend the assertion to non-blocking calls, too. Signed-off-by: Kevin Wolf ---

[Qemu-block] [PATCH 03/12] nbd: Restrict connection_co reentrance

2019-02-18 Thread Kevin Wolf
nbd_client_attach_aio_context() schedules connection_co in the new AioContext and this way reenters it in any arbitrary place that has yielded. We can restrict this a bit to the function call where the coroutine actually sits waiting when it's idle. This doesn't solve any bug yet, but it shows

[Qemu-block] [PATCH 04/12] io: Make qio_channel_yield() interruptible

2019-02-18 Thread Kevin Wolf
Similar to how qemu_co_sleep_ns() allows to be preempted by an external coroutine entry, allow reentering qio_channel_yield() early. Signed-off-by: Kevin Wolf --- include/io/channel.h | 9 ++--- io/channel.c | 10 ++ 2 files changed, 16 insertions(+), 3 deletions(-) diff

[Qemu-block] [PATCH 00/12] block: bdrv_set_aio_context() related fixes

2019-02-18 Thread Kevin Wolf
Background for this series is the following bug report, which is about a crash with virtio-blk + iothread and request resubmission for werror/rerror: https://bugzilla.redhat.com/show_bug.cgi?id=1671173 The reason is that bdrv_set_aio_context() didn't correctly quiesce everything. Instead, it had

[Qemu-block] [PATCH 06/12] nbd: Use low-level QIOChannel API in nbd_read_eof()

2019-02-18 Thread Kevin Wolf
Instead of using the convenience wrapper qio_channel_read_all_eof(), use the lower level QIOChannel API. This means duplicating some code, but we'll need this because this coroutine yield is special: We want it to be interruptible so that nbd_client_attach_aio_context() can correctly reenter the

[Qemu-block] [PATCH 08/12] block: Don't poll in bdrv_set_aio_context()

2019-02-18 Thread Kevin Wolf
The explicit aio_poll() call in bdrv_set_aio_context() was added in commit c2b6428d388 as a workaround for bdrv_drain() failing to achieve to actually quiesce everything (specifically the NBD client code to switch AioContext). Now that the NBD client has been fixed to complete this operation

[Qemu-block] [PATCH 05/12] nbd: Move nbd_read_eof() to nbd/client.c

2019-02-18 Thread Kevin Wolf
The only caller of nbd_read_eof() is nbd_receive_reply(), so it doesn't have to live in the header file, but can move next to its caller. Also add the missing coroutine_fn to the function and its caller. Signed-off-by: Kevin Wolf --- include/block/nbd.h | 3 ++- nbd/nbd-internal.h | 19

[Qemu-block] [PATCH 09/12] block: Fix AioContext switch for drained node

2019-02-18 Thread Kevin Wolf
When a drained node changes its AioContext, we need to move its aio_disable_external() to the new context, too. Without this fix, drain_end will try to reenable the new context, which has never been disabled, so an assertion failure is triggered. Signed-off-by: Kevin Wolf --- block.c | 7

[Qemu-block] [PATCH 02/12] virtio-blk: Increase in_flight for request restart BH

2019-02-18 Thread Kevin Wolf
virtio_blk_dma_restart_bh() submits new requests, so in order to make sure that these requests are not started inside a drained section of the attached BlockBackend, we need to make sure that draining the BlockBackend waits for the BH to be executed. This BH is still questionable because its

[Qemu-block] [PATCH 01/12] block-backend: Make blk_inc/dec_in_flight public

2019-02-18 Thread Kevin Wolf
For some users of BlockBackends, just increasing the in_flight counter is easier than implementing separate handlers in BlockDevOps. Make the helper functions for this public. Signed-off-by: Kevin Wolf --- include/sysemu/block-backend.h | 2 ++ block/block-backend.c | 4 ++-- 2 files

Re: [Qemu-block] [PATCH v14 3/3] qcow2: list of bitmaps new test 242

2019-02-18 Thread Kevin Wolf
Am 08.02.2019 um 16:06 hat Andrey Shinkevich geschrieben: > A new test file 242 added to the qemu-iotests set. It checks > the format of qcow2 specific information for the new added > section that lists details of bitmaps. > > Signed-off-by: Andrey Shinkevich This doesn't seem to be Python 3

Re: [Qemu-block] [PATCH 3/3] xen-block: report error condition from vbd_name_to_disk()

2019-02-18 Thread Anthony PERARD
On Fri, Feb 15, 2019 at 04:25:33PM +, Paul Durrant wrote: > The function needs to make sure it is passed a valid disk name. This is > easily done by making sure that the parsing loop results in a non-zero > value. > > Spotted by Coverity: CID 1398640 > > Reported-by: Peter Maydell >

Re: [Qemu-block] [PATCH] Discard old bitmap directories in QCOW2 image

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
12.02.2019 15:35, Andrey Shinkevich wrote: > Clean QCOW2 image from bitmap obsolete directory when a new one > is allocated and stored. It slows down the image growth a little bit. > The flag QCOW2_DISCARD_ALWAYS allows a call to raw_co_pdiscard() > that does the actual cleaning of the image on

Re: [Qemu-block] [Qemu-devel] [PATCH v5 00/10] virtio-blk: add DISCARD and WRITE_ZEROES features

2019-02-18 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20190218140301.197408-1-sgarz...@redhat.com/ Hi, This series seems to have some coding style problems. See output below for more information: Message-id: 20190218140301.197408-1-sgarz...@redhat.com Subject: [Qemu-devel] [PATCH v5 00/10] virtio-blk: add

Re: [Qemu-block] [PATCH 2/3] xen-block: remove redundant assignment

2019-02-18 Thread Anthony PERARD
On Fri, Feb 15, 2019 at 04:25:32PM +, Paul Durrant wrote: > The assignment to 'p' is unnecessary as the code will either goto 'invalid' > or p will get overwritten. > > Spotted by Coverity: CID 1398638 > > Reported-by: Peter Maydell > Signed-off-by: Paul Durrant Acked-by: Anthony PERARD

[Qemu-block] [PATCH v4 11/17] block/vmdk: use qemu_iovec_init_buf

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
Use new qemu_iovec_init_buf() instead of qemu_iovec_init_external( ... , 1), which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/vmdk.c | 7 +-- 1 file changed, 1 insertion(+), 6 deletions(-) diff --git

[Qemu-block] [PATCH v4 14/17] tests/test-bdrv-drain: use QEMU_IOVEC_INIT_BUF

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
Use new QEMU_IOVEC_INIT_BUF() instead of qemu_iovec_init_external( ... , 1), which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- tests/test-bdrv-drain.c | 29 - 1 file changed, 4

[Qemu-block] [PATCH v4 00/17] block: local qiov helper

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
Hi all! Here is a new simple helper for a very often patter around qemu_iovec_init_external, when we need simple qiov with only one iov, initialized from external buffer. v4: 01: tiny improvements by Eric + fix bug: s/niov/nalloc in assertion + rename

[Qemu-block] [PATCH v4 10/17] block/qed: use qemu_iovec_init_buf

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
Use new qemu_iovec_init_buf() instead of qemu_iovec_init_external( ... , 1), which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/qed-table.c | 16 +++- block/qed.c | 31

[Qemu-block] [PATCH v4 05/17] block/commit: use QEMU_IOVEC_INIT_BUF

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
Use new QEMU_IOVEC_INIT_BUF() instead of qemu_iovec_init_external( ... , 1), which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/commit.c | 7 +-- 1 file changed, 1 insertion(+), 6 deletions(-) diff --git

Re: [Qemu-block] [Qemu-devel] [PATCH 1/3] dataplane/xen-block: remove dead code

2019-02-18 Thread Anthony PERARD
On Fri, Feb 15, 2019 at 09:38:59PM +0100, Philippe Mathieu-Daudé wrote: > On 2/15/19 5:25 PM, Paul Durrant wrote: > > The if() statement is clearly bogus (dead code which should have been > > cleaned up when grant mapping was removed). > > "... was removed in 06454c24ad)." Actually, it looks

[Qemu-block] [PATCH v4 04/17] block/backup: use qemu_iovec_init_buf

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
Use new qemu_iovec_init_buf() instead of qemu_iovec_init_external( ... , 1), which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/backup.c | 5 + 1 file changed, 1 insertion(+), 4 deletions(-) diff --git

[Qemu-block] [PATCH v4 07/17] block/parallels: use QEMU_IOVEC_INIT_BUF

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
Use new QEMU_IOVEC_INIT_BUF() instead of qemu_iovec_init_external( ... , 1), which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/parallels.c | 13 + 1 file changed, 5 insertions(+), 8 deletions(-)

[Qemu-block] [PATCH v4 06/17] block/stream: use QEMU_IOVEC_INIT_BUF

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
Use new QEMU_IOVEC_INIT_BUF() instead of qemu_iovec_init_external( ... , 1), which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/stream.c | 7 +-- 1 file changed, 1 insertion(+), 6 deletions(-) diff --git

[Qemu-block] [PATCH v4 13/17] migration/block: use qemu_iovec_init_buf

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
Use new qemu_iovec_init_buf() instead of qemu_iovec_init_external( ... , 1), which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- migration/block.c | 10 +++--- 1 file changed, 3 insertions(+), 7 deletions(-) diff

[Qemu-block] [PATCH v4 17/17] hw/ide: drop iov field from IDEDMA

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
@iov is used only to initialize @qiov. Let's use new qemu_iovec_init_buf() instead, which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- include/hw/ide/internal.h | 1 - hw/ide/atapi.c| 5 ++--- 2 files

[Qemu-block] [PATCH v4 12/17] qemu-img: use qemu_iovec_init_buf

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
Use new qemu_iovec_init_buf() instead of qemu_iovec_init_external( ... , 1), which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- qemu-img.c | 10 ++ 1 file changed, 2 insertions(+), 8 deletions(-) diff --git

[Qemu-block] [PATCH v4 09/17] block/qcow2: use qemu_iovec_init_buf

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
Use new qemu_iovec_init_buf() instead of qemu_iovec_init_external( ... , 1), which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/qcow2.c | 12 ++-- 1 file changed, 2 insertions(+), 10 deletions(-) diff

[Qemu-block] [PATCH v4 15/17] hw/ide: drop iov field from IDEState

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
@iov is used only to initialize @qiov. Let's use new qemu_iovec_init_buf() instead, which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- include/hw/ide/internal.h | 1 - hw/ide/atapi.c| 9 -

[Qemu-block] [PATCH v4 03/17] block/block-backend: use QEMU_IOVEC_INIT_BUF

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
Use new QEMU_IOVEC_INIT_BUF() instead of qemu_iovec_init_external( ... , 1), which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/block-backend.c | 13 ++--- 1 file changed, 2 insertions(+), 11

[Qemu-block] [PATCH v4 01/17] block: enhance QEMUIOVector structure

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
Add a possibility of embedded iovec, for cases when we need only one local iov. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/qemu/iov.h | 64 -- 1 file changed, 62 insertions(+), 2 deletions(-) diff --git a/include/qemu/iov.h

[Qemu-block] [PATCH v4 08/17] block/qcow: use qemu_iovec_init_buf

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
Use new qemu_iovec_init_buf() instead of qemu_iovec_init_external( ... , 1), which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/qcow.c | 21 - 1 file changed, 4 insertions(+), 17

[Qemu-block] [PATCH v4 16/17] hw/ide: drop iov field from IDEBufferedRequest

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
@iov is used only to initialize @qiov. Let's use new qemu_iovec_init_buf() instead, which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- include/hw/ide/internal.h | 1 - hw/ide/core.c | 11 ++- 2

[Qemu-block] [PATCH v5 10/10] tests/virtio-blk: add test for DISCARD command

2019-02-18 Thread Stefano Garzarella
If the DISCARD feature is enabled, we try this command in the test_basic(), checking only the status returned by the request. Signed-off-by: Stefano Garzarella --- tests/virtio-blk-test.c | 27 +++ 1 file changed, 27 insertions(+) diff --git a/tests/virtio-blk-test.c

[Qemu-block] [PATCH v5 09/10] tests/virtio-blk: add test for WRITE_ZEROES command

2019-02-18 Thread Stefano Garzarella
If the WRITE_ZEROES feature is enabled, we check this command in the test_basic(). Reviewed-by: Michael S. Tsirkin Reviewed-by: Stefan Hajnoczi Acked-by: Thomas Huth Signed-off-by: Stefano Garzarella --- tests/virtio-blk-test.c | 62 + 1 file changed,

[Qemu-block] [PATCH v5 04/10] virtio-net: make VirtIOFeature usable for other virtio devices

2019-02-18 Thread Stefano Garzarella
In order to use VirtIOFeature also in other virtio devices, we move its declaration and the endof() macro (renamed in virtio_endof()) in virtio.h. We add virtio_feature_get_config_size() function to iterate the array of VirtIOFeature and to return the config size depending on the features enabled.

[Qemu-block] [PATCH v5 08/10] tests/virtio-blk: add virtio_blk_fix_dwz_hdr() function

2019-02-18 Thread Stefano Garzarella
This function is useful to fix the endianness of struct virtio_blk_discard_write_zeroes headers. Signed-off-by: Stefano Garzarella --- tests/virtio-blk-test.c | 23 +-- 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/tests/virtio-blk-test.c

[Qemu-block] [PATCH v5 03/10] virtio-blk: add "discard" and "write-zeroes" properties

2019-02-18 Thread Stefano Garzarella
In order to avoid migration issues, we enable DISCARD and WRITE_ZEROES features only for machine type >= 4.0 As discussed with Michael S. Tsirkin and Stefan Hajnoczi on the list [1], DISCARD operation should not have security implications (eg. page cache attacks), so we can enable it by default.

[Qemu-block] [PATCH v5 02/10] virtio-blk: add host_features field in VirtIOBlock

2019-02-18 Thread Stefano Garzarella
Since configurable features for virtio-blk are growing, this patch adds host_features field in the struct VirtIOBlock. (as in virtio-net) In this way, we can avoid to add new fields for new properties and we can directly set VIRTIO_BLK_F* flags in the host_features. We update "config-wce" and

[Qemu-block] [PATCH v5 07/10] tests/virtio-blk: change assert on data_size in virtio_blk_request()

2019-02-18 Thread Stefano Garzarella
The size of data in the virtio_blk_request must be a multiple of 512 bytes for IN and OUT requests, or a multiple of the size of struct virtio_blk_discard_write_zeroes for DISCARD and WRITE_ZEROES requests. Reviewed-by: Michael S. Tsirkin Reviewed-by: Stefan Hajnoczi Reviewed-by: Thomas Huth

[Qemu-block] [PATCH v5 01/10] virtio-blk: add acct_failed param to virtio_blk_handle_rw_error()

2019-02-18 Thread Stefano Garzarella
We add acct_failed param in order to use virtio_blk_handle_rw_error() also when is not required to call block_acct_failed(). (eg. a discard operation is failed) Reviewed-by: Michael S. Tsirkin Reviewed-by: Stefan Hajnoczi Signed-off-by: Stefano Garzarella --- hw/block/virtio-blk.c | 10

[Qemu-block] [PATCH v5 00/10] virtio-blk: add DISCARD and WRITE_ZEROES features

2019-02-18 Thread Stefano Garzarella
This series adds the support of DISCARD and WRITE_ZEROES commands and extends the virtio-blk-test to test these new commands. v5: - rebased on master - handled the config size for DISCARD and WRITE_ZEROES features as in virtio-net (patches 4 and 5) [Michael, Stefan] - fixed an endianness issues

Re: [Qemu-block] [PATCH v2 2/6] block/dirty-bitmaps: rename frozen predicate helper

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
14.02.2019 2:23, John Snow wrote: > "Frozen" was a good description a long time ago, but it isn't adequate now. > Rename the frozen predicate to has_successor to make the semantics of the > predicate more clear to outside callers. > > In the process, remove some calls to frozen() that no longer

Re: [Qemu-block] [PATCH v2 1/6] block/dirty-bitmap: add recording and busy properties

2019-02-18 Thread Vladimir Sementsov-Ogievskiy
14.02.2019 2:23, John Snow wrote: > The current API allows us to report a single status, which we've defined as: > > Frozen: has a successor, treated as qmp_locked, may or may not be enabled. > Locked: no successor, qmp_locked. may or may not be enabled. > Disabled: Not frozen or locked,

  1   2   >