Re: [PATCH v3] migration/block-dirty-bitmap: fix uninitialized variable warning

2020-10-20 Thread Laurent Vivier
Le 20/10/2020 à 09:36, Chenqun (kuhn) a écrit : > Ping! > > Hello, > > Maybe this patch, some people have any other suggestions? Or, maybe > missed to queue? Hi, As we've seen by reviewing previous versions, this change is not really trivial, so I will not take it via the trivial queue.

RFC: tracking valid backing chain issue

2020-10-20 Thread Nikolay Shirokovskiy
Hi, all. I recently found a corner case when it is impossible AFAIK to find out valid backing chain after block commit operation. Imagine committing top image. After commit ready state pivot is sent and then mgmt crashed. So far so good. Upon next start mgmt can either check block job status for

Re: [PATCH v4 4/7] nbd: Update qapi to support exporting multiple bitmaps

2020-10-20 Thread Markus Armbruster
Eric Blake writes: > On 10/14/20 7:15 AM, Vladimir Sementsov-Ogievskiy wrote: >> 10.10.2020 00:55, Eric Blake wrote: >>> Since 'nbd-server-add' is deprecated, and 'block-export-add' is new to >>> 5.2, we can still tweak the interface.  Allowing 'bitmaps':['str'] is >>> nicer than 'bitmap':'str'. 

Re: RFC: tracking valid backing chain issue

2020-10-20 Thread Kevin Wolf
Am 20.10.2020 um 10:21 hat Nikolay Shirokovskiy geschrieben: > Hi, all. > > I recently found a corner case when it is impossible AFAIK to find out valid > backing chain after block commit operation. Imagine committing top image. > After > commit ready state pivot is sent and then mgmt crashed.

Re: Plans to bring QMP 'x-blockdev-reopen' out of experimental?

2020-10-20 Thread Kashyap Chamarthy
On Mon, Oct 19, 2020 at 05:56:56PM +0200, Alberto Garcia wrote: > On Tue 06 Oct 2020 11:10:01 AM CEST, Kashyap Chamarthy wrote: > > Hi, folks > > > > If this was already discussed on the list, please point me to the > > thread. I took a quick look at my local archives, I didn't find any, > >

Re: [PATCH v7 10/11] hw/block/nvme: Separate read and write handlers

2020-10-20 Thread Klaus Jensen
On Oct 19 11:17, Dmitry Fomichev wrote: > With ZNS support in place, the majority of code in nvme_rw() has > become read- or write-specific. Move these parts to two separate > handlers, nvme_read() and nvme_write() to make the code more > readable and to remove multiple is_write checks that so far

Re: Plans to bring QMP 'x-blockdev-reopen' out of experimental?

2020-10-20 Thread Kevin Wolf
Am 19.10.2020 um 18:46 hat Alberto Garcia geschrieben: > On Mon 19 Oct 2020 05:56:56 PM CEST, Alberto Garcia wrote: > > And this one in particular: > > > >https://lists.gnu.org/archive/html/qemu-block/2020-02/msg00601.html > > I forgot to add, we still don't support changing bs->file with

Re: [PATCH v7 11/11] hw/block/nvme: Merge nvme_write_zeroes() with nvme_write()

2020-10-20 Thread Klaus Jensen
On Oct 19 11:17, Dmitry Fomichev wrote: > nvme_write() now handles WRITE, WRITE ZEROES and ZONE_APPEND. > Same here, Acked, but maybe move it in front as a preparation patch as well? signature.asc Description: PGP signature

Re: [PATCH v2 03/15] python: add VERSION file

2020-10-20 Thread Andrea Bolognani
On Mon, 2020-10-19 at 11:02 +0100, Daniel P. Berrangé wrote: > On Mon, Oct 19, 2020 at 11:45:09AM +0200, Andrea Bolognani wrote: > > I think this need to be considered very carefully. > > > > I'm not overly familiar with the Python ecosystem but it would appear > > that, despite PEP 440 not

Re: [PATCH v7 04/11] hw/block/nvme: Support allocated CNS command variants

2020-10-20 Thread Klaus Jensen
On Oct 19 11:17, Dmitry Fomichev wrote: (snip) > CAP.CSS (together with the I/O Command Set data structure) defines > what command sets are supported by the controller. > > CC.CSS (together with Set Profile) can be set to enable a subset of > the available command sets. > > Even if a user

Re: [PATCH v2 03/15] python: add VERSION file

2020-10-20 Thread Daniel P . Berrangé
On Tue, Oct 20, 2020 at 10:52:14AM +0200, Andrea Bolognani wrote: > On Mon, 2020-10-19 at 11:02 +0100, Daniel P. Berrangé wrote: > > On Mon, Oct 19, 2020 at 11:45:09AM +0200, Andrea Bolognani wrote: > > > With that in mind, I think it would be unwise for qemu.* not to do > > > the same; in

RE: [PATCH v3] migration/block-dirty-bitmap: fix uninitialized variable warning

2020-10-20 Thread Chenqun (kuhn)
Ping! Hello, Maybe this patch, some people have any other suggestions? Or, maybe missed to queue? Thanks, Chen Qun > -Original Message- > From: Vladimir Sementsov-Ogievskiy [mailto:vsement...@virtuozzo.com] > Sent: Wednesday, October 14, 2020 11:56 PM > To: Chenqun (kuhn) ;

Re: Question on Compression for Raw Image

2020-10-20 Thread Kevin Wolf
Am 20.10.2020 um 03:31 hat Wang, Wei W geschrieben: > Hi, > > Does anyone know the reason why raw-format.c doesn't have compression > support (but qcow has the supported added)? For example, raw image > backup with compression, "qemu-img convert -c -O raw origin.img > dist.img", doesn't work. A

Re: [PATCH v2 1/2] qemu-img: add support for offline rate limit in qemu-img commit

2020-10-20 Thread Alberto Garcia
On Tue 20 Oct 2020 01:39:05 PM CEST, Zhengui li wrote: > From: Zhengui > > Currently, there is no rate limit for qemu-img commit. This may > cause the task of qemu-img commit to consume all the bandwidth > of the storage. This will affect the IO performance of other processes > and virtual

Re: [PATCH v7 05/11] hw/block/nvme: Support Zoned Namespace Command Set

2020-10-20 Thread Klaus Jensen
On Oct 19 11:17, Dmitry Fomichev wrote: > diff --git a/hw/block/nvme-ns.c b/hw/block/nvme-ns.c > index 974aea33f7..fedfad595c 100644 > --- a/hw/block/nvme-ns.c > +++ b/hw/block/nvme-ns.c > @@ -133,6 +320,12 @@ static Property nvme_ns_props[] = { > DEFINE_PROP_UINT32("nsid", NvmeNamespace,

[PATCH v2 2/2] qemu-img: add support for rate limit in qemu-img convert

2020-10-20 Thread Zhengui li
From: Zhengui Currently, there is no rate limit for qemu-img convert. This may cause the task of qemu-img convert to consume all the bandwidth of the storage. This will affect the IO performance of other processes and virtual machines under shared storage. So we add support for offline rate

[PATCH v2 1/2] qemu-img: add support for offline rate limit in qemu-img commit

2020-10-20 Thread Zhengui li
From: Zhengui Currently, there is no rate limit for qemu-img commit. This may cause the task of qemu-img commit to consume all the bandwidth of the storage. This will affect the IO performance of other processes and virtual machines under shared storage. So we add support for offline rate limit

Re: RFC: tracking valid backing chain issue

2020-10-20 Thread Nikolay Shirokovskiy
On 20.10.2020 11:50, Kevin Wolf wrote: > Am 20.10.2020 um 10:21 hat Nikolay Shirokovskiy geschrieben: >> Hi, all. >> >> I recently found a corner case when it is impossible AFAIK to find out valid >> backing chain after block commit operation. Imagine committing top image. >> After >> commit

Re: [PATCH v3] migration/block-dirty-bitmap: fix uninitialized variable warning

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
20.10.2020 11:16, Laurent Vivier wrote: Le 20/10/2020 à 09:36, Chenqun (kuhn) a écrit : Ping! Hello, Maybe this patch, some people have any other suggestions? Or, maybe missed to queue? Hi, As we've seen by reviewing previous versions, this change is not really trivial, so I will not

Re: Plans to bring QMP 'x-blockdev-reopen' out of experimental?

2020-10-20 Thread Alberto Garcia
On Tue 20 Oct 2020 10:23:33 AM CEST, Kevin Wolf wrote: >> >https://lists.gnu.org/archive/html/qemu-block/2020-02/msg00601.html >> >> I forgot to add, we still don't support changing bs->file with this >> command, so I guess that would be one blocker? >> >> There's no other way of inserting

Re: [PATCH v2 2/2] qemu-img: add support for rate limit in qemu-img convert

2020-10-20 Thread Alberto Garcia
On Tue 20 Oct 2020 01:39:06 PM CEST, Zhengui li wrote: > +static void set_rate_limit(BlockBackend *blk, int64_t rate_limit) > +{ > +ThrottleConfig cfg; > + > +throttle_config_init(); > +cfg.buckets[THROTTLE_BPS_WRITE].avg = rate_limit; > + > +blk_io_limits_enable(blk,

Re: [PATCH v2 03/15] python: add VERSION file

2020-10-20 Thread Andrea Bolognani
On Tue, 2020-10-20 at 10:06 +0100, Daniel P. Berrangé wrote: > The QEMU python modules are not like other python modules though, > precisely because they are talking to QEMU. If we are shipping > QEMU python releases on the same schedule as QEMU, then we can > expect the normal ase to be updating

Re: RFC: tracking valid backing chain issue

2020-10-20 Thread Nikolay Shirokovskiy
On 20.10.2020 13:23, Nikolay Shirokovskiy wrote: > > > On 20.10.2020 11:50, Kevin Wolf wrote: >> Am 20.10.2020 um 10:21 hat Nikolay Shirokovskiy geschrieben: >>> Hi, all. >>> >>> I recently found a corner case when it is impossible AFAIK to find out valid >>> backing chain after block commit

[PATCH v4 03/14] block: check return value of bdrv_open_child and drop error propagation

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
This patch is generated by cocci script: @@ symbol bdrv_open_child, errp, local_err; expression file; @@ file = bdrv_open_child(..., -_err +errp ); - if (local_err) + if (!file) { ... - error_propagate(errp,

[PATCH v4 02/14] block: use return status of bdrv_append()

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
Now bdrv_append returns status and we can drop all the local_err things around it. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 6 ++ block/backup-top.c | 23 +++ block/commit.c | 6 ++ block/mirror.c

[PATCH v4 08/14] block/qcow2: qcow2_get_specific_info(): drop error propagation

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
Don't use error propagation in qcow2_get_specific_info(). For this refactor qcow2_get_bitmap_info_list, its current interface is rather weird. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Greg Kurz Reviewed-by: Alberto Garcia --- block/qcow2.h| 4 ++--

[PATCH v4 01/14] block: return status from bdrv_append and friends

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
The recommended use of qemu error api assumes returning status together with setting errp and avoid void functions with errp parameter. Let's improve bdrv_append and some friends to reduce error-propagation overhead in further patches. Choose int return status, because bdrv_replace_node() has

[PATCH v4 04/14] blockdev: fix drive_backup_prepare() missed error

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
We leak local_err and don't report failure to the caller. It's definitely wrong, let's fix. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Greg Kurz Reviewed-by: Alberto Garcia --- blockdev.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/blockdev.c

[PATCH v4 12/14] block/qcow2: simplify qcow2_co_invalidate_cache()

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
qcow2_do_open correctly sets errp on each failure path. So, we can simplify code in qcow2_co_invalidate_cache() and drop explicit error propagation. Add ERRP_GUARD() as mandated by the documentation in include/qapi/error.h so that error_prepend() is actually called even if errp is _fatal.

[PATCH v4 05/14] block: drop extra error propagation for bdrv_set_backing_hd

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
bdrv_set_backing_hd now returns status, let's use it. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Greg Kurz Reviewed-by: Alberto Garcia --- block.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/block.c b/block.c index 46d4ad0ea6..0ee36c1869 100644 ---

[PATCH v4 06/14] block/mirror: drop extra error propagation in commit_active_start()

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
Let's check return value of mirror_start_job to check for failure instead of local_err. Rename ret to job, as ret is usually integer variable. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Greg Kurz Reviewed-by: Alberto Garcia --- block/mirror.c | 12 +--- 1 file changed,

[PATCH v4 10/14] block/qcow2-bitmap: return status from qcow2_store_persistent_dirty_bitmaps

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
It's better to return status together with setting errp. It makes possible to avoid error propagation. While being here, put ERRP_GUARD() to fix error_prepend(errp, ...) usage inside qcow2_store_persistent_dirty_bitmaps() (see the comment above ERRP_GUARD() definition in include/qapi/error.h)

[PATCH v4 13/14] block/qed: bdrv_qed_do_open: deal with errp

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
Set errp always on failure. Generic bdrv_open_driver supports driver functions which can return negative value and forget to set errp. That's a strange thing.. Let's improve bdrv_qed_do_open to not behave this way. This allows to simplify code in bdrv_qed_co_invalidate_cache(). Signed-off-by:

[PATCH v4 11/14] block/qcow2: read_cache_sizes: return status value

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
It's better to return status together with setting errp. It allows to reduce error propagation. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Greg Kurz Reviewed-by: Alberto Garcia --- block/qcow2.c | 19 +-- 1 file changed, 9 insertions(+), 10 deletions(-) diff

Re: [PATCH v7 10/11] hw/block/nvme: Separate read and write handlers

2020-10-20 Thread Keith Busch
On Tue, Oct 20, 2020 at 10:28:22AM +0200, Klaus Jensen wrote: > On Oct 19 11:17, Dmitry Fomichev wrote: > > With ZNS support in place, the majority of code in nvme_rw() has > > become read- or write-specific. Move these parts to two separate > > handlers, nvme_read() and nvme_write() to make the

Re: [PATCH v3 12/13] block/qcow2: simplify qcow2_co_invalidate_cache()

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
19.10.2020 16:06, Kevin Wolf wrote: Am 16.10.2020 um 19:10 hat Vladimir Sementsov-Ogievskiy geschrieben: qcow2_do_open correctly sets errp on each failure path. So, we can simplify code in qcow2_co_invalidate_cache() and drop explicit error propagation. qcow2_update_options_prepare() can

[PATCH v4 00/14] block: deal with errp: part I

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
v4: 01: - use -EPERM on bdrv_attach_child failure - "ret = 0" above "out:" - drop r-bs 02: - use "ret = ...; if (ret < 0) { ... }" pattern - use g_assert_cmpint - drop r-bs 14: new Vladimir Sementsov-Ogievskiy (14): block: return status from bdrv_append and friends block: use

[PATCH v4 14/14] block/qcow2: refactor qcow2_update_options_prepare error paths

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
Keep setting ret close to setting errp and don't merge different error paths into one. This way it's more obvious that we don't return error without setting errp. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-)

[PATCH v4 09/14] block/qcow2-bitmap: improve qcow2_load_dirty_bitmaps() interface

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
It's recommended for bool functions with errp to return true on success and false on failure. Non-standard interfaces don't help to understand the code. The change is also needed to reduce error propagation. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by:

[PATCH v4 07/14] blockjob: return status from block_job_set_speed()

2020-10-20 Thread Vladimir Sementsov-Ogievskiy
Better to return status together with setting errp. It allows to avoid error propagation in the caller. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Greg Kurz Reviewed-by: Alberto Garcia --- include/block/blockjob.h | 2 +- blockjob.c | 18 -- 2

Re: [PATCH v3] migration/block-dirty-bitmap: fix uninitialized variable warning

2020-10-20 Thread Eric Blake
On 10/20/20 5:39 AM, Vladimir Sementsov-Ogievskiy wrote: > 20.10.2020 11:16, Laurent Vivier wrote: >> Le 20/10/2020 à 09:36, Chenqun (kuhn) a écrit : >>> Ping! >>> >>> Hello, >>> >>>    Maybe this patch, some people have any other suggestions?   Or, >>> maybe missed to queue? >> >> Hi, >> >> As

RE: Question on Compression for Raw Image

2020-10-20 Thread Wang, Wei W
On Tuesday, October 20, 2020 4:01 PM, Kevin Wolf wrote: > Am 20.10.2020 um 03:31 hat Wang, Wei W geschrieben: > > Hi, > > > > Does anyone know the reason why raw-format.c doesn't have > compression > > support (but qcow has the supported added)? For example, raw image > > backup with compression,

Re: Question on Compression for Raw Image

2020-10-20 Thread Eric Blake
On 10/20/20 9:22 AM, Wang, Wei W wrote: > On Tuesday, October 20, 2020 4:01 PM, Kevin Wolf wrote: >> Am 20.10.2020 um 03:31 hat Wang, Wei W geschrieben: >>> Hi, >>> >>> Does anyone know the reason why raw-format.c doesn't have >> compression >>> support (but qcow has the supported added)? For

RE: Question on Compression for Raw Image

2020-10-20 Thread Alberto Garcia
On Tue 20 Oct 2020 04:22:43 PM CEST, Wang, Wei W wrote: > Ok, thanks. I'm thinking QEMU could do decompression of the compressed > data in raw.img when guest reads data. The qcow2 format already supports compression and it's already transparent to the guest, so you can use that. As Kevin said

[PATCH v2 0/2] qemu-img: add add support for rate limit in commit and convert

2020-10-20 Thread Zhengui li
Currently, there is no rate limit for qemu-img commit and convert. This may cause the task of qemu-img commit and convert to consume all the bandwidth of the storage. This will affect the IO performance of other processes and virtual machines under shared storage. The following patches add

[PATCH v2 2/2] qemu-img: add support for rate limit in qemu-img convert

2020-10-20 Thread Zhengui li
From: Zhengui add support for rate limit in qemu-img convert. Signed-off-by: Zhengui --- docs/tools/qemu-img.rst | 6 +- qemu-img-cmds.hx| 4 ++-- qemu-img.c | 27 ++- 3 files changed, 33 insertions(+), 4 deletions(-) diff --git

[PATCH v2 1/2] qemu-img: add support for rate limit in qemu-img commit

2020-10-20 Thread Zhengui li
From: Zhengui add support for rate limit in qemu-img commit. Signed-off-by: Zhengui --- docs/tools/qemu-img.rst | 4 +++- qemu-img-cmds.hx| 4 ++-- qemu-img.c | 11 +-- 3 files changed, 14 insertions(+), 5 deletions(-) diff --git a/docs/tools/qemu-img.rst

Re: [PATCH 1/2] hw/block/m25p80: Fix Numonyx dummy cycle register behavior

2020-10-20 Thread Francisco Iglesias
Hi Joe, On Tue, Sep 29, 2020 at 05:28:35PM -0700, Joe Komlodi wrote: > Numonyx chips determine the number of cycles to wait based on bits 7:4 in the > volatile configuration register. > > However, if these bits are 0x0 or 0xF, the number of dummy cycles to wait is > 10 on a QIOR or QIOR4

Re: Question on Compression for Raw Image

2020-10-20 Thread Richard W.M. Jones
On Tue, Oct 20, 2020 at 09:32:23AM -0500, Eric Blake wrote: > On 10/20/20 9:22 AM, Wang, Wei W wrote: > > On Tuesday, October 20, 2020 4:01 PM, Kevin Wolf wrote: > >> Am 20.10.2020 um 03:31 hat Wang, Wei W geschrieben: > >>> Hi, > >>> > >>> Does anyone know the reason why raw-format.c doesn't have

[PATCH v3 0/2] qemu-img: add support for rate limit in commit and convert

2020-10-20 Thread Zhengui li
Currently, there is no rate limit for qemu-img commit and convert. This may cause the task of qemu-img commit and convert to consume all the bandwidth of the storage. This will affect the IO performance of other processes and virtual machines under shared storage. The following patches add

[PATCH v3 2/2] qemu-img: add support for rate limit in qemu-img convert

2020-10-20 Thread Zhengui li
From: Zhengui add support for rate limit in qemu-img convert. Signed-off-by: Zhengui --- docs/tools/qemu-img.rst | 6 +- qemu-img-cmds.hx| 4 ++-- qemu-img.c | 27 ++- 3 files changed, 33 insertions(+), 4 deletions(-) diff --git

[PATCH v3 1/2] qemu-img: add support for rate limit in qemu-img commit

2020-10-20 Thread Zhengui li
From: Zhengui add support for rate limit in qemu-img commit. Signed-off-by: Zhengui --- docs/tools/qemu-img.rst | 4 +++- qemu-img-cmds.hx| 4 ++-- qemu-img.c | 11 +-- 3 files changed, 14 insertions(+), 5 deletions(-) diff --git a/docs/tools/qemu-img.rst

Re: Question on Compression for Raw Image

2020-10-20 Thread Alberto Garcia
On Tue 20 Oct 2020 04:32:23 PM CEST, Eric Blake wrote: > My recommendation would be implementing a new BDS filter that does > uncompression. Then, you could do things like: > > raw -> decompress -> file.xz This would work, although read-only and you would need a compression format that supports

Re: [PATCH v3 0/2] qemu-img: add support for rate limit in commit and convert

2020-10-20 Thread Alberto Garcia
On Tue 20 Oct 2020 04:47:42 PM CEST, Zhengui li wrote: > Currently, there is no rate limit for qemu-img commit and convert. This may > cause the task of qemu-img commit and convert to consume all the bandwidth > of the storage. This will affect the IO performance of other processes > and virtual

Re: [PULL v2 00/30] Block patches

2020-10-20 Thread Stefan Hajnoczi
On Mon, Oct 12, 2020 at 10:48:17PM +0100, Peter Maydell wrote: > On Mon, 12 Oct 2020 at 19:28, Stefan Hajnoczi wrote: > > > > The following changes since commit 2387df497b4b4bcf754eb7398edca82889e2ef54: > > > > Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2020-10-10' > > into

[PATCH 00/16] util/vfio-helpers: Allow using multiple MSIX IRQs

2020-10-20 Thread Philippe Mathieu-Daudé
This series allow using multiple MSIX IRQs We currently share a single IRQ between 2 NVMe queues (ADMIN and I/O). This series still uses 1 shared IRQ but prepare for using multiple ones. The series is organized as: - Fix device minimum page size (prerequisite: patch 1) - Check IOMMU minimum page

[PULL 09/21] python/machine.py: fix _popen access

2020-10-20 Thread John Snow
As always, Optional[T] causes problems with unchecked access. Add a helper that asserts the pipe is present before we attempt to talk with it. Signed-off-by: John Snow Reviewed-by: Kevin Wolf Message-id: 20201006235817.3280413-9-js...@redhat.com Signed-off-by: John Snow ---

[PULL 10/21] python/qemu: make 'args' style arguments immutable

2020-10-20 Thread John Snow
These arguments don't need to be mutable and aren't really used as such. Clarify their types as immutable and adjust code to match where necessary. In general, It's probably best not to accept a user-defined mutable object and store it as internal object state unless there's a strong

[PULL 07/21] python/machine.py: use qmp.command

2020-10-20 Thread John Snow
machine.py and qmp.py both do the same thing here; refactor machine.py to use qmp.py's functionality more directly. Signed-off-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Philippe Mathieu-Daudé Message-id: 20201006235817.3280413-7-js...@redhat.com Signed-off-by: John Snow ---

Re: [PULL 20/21] python/qemu/qmp.py: re-raise OSError when encountered

2020-10-20 Thread John Snow
On 10/20/20 2:15 PM, Nir Soffer wrote: On Tue, Oct 20, 2020 at 8:52 PM John Snow wrote: Nested if conditions don't change when the exception block fires; we need to explicitly re-raise the error if we didn't intend to capture and suppress it. Signed-off-by: John Snow Reviewed-by: Philippe

[PATCH v3 07/15] python: move pylintrc into setup.cfg

2020-10-20 Thread John Snow
Delete the empty settings now that it's sharing a home with settings for other tools. pylint can now be run from this folder as "pylint qemu". Signed-off-by: John Snow --- python/qemu/machine/pylintrc | 58 python/setup.cfg | 29

[PATCH v3 08/15] python: add pylint to pipenv

2020-10-20 Thread John Snow
We are specifying >= pylint 2.6.x for two reasons: 1. For setup.cfg support, added in pylint 2.5.x 2. To clarify that we are using a version that has incompatibly dropped bad-whitespace checks. Signed-off-by: John Snow --- python/Pipfile | 1 + python/Pipfile.lock | 127

[PATCH 01/16] block/nvme: Correct minimum device page size

2020-10-20 Thread Philippe Mathieu-Daudé
While trying to simplify the code using a macro, we forgot the 12-bit shift... Correct that. Fixes: fad1eb68862 ("block/nvme: Use register definitions from 'block/nvme.h'") Reported-by: Eric Auger Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 2 +- 1 file changed, 1 insertion(+), 1

[PATCH 09/16] util/vfio-helpers: Convert vfio_dump_mapping to trace events

2020-10-20 Thread Philippe Mathieu-Daudé
The QEMU_VFIO_DEBUG definition is only modifiable at build-time. Trace events can be enabled at run-time. As we prefer the latter, convert qemu_vfio_dump_mappings() to use trace events instead of fprintf(). Reviewed-by: Fam Zheng Signed-off-by: Philippe Mathieu-Daudé --- util/vfio-helpers.c |

[PATCH 13/16] util/vfio-helpers: Introduce qemu_vfio_pci_msix_init_irqs()

2020-10-20 Thread Philippe Mathieu-Daudé
qemu_vfio_pci_init_irq() allows us to initialize any type of IRQ, but only one. Introduce qemu_vfio_pci_msix_init_irqs() which is specific to MSIX IRQ type, and allow us to use multiple IRQs (thus passing multiple eventfd notifiers). All eventfd notifiers are initialized with the special '-1'

[PATCH 12/16] util/vfio-helpers: Let qemu_vfio_verify_mappings() use error_report()

2020-10-20 Thread Philippe Mathieu-Daudé
Instead of displaying the error on stderr, use error_report() which also report to the monitor. Reviewed-by: Fam Zheng Signed-off-by: Philippe Mathieu-Daudé --- util/vfio-helpers.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c

[PATCH 02/16] util/vfio-helpers: Improve reporting unsupported IOMMU type

2020-10-20 Thread Philippe Mathieu-Daudé
Change the confuse "VFIO IOMMU check failed" error message by the explicit "VFIO IOMMU Type1 is not supported" once. Example on POWER: $ qemu-system-ppc64 -drive if=none,id=nvme0,file=nvme://0001:01:00.0/1,format=raw qemu-system-ppc64: -drive

[PATCH 05/16] util/vfio-helpers: Trace PCI I/O config accesses

2020-10-20 Thread Philippe Mathieu-Daudé
We sometime get kernel panic with some devices on Aarch64 hosts. Alex Williamson suggests it might be broken PCIe root complex. Add trace event to record the latest I/O access before crashing. In case, assert our accesses are aligned. Reviewed-by: Fam Zheng Signed-off-by: Philippe Mathieu-Daudé

[PATCH 10/16] util/vfio-helpers: Let qemu_vfio_dma_map() propagate Error

2020-10-20 Thread Philippe Mathieu-Daudé
Currently qemu_vfio_dma_map() displays errors on stderr. When using management interface, this information is simply lost. Pass qemu_vfio_dma_map() an Error* argument so it can propagate the error to callers. Reviewed-by: Fam Zheng Signed-off-by: Philippe Mathieu-Daudé ---

[PATCH 07/16] util/vfio-helpers: Trace where BARs are mapped

2020-10-20 Thread Philippe Mathieu-Daudé
For debugging purpose, trace where a BAR is mapped. Reviewed-by: Fam Zheng Signed-off-by: Philippe Mathieu-Daudé --- util/vfio-helpers.c | 2 ++ util/trace-events | 1 + 2 files changed, 3 insertions(+) diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c index 4204ce55445..95e010bed6d

[PATCH 03/16] util/vfio-helpers: Pass minimum page size to qemu_vfio_open_pci()

2020-10-20 Thread Philippe Mathieu-Daudé
The block driver asks for a minimum page size, but it might not match the minimum IOMMU requirement. In the next commit qemu_vfio_init_pci() will be able to report the minimum IOMMU page size back to the block driver. In preparation, pass the minimum page size as argument to qemu_vfio_open_pci().

[PULL 02/21] python/qemu: use isort to lay out imports

2020-10-20 Thread John Snow
Borrowed from the QAPI cleanup series, use the same configuration to standardize the way we write and sort imports. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Kevin Wolf Message-id: 20201006235817.3280413-2-js...@redhat.com Signed-off-by: John Snow ---

[PULL 04/21] python/machine.py: reorder __init__

2020-10-20 Thread John Snow
Put the init arg handling all at the top, and mostly in order (deviating when one is dependent on another), and put what is effectively runtime state declaration at the bottom. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Kevin Wolf Message-id:

[PULL 16/21] python/qemu/console_socket.py: Add type hint annotations

2020-10-20 Thread John Snow
Finish the typing of console_socket.py with annotations and no code changes. Signed-off-by: John Snow Reviewed-by: Kevin Wolf Message-id: 20201006235817.3280413-16-js...@redhat.com Signed-off-by: John Snow --- python/qemu/console_socket.py | 18 +- 1 file changed, 9

[PULL v2 00/21] Python patches

2020-10-20 Thread John Snow
The following changes since commit 4c41341af76cfc85b5a6c0f87de4838672ab9f89: Merge remote-tracking branch 'remotes/aperard/tags/pull-xen-20201020' into staging (2020-10-20 11:20:36 +0100) are available in the Git repository at: https://gitlab.com/jsnow/qemu.git tags/python-pull-request

Re: [PATCH v11 13/13] block: apply COR-filter to block-stream jobs

2020-10-20 Thread Andrey Shinkevich
On 16.10.2020 18:45, Vladimir Sementsov-Ogievskiy wrote: 15.10.2020 20:16, Andrey Shinkevich wrote: On 14.10.2020 19:24, Max Reitz wrote: On 12.10.20 19:43, Andrey Shinkevich wrote: [...] ---   block/stream.c | 93 +-  

[PATCH 3/3] ide: clear SRST after SRST finishes

2020-10-20 Thread John Snow
The SRST protocol states that after diagnostics are complete and the status is posted, we should clear the SRST bit if it should so happen to be set. The reset method itself should handle this, but just in case -- make our intention explicit here. Signed-off-by: John Snow --- hw/ide/core.c | 2

[PATCH 2/3] ide: perform SRST as early as possible

2020-10-20 Thread John Snow
We don't need to wait for the falling edge. We can set BSY as soon as possible and begin immediately resetting the drive. Devices don't appear to need to take any specific action on the falling edge. Signed-off-by: John Snow --- hw/ide/core.c | 4 +--- 1 file changed, 1 insertion(+), 3

[PATCH 11/16] util/vfio-helpers: Let qemu_vfio_do_mapping() propagate Error

2020-10-20 Thread Philippe Mathieu-Daudé
Pass qemu_vfio_do_mapping() an Error* argument so it can propagate any error to callers. Replace error_report() which only report to the monitor by the more generic error_setg_errno(). Reviewed-by: Fam Zheng Signed-off-by: Philippe Mathieu-Daudé --- util/vfio-helpers.c | 8 1 file

[PULL 05/21] python/machine.py: Don't modify state in _base_args()

2020-10-20 Thread John Snow
Don't append to the _remove_files list during _base_args; instead do so during _launch. Rework _base_args as a @property to help facilitate this impression. This has the additional benefit of making the type of _console_address easier to analyze statically. Signed-off-by: John Snow Reviewed-by:

[PATCH 15/16] block/nvme: Switch to using the MSIX API

2020-10-20 Thread Philippe Mathieu-Daudé
In preparation of using multiple IRQs, switch to using the recently introduced MSIX API. Instead of allocating and assigning IRQ in a single step, we now have to use two distinct calls. Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 14 -- 1 file changed, 12 insertions(+),

[PULL 00/21] Python patches

2020-10-20 Thread John Snow
The following changes since commit 4c41341af76cfc85b5a6c0f87de4838672ab9f89: Merge remote-tracking branch 'remotes/aperard/tags/pull-xen-20201020' into staging (2020-10-20 11:20:36 +0100) are available in the Git repository at: https://gitlab.com/jsnow/qemu.git tags/python-pull-request

Re: [PATCH v4 4/7] nbd: Update qapi to support exporting multiple bitmaps

2020-10-20 Thread Eric Blake
On 10/20/20 3:51 AM, Markus Armbruster wrote: >> #define QAPI_LIST_ADD(list, element) do { \ >> typeof(list) _tmp = g_new(typeof(*(list)), 1); \ >> _tmp->value = (element); \ >> _tmp->next = (list); \ >> (list) = _tmp; \ >> } while (0) >> >> >> Markus, thoughts on if we should

[PATCH 1/3] ide: run diagnostic after SRST

2020-10-20 Thread John Snow
Software reset (SRST) should cause the diagnostic command to be run. Make an explicit call to that routine. Reported-by: Mark Cave-Ayland Fixes: 55adb3c45620c31f29978f209e2a44a08d34e2da Fixes: https://bugs.launchpad.net/bugs/1900155 Tested-by: Mark Cave-Ayland Signed-off-by: John Snow ---

[PULL 17/21] python/qemu/console_socket.py: avoid encoding to/from string

2020-10-20 Thread John Snow
We can work directly in bytes instead of translating back and forth to string, which removes the question of which encodings to use. Signed-off-by: John Snow Reviewed-by: Kevin Wolf Message-id: 20201006235817.3280413-17-js...@redhat.com Signed-off-by: John Snow ---

[PULL 21/21] python/qemu/qmp.py: Fix settimeout operation

2020-10-20 Thread John Snow
We enabled callers to interface directly with settimeout, but this reacts poorly with blocking/nonblocking operation; as they are using the same internal mechanism. 1. Whenever we change the blocking mechanism temporarily, always set it back to what it was afterwards. 2. Disallow callers from

[PULL 19/21] python: add mypy config

2020-10-20 Thread John Snow
Formalize the options used for checking the python library. You can run mypy from the directory that mypy.ini is in by typing `mypy qemu/`. Signed-off-by: John Snow Message-id: 20201009175123.249009-2-js...@redhat.com [Edit: Added newline; thanks Bin Meng --js] Signed-off-by: John Snow ---

[PULL 14/21] python/qemu/console_socket.py: fix typing of settimeout

2020-10-20 Thread John Snow
The types and names of the parameters must match the socket.socket interface. Signed-off-by: John Snow Reviewed-by: Kevin Wolf Message-id: 20201006235817.3280413-14-js...@redhat.com Signed-off-by: John Snow --- python/qemu/console_socket.py | 10 ++ 1 file changed, 6 insertions(+), 4

[PATCH v3 01/15] python: create qemu packages

2020-10-20 Thread John Snow
move python/qemu/*.py to python/qemu/[machine, qmp]/*.py and update import directives across the tree. This is done to create a PEP420 namespace package, in which we may create subpackages. To do this, the namespace directory ("qemu") should not have any modules in it. Those files will go into

[PATCH 0/3] ide: fix SRST (again)

2020-10-20 Thread John Snow
My last fix for SRST wasn't entirely correct and caused a regression; namely we need to mime execution of the diagnostic command. If we don't, Linux bins the IDE device as 'horked'. For detailed reading on the SRST protocol, I recommend checking out the ATA8-APT specification. John Snow (3):

[PATCH 08/16] util/vfio-helpers: Improve DMA trace events

2020-10-20 Thread Philippe Mathieu-Daudé
For debugging purpose, trace where DMA regions are mapped. Reviewed-by: Fam Zheng Signed-off-by: Philippe Mathieu-Daudé --- util/vfio-helpers.c | 3 ++- util/trace-events | 5 +++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c index

[PULL 01/21] MAINTAINERS: Add Python library stanza

2020-10-20 Thread John Snow
I'm proposing that I split the actual Python library off from the other miscellaneous python scripts we have and declare it maintained. Add myself as a maintainer of this folder, along with Cleber. I will be actively working to add CI style guide checks, strict typing, and an actual package

[PATCH v3 09/15] python: move flake8 config to setup.cfg

2020-10-20 Thread John Snow
Update the comment concerning the flake8 exception to match commit 42c0dd12, whose commit message stated: A note on the flake8 exception: flake8 will warn on *any* bare except, but pylint's is context-aware and will suppress the warning if you re-raise the exception. Signed-off-by: John Snow

[PATCH 04/16] util/vfio-helpers: Report error when IOMMU page size is not supported

2020-10-20 Thread Philippe Mathieu-Daudé
This driver uses the host page size to align its memory regions, but this size is not always compatible with the IOMMU. Add a check if the size matches, and bails out providing a hint what is the minimum page size the driver should use. Suggested-by: Alex Williamson Signed-off-by: Philippe

[PULL 18/21] python/qemu/qmp.py: Preserve error context on re-raise

2020-10-20 Thread John Snow
Use the "from ..." phrasing when re-raising errors to preserve their initial context, to help aid debugging when things go wrong. This also silences a pylint 2.6.0+ error. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Kevin Wolf Message-id:

[PULL 08/21] python/machine.py: Add _qmp access shim

2020-10-20 Thread John Snow
Like many other Optional[] types, it's not always a given that this object will be set. Wrap it in a type-shim that raises a meaningful error and will always return a concrete type. Signed-off-by: John Snow Reviewed-by: Kevin Wolf Message-id: 20201006235817.3280413-8-js...@redhat.com

[PATCH v3 10/15] python: Add flake8 to pipenv

2020-10-20 Thread John Snow
flake8 3.5.x does not support the --extend-ignore syntax used in the .flake8 file to gracefully extend default ignores, so 3.6.x is our minimum requirement. There is no known upper bound. flake8 can be run from the python/ directory with no arguments. Signed-off-by: John Snow ---

[PATCH 14/16] util/vfio-helpers: Introduce qemu_vfio_pci_msix_set_irq()

2020-10-20 Thread Philippe Mathieu-Daudé
Introduce qemu_vfio_pci_msix_set_irq() to set the event notifier of a specific MSIX IRQ. All other registered IRQs are left unmodified. Signed-off-by: Philippe Mathieu-Daudé --- include/qemu/vfio-helpers.h | 2 ++ util/vfio-helpers.c | 35 +++

[PULL 06/21] python/machine.py: Handle None events in events_wait

2020-10-20 Thread John Snow
If the timeout is 0, we can get None back. Handle this explicitly. Signed-off-by: John Snow Reviewed-by: Kevin Wolf Message-id: 20201006235817.3280413-6-js...@redhat.com Signed-off-by: John Snow --- python/qemu/machine.py | 27 --- 1 file changed, 20 insertions(+), 7

[PATCH v3 11/15] python: move mypy.ini into setup.cfg

2020-10-20 Thread John Snow
mypy supports reading its configuration values from a central project configuration file; do so. Signed-off-by: John Snow --- python/mypy.ini | 4 python/setup.cfg | 5 + 2 files changed, 5 insertions(+), 4 deletions(-) delete mode 100644 python/mypy.ini diff --git a/python/mypy.ini

  1   2   >