[PULL 07/10] block/dirty-bitmap: improve _next_dirty_area API

2020-03-16 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Firstly, _next_dirty_area is for scenarios when we may contiguously search for next dirty area inside some limited region, so it is more comfortable to specify "end" which should not be recalculated on each iteration. Secondly, let's add a possibility to limit

[PULL 05/10] block/dirty-bitmap: switch _next_dirty_area and _next_zero to int64_t

2020-03-16 Thread John Snow
From: Vladimir Sementsov-Ogievskiy We are going to introduce bdrv_dirty_bitmap_next_dirty so that same variable may be used to store its return value and to be its parameter, so it would int64_t. Similarly, we are going to refactor hbitmap_next_dirty_area to use hbitmap_next_dirty together with

[PULL 10/10] block/qcow2-bitmap: use bdrv_dirty_bitmap_next_dirty

2020-03-16 Thread John Snow
From: Vladimir Sementsov-Ogievskiy store_bitmap_data() loop does bdrv_set_dirty_iter() on each iteration, which means that we actually don't need iterator itself and we can use simpler bitmap API. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow

[PULL 09/10] nbd/server: use bdrv_dirty_bitmap_next_dirty_area

2020-03-16 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Use bdrv_dirty_bitmap_next_dirty_area for bitmap_to_extents. Since bdrv_dirty_bitmap_next_dirty_area is very accurate in its interface, we'll never exceed requested region with last chunk. So, we don't need dont_fragment, and bitmap_to_extents() interface

[PULL 04/10] hbitmap: drop meta bitmaps as they are unused

2020-03-16 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow Message-id: 20200205112041.6003-5-vsement...@virtuozzo.com Signed-off-by: John Snow --- include/qemu/hbitmap.h | 21 tests/test-hbitmap.c | 115

[PULL 08/10] nbd/server: introduce NBDExtentArray

2020-03-16 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Introduce NBDExtentArray class, to handle extents list creation in more controlled way and with fewer OUT parameters in functions. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Message-id: 20200205112041.6003-9-vsement...@virtuozzo.com

[PULL 02/10] hbitmap: move hbitmap_iter_next_word to hbitmap.c

2020-03-16 Thread John Snow
From: Vladimir Sementsov-Ogievskiy The function is definitely internal (it's not used by third party and it has complicated interface). Move it to .c file. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow Message-id:

[PULL 06/10] block/dirty-bitmap: add _next_dirty API

2020-03-16 Thread John Snow
From: Vladimir Sementsov-Ogievskiy We have bdrv_dirty_bitmap_next_zero, let's add corresponding bdrv_dirty_bitmap_next_dirty, which is more comfortable to use than bitmap iterators in some cases. For test modify test_hbitmap_next_zero_check_range to check both next_zero and next_dirty and add

[PULL 00/10] Bitmaps patches

2020-03-16 Thread John Snow
The following changes since commit 6e8a73e911f066527e775e04b98f31ebd19db600: Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging (2020-03-11 14:41:27 +) are available in the Git repository at: https://github.com/jnsnow/qemu.git tags/bitmaps-pull-request

[PULL 03/10] hbitmap: unpublish hbitmap_iter_skip_words

2020-03-16 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Function is internal and even commented as internal. Drop its definition from .h file. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow Message-id: 20200205112041.6003-4-vsement...@virtuozzo.com Signed-off-by: John

[PULL 01/10] hbitmap: assert that we don't create bitmap larger than INT64_MAX

2020-03-16 Thread John Snow
From: Vladimir Sementsov-Ogievskiy We have APIs which returns signed int64_t, to be able to return error. Therefore we can't handle bitmaps with absolute size larger than (INT64_MAX+1). Still, keep maximum to be INT64_MAX which is a bit safer. Note, that bitmaps are used to represent disk

[PATCH v3 3/3] iotests: modify test 040 to use JobRunner

2020-03-16 Thread John Snow
Instead of having somewhat reproduced it for itself. Signed-off-by: John Snow --- tests/qemu-iotests/040 | 51 +- 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 index 90b59081ff..e2ef3bb812

[PATCH v3 1/3] qmp.py: change event_wait to use a dict

2020-03-16 Thread John Snow
It's easier to work with than a list of tuples, because we can check the keys for membership. Signed-off-by: John Snow --- python/qemu/machine.py| 10 +- tests/qemu-iotests/040| 12 ++-- tests/qemu-iotests/260| 5 +++-- tests/qemu-iotests/iotests.py | 16

[PATCH v3 2/3] iotests: add JobRunner class

2020-03-16 Thread John Snow
The idea is that instead of increasing the arguments to job_run all the time, create a more general-purpose job runner that can be subclassed to do interesting things with. pylint note: the 'callbacks' option guards against unused warning arguments in functions designated as callbacks. It does

[PATCH v3 0/3] iotests: add JobRunner framework

2020-03-16 Thread John Snow
Requires: 20200317004105.27059-1-js...@redhat.com (This requires the iotests pylint & logging series.) The basic idea is to make a generic job runtime manager and allow callers to subclass the manager. Then, instead of adding callback arguments to the function all the time, we have à la carte

Re: [PATCH 0/8] Misc hw/ide legacy clean up

2020-03-16 Thread John Snow
On 3/16/20 9:41 AM, BALATON Zoltan wrote: > On Mon, 16 Mar 2020, BALATON Zoltan wrote: >> On Mon, 16 Mar 2020, Markus Armbruster wrote: >>> BALATON Zoltan writes: These are some clean ups to remove more legacy init functions and lessen dependence on include/hw/ide.h with some

Re: [PATCH 0/7] via-ide: fixes and improvements

2020-03-16 Thread John Snow
On 3/14/20 5:15 AM, Mark Cave-Ayland wrote: > On 13/03/2020 17:57, John Snow wrote: > >> On 3/13/20 4:24 AM, Mark Cave-Ayland wrote: >>> Following on from the earlier thread "Implement "non 100% native mode" >>> in via-ide", here is an updated patchset based upon the test cases >>> sent to me

[PATCH v8 08/11] iotests: add script_initialize

2020-03-16 Thread John Snow
Like script_main, but doesn't require a single point of entry. Replace all existing initialization sections with this drop-in replacement. This brings debug support to all existing script-style iotests. Signed-off-by: John Snow Reviewed-by: Max Reitz --- tests/qemu-iotests/149| 3 +-

[PATCH v8 10/11] iotests: Mark verify functions as private

2020-03-16 Thread John Snow
Mark the verify functions as "private" with a leading underscore, to discourage their use. (Also, make pending patches not yet using the new entry points fail in a very obvious way.) Signed-off-by: John Snow Reviewed-by: Max Reitz --- tests/qemu-iotests/iotests.py | 20 ++-- 1

[PATCH v8 01/11] iotests: do a light delinting

2020-03-16 Thread John Snow
This doesn't fix everything in here, but it does help clean up the pylint report considerably. This should be 100% style changes only; the intent is to make pylint more useful by working on establishing a baseline for iotests that we can gate against in the future. Signed-off-by: John Snow

[PATCH v8 06/11] iotests: drop Python 3.4 compatibility code

2020-03-16 Thread John Snow
We no longer need to accommodate 3.4, drop this code. (Also, the line is over 79 characters, so drop it.) Touch up the docstring a little bit while we're here. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff

[PATCH v8 11/11] iotests: use python logging for iotests.log()

2020-03-16 Thread John Snow
We can turn logging on/off globally instead of per-function. Remove use_log from run_job, and use python logging to turn on diffable output when we run through a script entry point. iotest 245 changes output order due to buffering reasons. An extended note on python logging: A NullHandler is

[PATCH v8 03/11] iotests: ignore import warnings from pylint

2020-03-16 Thread John Snow
The right way to solve this is to come up with a virtual environment infrastructure that sets all the paths correctly, and/or to create installable python modules that can be imported normally. That's hard, so just silence this error for now. Signed-off-by: John Snow ---

[PATCH v8 09/11] iotest 258: use script_main

2020-03-16 Thread John Snow
Since this one is nicely factored to use a single entry point, use script_main to run the tests. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Max Reitz --- tests/qemu-iotests/258 | 11 --- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git

[PATCH v8 07/11] iotests: limit line length to 79 chars

2020-03-16 Thread John Snow
79 is the PEP8 recommendation. This recommendation works well for reading patch diffs in TUI email clients. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 64 +++ tests/qemu-iotests/pylintrc | 6 +++- 2 files changed, 47 insertions(+), 23

[PATCH v8 05/11] iotests: add pylintrc file

2020-03-16 Thread John Snow
This allows others to get repeatable results with pylint. If you run `pylint iotests.py`, you should see a 100% pass. Signed-off-by: John Snow --- tests/qemu-iotests/pylintrc | 22 ++ 1 file changed, 22 insertions(+) create mode 100644 tests/qemu-iotests/pylintrc diff

[PATCH v8 04/11] iotests: replace mutable list default args

2020-03-16 Thread John Snow
It's bad hygiene: if we modify this list, it will be modified across all invocations. (Remaining bad usages are fixed in a subsequent patch which changes the function signature anyway.) Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé --- tests/qemu-iotests/iotests.py | 24

[PATCH v8 00/11] iotests: use python logging

2020-03-16 Thread John Snow
This series uses python logging to enable output conditionally on iotests.log(). We unify an initialization call (which also enables debugging output for those tests with -d) and then make the switch inside of iotests. It will help alleviate the need to create logged/unlogged versions of all the

[PATCH v8 02/11] iotests: don't use 'format' for drive_add

2020-03-16 Thread John Snow
It shadows (with a different type) the built-in format. Use something else. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Max Reitz --- tests/qemu-iotests/055| 3 ++- tests/qemu-iotests/iotests.py | 6 +++--- 2 files changed, 5 insertions(+), 4 deletions(-)

Re: [PATCH] scsi/qemu-pr-helper: Fix out-of-bounds access to trnptid_list[]

2020-03-16 Thread Paolo Bonzini
On 28/02/20 16:00, Christophe de Dinechin wrote: > Compile error reported by gcc 10.0.1: > > scsi/qemu-pr-helper.c: In function ‘multipath_pr_out’: > scsi/qemu-pr-helper.c:523:32: error: array subscript is outside > array bounds of ‘struct transportid *[0]’ [-Werror=array-bounds] > 523 |

Re: [PATCH 3/3] iotests: Increase pause_wait() timeout

2020-03-16 Thread John Snow
On 3/13/20 4:36 AM, Kevin Wolf wrote: > Waiting for only 1 second proved to be too short on a loaded system, > resulting in false positives when testing pull requests. Increase the > timeout a bit to make this less likely. > > Signed-off-by: Kevin Wolf Reviewed-by: John Snow > --- >

Re: [PATCH 2/3] python/qemu: Kill QEMU process if 'quit' doesn't work

2020-03-16 Thread John Snow
On 3/13/20 4:36 AM, Kevin Wolf wrote: > With a QEMU bug, it can happen that the QEMU process doesn't react to a > 'quit' QMP command. If we got an exception during previous QMP > communication (e.g. iotests Timeout expiring), we could also be in an > inconsistent state where after sending

Re: [PATCH 1/3] iotests.py: Enable faulthandler

2020-03-16 Thread John Snow
On 3/13/20 4:36 AM, Kevin Wolf wrote: > With this, you can send SIGABRT to a hanging test case and you'll get a > Python stack trace so you know where it was hanging. > > Signed-off-by: Kevin Wolf Reviewed-by: John Snow

Re: [PATCH v6 00/42] nvme: support NVMe v1.3d, SGLs and multiple namespaces

2020-03-16 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200316142928.153431-1-...@irrelevant.dk/ Hi, This series seems to have some coding style problems. See output below for more information: Subject: [PATCH v6 00/42] nvme: support NVMe v1.3d, SGLs and multiple namespaces Message-id:

Re: [PATCH 2/2] block/qcow2: zero data_file child after free

2020-03-16 Thread John Snow
On 3/16/20 2:06 AM, Vladimir Sementsov-Ogievskiy wrote: > data_file being NULL doesn't seem to be a correct state, but it's > better than dead pointer and simpler to debug. > How important is it to have correct state in the middle of teardown? > Signed-off-by: Vladimir Sementsov-Ogievskiy >

Re: [PATCH 1/2] block: bdrv_set_backing_bs: fix use-after-free

2020-03-16 Thread John Snow
On 3/16/20 4:47 AM, Philippe Mathieu-Daudé wrote: > On 3/16/20 7:06 AM, Vladimir Sementsov-Ogievskiy wrote: >> There is a use-after-free possible: bdrv_unref_child() leaves >> bs->backing freed but not NULL. bdrv_attach_child may produce nested >> polling loop due to drain, than access of freed

Re: [PATCH RESEND v2] block/nvme: introduce PMR support from NVMe 1.4 spec

2020-03-16 Thread Andrzej Jakowski
On 3/16/20 4:32 AM, Stefan Hajnoczi wrote: > On Wed, Mar 11, 2020 at 11:08:27PM -0700, Klaus Birkelund Jensen wrote: >> On Mar 11 15:54, Andrzej Jakowski wrote: >>> On 3/11/20 2:20 AM, Stefan Hajnoczi wrote: Please try: $ git grep pmem backends/hostmem-file.c is the

Re: [PATCH v2 1/4] m25p80: Convert to support tracing

2020-03-16 Thread Peter Maydell
On Mon, 16 Mar 2020 at 15:11, Cédric Le Goater wrote: > > On 3/16/20 3:58 PM, Peter Maydell wrote: > > On Mon, 16 Mar 2020 at 14:14, Cédric Le Goater wrote: > >> > >> Hello, > >> > >> On 2/17/20 4:47 PM, Cédric Le Goater wrote: > >>> Hello all, > >>> > >>> On 2/6/20 7:32 PM, Guenter Roeck wrote:

Re: [PULL 04/31] util/async: make bh_aio_poll() O(1)

2020-03-16 Thread Marc-André Lureau
Hi On Sat, Feb 22, 2020 at 9:51 AM Stefan Hajnoczi wrote: > > The ctx->first_bh list contains all created BHs, including those that > are not scheduled. The list is iterated by the event loop and therefore > has O(n) time complexity with respected to the number of created BHs. > > Rewrite BHs

Re: [PATCH v2 1/4] m25p80: Convert to support tracing

2020-03-16 Thread Cédric Le Goater
On 3/16/20 3:58 PM, Peter Maydell wrote: > On Mon, 16 Mar 2020 at 14:14, Cédric Le Goater wrote: >> >> Hello, >> >> On 2/17/20 4:47 PM, Cédric Le Goater wrote: >>> Hello all, >>> >>> On 2/6/20 7:32 PM, Guenter Roeck wrote: While at it, add some trace messages to help debug problems seen

Re: [PATCH v7 00/11] error: auto propagated local_err part I

2020-03-16 Thread Markus Armbruster
Vladimir Sementsov-Ogievskiy writes: > 03.03.2020 11:01, Markus Armbruster wrote: >> Hi Vladimir, >> >> I've come to rather like your ERRP_AUTO_PROPAGATE() idea. What I >> wouldn't like is a protracted conversion. >> >> Once we're happy with PATCH 1-3, it's a matter of running Coccinelle and >>

[PATCH v6 41/42] nvme: remove redundant NvmeCmd pointer parameter

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen The command struct is available in the NvmeRequest that we generally pass around anyway. Signed-off-by: Klaus Jensen Acked-by: Keith Busch Reviewed-by: Maxim Levitsky --- hw/block/nvme.c | 164 +++- 1 file changed, 78

[PATCH v6 42/42] nvme: make lba data size configurable

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Acked-by: Keith Busch --- hw/block/nvme-ns.c | 7 ++- hw/block/nvme-ns.h | 4 +++- hw/block/nvme.c| 1 + 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/hw/block/nvme-ns.c b/hw/block/nvme-ns.c index

[PATCH v6 38/42] nvme: support multiple namespaces

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen This adds support for multiple namespaces by introducing a new 'nvme-ns' device model. The nvme device creates a bus named from the device name ('id'). The nvme-ns devices then connect to this and registers themselves with the nvme device. This changes how an nvme device is

[PATCH v6 39/42] pci: allocate pci id for nvme

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen The emulated nvme device (hw/block/nvme.c) is currently using an internal Intel device id. Prepare to change that by allocating a device id under the 1b36 (Red Hat, Inc.) vendor id. Signed-off-by: Klaus Jensen Acked-by: Keith Busch Reviewed-by: Maxim Levitsky ---

[PATCH v6 37/42] nvme: refactor identify active namespace id list

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Prepare to support inactive namespaces. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index b89b96990f52..bf9fb500842a 100644 --- a/hw/block/nvme.c +++

[PATCH v6 32/42] nvme: allow multiple aios per command

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen This refactors how the device issues asynchronous block backend requests. The NvmeRequest now holds a queue of NvmeAIOs that are associated with the command. This allows multiple aios to be issued for a command. Only when all requests have been completed will the device post a

[PATCH v6 35/42] nvme: handle dma errors

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Handling DMA errors gracefully is required for the device to pass the block/011 test ("disable PCI device while doing I/O") in the blktests suite. With this patch the device passes the test by retrying "critical" transfers (posting of completion entries and processing of

[PATCH v6 24/42] nvme: remove redundant has_sg member

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Remove the has_sg member from NvmeRequest since it's redundant. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 18 -- hw/block/nvme.h | 1 - 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index

[PATCH v6 19/42] nvme: enforce valid queue creation sequence

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Support returning Command Sequence Error if Set Features on Number of Queues is called after queues have been created. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 7 +++ hw/block/nvme.h | 1 + 2 files changed, 8 insertions(+) diff --git a/hw/block/nvme.c

[PATCH v6 27/42] nvme: add request mapping helper

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Introduce the nvme_map helper to remove some noise in the main nvme_rw function. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index

[PATCH v6 28/42] nvme: verify validity of prp lists in the cmb

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Before this patch the device already supported this, but it did not check for the validity of it nor announced the support in the LISTS field. If some of the PRPs in a PRP list are in the CMB, then ALL entries must be there. This patch makes sure that is verified as well as

[PATCH v6 18/42] nvme: support identify namespace descriptor list

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Since we are not providing the NGUID or EUI64 fields, we must support the Namespace UUID. We do not have any way of storing a persistent unique identifier, so conjure up a UUID that is just the namespace id. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 38

[PATCH v6 36/42] nvme: add support for scatter gather lists

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen For now, support the Data Block, Segment and Last Segment descriptor types. See NVM Express 1.3d, Section 4.4 ("Scatter Gather List (SGL)"). Signed-off-by: Klaus Jensen Acked-by: Keith Busch --- hw/block/nvme.c | 310 +++---

[PATCH v6 16/42] nvme: make sure ncqr and nsqr is valid

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen 0x is not an allowed value for NCQR and NSQR in Set Features on Number of Queues. Signed-off-by: Klaus Jensen Acked-by: Keith Busch Reviewed-by: Maxim Levitsky --- hw/block/nvme.c | 8 1 file changed, 8 insertions(+) diff --git a/hw/block/nvme.c

[PATCH v6 30/42] nvme: add check for mdts

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Add 'mdts' device parameter to control the Maximum Data Transfer Size of the controller and check that it is respected. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 29 - hw/block/nvme.h | 4 +++- hw/block/trace-events | 1 + 3

[PATCH v6 23/42] nvme: add mapping helpers

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Add nvme_map_addr, nvme_map_addr_cmb and nvme_addr_to_cmb helpers and use them in nvme_map_prp. This fixes a bug where in the case of a CMB transfer, the device would map to the buffer with a wrong length. Fixes: b2b2b67a00574 ("nvme: Add support for Read Data and Write Data

[PATCH v6 21/42] nvme: bump supported version to v1.3

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 74061d08fd2e..26c4b6e69f72 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -44,6 +44,7 @@ #include "trace.h"

Re: [PATCH v6 3/4] qcow2: add zstd cluster compression

2020-03-16 Thread Denis Plotnikov
On 16.03.2020 17:01, Eric Blake wrote: On 3/12/20 4:22 AM, Denis Plotnikov wrote: zstd significantly reduces cluster compression time. It provides better compression performance maintaining the same level of the compression ratio in comparison with zlib, which, at the moment, is the only

[PATCH v6 25/42] nvme: refactor dma read/write

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Refactor the nvme_dma_{read,write}_prp functions into a common function taking a DMADirection parameter. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 89 - 1 file changed, 43 insertions(+), 46 deletions(-) diff --git

[PATCH v6 04/42] nvme: bump spec data structures to v1.3

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Add missing fields in the Identify Controller and Identify Namespace data structures to bring them in line with NVMe v1.3. This also adds data structures and defines for SGL support which requires a couple of trivial changes to the nvme block driver as well. Signed-off-by:

[PATCH v6 03/42] nvme: move device parameters to separate struct

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Move device configuration parameters to separate struct to make it explicit what is configurable and what is set internally. Signed-off-by: Klaus Jensen Acked-by: Keith Busch Reviewed-by: Maxim Levitsky --- hw/block/nvme.c | 44 ++--

[PATCH v6 00/42] nvme: support NVMe v1.3d, SGLs and multiple namespaces

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Hi, So this patchset kinda blew up in size (wrt. number of patches) after Maxim's comments (26 -> 42), but Maxim's comments about splitting up a bunch of the patches made a lot of sense. v6 primarily splits up the big nasty patches into more digestible parts. Specifically

[PATCH v6 05/42] nvme: use constant for identify data size

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 40cb176dea3c..f716f690a594 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -679,7 +679,7 @@ static uint16_t

[PATCH v6 08/42] nvme: add support for the abort command

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Required for compliance with NVMe revision 1.2.1. See NVM Express 1.2.1, Section 5.1 ("Abort command"). The Abort command is a best effort command; for now, the device always fails to abort the given command. Signed-off-by: Klaus Jensen Acked-by: Keith Busch ---

[PATCH v6 34/42] pci: pass along the return value of dma_memory_rw

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen The nvme device needs to know the return value of dma_memory_rw to pass block/011 from blktests. So pass it along instead of ignoring it. There are no existing users of the return value, so this patch should be safe. Signed-off-by: Klaus Jensen Reviewed-by: Philippe

[PATCH v6 29/42] nvme: refactor request bounds checking

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 28 ++-- 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index eecfad694bf8..ba520c76bae5 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -491,6

[PATCH v6 12/42] nvme: add support for the get log page command

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Add support for the Get Log Page command and basic implementations of the mandatory Error Information, SMART / Health Information and Firmware Slot Information log pages. In violation of the specification, the SMART / Health Information log page does not persist information

[PATCH v6 22/42] nvme: memset preallocated requests structures

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen This is preparatory to subsequent patches that change how QSGs/IOVs are handled. It is important that the qsg and iov members of the NvmeRequest are initially zeroed. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

[PATCH v6 01/42] nvme: rename trace events to nvme_dev

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Change the prefix of all nvme device related trace events to 'nvme_dev' to not clash with trace events from the nvme block driver. Signed-off-by: Klaus Jensen Acked-by: Keith Busch Reviewed-by: Maxim Levitsky --- hw/block/nvme.c | 188

[PATCH v6 10/42] nvme: refactor device realization

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen This patch splits up nvme_realize into multiple individual functions, each initializing a different subset of the device. Signed-off-by: Klaus Jensen Acked-by: Keith Busch --- hw/block/nvme.c | 178 ++-- hw/block/nvme.h | 23

[PATCH v6 13/42] nvme: add support for the asynchronous event request command

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Required for compliance with NVMe revision 1.2.1. See NVM Express 1.2.1, Section 5.2 ("Asynchronous Event Request command"). Mostly imported from Keith's qemu-nvme tree. Modified with a max number of queued events (controllable with the aer_max_queued device parameter). The

Re: [PATCH 6/8] hw/ide: Do ide_drive_get() within pci_ide_create_devs()

2020-03-16 Thread Markus Armbruster
Philippe Mathieu-Daudé writes: > On 3/16/20 7:23 AM, Markus Armbruster wrote: >> Paolo Bonzini writes: >> >>> On 13/03/20 23:16, BALATON Zoltan wrote: > > +    pci_dev = pci_create_simple(pci_bus, -1, "cmd646-ide"); > +    pci_ide_create_devs(pci_dev); Additionally, I

[PATCH v6 02/42] nvme: remove superfluous breaks

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen These break statements was left over when commit 3036a626e9ef ("nvme: add Get/Set Feature Timestamp support") was merged. Signed-off-by: Klaus Jensen Acked-by: Keith Busch Reviewed-by: Maxim Levitsky --- hw/block/nvme.c | 4 1 file changed, 4 deletions(-) diff --git

Re: [PATCH v2 1/4] m25p80: Convert to support tracing

2020-03-16 Thread Peter Maydell
On Mon, 16 Mar 2020 at 14:14, Cédric Le Goater wrote: > > Hello, > > On 2/17/20 4:47 PM, Cédric Le Goater wrote: > > Hello all, > > > > On 2/6/20 7:32 PM, Guenter Roeck wrote: > >> While at it, add some trace messages to help debug problems > >> seen when running the latest Linux kernel. > > > >

[PATCH v6 09/42] nvme: add max_ioqpairs device parameter

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen The num_queues device paramater has a slightly confusing meaning because it accounts for the admin queue pair which is not really optional. Secondly, it is really a maximum value of queues allowed. Add a new max_ioqpairs parameter that only accounts for I/O queue pairs, but

Re: [PATCH 0/8] Misc hw/ide legacy clean up

2020-03-16 Thread BALATON Zoltan
On Mon, 16 Mar 2020, BALATON Zoltan wrote: On Mon, 16 Mar 2020, Markus Armbruster wrote: BALATON Zoltan writes: These are some clean ups to remove more legacy init functions and lessen dependence on include/hw/ide.h with some simplifications in board code. There should be no functional

[PATCH v6 40/42] nvme: change controller pci id

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen There are two reasons for changing this: 1. The nvme device currently uses an internal Intel device id. 2. Since commits "nvme: fix write zeroes offset and count" and "nvme: support multiple namespaces" the controller device no longer has the quirks that the

[PATCH v6 26/42] nvme: pass request along for tracing

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 67 +-- hw/block/trace-events | 2 +- 2 files changed, 40 insertions(+), 29 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 809d00443369..3e9c2ed434c2 100644

Re: [PATCH v6 3/4] qcow2: add zstd cluster compression

2020-03-16 Thread Eric Blake
On 3/12/20 4:22 AM, Denis Plotnikov wrote: zstd significantly reduces cluster compression time. It provides better compression performance maintaining the same level of the compression ratio in comparison with zlib, which, at the moment, is the only compression method available. +++

[PATCH v6 20/42] nvme: provide the mandatory subnqn field

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index b40d27cddc46..74061d08fd2e 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -1925,6 +1925,9 @@ static void

[PATCH v6 31/42] nvme: add check for prinfo

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Check the validity of the PRINFO field. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 50 --- hw/block/trace-events | 1 + include/block/nvme.h | 1 + 3 files changed, 44 insertions(+), 8 deletions(-) diff --git

[PATCH v6 33/42] nvme: use preallocated qsg/iov in nvme_dma_prp

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Since clean up of the request qsg/iov has been moved to the common nvme_enqueue_req_completion function, there is no need to use a stack allocated qsg/iov in nvme_dma_prp. Signed-off-by: Klaus Jensen Acked-by: Keith Busch Reviewed-by: Maxim Levitsky --- hw/block/nvme.c |

[PATCH v6 17/42] nvme: add log specific field to trace events

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen The LSP field is not used directly now, but include it in the trace. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 3 ++- hw/block/trace-events | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index

[PATCH v6 15/42] nvme: additional tracing

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Add additional trace calls for nvme_enqueue_req_completion, mmio and doorbell writes. Also, streamline nvme_identify_ns and nvme_identify_ns_list. They do not need to repeat the command, it is already in the trace name. Signed-off-by: Klaus Jensen Acked-by: Keith Busch ---

[PATCH v6 14/42] nvme: add missing mandatory features

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Add support for returning a resonable response to Get/Set Features of mandatory features. Signed-off-by: Klaus Jensen Acked-by: Keith Busch --- hw/block/nvme.c | 60 ++- hw/block/trace-events | 2 ++ include/block/nvme.h | 6

[PATCH v6 11/42] nvme: add temperature threshold feature

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen It might seem wierd to implement this feature for an emulated device, but it is mandatory to support and the feature is useful for testing asynchronous event request support, which will be added in a later patch. Signed-off-by: Klaus Jensen Acked-by: Keith Busch ---

[PATCH v6 07/42] nvme: refactor nvme_addr_read

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Pull the controller memory buffer check to its own function. The check will be used on its own in later patches. Signed-off-by: Klaus Jensen Acked-by: Keith Busch --- hw/block/nvme.c | 16 1 file changed, 12 insertions(+), 4 deletions(-) diff --git

[PATCH v6 06/42] nvme: add identify cns values in header

2020-03-16 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index f716f690a594..b38d7e548a60 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -709,11 +709,11 @@ static

Re: [PATCH v2 1/4] m25p80: Convert to support tracing

2020-03-16 Thread Cédric Le Goater
Hello, On 2/17/20 4:47 PM, Cédric Le Goater wrote: > Hello all, > > On 2/6/20 7:32 PM, Guenter Roeck wrote: >> While at it, add some trace messages to help debug problems >> seen when running the latest Linux kernel. > > Through which tree do you think it is best to merge this patchset ? >

Re: [PATCH v2 5/8] qapi/misc: Restrict query-vm-generation-id command to machine code

2020-03-16 Thread Igor Mammedov
On Mon, 16 Mar 2020 01:03:45 +0100 Philippe Mathieu-Daudé wrote: > Signed-off-by: Philippe Mathieu-Daudé Acked-by: Igor Mammedov > --- > qapi/machine.json | 20 > qapi/misc.json| 21 - > hw/acpi/vmgenid.c | 2 +- > stubs/vmgenid.c | 2 +- > 4

Re: [PATCH 0/8] Misc hw/ide legacy clean up

2020-03-16 Thread BALATON Zoltan
On Mon, 16 Mar 2020, Markus Armbruster wrote: BALATON Zoltan writes: These are some clean ups to remove more legacy init functions and lessen dependence on include/hw/ide.h with some simplifications in board code. There should be no functional change. PATCH 1 could quote precedence more

Re: [PATCH v2 4/8] qapi/misc: Move query-uuid command with block code

2020-03-16 Thread Philippe Mathieu-Daudé
On 3/16/20 2:09 PM, Igor Mammedov wrote: On Mon, 16 Mar 2020 01:03:44 +0100 Philippe Mathieu-Daudé wrote: here should be why Indeed. Daniel explained on IRC the GUID structure is "standardized by microsoft as a way to detect when a guest has certain operations applied" to a saved

[PATCH v2] hw/ide: Do ide_drive_get() within pci_ide_create_devs()

2020-03-16 Thread BALATON Zoltan
The pci_ide_create_devs() function takes a hd_table parameter but all callers just pass what ide_drive_get() returns so we can do it locally simplifying callers and removing hd_table parameter. Signed-off-by: BALATON Zoltan --- v2: Alternative version that applies without [PATCH 4/8] hw/ide:

[PATCH v2] hw/ide: Move MAX_IDE_DEVS define to hw/ide/internal.h

2020-03-16 Thread BALATON Zoltan
We can move this define now that less files use it to internal.h to further reduce dependency on hw/ide.h. Signed-off-by: BALATON Zoltan --- v2: Alternative version of patch 7 that applies without [PATCH 4/8] hw/ide: Move MAX_IDE_BUS define to one header include/hw/ide.h | 2 --

Re: [PATCH v2 8/8] qapi/misc: Restrict device memory commands to machine code

2020-03-16 Thread Igor Mammedov
On Mon, 16 Mar 2020 01:03:48 +0100 Philippe Mathieu-Daudé wrote: > Signed-off-by: Philippe Mathieu-Daudé Acked-by: Igor Mammedov > --- > qapi/machine.json | 131 +++ > qapi/misc.json | 132 >

Re: [PATCH v2 4/8] qapi/misc: Move query-uuid command with block code

2020-03-16 Thread Igor Mammedov
On Mon, 16 Mar 2020 01:03:44 +0100 Philippe Mathieu-Daudé wrote: here should be why PS: I don't see a reason to move it to block code at all if this command is moved then it should be machine code > Signed-off-by: Philippe Mathieu-Daudé > --- > qapi/block-core.json | 30

Re: [PATCH v2 6/8] qapi/misc: Restrict ACPI commands to machine code

2020-03-16 Thread Igor Mammedov
On Mon, 16 Mar 2020 01:03:46 +0100 Philippe Mathieu-Daudé wrote: > Signed-off-by: Philippe Mathieu-Daudé Acked-by: Igor Mammedov > --- > qapi/machine.json| 154 +++ > qapi/misc.json | 154 --- >

Re: [PATCH 1/8] hw/ide: Get rid of piix3_init functions

2020-03-16 Thread BALATON Zoltan
On Mon, 16 Mar 2020, Markus Armbruster wrote: BALATON Zoltan writes: This removes pci_piix3_ide_init() and pci_piix3_xen_ide_init() functions similar to clean up done to other ide devices. Got a commit hash for "done to other ide devices"? Not yet in master, patches to CMD646 from Mark are

Re: [PATCH] block/io: fix bdrv_co_do_copy_on_readv

2020-03-16 Thread Stefan Hajnoczi
On Thu, Mar 12, 2020 at 11:19:49AM +0300, Vladimir Sementsov-Ogievskiy wrote: > Prior to 1143ec5ebf4 it was OK to qemu_iovec_from_buf() from aligned-up > buffer to original qiov, as qemu_iovec_from_buf() will stop at qiov end > anyway. > > But after 1143ec5ebf4 we assume that

  1   2   >