Re: [for-6.0] Non-deterministic qemu-iotests qsd-jobs failures

2021-03-30 Thread Vladimir Sementsov-Ogievskiy
29.03.2021 18:05, Stefan Hajnoczi wrote: Hi Kevin, Peter hit an s390 qemu-iotests failure. I was able to reproduce it easily. I haven't checked if it reproduce on other platforms too, but it seems likely. Here is what I found. qsd-jobs has race conditions: # Just make sure that this doesn't

Re: [PATCH v4 for-6.0? 0/3] qcow2: fix parallel rewrite and discard (rw-lock)

2021-03-30 Thread Vladimir Sementsov-Ogievskiy
30.03.2021 19:39, Max Reitz wrote: On 30.03.21 15:25, Vladimir Sementsov-Ogievskiy wrote: 30.03.2021 15:51, Max Reitz wrote: On 30.03.21 12:51, Vladimir Sementsov-Ogievskiy wrote: 30.03.2021 12:49, Max Reitz wrote: On 25.03.21 20:12, Vladimir Sementsov-Ogievskiy wrote: ping. Do we want it

Re: [PATCH] nvme: expose 'bootindex' property

2021-03-30 Thread Klaus Jensen
On Mar 22 01:24, Joelle van Dyne wrote: > The check for `n->namespace.blkconf.blk` always fails because > this is in the initialization function. > > Signed-off-by: Joelle van Dyne > --- > hw/block/nvme.c | 8 +++- > 1 file changed, 3 insertions(+), 5 deletions(-) > > diff --git

Re: [PATCH 2/4] migrate-bitmaps-postcopy-test: Fix pylint warnings

2021-03-30 Thread Vladimir Sementsov-Ogievskiy
30.03.2021 20:18, Max Reitz wrote: On 30.03.21 18:47, Vladimir Sementsov-Ogievskiy wrote: 29.03.2021 16:26, Max Reitz wrote: pylint complains that discards1_sha256 and all_discards_sha256 are first set in non-__init__ methods.  Let's make it happy. Signed-off-by: Max Reitz ---  

Re: [PATCH 2/4] migrate-bitmaps-postcopy-test: Fix pylint warnings

2021-03-30 Thread Max Reitz
On 30.03.21 18:47, Vladimir Sementsov-Ogievskiy wrote: 29.03.2021 16:26, Max Reitz wrote: pylint complains that discards1_sha256 and all_discards_sha256 are first set in non-__init__ methods.  Let's make it happy. Signed-off-by: Max Reitz ---  

Re: [PATCH 4/4] iotests/297: Cover tests/

2021-03-30 Thread Vladimir Sementsov-Ogievskiy
29.03.2021 16:26, Max Reitz wrote: 297 so far does not check the named tests, which reside in the tests/ directory (i.e. full path tests/qemu-iotests/tests). Fix it. Thanks to the previous two commits, all named tests pass its scrutiny, so we do not have to add anything to SKIP_FILES.

Re: [for-6.0] Non-deterministic qemu-iotests qsd-jobs failures

2021-03-30 Thread John Snow
On 3/30/21 10:38 AM, Stefan Hajnoczi wrote: On Mon, Mar 29, 2021 at 05:43:28PM +0200, Max Reitz wrote: On 29.03.21 17:05, Stefan Hajnoczi wrote: Hi Kevin, Peter hit an s390 qemu-iotests failure. I was able to reproduce it easily. I haven't checked if it reproduce on other platforms too, but it

Re: [PATCH 3/4] migrate-bitmaps-test: Fix pylint warnings

2021-03-30 Thread Vladimir Sementsov-Ogievskiy
29.03.2021 16:26, Max Reitz wrote: There are a couple of things pylint takes issue with: - The "time" import is unused - The import order (iotests should come last) - get_bitmap_hash() doesn't use @self and so should be a function - Semicolons at the end of some lines Wow that's funny :) My

Re: [PATCH 2/4] migrate-bitmaps-postcopy-test: Fix pylint warnings

2021-03-30 Thread Vladimir Sementsov-Ogievskiy
29.03.2021 16:26, Max Reitz wrote: pylint complains that discards1_sha256 and all_discards_sha256 are first set in non-__init__ methods. Let's make it happy. Signed-off-by: Max Reitz --- tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test | 3 +++ 1 file changed, 3 insertions(+) diff

Re: [PATCH 1/4] iotests/297: Drop 169 and 199 from the skip list

2021-03-30 Thread Vladimir Sementsov-Ogievskiy
29.03.2021 16:26, Max Reitz wrote: 169 and 199 have been renamed and moved to tests/ (commit a44be0334be: "iotests: rename and move 169 and 199 tests"), so we can drop them from the skip list. Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy -- Best regards, Vladimir

Re: [PATCH v4 for-6.0? 0/3] qcow2: fix parallel rewrite and discard (rw-lock)

2021-03-30 Thread Max Reitz
On 30.03.21 15:25, Vladimir Sementsov-Ogievskiy wrote: 30.03.2021 15:51, Max Reitz wrote: On 30.03.21 12:51, Vladimir Sementsov-Ogievskiy wrote: 30.03.2021 12:49, Max Reitz wrote: On 25.03.21 20:12, Vladimir Sementsov-Ogievskiy wrote: ping. Do we want it for 6.0? I’d rather wait.  I think

Re: [PULL 0/9] Block patches for 6.0-rc1

2021-03-30 Thread Peter Maydell
On Tue, 30 Mar 2021 at 13:40, Max Reitz wrote: > > The following changes since commit ec2e6e016d24bd429792d08cf607e4c5350dcdaa: > > Merge remote-tracking branch > 'remotes/vivier2/tags/linux-user-for-6.0-pull-request' into staging > (2021-03-28 19:49:57 +0100) > > are available in the Git

Re: [for-6.0] Non-deterministic qemu-iotests qsd-jobs failures

2021-03-30 Thread Stefan Hajnoczi
On Mon, Mar 29, 2021 at 05:43:28PM +0200, Max Reitz wrote: > On 29.03.21 17:05, Stefan Hajnoczi wrote: > > Hi Kevin, > > Peter hit an s390 qemu-iotests failure. I was able to reproduce it > > easily. I haven't checked if it reproduce on other platforms too, but it > > seems likely. Here is what I

Re: [RFC 3/8] virtio: Add API to batch set host notifiers

2021-03-30 Thread Greg Kurz
On Tue, 30 Mar 2021 14:55:42 +0100 Stefan Hajnoczi wrote: > On Tue, Mar 30, 2021 at 12:17:40PM +0200, Greg Kurz wrote: > > On Mon, 29 Mar 2021 18:10:57 +0100 > > Stefan Hajnoczi wrote: > > > On Thu, Mar 25, 2021 at 04:07:30PM +0100, Greg Kurz wrote: > > > > @@ -315,6 +338,10 @@ static void > >

Re: [RFC 3/8] virtio: Add API to batch set host notifiers

2021-03-30 Thread Stefan Hajnoczi
On Tue, Mar 30, 2021 at 12:17:40PM +0200, Greg Kurz wrote: > On Mon, 29 Mar 2021 18:10:57 +0100 > Stefan Hajnoczi wrote: > > On Thu, Mar 25, 2021 at 04:07:30PM +0100, Greg Kurz wrote: > > > @@ -315,6 +338,10 @@ static void > > > virtio_bus_unset_and_cleanup_host_notifiers(VirtioBusState *bus, >

Re: [PATCH v4 for-6.0? 0/3] qcow2: fix parallel rewrite and discard (rw-lock)

2021-03-30 Thread Vladimir Sementsov-Ogievskiy
30.03.2021 15:51, Max Reitz wrote: On 30.03.21 12:51, Vladimir Sementsov-Ogievskiy wrote: 30.03.2021 12:49, Max Reitz wrote: On 25.03.21 20:12, Vladimir Sementsov-Ogievskiy wrote: ping. Do we want it for 6.0? I’d rather wait.  I think the conclusion was that guests shouldn’t hit this

Re: [RFC 0/8] virtio: Improve boot time of virtio-scsi-pci and virtio-blk-pci

2021-03-30 Thread Greg Kurz
On Mon, 29 Mar 2021 18:35:16 +0100 Stefan Hajnoczi wrote: > On Thu, Mar 25, 2021 at 04:07:27PM +0100, Greg Kurz wrote: > > Now that virtio-scsi-pci and virtio-blk-pci map 1 virtqueue per vCPU, > > a serious slow down may be observed on setups with a big enough number > > of vCPUs. > > > >

Re: [PATCH for-6.0 5/7] hw/block/nvme: fix warning about legacy namespace configuration

2021-03-30 Thread Gollu Appalanaidu
On Wed, Mar 24, 2021 at 09:09:05PM +0100, Klaus Jensen wrote: From: Klaus Jensen Remove the unused BlockConf from the controller structure and fix the constraint checking to actually check the right BlockConf and issue the warning. Signed-off-by: Klaus Jensen --- hw/block/nvme.h | 1 -

Re: [PATCH for-6.0 6/7] hw/block/nvme: update dmsrl limit on namespace detachment

2021-03-30 Thread Gollu Appalanaidu
On Wed, Mar 24, 2021 at 09:09:06PM +0100, Klaus Jensen wrote: From: Klaus Jensen The Non-MDTS DMSRL limit must be recomputed when namespaces are detached. Fixes: 645ce1a70cb6 ("hw/block/nvme: support namespace attachment command") Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 17

Re: [PATCH for-6.0 1/7] hw/block/nvme: fix pi constraint check

2021-03-30 Thread Gollu Appalanaidu
On Tue, Mar 30, 2021 at 09:24:59AM +0200, Klaus Jensen wrote: On Mar 29 19:52, Gollu Appalanaidu wrote: On Wed, Mar 24, 2021 at 09:09:01PM +0100, Klaus Jensen wrote: > From: Klaus Jensen > > Protection Information can only be enabled if there is at least 8 bytes > of metadata. > >

Re: [PATCH v4 for-6.0? 0/3] qcow2: fix parallel rewrite and discard (rw-lock)

2021-03-30 Thread Max Reitz
On 30.03.21 12:51, Vladimir Sementsov-Ogievskiy wrote: 30.03.2021 12:49, Max Reitz wrote: On 25.03.21 20:12, Vladimir Sementsov-Ogievskiy wrote: ping. Do we want it for 6.0? I’d rather wait.  I think the conclusion was that guests shouldn’t hit this because they serialize discards? I

[PULL 2/9] iotests: fix 051.out expected output after error text touchups

2021-03-30 Thread Max Reitz
From: Connor Kuehl A patch was recently applied that touched up some error messages that pertained to key names like 'node-name'. The trouble is it only updated tests/qemu-iotests/051.pc.out and not tests/qemu-iotests/051.out as well. Do that now. Fixes: 785ec4b1b9 ("block: Clarify error

[PULL 3/9] iotests/116: Fix reference output

2021-03-30 Thread Max Reitz
15ce94a68ca ("block/qed: bdrv_qed_do_open: deal with errp") has improved the qed driver's error reporting, though sadly did not add a test for it. The good news are: There already is such a test, namely 116. The bad news are: Its reference output was not adjusted, and so now it fails. Let's fix

[PULL 7/9] qsd: Document FUSE exports

2021-03-30 Thread Max Reitz
Implementing FUSE exports required no changes to the storage daemon, so we forgot to document them there. Considering that both NBD and vhost-user-blk exports are documented in its man page (and NBD exports in its --help text), we should probably do the same for FUSE. Signed-off-by: Max Reitz

[PULL 5/9] iotests/046: Filter request length

2021-03-30 Thread Max Reitz
For its concurrent requests, 046 has always filtered the offset, probably because concurrent requests may settle in any order. However, it did not filter the request length, and so if requests with different lengths settle in an unexpected order (notably the longer request before the shorter

[PULL 9/9] iotests/244: Test preallocation for data-file-raw

2021-03-30 Thread Max Reitz
Three test cases: (1) Adding a qcow2 (metadata) file to an existing data file, see whether we can read the existing data through the qcow2 image. (2) Append data to the data file, grow the qcow2 image accordingly, see whether we can read the new data through the qcow2 image. (3) At

[PULL 8/9] qcow2: Force preallocation with data-file-raw

2021-03-30 Thread Max Reitz
Setting the qcow2 data-file-raw bit means that you can ignore the qcow2 metadata when reading from the external data file. It does not mean that you have to ignore it, though. Therefore, the data read must be the same regardless of whether you interpret the metadata or whether you ignore it, and

[PULL 4/9] qcow2: use external virtual timers

2021-03-30 Thread Max Reitz
From: Pavel Dovgalyuk Regular virtual timers are used to emulate timings related to vCPU and peripheral states. QCOW2 uses timers to clean the cache. These timers should have external flag. In the opposite case they affect the execution and it can't be recorded and replayed. This patch adds

[PULL 6/9] block/mirror: Fix mirror_top's permissions

2021-03-30 Thread Max Reitz
mirror_top currently shares all permissions, and takes only the WRITE permission (if some parent has taken that permission, too). That is wrong, though; mirror_top is a filter, so it should take permissions like any other filter does. For example, if the parent needs CONSISTENT_READ, we need to

[PULL 1/9] iotests: Fix typo in iotest 051

2021-03-30 Thread Max Reitz
From: Tao Xu There is an typo in iotest 051, correct it. Signed-off-by: Tao Xu Message-Id: <20210324084321.90952-1-tao3...@intel.com> Signed-off-by: Max Reitz --- tests/qemu-iotests/051| 2 +- tests/qemu-iotests/051.pc.out | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)

[PULL 0/9] Block patches for 6.0-rc1

2021-03-30 Thread Max Reitz
The following changes since commit ec2e6e016d24bd429792d08cf607e4c5350dcdaa: Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-6.0-pull-request' into staging (2021-03-28 19:49:57 +0100) are available in the Git repository at: https://github.com/XanClic/qemu.git

Re: [PATCH] hw/block/nvme: remove description for zoned.append_size_limit

2021-03-30 Thread Klaus Jensen
On Mar 30 10:19, Niklas Cassel wrote: > On Tue, Mar 23, 2021 at 12:20:32PM +0100, Klaus Jensen wrote: > > On Mar 23 11:18, Niklas Cassel wrote: > > > From: Niklas Cassel > > > > > > The description was originally removed in commit 578d914b263c > > > ("hw/block/nvme: align zoned.zasl with mdts")

Re: [PULL for-6.0 0/2] emulated nvme fixes

2021-03-30 Thread Peter Maydell
On Mon, 29 Mar 2021 at 18:04, Klaus Jensen wrote: > > From: Klaus Jensen > > Hi Peter, > > The following changes since commit ec2e6e016d24bd429792d08cf607e4c5350dcdaa: > > Merge remote-tracking branch > 'remotes/vivier2/tags/linux-user-for-6.0-pull-request' into staging > (2021-03-28

Re: [PATCH v5 4/6] qcow2: introduce inflight-write-counters

2021-03-30 Thread Vladimir Sementsov-Ogievskiy
26.03.2021 23:00, Vladimir Sementsov-Ogievskiy wrote: We have a bug in qcow2: assume we've started data write into host cluster A. s->lock is unlocked. During the write the refcount of cluster A may become zero, cluster may be reallocated for other needs, and our in-flight write become a

Re: [PATCH v3 0/5] qemu-iotests: quality of life improvements

2021-03-30 Thread Paolo Bonzini
On 30/03/21 13:44, Max Reitz wrote: On 30.03.21 13:32, Max Reitz wrote: On 26.03.21 15:23, Paolo Bonzini wrote: This series adds a few usability improvements to qemu-iotests, in particular: - arguments can be passed to Python unittests scripts, for example    to run only a subset of the test

Re: [PATCH v3 0/5] qemu-iotests: quality of life improvements

2021-03-30 Thread Max Reitz
On 30.03.21 13:32, Max Reitz wrote: On 26.03.21 15:23, Paolo Bonzini wrote: This series adds a few usability improvements to qemu-iotests, in particular: - arguments can be passed to Python unittests scripts, for example    to run only a subset of the test cases (patches 1-2) - it is possible

Re: [PATCH v3 0/5] qemu-iotests: quality of life improvements

2021-03-30 Thread Max Reitz
On 26.03.21 15:23, Paolo Bonzini wrote: This series adds a few usability improvements to qemu-iotests, in particular: - arguments can be passed to Python unittests scripts, for example to run only a subset of the test cases (patches 1-2) - it is possible to do "./check --

Re: [PATCH v3 4/5] qemu-iotests: let "check" spawn an arbitrary test command

2021-03-30 Thread Paolo Bonzini
On 30/03/21 12:57, Max Reitz wrote: 297 says: check:125: error: Argument 1 to "Path" has incompatible type "Optional[str]"; expected "Union[str, _PathLike[str]]" Found 1 error in 1 file (checked 1 source file) Weird, I had tested this and I cannot reproduce it. diff --git

Re: [PATCH v2 0/2] qcow2: Force preallocation with data-file-raw

2021-03-30 Thread Max Reitz
On 26.03.21 15:55, Max Reitz wrote: v1: https://lists.nongnu.org/archive/html/qemu-block/2020-06/msg00992.html Hi, I think that qcow2 images with data-file-raw should always have preallocated 1:1 L1/L2 tables, so that the image always looks the same whether you respect or ignore the qcow2

Re: [PATCH v3 4/5] qemu-iotests: let "check" spawn an arbitrary test command

2021-03-30 Thread Max Reitz
On 30.03.21 12:44, Max Reitz wrote: On 30.03.21 12:38, Max Reitz wrote: On 26.03.21 16:05, Max Reitz wrote: On 26.03.21 15:23, Paolo Bonzini wrote: Right now there is no easy way for "check" to print a reproducer command. Because such a reproducer command line would be huge, we can instead

Re: [PATCH v4 for-6.0? 0/3] qcow2: fix parallel rewrite and discard (rw-lock)

2021-03-30 Thread Vladimir Sementsov-Ogievskiy
30.03.2021 12:49, Max Reitz wrote: On 25.03.21 20:12, Vladimir Sementsov-Ogievskiy wrote: ping. Do we want it for 6.0? I’d rather wait.  I think the conclusion was that guests shouldn’t hit this because they serialize discards? I think, that we never had bugs, so we of course can wait.

Re: [PATCH v3 4/5] qemu-iotests: let "check" spawn an arbitrary test command

2021-03-30 Thread Max Reitz
On 30.03.21 12:38, Max Reitz wrote: On 26.03.21 16:05, Max Reitz wrote: On 26.03.21 15:23, Paolo Bonzini wrote: Right now there is no easy way for "check" to print a reproducer command. Because such a reproducer command line would be huge, we can instead teach check to start a command of our

Re: [PATCH v3 4/5] qemu-iotests: let "check" spawn an arbitrary test command

2021-03-30 Thread Max Reitz
On 26.03.21 16:05, Max Reitz wrote: On 26.03.21 15:23, Paolo Bonzini wrote: Right now there is no easy way for "check" to print a reproducer command. Because such a reproducer command line would be huge, we can instead teach check to start a command of our choice.  This can be for example a

Re: [RFC 4/8] virtio-pci: Batch add/del ioeventfds in a single MR transaction

2021-03-30 Thread Greg Kurz
On Mon, 29 Mar 2021 18:24:40 +0100 Stefan Hajnoczi wrote: > On Thu, Mar 25, 2021 at 04:07:31PM +0100, Greg Kurz wrote: > > diff --git a/softmmu/memory.c b/softmmu/memory.c > > index 1b1942d521cc..0279e5671bcb 100644 > > --- a/softmmu/memory.c > > +++ b/softmmu/memory.c > > @@ -2368,7 +2368,7 @@

Re: [PATCH] hw/block/nvme: remove description for zoned.append_size_limit

2021-03-30 Thread Niklas Cassel
On Tue, Mar 23, 2021 at 12:20:32PM +0100, Klaus Jensen wrote: > On Mar 23 11:18, Niklas Cassel wrote: > > From: Niklas Cassel > > > > The description was originally removed in commit 578d914b263c > > ("hw/block/nvme: align zoned.zasl with mdts") together with the removal > > of the

Re: [RFC 3/8] virtio: Add API to batch set host notifiers

2021-03-30 Thread Greg Kurz
On Mon, 29 Mar 2021 18:10:57 +0100 Stefan Hajnoczi wrote: > On Thu, Mar 25, 2021 at 04:07:30PM +0100, Greg Kurz wrote: > > Introduce VirtioBusClass methods to begin and commit a transaction > > of setting/unsetting host notifiers. These handlers will be implemented > > by virtio-pci to batch

Re: [RFC 1/8] memory: Allow eventfd add/del without starting a transaction

2021-03-30 Thread Stefan Hajnoczi
On Tue, Mar 30, 2021 at 09:47:49AM +0200, Greg Kurz wrote: > On Mon, 29 Mar 2021 18:03:49 +0100 > Stefan Hajnoczi wrote: > > > On Thu, Mar 25, 2021 at 04:07:28PM +0100, Greg Kurz wrote: > > > diff --git a/include/exec/memory.h b/include/exec/memory.h > > > index 5728a681b27d..98ed552e001c 100644

Re: [PATCH v4 for-6.0? 0/3] qcow2: fix parallel rewrite and discard (rw-lock)

2021-03-30 Thread Max Reitz
On 25.03.21 20:12, Vladimir Sementsov-Ogievskiy wrote: ping. Do we want it for 6.0? I’d rather wait. I think the conclusion was that guests shouldn’t hit this because they serialize discards? There’s also something Kevin wrote on IRC a couple of weeks ago, for which I had hoped he’d sent

Re: [RFC 1/8] memory: Allow eventfd add/del without starting a transaction

2021-03-30 Thread Greg Kurz
On Mon, 29 Mar 2021 18:03:49 +0100 Stefan Hajnoczi wrote: > On Thu, Mar 25, 2021 at 04:07:28PM +0100, Greg Kurz wrote: > > diff --git a/include/exec/memory.h b/include/exec/memory.h > > index 5728a681b27d..98ed552e001c 100644 > > --- a/include/exec/memory.h > > +++ b/include/exec/memory.h > > @@

Re: [PATCH 0/6] Add debug interface to kick/call on purpose

2021-03-30 Thread Dongli Zhang
On 3/28/21 8:56 PM, Jason Wang wrote: > > 在 2021/3/27 上午5:16, Dongli Zhang 写道: >> Hi Jason, >> >> On 3/26/21 12:24 AM, Jason Wang wrote: >>> 在 2021/3/26 下午1:44, Dongli Zhang 写道: The virtio device/driver (e.g., vhost-scsi or vhost-net) may hang due to the loss of doorbell kick, e.g.,

Re: [PATCH for-6.0 1/7] hw/block/nvme: fix pi constraint check

2021-03-30 Thread Klaus Jensen
On Mar 29 19:52, Gollu Appalanaidu wrote: > On Wed, Mar 24, 2021 at 09:09:01PM +0100, Klaus Jensen wrote: > > From: Klaus Jensen > > > > Protection Information can only be enabled if there is at least 8 bytes > > of metadata. > > > > Signed-off-by: Klaus Jensen > > --- > > hw/block/nvme-ns.c |