Re: [PATCH v6 07/11] iotests: add findtests.py

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
12.01.2021 19:42, Kevin Wolf wrote: +def find_tests(self, groups: Optional[List[str]] = None, + exclude_groups: Optional[List[str]] = None, + tests: Optional[List[str]] = None, group and tests seem to be read-only, so this can be simplified to 'groups:

[PATCH v3 3/4] hw/block/nvme: add smart_critical_warning property

2021-01-13 Thread zhenwei pi
There is a very low probability that hitting physical NVMe disk hardware critical warning case, it's hard to write & test a monitor agent service. For debugging purposes, add a new 'smart_critical_warning' property to emulate this situation. The orignal version of this change is implemented by

[PATCH v3 4/4] hw/blocl/nvme: trigger async event during injecting smart warning

2021-01-13 Thread zhenwei pi
During smart critical warning injection by setting property from QMP command, also try to trigger asynchronous event. Signed-off-by: zhenwei pi --- hw/block/nvme.c | 47 --- 1 file changed, 40 insertions(+), 7 deletions(-) diff --git

[PATCH v3 2/4] hw/block/nvme: fix overwritten bar.cap

2021-01-13 Thread zhenwei pi
After PMR initialization, bar.cap should not be clear in function nvme_init_ctrl. Otherwise the PMR cap would be always disabled. Signed-off-by: zhenwei pi --- hw/block/nvme.c | 1 - 1 file changed, 1 deletion(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 27d2c72716..f361103bb4

[PATCH v3 0/4] support NVMe smart critial warning injection

2021-01-13 Thread zhenwei pi
v2 -> v3: - Introduce "Persistent Memory Region has become read-only or unreliable" - Fix overwritten bar.cap - Check smart critical warning value from QOM. - Trigger asynchronous event during smart warning injection. v1 -> v2: - Suggested by Philippe & Klaus, set/get smart_critical_warning

[PATCH v3 1/4] block/nvme: introduce bit 5 for critical warning

2021-01-13 Thread zhenwei pi
According to NVMe spec 1.4 section , introduce bit 5 for "Persistent Memory Region has become read-only or unreliable". Signed-off-by: zhenwei pi --- include/block/nvme.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/block/nvme.h b/include/block/nvme.h index 3e02d9ca98..f68a88c712

Re: [PATCH v6 08/11] iotests: add testenv.py

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
14.01.2021 09:14, Vladimir Sementsov-Ogievskiy wrote: 12.01.2021 20:36, Kevin Wolf wrote: Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben: Add TestEnv class, which will handle test environment in a new python iotests running framework. Difference with current ./check

Re: [PATCH v6 08/11] iotests: add testenv.py

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
12.01.2021 20:36, Kevin Wolf wrote: Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben: Add TestEnv class, which will handle test environment in a new python iotests running framework. Difference with current ./check interface: - -v (verbose) option dropped, as it is unused -

Re: [PATCH v6 08/11] iotests: add testenv.py

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
09.01.2021 15:26, Vladimir Sementsov-Ogievskiy wrote: Add TestEnv class, which will handle test environment in a new python iotests running framework. Difference with current ./check interface: - -v (verbose) option dropped, as it is unused - -xdiff option is dropped, until somebody complains

Re: [PATCH 22/22] docs/system: riscv: Add documentation for sifive_u machine

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:42 AM Bin Meng wrote: > > From: Bin Meng > > This adds detailed documentation for RISC-V `sifive_u` machine, > including the following information: > > - Supported devices > - Hardware configuration information > - Boot options > - Machine-specific options > - Running

Re: [PATCH v6 10/11] iotests: rewrite check into python

2021-01-13 Thread John Snow
On 1/13/21 6:20 PM, Eric Blake wrote: On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote: Just use classes introduced in previous three commits. Behavior difference is described in these three commits. Drop group file, as it becomes unused. Signed-off-by: Vladimir Sementsov-Ogievskiy ---

Re: [PATCH 21/22] docs/system: Add RISC-V documentation

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:53 AM Bin Meng wrote: > > From: Bin Meng > > Add RISC-V system emulator documentation for generic information. > `Board-specific documentation` and `RISC-V CPU features` are only > a placeholder and will be added in the future. > > Signed-off-by: Bin Meng This is

Re: [PATCH v6 10/11] iotests: rewrite check into python

2021-01-13 Thread Eric Blake
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote: > Just use classes introduced in previous three commits. Behavior > difference is described in these three commits. > > Drop group file, as it becomes unused. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > tests/qemu-iotests/check

Re: [PATCH v6 07/11] iotests: add findtests.py

2021-01-13 Thread Eric Blake
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote: > Add python script with new logic of searching for tests: > > Current ./check behavior: > - tests are named [0-9][0-9][0-9] > - tests must be registered in group file (even if test doesn't belong >to any group, like 142) > > > This

Re: [PATCH v6 06/11] iotests: define group in each iotest

2021-01-13 Thread Eric Blake
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote: > We are going to drop group file. Define group in tests as a preparatory > step. > > The patch is generated by > > cd tests/qemu-iotests > > grep '^[0-9]\{3\} ' group | while read line; do > file=$(awk '{print $1}' <<<

Re: [PATCH v6 05/11] iotests/294: add shebang line

2021-01-13 Thread Eric Blake
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote: > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > tests/qemu-iotests/294 | 1 + > 1 file changed, 1 insertion(+) Reviewed-by: Eric Blake > > diff --git a/tests/qemu-iotests/294 b/tests/qemu-iotests/294 > index 87da35db49..4c375ed609

Re: [PATCH v6 04/11] iotests: make tests executable

2021-01-13 Thread Eric Blake
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote: > All other test files are executable. Fix these. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > tests/qemu-iotests/283 | 0 > tests/qemu-iotests/298 | 0 > tests/qemu-iotests/299 | 0 > 3 files changed, 0 insertions(+), 0

Re: [PATCH v6 03/11] iotests: fix some whitespaces in test output files

2021-01-13 Thread Eric Blake
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote: > We are going to be stricter about comparing test result with .out > files. So, fix some whitespaces now. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > tests/qemu-iotests/175.out | 2 +- > tests/qemu-iotests/271.out | 12

Re: [PATCH v6 02/11] iotests/303: use dot slash for qcow2.py running

2021-01-13 Thread Eric Blake
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote: > If you run './check 303', check includes common.config which adjusts > $PATH to include '.' first, and therefore finds qcow2.py on PATH. But > if you run './303' directly, there is nothing to adjust PATH, and if > '.' is not already on your

Re: [PATCH v6 01/11] iotests/277: use dot slash for nbd-fault-injector.py running

2021-01-13 Thread Eric Blake
On 1/9/21 6:26 AM, Vladimir Sementsov-Ogievskiy wrote: > If you run './check 277', check includes common.config which adjusts > $PATH to include '.' first, and therefore finds nbd-fault-injector.py > on PATH. But if you run './277' directly, there is nothing to adjust > PATH, and if '.' is not

[PATCH v4 5/5] qapi: More complex uses of QAPI_LIST_APPEND

2021-01-13 Thread Eric Blake
These cases require a bit more thought to review; in each case, the code was appending to a list, but not with a FOOList **tail variable. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- fix qmp_guest_network_get_interfaces [Vladimir] --- block/gluster.c| 13

[PATCH v4 4/5] qapi: Use QAPI_LIST_APPEND in trivial cases

2021-01-13 Thread Eric Blake
The easiest spots to use QAPI_LIST_APPEND are where we already have an obvious pointer to the tail of a list. While at it, consistently use the variable name 'tail' for that purpose. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Markus Armbruster --- move

Re: [PATCH v5] block: report errno when flock fcntl fails

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 19:44, David Edmondson wrote: When a call to fcntl(2) for the purpose of adding file locks fails with an error other than EAGAIN or EACCES, report the error returned by fcntl. EAGAIN or EACCES are elided as they are considered to be common failures, indicating that a conflicting lock

Re: [PATCH v2 8/8] iotests/129: Clean up pylint and mypy complaints

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 20:57, Max Reitz wrote: Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 6 -- tests/qemu-iotests/297 | 2 +- tests/qemu-iotests/297.out | 1 + 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index

Re: [PATCH v2 5/8] iotests/129: Use throttle node

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 20:57, Max Reitz wrote: Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz Reviewed-by: Vladimir

Re: [PATCH v2 1/8] iotests/297: Allow checking all Python test files

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 22:27, Vladimir Sementsov-Ogievskiy wrote: 13.01.2021 20:57, Max Reitz wrote: I.e., all Python files in the qemu-iotests/ directory. Most files of course do not pass, so there is an extensive skip list for now.  (The only files that do pass are 209, 254, 283, and iotests.py.)

Re: [PATCH v2 2/8] iotests: Move try_remove to iotests.py

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 20:57, Max Reitz wrote: Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy -- Best regards, Vladimir

Re: [PATCH v2 1/8] iotests/297: Allow checking all Python test files

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 20:57, Max Reitz wrote: I.e., all Python files in the qemu-iotests/ directory. Most files of course do not pass, so there is an extensive skip list for now. (The only files that do pass are 209, 254, 283, and iotests.py.) (Alternatively, we could have the opposite, i.e. an explicit

Re: [PATCH v2 5/8] iotests/129: Use throttle node

2021-01-13 Thread Eric Blake
On 1/13/21 11:57 AM, Max Reitz wrote: > Throttling on the BB has not affected block jobs in a while, so it is > possible that one of the jobs in 129 finishes before the VM is stopped. > We can fix that by running the job from a throttle node. > > Signed-off-by: Max Reitz > --- >

Re: [PATCH v2 8/8] iotests/129: Clean up pylint and mypy complaints

2021-01-13 Thread Eric Blake
On 1/13/21 11:57 AM, Max Reitz wrote: > Signed-off-by: Max Reitz > --- > tests/qemu-iotests/129 | 6 -- > tests/qemu-iotests/297 | 2 +- > tests/qemu-iotests/297.out | 1 + > 3 files changed, 6 insertions(+), 3 deletions(-) > Reviewed-by: Eric Blake > +++

Re: [PATCH v2 1/8] iotests/297: Allow checking all Python test files

2021-01-13 Thread Eric Blake
On 1/13/21 11:57 AM, Max Reitz wrote: > I.e., all Python files in the qemu-iotests/ directory. > > Most files of course do not pass, so there is an extensive skip list for > now. (The only files that do pass are 209, 254, 283, and iotests.py.) > > (Alternatively, we could have the opposite,

Re: [PATCH 20/22] docs/system: Sort targets in alphabetical order

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:50 AM Bin Meng wrote: > > From: Bin Meng > > Signed-off-by: Bin Meng Reviewed-by: Alistair Francis Alistair > --- > > docs/system/targets.rst | 19 --- > 1 file changed, 12 insertions(+), 7 deletions(-) > > diff --git a/docs/system/targets.rst

Re: [PATCH 19/22] hw/riscv: sifive_u: Change SIFIVE_U_GEM_IRQ to decimal value

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:38 AM Bin Meng wrote: > > From: Bin Meng > > All other peripherals' IRQs are in the format of decimal value. > Change SIFIVE_U_GEM_IRQ to be consistent. > > Signed-off-by: Bin Meng Reviewed-by: Alistair Francis Alistair > --- > > include/hw/riscv/sifive_u.h | 2 +-

Re: [PATCH 18/22] hw/riscv: sifive_u: Add QSPI2 controller and connect an SD card

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:45 AM Bin Meng wrote: > > From: Bin Meng > > This adds the QSPI2 controller to the SoC, and connnects an SD > card to it. The generation of corresponding device tree source > fragment is also added. > > Specify machine property `msel` to 11 to boot the same upstream >

Re: [PATCH 17/22] hw/riscv: sifive_u: Add QSPI0 controller and connect a flash

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:51 AM Bin Meng wrote: > > From: Bin Meng > > This adds the QSPI0 controller to the SoC, and connnects an ISSI > 25WP256 flash to it. The generation of corresponding device tree > source fragment is also added. > > With this commit, upstream U-Boot for the SiFive HiFive

Re: [PATCH 16/22] hw/ssi: Add SiFive SPI controller support

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:36 AM Bin Meng wrote: > > From: Bin Meng > > This adds the SiFive SPI controller model for the FU540 SoC. > The direct memory-mapped SPI flash mode is unsupported. > > Signed-off-by: Bin Meng > --- > > include/hw/ssi/sifive_spi.h | 47 ++ > hw/ssi/sifive_spi.c

Re: [PATCH 15/22] hw/sd: ssi-sd: Support multiple block write

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:47 AM Bin Meng wrote: > > From: Bin Meng > > For a multiple block write operation, each block begins with a multi > write start token. Unlike the SD mode that the multiple block write > ends when receiving a STOP_TRAN command (CMD12), a special stop tran > tocken is

Re: [PATCH 12/22] hw/sd: sd.h: Cosmetic change of using spaces

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:46 AM Bin Meng wrote: > > From: Bin Meng > > QEMU conding convention prefers spaces over tabs. > > Signed-off-by: Bin Meng Reviewed-by: Alistair Francis Alistair > --- > > include/hw/sd/sd.h | 42 +- > 1 file changed, 21

Re: [PATCH 14/22] hw/sd: ssi-sd: Support single block write

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:43 AM Bin Meng wrote: > > From: Bin Meng > > Add 2 more states for the block write operation. The SPI host needs > to send a data start tocken to start the transfer, and the data block > written to the card will be acknowledged by a data response tocken. > >

[PATCH v2 4/8] iotests/129: Do not check @busy

2021-01-13 Thread Max Reitz
@busy is false when the job is paused, which happens all the time because that is how jobs yield (e.g. for mirror at least since commit 565ac01f8d3). Back when 129 was added (2015), perhaps there was no better way of checking whether the job was still actually running. Now we have the @status

[PATCH v2 8/8] iotests/129: Clean up pylint and mypy complaints

2021-01-13 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 6 -- tests/qemu-iotests/297 | 2 +- tests/qemu-iotests/297.out | 1 + 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 6d21470cd7..64578493c1 100755 ---

[PATCH v2 5/8] iotests/129: Use throttle node

2021-01-13 Thread Max Reitz
Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 37 +

[PATCH v2 7/8] iotests/129: Limit mirror job's buffer size

2021-01-13 Thread Max Reitz
Issuing 'stop' on the VM drains all nodes. If the mirror job has many large requests in flight, this may lead to significant I/O that looks a bit like 'stop' would make the job try to complete (which is what 129 should verify not to happen). We can limit the I/O in flight by limiting the buffer

[PATCH v2 6/8] iotests/129: Actually test a commit job

2021-01-13 Thread Max Reitz
Before this patch, test_block_commit() performs an active commit, which under the hood is a mirror job. If we want to test various different block jobs, we should perhaps run an actual commit job instead. Doing so requires adding an overlay above the source node before the commit is done (and

[PATCH v2 3/8] iotests/129: Remove test images in tearDown()

2021-01-13 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- tests/qemu-iotests/129 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 0e13244d85..2fc65ada6a 100755 --- a/tests/qemu-iotests/129 +++

[PATCH v2 1/8] iotests/297: Allow checking all Python test files

2021-01-13 Thread Max Reitz
I.e., all Python files in the qemu-iotests/ directory. Most files of course do not pass, so there is an extensive skip list for now. (The only files that do pass are 209, 254, 283, and iotests.py.) (Alternatively, we could have the opposite, i.e. an explicit list of files that we do want to

[PATCH v2 2/8] iotests: Move try_remove to iotests.py

2021-01-13 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- tests/qemu-iotests/124| 8 +--- tests/qemu-iotests/iotests.py | 11 +++ 2 files changed, 8 insertions(+), 11 deletions(-) diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 index 3705cbb6b3..e40eeb50b9 100755

[PATCH v2 0/8] iotests: Fix 129 and expand 297’s reach

2021-01-13 Thread Max Reitz
v1 cover letter: https://lists.nongnu.org/archive/html/qemu-block/2021-01/msg00254.html Hi, See the cover letter above for the main point of this series (it’s just that all patch indices are shifted up by one in v2). In addition to that, I’ve added patch 1 that makes some changes to 297 so it

Re: [PATCH 13/22] hw/sd: Introduce receive_ready() callback

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:44 AM Bin Meng wrote: > > From: Bin Meng > > At present there is a data_ready() callback for the SD data read > path. Let's add a receive_ready() for the SD data write path. > > Signed-off-by: Bin Meng Acked-by: Alistair Francis Alistair > --- > >

Re: [PATCH 07/22] hw/sd: ssi-sd: Suffix a data block with CRC16

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:38 AM Bin Meng wrote: > > From: Bin Meng > > Per the SD spec, a valid data block is suffixed with a 16-bit CRC > generated by the standard CCITT polynomial x16+x12+x5+1. This part > is currently missing in the ssi-sd state machine. Without it, all > data block transfer

Re: [PATCH 4/7] iotests/129: Use throttle node

2021-01-13 Thread Max Reitz
On 13.01.21 17:46, Vladimir Sementsov-Ogievskiy wrote: 13.01.2021 17:06, Max Reitz wrote: Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node.

Re: [PATCH 4/7] iotests/129: Use throttle node

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 20:02, Max Reitz wrote: On 13.01.21 17:46, Vladimir Sementsov-Ogievskiy wrote: 13.01.2021 17:06, Max Reitz wrote: Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running

Re: [PATCH 11/22] hw/sd: sd: Allow single/multiple block write for SPI mode

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:42 AM Bin Meng wrote: > > From: Bin Meng > > At present the single/multiple block write in SPI mode is blocked > by sd_normal_command(). Remove the limitation. > > Signed-off-by: Bin Meng Acked-by: Alistair Francis Alistair > --- > > hw/sd/sd.c | 3 --- > 1 file

Re: [PATCH 6/7] iotests/129: Limit mirror job's buffer size

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 17:06, Max Reitz wrote: Issuing 'stop' on the VM drains all nodes. If the mirror job has many large requests in flight, this may lead to significant I/O that looks a bit like the job is being drained. what do you mean by "looks like the job is being drained"? If "drain" in Qemu

Re: [PATCH 09/22] hw/sd: ssi-sd: Use macros for the dummy value and tokens in the transfer

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:38 AM Bin Meng wrote: > > From: Bin Meng > > At present the codes use hardcoded numbers (0xff/0xfe) for the dummy > value and block start token. Replace them with macros, and add more > tokens for multiple block write. > > Signed-off-by: Bin Meng Reviewed-by: Alistair

Re: [PATCH 10/22] hw/sd: sd: Remove duplicated codes in single/multiple block read/write

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:42 AM Bin Meng wrote: > > From: Bin Meng > > The single block read (CMD17) codes are the same as the multiple > block read (CMD18). Merge them into one. The same applies to single > block write (CMD24) and multiple block write (CMD25). > > Signed-off-by: Bin Meng

Re: [PATCH 08/22] hw/sd: ssi-sd: Support multiple block read (CMD18)

2021-01-13 Thread Alistair Francis
On Thu, Dec 31, 2020 at 3:41 AM Bin Meng wrote: > > From: Bin Meng > > In the case of a multiple block read operation every transfered > block has its suffix of CRC16. Update the state machine logic to > handle multiple block read. > > This also fixed the wrong command index for

Re: [PATCH 5/7] iotests/129: Actually test a commit job

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 17:06, Max Reitz wrote: Before this patch, test_block_commit() performs an active commit, which under the hood is a mirror job. If we want to test various different block jobs, we should perhaps run an actual commit job instead. Doing so requires adding an overlay above the source

Re: [PATCH 4/7] iotests/129: Use throttle node

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 17:06, Max Reitz wrote: Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 39

[PATCH v5] block: report errno when flock fcntl fails

2021-01-13 Thread David Edmondson
When a call to fcntl(2) for the purpose of adding file locks fails with an error other than EAGAIN or EACCES, report the error returned by fcntl. EAGAIN or EACCES are elided as they are considered to be common failures, indicating that a conflicting lock is held by another process. No errors are

Re: [PATCH 7/7] iotests/129: Clean up pylint and mypy complaints

2021-01-13 Thread Eric Blake
On 1/13/21 8:06 AM, Max Reitz wrote: > Signed-off-by: Max Reitz > --- > tests/qemu-iotests/129 | 6 -- > 1 file changed, 4 insertions(+), 2 deletions(-) > Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org

Re: [PATCH 6/7] iotests/129: Limit mirror job's buffer size

2021-01-13 Thread Eric Blake
On 1/13/21 8:06 AM, Max Reitz wrote: > Issuing 'stop' on the VM drains all nodes. If the mirror job has many > large requests in flight, this may lead to significant I/O that looks a > bit like the job is being drained. > > We can limit the I/O in flight by limiting the buffer size, so mirror >

Re: [PATCH 5/7] iotests/129: Actually test a commit job

2021-01-13 Thread Eric Blake
On 1/13/21 8:06 AM, Max Reitz wrote: > Before this patch, test_block_commit() performs an active commit, which > under the hood is a mirror job. If we want to test various different > block jobs, we should perhaps run an actual commit job instead. > > Doing so requires adding an overlay above

Re: [PATCH 0/7] iotests/129: Fix it

2021-01-13 Thread Kevin Wolf
Am 13.01.2021 um 16:26 hat Max Reitz geschrieben: > On 13.01.21 15:38, Kevin Wolf wrote: > > Am 13.01.2021 um 15:06 hat Max Reitz geschrieben: > > > - pylint and mypy complain. > > >(Running mypy with the options given in 297.) > > >[Patch 4 removes one pylint complaint; patch 7 the rest.]

Re: [PATCH 0/7] iotests/129: Fix it

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 18:19, Max Reitz wrote: On 13.01.21 15:31, Vladimir Sementsov-Ogievskiy wrote: 13.01.2021 17:06, Max Reitz wrote: Hi, There are some problems with iotests 129 (perhaps more than these, but these are the ones I know of): 1. It checks @busy to see whether a block job is still

Re: [PATCH 0/7] iotests/129: Fix it

2021-01-13 Thread Max Reitz
On 13.01.21 15:31, Vladimir Sementsov-Ogievskiy wrote: 13.01.2021 17:06, Max Reitz wrote: Hi, There are some problems with iotests 129 (perhaps more than these, but these are the ones I know of): 1. It checks @busy to see whether a block job is still running; however,     block jobs tend to

Re: [PATCH 0/7] iotests/129: Fix it

2021-01-13 Thread Max Reitz
On 13.01.21 15:38, Kevin Wolf wrote: Am 13.01.2021 um 15:06 hat Max Reitz geschrieben: - pylint and mypy complain. (Running mypy with the options given in 297.) [Patch 4 removes one pylint complaint; patch 7 the rest.] Should we add it to 297 then to make sure we won't regress? Sounds

Re: [PATCH 4/7] iotests/129: Use throttle node

2021-01-13 Thread Max Reitz
On 13.01.21 16:07, Vladimir Sementsov-Ogievskiy wrote: 13.01.2021 17:10, Max Reitz wrote: On 13.01.21 15:06, Max Reitz wrote: Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running

Re: [PATCH 4/7] iotests/129: Use throttle node

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 17:10, Max Reitz wrote: On 13.01.21 15:06, Max Reitz wrote: Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz

Re: [PATCH 3/7] iotests/129: Do not check @busy

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 17:06, Max Reitz wrote: @busy is false when the job is paused, which happens all the time because that is how jobs yield (e.g. for mirror at least since commit 565ac01f8d3). Back when 129 was added (2015), perhaps there was no better way of checking whether the job was still actually

Re: [PATCH 0/7] iotests/129: Fix it

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 17:38, Kevin Wolf wrote: Am 13.01.2021 um 15:06 hat Max Reitz geschrieben: - pylint and mypy complain. (Running mypy with the options given in 297.) [Patch 4 removes one pylint complaint; patch 7 the rest.] Should we add it to 297 then to make sure we won't regress? At some

Re: [PATCH 4/7] iotests/129: Use throttle node

2021-01-13 Thread Eric Blake
On 1/13/21 8:10 AM, Max Reitz wrote: > On 13.01.21 15:06, Max Reitz wrote: >> Throttling on the BB has not affected block jobs in a while, so it is >> possible that one of the jobs in 129 finishes before the VM is stopped. >> We can fix that by running the job from a throttle node. >> >>

Re: [PATCH 3/7] iotests/129: Do not check @busy

2021-01-13 Thread Eric Blake
On 1/13/21 8:06 AM, Max Reitz wrote: > @busy is false when the job is paused, which happens all the time > because that is how jobs yield (e.g. for mirror at least since commit > 565ac01f8d3). > > Back when 129 was added (2015), perhaps there was no better way of > checking whether the job was

Re: [PATCH 2/7] iotests/129: Remove test images in tearDown()

2021-01-13 Thread Eric Blake
On 1/13/21 8:06 AM, Max Reitz wrote: > Signed-off-by: Max Reitz > --- > tests/qemu-iotests/129 | 2 ++ > 1 file changed, 2 insertions(+) Reviewed-by: Eric Blake > > diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 > index 0e13244d85..2fc65ada6a 100755 > ---

Re: [PATCH 2/7] iotests/129: Remove test images in tearDown()

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 17:06, Max Reitz wrote: Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 0e13244d85..2fc65ada6a 100755 ---

Re: [PATCH 0/7] iotests/129: Fix it

2021-01-13 Thread Kevin Wolf
Am 13.01.2021 um 15:06 hat Max Reitz geschrieben: > - pylint and mypy complain. > (Running mypy with the options given in 297.) > [Patch 4 removes one pylint complaint; patch 7 the rest.] Should we add it to 297 then to make sure we won't regress? At some point, I guess we'll want to cover

Re: [PATCH 0/7] iotests/129: Fix it

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 17:06, Max Reitz wrote: Hi, There are some problems with iotests 129 (perhaps more than these, but these are the ones I know of): 1. It checks @busy to see whether a block job is still running; however, block jobs tend to unset @busy all the time (when they yield). [Fixed by

Re: [PATCH 1/7] iotests: Move try_remove to iotests.py

2021-01-13 Thread Eric Blake
On 1/13/21 8:06 AM, Max Reitz wrote: > Signed-off-by: Max Reitz > --- > tests/qemu-iotests/124| 8 +--- > tests/qemu-iotests/iotests.py | 11 +++ > 2 files changed, 8 insertions(+), 11 deletions(-) > Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red

[PATCH 3/7] iotests/129: Do not check @busy

2021-01-13 Thread Max Reitz
@busy is false when the job is paused, which happens all the time because that is how jobs yield (e.g. for mirror at least since commit 565ac01f8d3). Back when 129 was added (2015), perhaps there was no better way of checking whether the job was still actually running. Now we have the @status

Re: [PATCH v3 6/7] qapi: Use QAPI_LIST_APPEND in trivial cases

2021-01-13 Thread Eric Blake
On 1/13/21 7:16 AM, Markus Armbruster wrote: > Eric Blake writes: > >> The easiest spots to use QAPI_LIST_APPEND are where we already have an >> obvious pointer to the tail of a list. While at it, consistently use >> the variable name 'tail' for that purpose. >> >> Signed-off-by: Eric Blake

Re: [PATCH v3 6/7] qapi: Use QAPI_LIST_APPEND in trivial cases

2021-01-13 Thread Eric Blake
On 12/24/20 3:56 AM, Vladimir Sementsov-Ogievskiy wrote: > 24.12.2020 01:11, Eric Blake wrote: >> The easiest spots to use QAPI_LIST_APPEND are where we already have an >> obvious pointer to the tail of a list.  While at it, consistently use >> the variable name 'tail' for that purpose. >> >>

[PATCH 5/7] iotests/129: Actually test a commit job

2021-01-13 Thread Max Reitz
Before this patch, test_block_commit() performs an active commit, which under the hood is a mirror job. If we want to test various different block jobs, we should perhaps run an actual commit job instead. Doing so requires adding an overlay above the source node before the commit is done (and

[PATCH 7/7] iotests/129: Clean up pylint and mypy complaints

2021-01-13 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index c3ad584ba2..ec303069e9 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -20,9 +20,10 @@ import

[PATCH 6/7] iotests/129: Limit mirror job's buffer size

2021-01-13 Thread Max Reitz
Issuing 'stop' on the VM drains all nodes. If the mirror job has many large requests in flight, this may lead to significant I/O that looks a bit like the job is being drained. We can limit the I/O in flight by limiting the buffer size, so mirror will make very little progress during the 'stop'

Re: [PATCH 4/7] iotests/129: Use throttle node

2021-01-13 Thread Max Reitz
On 13.01.21 15:06, Max Reitz wrote: Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 39

[PATCH 4/7] iotests/129: Use throttle node

2021-01-13 Thread Max Reitz
Throttling on the BB has not affected block jobs in a while, so it is possible that one of the jobs in 129 finishes before the VM is stopped. We can fix that by running the job from a throttle node. Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 39 +++

Re: iotest 129

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 16:59, Vladimir Sementsov-Ogievskiy wrote: 13.01.2021 14:05, Max Reitz wrote: On 13.01.21 10:53, Vladimir Sementsov-Ogievskiy wrote: 12.01.2021 20:44, Max Reitz wrote: Hi, tl;dr: I have some troubles debugging what’s wrong with iotest 129. It wants to check that 'stop' does not

[PATCH 2/7] iotests/129: Remove test images in tearDown()

2021-01-13 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/129 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 0e13244d85..2fc65ada6a 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -47,6 +47,8 @@ class

[PATCH 1/7] iotests: Move try_remove to iotests.py

2021-01-13 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/124| 8 +--- tests/qemu-iotests/iotests.py | 11 +++ 2 files changed, 8 insertions(+), 11 deletions(-) diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 index 3705cbb6b3..e40eeb50b9 100755 ---

[PATCH 0/7] iotests/129: Fix it

2021-01-13 Thread Max Reitz
Hi, There are some problems with iotests 129 (perhaps more than these, but these are the ones I know of): 1. It checks @busy to see whether a block job is still running; however, block jobs tend to unset @busy all the time (when they yield). [Fixed by patch 3] 2. It uses blockdev

Re: iotest 129

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
13.01.2021 14:05, Max Reitz wrote: On 13.01.21 10:53, Vladimir Sementsov-Ogievskiy wrote: 12.01.2021 20:44, Max Reitz wrote: Hi, tl;dr: I have some troubles debugging what’s wrong with iotest 129. It wants to check that 'stop' does not drain a block job, but to me it seems like that’s

Re: [PATCH v3 7/7] qapi: More complex uses of QAPI_LIST_APPEND

2021-01-13 Thread Markus Armbruster
Eric Blake writes: > These cases require a bit more thought to review; in each case, the > code was appending to a list, but not with a FOOList **tail variable. > > Signed-off-by: Eric Blake > --- [...] > diff --git a/qga/commands-posix.c b/qga/commands-posix.c > index

Re: [PATCH v3 6/7] qapi: Use QAPI_LIST_APPEND in trivial cases

2021-01-13 Thread Markus Armbruster
Eric Blake writes: > The easiest spots to use QAPI_LIST_APPEND are where we already have an > obvious pointer to the tail of a list. While at it, consistently use > the variable name 'tail' for that purpose. > > Signed-off-by: Eric Blake > --- [...] > diff --git a/hw/acpi/cpu.c b/hw/acpi/cpu.c

Re: [PATCH v3 4/7] qapi: Use QAPI_LIST_PREPEND() where possible

2021-01-13 Thread Markus Armbruster
Eric Blake writes: > Anywhere we create a list of just one item or by prepending items > (typically because order doesn't matter), we can use the > QAPI_LIST_PREPEND macro. But places where we must keep the list in > order by appending remain open-coded until later patches. > > Note that as a

Re: [PATCH 3/9] configure/meson: Only check sys/signal.h on non-Linux

2021-01-13 Thread Peter Maydell
On Wed, 13 Jan 2021 at 07:06, Thomas Huth wrote: > > On 21/12/2020 01.53, Jiaxun Yang wrote: > > signal.h is equlevant of sys/signal.h on Linux, musl would complain > > wrong usage of sys/signal.h. > > > > In file included from /builds/FlyGoat/qemu/include/qemu/osdep.h:108, > >

Re: [PATCH v6 07/11] iotests: add findtests.py

2021-01-13 Thread Kevin Wolf
Am 13.01.2021 um 11:37 hat Vladimir Sementsov-Ogievskiy geschrieben: > 12.01.2021 19:42, Kevin Wolf wrote: > > Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben: > > > +def __init__(self, test_dir: Optional[str] = None) -> None: > > > +self.groups = defaultdict(set) >

Re: iotest 129

2021-01-13 Thread Max Reitz
On 13.01.21 10:53, Vladimir Sementsov-Ogievskiy wrote: 12.01.2021 20:44, Max Reitz wrote: Hi, tl;dr: I have some troubles debugging what’s wrong with iotest 129. It wants to check that 'stop' does not drain a block job, but to me it seems like that’s exactly what’s happening with the mirror

Re: [PATCH v4] block: report errno when flock fcntl fails

2021-01-13 Thread David Edmondson
On Wednesday, 2021-01-13 at 13:26:48 +03, Vladimir Sementsov-Ogievskiy wrote: > 12.01.2021 18:27, David Edmondson wrote: >> When a call to fcntl(2) for the purpose of manipulating file locks >> fails with an error other than EAGAIN or EACCES, report the error >> returned by fcntl. >> >> EAGAIN

Re: [PATCH v6 07/11] iotests: add findtests.py

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
12.01.2021 19:42, Kevin Wolf wrote: Am 09.01.2021 um 13:26 hat Vladimir Sementsov-Ogievskiy geschrieben: Add python script with new logic of searching for tests: Current ./check behavior: - tests are named [0-9][0-9][0-9] - tests must be registered in group file (even if test doesn't

Re: [PATCH v4] block: report errno when flock fcntl fails

2021-01-13 Thread Vladimir Sementsov-Ogievskiy
12.01.2021 18:27, David Edmondson wrote: When a call to fcntl(2) for the purpose of manipulating file locks fails with an error other than EAGAIN or EACCES, report the error returned by fcntl. EAGAIN or EACCES are elided as they are considered to be common failures, indicating that a

  1   2   >