Re: [PATCH v2 5/5] vhost: add device started check in migration set log

[Li Feng]

Hi, Dima.
This abort is what I have mentioned in my previous email.
I have triggered this crash without any fix a week ago.
And I have written a test patch to let vhost_log_global_start return
int and propagate the error to up layer.
However, my change is a little large, because the origin callback
return void, and don't do some rollback.
After test, the migration could migrate to dst successfully, and fio
is still running perfectly, but the src vm is still stuck here, no
crash.

Is it right to return this error to the up layer?

Thanks,
Feng Li

Dima Stepanov  于2020年5月16日周六 上午12:55写道：
>
> On Thu, May 14, 2020 at 03:34:24PM +0800, Jason Wang wrote:
> >
> > On 2020/5/13 下午5:47, Dima Stepanov wrote:
> > >>> case CHR_EVENT_CLOSED:
> > >>> /* a close event may happen during a read/write, but vhost
> > >>>  * code assumes the vhost_dev remains setup, so delay the
> > >>>  * stop & clear to idle.
> > >>>  * FIXME: better handle failure in vhost code, remove bh
> > >>>  */
> > >>> if (s->watch) {
> > >>> AioContext *ctx = qemu_get_current_aio_context();
> > >>>
> > >>> g_source_remove(s->watch);
> > >>> s->watch = 0;
> > >>> qemu_chr_fe_set_handlers(>chr, NULL, NULL, NULL, NULL,
> > >>>  NULL, NULL, false);
> > >>>
> > >>> aio_bh_schedule_oneshot(ctx, chr_closed_bh, opaque);
> > >>> }
> > >>> break;
> > >>>
> > >>>I think it's time we dropped the FIXME and moved the handling to common
> > >>>code. Jason? Marc-André?
> > >>I agree. Just to confirm, do you prefer bh or doing changes like what is
> > >>done in this series? It looks to me bh can have more easier codes.
> > >Could it be a good idea just to make disconnect in the char device but
> > >postphone clean up in the vhost-user-blk (or any other vhost-user
> > >device) itself? So we are moving the postphone logic and decision from
> > >the char device to vhost-user device. One of the idea i have is as
> > >follows:
> > >   - Put ourself in the INITIALIZATION state
> > >   - Start these vhost-user "handshake" commands
> > >   - If we got a disconnect error, perform disconnect, but don't clean up
> > > device (it will be clean up on the roll back). I can be done by
> > > checking the state in vhost_user_..._disconnect routine or smth like 
> > > it
> >
> >
> > Any issue you saw just using the aio bh as Michael posted above.
> >
> > Then we don't need to deal with the silent vhost_dev_stop() and we will have
> > codes that is much more easier to understand.
> I've implemented this solution inside
> hw/block/vhost-user-blk.c:vhost_user_blk_event() in the similar way by
> using the s->connected field. Looks good and more correct fix ). I have
> two questions here before i'll rework the fixes:
> 1. Is it okay to make the similar fix inside vhost_user_blk_event() or
> we are looking for more generic vhost-user solution? What do you think?
> 2. For migration we require an additional information that for the
> vhost-user device it isn't an error, because i'm trigerring the
> following assert error:
>   Core was generated by `x86_64-softmmu/qemu-system-x86_64 -nodefaults 
> -no-user-config -M q35,sata=false'.
>   Program terminated with signal SIGABRT, Aborted.
>   #0  0x7fb56e729428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
>   [Current thread is 1 (Thread 0x7fb486ef5700 (LWP 527734))]
>
>   (gdb) bt
>   #0  0x7fb56e729428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
>   #1  0x7fb56e72b02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
>   #2  0x5648ea376ee6 in vhost_log_global_start
>   (listener=0x5648ece4eb08) at ./hw/virtio/vhost.c:857
>   #3  0x5648ea2dde7e in memory_global_dirty_log_start ()
>   at ./memory.c:2611
>   #4  0x5648ea2e68e7 in ram_init_bitmaps (rs=0x7fb4740008c0)
>   at ./migration/ram.c:2305
>   #5  0x5648ea2e698b in ram_init_all (rsp=0x5648eb1f0f20 )
>   at ./migration/ram.c:2323
>   #6  0x5648ea2e6cc5 in ram_save_setup (f=0x5648ec609e00,
>   opaque=0x5648eb1f0f20 )
>   at ./migration/ram.c:2436
>   #7  0x5648ea67b7d3 in qemu_savevm_state_setup (f=0x5648ec609e00) at
>   migration/savevm.c:1176
>   #8  0x5648ea674511 in migration_thread (opaque=0x5648ec031ff0) at
>   migration/migration.c:3416
>   #9  0x5648ea85d65d in qemu_thread_start (args=0x5648ec6057f0) at
>   util/qemu-thread-posix.c:519
>   #10 0x7fb56eac56ba in start_thread () from
>   /lib/x86_64-linux-gnu/libpthread.so.0
>   #11 0x7fb56e7fb41d in clone () from /lib/x86_64-linux-gnu/libc.so.6
>   (gdb) frame 2
>   #2  0x5648ea376ee6 in vhost_log_global_start
>  (listener=0x5648ece4eb08) at ./hw/virtio/vhost.c:857
>   857 abort();
>   (gdb) list
>   852 {
>   853 int r;
>   854
>   855 r = vhost_migration_log(listener, true);
>   856 if (r < 0) {
>   857 

Re: [PATCH v6 00/16] acpi: i386 tweaks

[no-reply]

Patchew URL: https://patchew.org/QEMU/20200515150421.25479-1-kra...@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Message-id: 20200515150421.25479-1-kra...@redhat.com
Subject: [PATCH v6 00/16] acpi: i386 tweaks
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
50ccf98 acpi: q35: drop _SB.PCI0.ISA.LPCD opregion.
9df782c acpi: drop build_piix4_pm()
3dde274 acpi: drop serial/parallel enable bits from dsdt
ba91b3c acpi: simplify build_isa_devices_aml()
b3c142b acpi: factor out fw_cfg_add_acpi_dsdt()
c323a33 acpi: move aml builder code for i8042 (kbd+mouse) device
3e7bae0 floppy: move cmos_get_fd_drive_type() from pc
e9149b0 floppy: make isa_fdc_get_drive_max_chs static
db4dee8 acpi: move aml builder code for floppy device
0260d46 acpi: move aml builder code for parallel device
ddf467f acpi: parallel: don't use _STA method
64d08f8 acpi: move aml builder code for serial device
8104cb6 acpi: serial: don't use _STA method
6d7f2e7 acpi: rtc: use a single crs range
fa494c5 acpi: move aml builder code for rtc device
7300b89 qtest: allow DSDT acpi table changes

=== OUTPUT BEGIN ===
1/16 Checking commit 7300b8986cb8 (qtest: allow DSDT acpi table changes)
2/16 Checking commit fa494c572f21 (acpi: move aml builder code for rtc device)
3/16 Checking commit 6d7f2e7fcb02 (acpi: rtc: use a single crs range)
4/16 Checking commit 8104cb67898a (acpi: serial: don't use _STA method)
5/16 Checking commit 64d08f8cd884 (acpi: move aml builder code for serial 
device)
6/16 Checking commit ddf467f98a4a (acpi: parallel: don't use _STA method)
7/16 Checking commit 0260d46f99be (acpi: move aml builder code for parallel 
device)
8/16 Checking commit db4dee8bd791 (acpi: move aml builder code for floppy 
device)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#245: 
new file mode 100644

total: 0 errors, 1 warnings, 221 lines checked

Patch 8/16 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
9/16 Checking commit e9149b0c087e (floppy: make isa_fdc_get_drive_max_chs 
static)
10/16 Checking commit 3e7bae0c4433 (floppy: move cmos_get_fd_drive_type() from 
pc)
ERROR: Missing Signed-off-by: line(s)

total: 1 errors, 0 warnings, 82 lines checked

Patch 10/16 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

11/16 Checking commit c323a336b719 (acpi: move aml builder code for i8042 
(kbd+mouse) device)
12/16 Checking commit b3c142be77e7 (acpi: factor out fw_cfg_add_acpi_dsdt())
13/16 Checking commit ba91b3cd8ac4 (acpi: simplify build_isa_devices_aml())
14/16 Checking commit 3dde274c7e73 (acpi: drop serial/parallel enable bits from 
dsdt)
15/16 Checking commit 9df782c3c334 (acpi: drop build_piix4_pm())
16/16 Checking commit 50ccf983fc52 (acpi: q35: drop _SB.PCI0.ISA.LPCD opregion.)
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20200515150421.25479-1-kra...@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

[PATCH v4 4/9] iotests: define group in each iotest

[Vladimir Sementsov-Ogievskiy]

We are going to drop group file. Define group in tests as a preparatory
step.

The patch is generated by

cd tests/qemu-iotests

grep '^[0-9]\{3\} ' group | while read line; do
file=$(awk '{print $1}' <<< "$line");
groups=$(sed -e 's/^... //' <<< "$line");
awk "NR==2{print \"# group: $groups\"}1" $file > tmp;
cat tmp > $file;
done

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/001 | 1 +
 tests/qemu-iotests/002 | 1 +
 tests/qemu-iotests/003 | 1 +
 tests/qemu-iotests/004 | 1 +
 tests/qemu-iotests/005 | 1 +
 tests/qemu-iotests/007 | 1 +
 tests/qemu-iotests/008 | 1 +
 tests/qemu-iotests/009 | 1 +
 tests/qemu-iotests/010 | 1 +
 tests/qemu-iotests/011 | 1 +
 tests/qemu-iotests/012 | 1 +
 tests/qemu-iotests/013 | 1 +
 tests/qemu-iotests/014 | 1 +
 tests/qemu-iotests/015 | 1 +
 tests/qemu-iotests/017 | 1 +
 tests/qemu-iotests/018 | 1 +
 tests/qemu-iotests/019 | 1 +
 tests/qemu-iotests/020 | 1 +
 tests/qemu-iotests/021 | 1 +
 tests/qemu-iotests/022 | 1 +
 tests/qemu-iotests/023 | 1 +
 tests/qemu-iotests/024 | 1 +
 tests/qemu-iotests/025 | 1 +
 tests/qemu-iotests/026 | 1 +
 tests/qemu-iotests/027 | 1 +
 tests/qemu-iotests/028 | 1 +
 tests/qemu-iotests/029 | 1 +
 tests/qemu-iotests/030 | 1 +
 tests/qemu-iotests/031 | 1 +
 tests/qemu-iotests/032 | 1 +
 tests/qemu-iotests/033 | 1 +
 tests/qemu-iotests/034 | 1 +
 tests/qemu-iotests/035 | 1 +
 tests/qemu-iotests/036 | 1 +
 tests/qemu-iotests/037 | 1 +
 tests/qemu-iotests/038 | 1 +
 tests/qemu-iotests/039 | 1 +
 tests/qemu-iotests/040 | 1 +
 tests/qemu-iotests/041 | 1 +
 tests/qemu-iotests/042 | 1 +
 tests/qemu-iotests/043 | 1 +
 tests/qemu-iotests/044 | 1 +
 tests/qemu-iotests/045 | 1 +
 tests/qemu-iotests/046 | 1 +
 tests/qemu-iotests/047 | 1 +
 tests/qemu-iotests/048 | 1 +
 tests/qemu-iotests/049 | 1 +
 tests/qemu-iotests/050 | 1 +
 tests/qemu-iotests/051 | 1 +
 tests/qemu-iotests/052 | 1 +
 tests/qemu-iotests/053 | 1 +
 tests/qemu-iotests/054 | 1 +
 tests/qemu-iotests/055 | 1 +
 tests/qemu-iotests/056 | 1 +
 tests/qemu-iotests/057 | 1 +
 tests/qemu-iotests/058 | 1 +
 tests/qemu-iotests/059 | 1 +
 tests/qemu-iotests/060 | 1 +
 tests/qemu-iotests/061 | 1 +
 tests/qemu-iotests/062 | 1 +
 tests/qemu-iotests/063 | 1 +
 tests/qemu-iotests/064 | 1 +
 tests/qemu-iotests/065 | 1 +
 tests/qemu-iotests/066 | 1 +
 tests/qemu-iotests/067 | 1 +
 tests/qemu-iotests/068 | 1 +
 tests/qemu-iotests/069 | 1 +
 tests/qemu-iotests/070 | 1 +
 tests/qemu-iotests/071 | 1 +
 tests/qemu-iotests/072 | 1 +
 tests/qemu-iotests/073 | 1 +
 tests/qemu-iotests/074 | 1 +
 tests/qemu-iotests/075 | 1 +
 tests/qemu-iotests/076 | 1 +
 tests/qemu-iotests/077 | 1 +
 tests/qemu-iotests/078 | 1 +
 tests/qemu-iotests/079 | 1 +
 tests/qemu-iotests/080 | 1 +
 tests/qemu-iotests/081 | 1 +
 tests/qemu-iotests/082 | 1 +
 tests/qemu-iotests/083 | 1 +
 tests/qemu-iotests/084 | 1 +
 tests/qemu-iotests/085 | 1 +
 tests/qemu-iotests/086 | 1 +
 tests/qemu-iotests/087 | 1 +
 tests/qemu-iotests/088 | 1 +
 tests/qemu-iotests/089 | 1 +
 tests/qemu-iotests/090 | 1 +
 tests/qemu-iotests/091 | 1 +
 tests/qemu-iotests/092 | 1 +
 tests/qemu-iotests/093 | 1 +
 tests/qemu-iotests/094 | 1 +
 tests/qemu-iotests/095 | 1 +
 tests/qemu-iotests/096 | 1 +
 tests/qemu-iotests/097 | 1 +
 tests/qemu-iotests/098 | 1 +
 tests/qemu-iotests/099 | 1 +
 tests/qemu-iotests/101 | 1 +
 tests/qemu-iotests/102 | 1 +
 tests/qemu-iotests/103 | 1 +
 tests/qemu-iotests/104 | 1 +
 tests/qemu-iotests/105 | 1 +
 tests/qemu-iotests/106 | 1 +
 tests/qemu-iotests/107 | 1 +
 tests/qemu-iotests/108 | 1 +
 tests/qemu-iotests/109 | 1 +
 tests/qemu-iotests/110 | 1 +
 tests/qemu-iotests/111 | 1 +
 tests/qemu-iotests/112 | 1 +
 tests/qemu-iotests/113 | 1 +
 tests/qemu-iotests/114 | 1 +
 tests/qemu-iotests/115 | 1 +
 tests/qemu-iotests/116 | 1 +
 tests/qemu-iotests/117 | 1 +
 tests/qemu-iotests/118 | 1 +
 tests/qemu-iotests/119 | 1 +
 tests/qemu-iotests/120 | 1 +
 tests/qemu-iotests/121 | 1 +
 tests/qemu-iotests/122 | 1 +
 tests/qemu-iotests/123 | 1 +
 tests/qemu-iotests/124 | 1 +
 tests/qemu-iotests/125 | 1 +
 tests/qemu-iotests/126 | 1 +
 tests/qemu-iotests/127 | 1 +
 tests/qemu-iotests/128 | 1 +
 tests/qemu-iotests/129 | 1 +
 tests/qemu-iotests/130 | 1 +
 tests/qemu-iotests/131 | 1 +
 tests/qemu-iotests/132 | 1 +
 tests/qemu-iotests/133 | 1 +
 tests/qemu-iotests/134 | 1 +
 tests/qemu-iotests/135 | 1 +
 tests/qemu-iotests/136 | 1 +
 tests/qemu-iotests/137 | 1 +
 tests/qemu-iotests/138 | 1 +
 tests/qemu-iotests/139 | 1 +
 tests/qemu-iotests/140 | 1 +
 tests/qemu-iotests/141 | 1 +
 tests/qemu-iotests/143 | 1 +
 tests/qemu-iotests/144 | 1 +
 tests/qemu-iotests/145 | 1 +
 tests/qemu-iotests/146 | 1 +
 tests/qemu-iotests/147 | 1 +
 tests/qemu-iotests/148 | 1 +
 tests/qemu-iotests/149 | 1 +
 tests/qemu-iotests/150 | 1 +
 tests/qemu-iotests/151 | 1 +
 tests/qemu-iotests/152 | 1 +
 tests/qemu-iotests/153 | 1 +
 tests/qemu-iotests/154 | 1 +
 tests/qemu-iotests/155 | 1 +
 tests/qemu-iotests/156 | 1 +
 

[PATCH v4 7/9] iotests: add testrunner.py

[Vladimir Sementsov-Ogievskiy]

Add TestRunner class, which will run tests in a new python iotests
running framework.

There are some differences with current ./check behavior, most
significant are:
- Consider all tests self-executable, just run them, don't run python
  by hand.
- Elapsed time is cached in json file
- Elapsed time precision increased a bit
- use python difflib instead of "diff -w", to ignore spaces at line
  ends strip lines by hand. Do not ignore other spaces.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/testrunner.py | 351 +++
 1 file changed, 351 insertions(+)
 create mode 100644 tests/qemu-iotests/testrunner.py

diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
new file mode 100644
index 00..e395877882
--- /dev/null
+++ b/tests/qemu-iotests/testrunner.py
@@ -0,0 +1,351 @@
+# Class for actual tests running.
+#
+# Copyright (c) 2020 Virtuozzo International GmbH
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+import os
+import random
+from pathlib import Path
+import datetime
+import time
+import difflib
+import subprocess
+import collections
+import contextlib
+import json
+import argparse
+import termios
+import sys
+from contextlib import contextmanager
+from typing import List, Optional, Iterator
+
+from testenv import TestEnv
+
+
+def silent_unlink(path: Path) -> None:
+try:
+path.unlink()
+except OSError:
+pass
+
+
+def file_diff(file1: str, file2: str) -> List[str]:
+with open(file1) as f1, open(file2) as f2:
+# We want to ignore spaces at line ends. There are a lot of mess about
+# it in iotests.
+# TODO: fix all tests to not produce extra spaces, fix all .out files
+# and use strict diff here!
+seq1 = [line.rstrip() for line in f1]
+seq2 = [line.rstrip() for line in f2]
+return list(difflib.unified_diff(seq1, seq2, file1, file2))
+
+
+# We want to save current tty settings during test run,
+# since an aborting qemu call may leave things screwed up.
+@contextmanager
+def savetty() -> Iterator:
+isterm = sys.stdin.isatty()
+if isterm:
+fd = sys.stdin.fileno()
+attr = termios.tcgetattr(0)
+
+try:
+yield
+finally:
+if isterm:
+termios.tcsetattr(fd, termios.TCSADRAIN, attr)
+
+
+class LastElapsedTime:
+""" Cache for elapsed time for tests, to show it during new test run
+
+Use get() in any time. But, if use update you should then call save(),
+or use update() inside with-block.
+"""
+def __init__(self, cache_file: str, env: TestEnv) -> None:
+self.env = env
+self.cache_file = cache_file
+
+try:
+with open(cache_file) as f:
+self.cache = json.load(f)
+except (OSError, ValueError):
+self.cache = {}
+
+def get(self, test: str,
+default: Optional[float] = None) -> Optional[float]:
+if test not in self.cache:
+return default
+
+if self.env.imgproto not in self.cache[test]:
+return default
+
+return self.cache[test][self.env.imgproto].get(self.env.imgfmt,
+   default)
+
+def update(self, test: str, elapsed: float) -> None:
+d = self.cache.setdefault(test, {})
+d = d.setdefault(self.env.imgproto, {})
+d[self.env.imgfmt] = elapsed
+
+def save(self) -> None:
+with open(self.cache_file, 'w') as f:
+json.dump(self.cache, f)
+
+def __enter__(self) -> 'LastElapsedTime':
+return self
+
+def __exit__(self, *args) -> None:
+self.save()
+
+
+TestResult = collections.namedtuple(
+'TestResult',
+['status', 'description', 'elapsed', 'diff', 'casenotrun'],
+defaults=('', '', '', ''))
+
+
+class TestRunner:
+_argparser = None
+@classmethod
+def get_argparser(cls) -> argparse.ArgumentParser:
+if cls._argparser is not None:
+return cls._argparser
+
+p = argparse.ArgumentParser(description="= test running options =",
+add_help=False, usage=argparse.SUPPRESS)
+
+p.add_argument('-makecheck', action='store_true',
+   help='pretty print output for make check')
+
+cls._argparser = p
+return p
+
+def 

[PATCH v4 8/9] iotests: rewrite check into python

[Vladimir Sementsov-Ogievskiy]

Just use classes introduced in previous three commits. Behavior
difference is described in these three commits.

Drop group file, as it becomes unused.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/check | 983 ++-
 tests/qemu-iotests/group | 302 
 2 files changed, 28 insertions(+), 1257 deletions(-)
 delete mode 100644 tests/qemu-iotests/group

diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index 9c461cf76d..48bb3128c3 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -1,7 +1,8 @@
-#!/usr/bin/env bash
+#!/usr/bin/env python3
 #
-# Copyright (C) 2009 Red Hat, Inc.
-# Copyright (c) 2000-2002,2006 Silicon Graphics, Inc.  All Rights Reserved.
+# Configure environment and run group of tests in it.
+#
+# Copyright (c) 2020 Virtuozzo International GmbH
 #
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License as
@@ -14,956 +15,28 @@
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see .
-#
-#
-# Control script for QA
-#
-
-status=0
-needwrap=true
-try=0
-n_bad=0
-bad=""
-notrun=""
-casenotrun=""
-interrupt=true
-makecheck=false
-
-_init_error()
-{
-echo "check: $1" >&2
-exit 1
-}
-
-if [ -L "$0" ]
-then
-# called from the build tree
-source_iotests=$(dirname "$(readlink "$0")")
-if [ -z "$source_iotests" ]
-then
-_init_error "failed to obtain source tree name from check symlink"
-fi
-source_iotests=$(cd "$source_iotests"; pwd) || _init_error "failed to 
enter source tree"
-build_iotests=$PWD
-else
-# called from the source tree
-source_iotests=$PWD
-# this may be an in-tree build (note that in the following code we may not
-# assume that it truly is and have to test whether the build results
-# actually exist)
-build_iotests=$PWD
-fi
-
-build_root="$build_iotests/../.."
-
-# we need common.env
-if ! . "$build_iotests/common.env"
-then
-_init_error "failed to source common.env (make sure the qemu-iotests are 
run from tests/qemu-iotests in the build tree)"
-fi
-
-# we need common.config
-if ! . "$source_iotests/common.config"
-then
-_init_error "failed to source common.config"
-fi
-
-_full_imgfmt_details()
-{
-if [ -n "$IMGOPTS" ]; then
-echo "$IMGFMT ($IMGOPTS)"
-else
-echo "$IMGFMT"
-fi
-}
-
-_full_platform_details()
-{
-os=$(uname -s)
-host=$(hostname -s)
-kernel=$(uname -r)
-platform=$(uname -m)
-echo "$os/$platform $host $kernel"
-}
-
-_full_env_details()
-{
-cat < /dev/null)
-if [ -n "$p" -a -x "$p" ]; then
-type -p "$p"
-else
-return 1
-fi
-}
-
-if [ -z "$TEST_DIR" ]; then
-TEST_DIR=$PWD/scratch
-fi
-mkdir -p "$TEST_DIR" || _init_error 'Failed to create TEST_DIR'
-
-tmp_sock_dir=false
-if [ -z "$SOCK_DIR" ]; then
-SOCK_DIR=$(mktemp -d)
-tmp_sock_dir=true
-fi
-mkdir -p "$SOCK_DIR" || _init_error 'Failed to create SOCK_DIR'
-
-diff="diff -u"
-verbose=false
-debug=false
-group=false
-xgroup=false
-imgopts=false
-showme=false
-sortme=false
-expunge=true
-have_test_arg=false
-cachemode=false
-aiomode=false
-
-tmp="${TEST_DIR}"/$$
-rm -f $tmp.list $tmp.tmp $tmp.sed
-
-export IMGFMT=raw
-export IMGFMT_GENERIC=true
-export IMGPROTO=file
-export IMGOPTS=""
-export CACHEMODE="writeback"
-export AIOMODE="threads"
-export QEMU_IO_OPTIONS=""
-export QEMU_IO_OPTIONS_NO_FMT=""
-export CACHEMODE_IS_DEFAULT=true
-export VALGRIND_QEMU=
-export IMGKEYSECRET=
-export IMGOPTSSYNTAX=false
-
-# Save current tty settings, since an aborting qemu call may leave things
-# screwed up
-STTY_RESTORE=
-if test -t 0; then
-STTY_RESTORE=$(stty -g)
-fi
-
-for r
-do
-
-if $group
-then
-# arg after -g
-group_list=$(sed -n <"$source_iotests/group" -e 's/$/ /' -e 
"/^[0-9][0-9][0-9].* $r /"'{
-s/ .*//p
-}')
-if [ -z "$group_list" ]
-then
-echo "Group \"$r\" is empty or not defined?"
-exit 1
-fi
-[ ! -s $tmp.list ] && touch $tmp.list
-for t in $group_list
-do
-if grep -s "^$t\$" $tmp.list >/dev/null
-then
-:
-else
-echo "$t" >>$tmp.list
-fi
-done
-group=false
-continue
-
-elif $xgroup
-then
-# arg after -x
-# Populate $tmp.list with all tests
-awk '/^[0-9]{3,}/ {print $1}' "${source_iotests}/group" > $tmp.list 
2>/dev/null
-group_list=$(sed -n <"$source_iotests/group" -e 's/$/ /' -e 
"/^[0-9][0-9][0-9].* $r /"'{
-s/ .*//p
-}')
-if [ -z "$group_list" ]
-then
-echo "Group \"$r\" is empty or not defined?"
-exit 1
-fi
-numsed=0
-rm -f $tmp.sed
-for t in $group_list
-do
-if [ 

[PATCH v4 6/9] iotests: add testenv.py

[Vladimir Sementsov-Ogievskiy]

Add TestEnv class, which will handle test environment in a new python
iotests running framework.

Difference with current ./check interface:
- -v (verbose) option dropped, as it is unused

- -xdiff option is dropped, until somebody complains that it is needed
- same for -n option

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/testenv.py | 325 ++
 1 file changed, 325 insertions(+)
 create mode 100755 tests/qemu-iotests/testenv.py

diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py
new file mode 100755
index 00..ba396d5a97
--- /dev/null
+++ b/tests/qemu-iotests/testenv.py
@@ -0,0 +1,325 @@
+#!/usr/bin/env python3
+#
+# Parse command line options to manage test environment variables.
+#
+# Copyright (c) 2020 Virtuozzo International GmbH
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+import os
+import sys
+import tempfile
+from pathlib import Path
+import shutil
+import collections
+import subprocess
+import argparse
+from typing import List, Dict
+
+
+def get_default_machine(qemu_prog: str) -> str:
+outp = subprocess.run([qemu_prog, '-machine', 'help'], check=True,
+  text=True, stdout=subprocess.PIPE).stdout
+
+machines = outp.split('\n')
+default_machine = next(m for m in machines if m.endswith(' (default)'))
+default_machine = default_machine.split(' ', 1)[0]
+
+alias_suf = ' (alias of {})'.format(default_machine)
+alias = next((m for m in machines if m.endswith(alias_suf)), None)
+if alias is not None:
+default_machine = alias.split(' ', 1)[0]
+
+return default_machine
+
+
+class TestEnv:
+"""
+Manage system environment for running tests
+
+The following variables are supported/provided. They are represented by
+lower-cased TestEnv attributes.
+"""
+env_variables = ['PYTHONPATH', 'TEST_DIR', 'SOCK_DIR', 'SAMPLE_IMG_DIR',
+ 'OUTPUT_DIR', 'PYTHON', 'QEMU_PROG', 'QEMU_IMG_PROG',
+ 'QEMU_IO_PROG', 'QEMU_NBD_PROG', 'QEMU_VXHS_PROG',
+ 'SOCKET_SCM_HELPER', 'QEMU_OPTIONS', 'QEMU_IMG_OPTIONS',
+ 'QEMU_IO_OPTIONS', 'QEMU_NBD_OPTIONS', 'IMGOPTS',
+ 'IMGFMT', 'IMGPROTO', 'AIOMODE', 'CACHEMODE',
+ 'VALGRIND_QEMU', 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC',
+ 'IMGOPTSSYNTAX', 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE']
+
+def get_env(self) -> Dict[str, str]:
+env = {}
+for v in self.env_variables:
+val = getattr(self, v.lower(), None)
+if val is not None:
+env[v] = val
+
+return env
+
+_argparser = None
+@classmethod
+def get_argparser(cls) -> argparse.ArgumentParser:
+if cls._argparser is not None:
+return cls._argparser
+
+p = argparse.ArgumentParser(description="= test environment options =",
+add_help=False, usage=argparse.SUPPRESS)
+
+p.add_argument('-d', dest='debug', action='store_true', help='debug')
+p.add_argument('-misalign', action='store_true',
+   help='misalign memory allocations')
+
+p.set_defaults(imgfmt='raw', imgproto='file')
+
+format_list = ['raw', 'bochs', 'cloop', 'parallels', 'qcow', 'qcow2',
+   'qed', 'vdi', 'vpc', 'vhdx', 'vmdk', 'luks', 'dmg']
+g = p.add_argument_group(
+'image format options',
+'The following options sets IMGFMT environment variable. '
+'At most one chose is allowed, default is "raw"')
+g = g.add_mutually_exclusive_group()
+for fmt in format_list:
+g.add_argument('-' + fmt, dest='imgfmt', action='store_const',
+   const=fmt)
+
+protocol_list = ['file', 'rbd', 'sheepdoc', 'nbd', 'ssh', 'nfs',
+ 'vxhs']
+g = p.add_argument_group(
+'image protocol options',
+'The following options sets IMGPROTO environment variably. '
+'At most one chose is allowed, default is "file"')
+g = g.add_mutually_exclusive_group()
+for prt in protocol_list:
+g.add_argument('-' + prt, dest='imgproto', action='store_const',
+   const=prt)
+
+g = 

[PATCH v4 3/9] iotests/283: make executable

[Vladimir Sementsov-Ogievskiy]

All other test files are executable, except for this one. Fix that.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Eric Blake 
Reviewed-by: Philippe Mathieu-Daudé 
---
 tests/qemu-iotests/283 | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 mode change 100644 => 100755 tests/qemu-iotests/283

diff --git a/tests/qemu-iotests/283 b/tests/qemu-iotests/283
old mode 100644
new mode 100755
-- 
2.21.0

[PATCH v4 9/9] iotests: rename and move 169 and 199 tests

[Vladimir Sementsov-Ogievskiy]

Rename bitmaps migration tests and move them to tests subdirectory to
demonstrate new human-friendly test naming.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/{199 => tests/migrate-bitmaps-postcopy-test}   | 0
 .../{199.out => tests/migrate-bitmaps-postcopy-test.out}  | 0
 tests/qemu-iotests/{169 => tests/migrate-bitmaps-test}| 0
 tests/qemu-iotests/{169.out => tests/migrate-bitmaps-test.out}| 0
 4 files changed, 0 insertions(+), 0 deletions(-)
 rename tests/qemu-iotests/{199 => tests/migrate-bitmaps-postcopy-test} (100%)
 rename tests/qemu-iotests/{199.out => tests/migrate-bitmaps-postcopy-test.out} 
(100%)
 rename tests/qemu-iotests/{169 => tests/migrate-bitmaps-test} (100%)
 rename tests/qemu-iotests/{169.out => tests/migrate-bitmaps-test.out} (100%)

diff --git a/tests/qemu-iotests/199 
b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
similarity index 100%
rename from tests/qemu-iotests/199
rename to tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
diff --git a/tests/qemu-iotests/199.out 
b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test.out
similarity index 100%
rename from tests/qemu-iotests/199.out
rename to tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test.out
diff --git a/tests/qemu-iotests/169 
b/tests/qemu-iotests/tests/migrate-bitmaps-test
similarity index 100%
rename from tests/qemu-iotests/169
rename to tests/qemu-iotests/tests/migrate-bitmaps-test
diff --git a/tests/qemu-iotests/169.out 
b/tests/qemu-iotests/tests/migrate-bitmaps-test.out
similarity index 100%
rename from tests/qemu-iotests/169.out
rename to tests/qemu-iotests/tests/migrate-bitmaps-test.out
-- 
2.21.0

[PATCH v4 5/9] iotests: add findtests.py

[Vladimir Sementsov-Ogievskiy]

Add python script with new logic of searching for tests:

Current ./check behavior:
 - tests are named [0-9][0-9][0-9]
 - tests must be registered in group file (even if test doesn't belong
   to any group, like 142)

Behavior of findtests.py:
 - group file is dropped
 - tests are all files in tests/ subdirectory (except for .out files),
   so it's not needed more to "register the test", just create it with
   appropriate name in tests/ subdirectory. Old names like
   [0-9][0-9][0-9] (in root iotests directory) are supported too, but
   not recommended for new tests
 - groups are parsed from '# group: ' line inside test files
 - optional file group.local may be used to define some additional
   groups for downstreams
 - 'disabled' group is used to temporary disable tests. So instead of
   commenting tests in old 'group' file you now can add them to
   disabled group with help of 'group.local' file
 - selecting test ranges like 5-15 are not supported more
   (to support restarting failed ./check command from the middle of the
process, new argument is added: --start-from)

Benefits:
 - no rebase conflicts in group file on patch porting from branch to
   branch
 - no conflicts in upstream, when different series want to occupy same
   test number
 - meaningful names for test files
   For example, with digital number, when some person wants to add some
   test about block-stream, he most probably will just create a new
   test. But if there would be test-block-stream test already, he will
   at first look at it and may be just add a test-case into it.
   And anyway meaningful names are better.

This commit don't update check behavior (which will be don in further
commit), still, the documentation changed like new behavior is already
here.  Let's live with this small inconsistency for the following few
commits, until final change.

The file findtests.py is self-executable and may be used for debugging
purposes.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 docs/devel/testing.rst  |  50 ++-
 tests/qemu-iotests/findtests.py | 229 
 2 files changed, 278 insertions(+), 1 deletion(-)
 create mode 100755 tests/qemu-iotests/findtests.py

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 770a987ea4..376e2f5086 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -153,7 +153,7 @@ check-block
 ---
 
 ``make check-block`` runs a subset of the block layer iotests (the tests that
-are in the "auto" group in ``tests/qemu-iotests/group``).
+are in the "auto" group).
 See the "QEMU iotests" section below for more information.
 
 GCC gcov support
@@ -267,6 +267,54 @@ another application on the host may have locked the file, 
possibly leading to a
 test failure.  If using such devices are explicitly desired, consider adding
 ``locking=off`` option to disable image locking.
 
+Test case groups
+
+
+Test may belong to some groups, you may define it in the comment inside the
+test. By convention, test groups are listed in the second line of the test
+file, after "#!/..." line, like this:
+
+.. code::
+
+  #!/usr/bin/env python3
+  # group: auto quick
+  #
+  ...
+
+Additional way of defining groups is creating tests/qemu-iotests/group.local
+file. This should be used only for downstream (this file should never appear
+in upstream). This file may be used for defining some downstream test groups
+or for temporary disable tests, like this:
+
+.. code::
+
+  # groups for some company downstream process
+  #
+  # ci - tests to run on build
+  # down - our downstream tests, not for upstream
+  #
+  # Format of each line is:
+  # TEST_NAME TEST_GROUP [TEST_GROUP ]...
+
+  013 ci
+  210 disabled
+  215 disabled
+  our-ugly-workaround-test down ci
+
+The (not exhaustive) list of groups:
+
+- quick : Tests in this group should finish within some few seconds.
+
+- auto : Tests in this group are used during "make check" and should be
+  runnable in any case. That means they should run with every QEMU binary
+  (also non-x86), with every QEMU configuration (i.e. must not fail if
+  an optional feature is not compiled in - but reporting a "skip" is ok),
+  work at least with the qcow2 file format, work with all kind of host
+  filesystems and users (e.g. "nobody" or "root") and must not take too
+  much memory and disk space (since CI pipelines tend to fail otherwise).
+
+- disabled : Tests in this group are disabled and ignored by check.
+
 .. _docker-ref:
 
 Docker based tests
diff --git a/tests/qemu-iotests/findtests.py b/tests/qemu-iotests/findtests.py
new file mode 100755
index 00..b053db48e8
--- /dev/null
+++ b/tests/qemu-iotests/findtests.py
@@ -0,0 +1,229 @@
+#!/usr/bin/env python3
+#
+# Parse command line options to define set of tests to run.
+#
+# Copyright (c) 2020 Virtuozzo International GmbH
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public 

[PATCH v4 2/9] iotests: fix some whitespaces in test output files

[Vladimir Sementsov-Ogievskiy]

We are going to be stricter about comparing test result with .out
files. So, fix some whitespaces now.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/175.out |  2 +-
 tests/qemu-iotests/267.out | 48 +++---
 tests/qemu-iotests/287.out | 10 
 3 files changed, 30 insertions(+), 30 deletions(-)

diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
index 39c2ee0f62..40a5bd1ce6 100644
--- a/tests/qemu-iotests/175.out
+++ b/tests/qemu-iotests/175.out
@@ -23,4 +23,4 @@ size=4096, min allocation
 == resize empty image with block_resize ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=0
 size=1048576, min allocation
- *** done
+*** done
diff --git a/tests/qemu-iotests/267.out b/tests/qemu-iotests/267.out
index 8dddb4baa4..26ef5f64b9 100644
--- a/tests/qemu-iotests/267.out
+++ b/tests/qemu-iotests/267.out
@@ -33,8 +33,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) savevm snap0
 (qemu) info snapshots
 List of snapshots present on all disks:
-IDTAG VM SIZEDATE   VM CLOCK
---snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
+IDTAG VM SIZEDATE   VM CLOCK
+--snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
 (qemu) loadvm snap0
 (qemu) quit
 
@@ -44,8 +44,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) savevm snap0
 (qemu) info snapshots
 List of snapshots present on all disks:
-IDTAG VM SIZEDATE   VM CLOCK
---snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
+IDTAG VM SIZEDATE   VM CLOCK
+--snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
 (qemu) loadvm snap0
 (qemu) quit
 
@@ -69,8 +69,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) savevm snap0
 (qemu) info snapshots
 List of snapshots present on all disks:
-IDTAG VM SIZEDATE   VM CLOCK
---snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
+IDTAG VM SIZEDATE   VM CLOCK
+--snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
 (qemu) loadvm snap0
 (qemu) quit
 
@@ -94,8 +94,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) savevm snap0
 (qemu) info snapshots
 List of snapshots present on all disks:
-IDTAG VM SIZEDATE   VM CLOCK
---snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
+IDTAG VM SIZEDATE   VM CLOCK
+--snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
 (qemu) loadvm snap0
 (qemu) quit
 
@@ -105,8 +105,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) savevm snap0
 (qemu) info snapshots
 List of snapshots present on all disks:
-IDTAG VM SIZEDATE   VM CLOCK
---snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
+IDTAG VM SIZEDATE   VM CLOCK
+--snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
 (qemu) loadvm snap0
 (qemu) quit
 
@@ -119,8 +119,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) savevm snap0
 (qemu) info snapshots
 List of snapshots present on all disks:
-IDTAG VM SIZEDATE   VM CLOCK
---snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
+IDTAG VM SIZEDATE   VM CLOCK
+--snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
 (qemu) loadvm snap0
 (qemu) quit
 
@@ -134,8 +134,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) savevm snap0
 (qemu) info snapshots
 List of snapshots present on all disks:
-IDTAG VM SIZEDATE   VM CLOCK
---snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
+IDTAG VM SIZEDATE   VM CLOCK
+--snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
 (qemu) loadvm snap0
 (qemu) quit
 
@@ -145,15 +145,15 @@ QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) savevm snap0
 (qemu) info snapshots
 List of snapshots present on all disks:
-IDTAG VM SIZEDATE   VM CLOCK
---snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
+IDTAG VM SIZEDATE   VM CLOCK
+--snap0  SIZE -mm-dd hh:mm:ss   00:00:00.000
 (qemu) loadvm snap0
 (qemu) quit
 
 Internal snapshots on overlay:
 Snapshot list:
-IDTAG VM SIZEDATE   VM CLOCK
-1 

[PATCH v4 1/9] iotests/277: use dot slash for nbd-fault-injector.py running

[Vladimir Sementsov-Ogievskiy]

If you run './check 277', check includes common.config which adjusts
$PATH to include '.' first, and therefore finds nbd-fault-injector.py
on PATH.  But if you run './277' directly, there is nothing to adjust
PATH, and if '.' is not already on your PATH by other means, the test
fails because the executable is not found.  Adjust how we invoke the
helper executable to avoid needing a PATH search in the first place.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Eric Blake 
---
 tests/qemu-iotests/277 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/277 b/tests/qemu-iotests/277
index d34f87021f..a39ce2d873 100755
--- a/tests/qemu-iotests/277
+++ b/tests/qemu-iotests/277
@@ -42,7 +42,7 @@ def make_conf_file(event):
 def start_server_NBD(event):
 make_conf_file(event)
 
-srv = subprocess.Popen(['nbd-fault-injector.py', '--classic-negotiation',
+srv = subprocess.Popen(['./nbd-fault-injector.py', '--classic-negotiation',
nbd_sock, conf_file], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, universal_newlines=True)
 line = srv.stdout.readline()
-- 
2.21.0

[PATCH v4 0/9] Rework iotests/check

[Vladimir Sementsov-Ogievskiy]

Hi all!

These series has 3 goals:

 - get rid of group file
 - introduce human-readable names for tests
 - rewrite check into python

v4:
   - human readable tests in  ..qemu-iotests/tests subdirectory
   - add --start-from argument for findtests
   - add type hinting

Vladimir Sementsov-Ogievskiy (9):
  iotests/277: use dot slash for nbd-fault-injector.py running
  iotests: fix some whitespaces in test output files
  iotests/283: make executable
  iotests: define group in each iotest
  iotests: add findtests.py
  iotests: add testenv.py
  iotests: add testrunner.py
  iotests: rewrite check into python
  iotests: rename and move 169 and 199 tests

 docs/devel/testing.rst|  50 +-
 tests/qemu-iotests/001|   1 +
 tests/qemu-iotests/002|   1 +
 tests/qemu-iotests/003|   1 +
 tests/qemu-iotests/004|   1 +
 tests/qemu-iotests/005|   1 +
 tests/qemu-iotests/007|   1 +
 tests/qemu-iotests/008|   1 +
 tests/qemu-iotests/009|   1 +
 tests/qemu-iotests/010|   1 +
 tests/qemu-iotests/011|   1 +
 tests/qemu-iotests/012|   1 +
 tests/qemu-iotests/013|   1 +
 tests/qemu-iotests/014|   1 +
 tests/qemu-iotests/015|   1 +
 tests/qemu-iotests/017|   1 +
 tests/qemu-iotests/018|   1 +
 tests/qemu-iotests/019|   1 +
 tests/qemu-iotests/020|   1 +
 tests/qemu-iotests/021|   1 +
 tests/qemu-iotests/022|   1 +
 tests/qemu-iotests/023|   1 +
 tests/qemu-iotests/024|   1 +
 tests/qemu-iotests/025|   1 +
 tests/qemu-iotests/026|   1 +
 tests/qemu-iotests/027|   1 +
 tests/qemu-iotests/028|   1 +
 tests/qemu-iotests/029|   1 +
 tests/qemu-iotests/030|   1 +
 tests/qemu-iotests/031|   1 +
 tests/qemu-iotests/032|   1 +
 tests/qemu-iotests/033|   1 +
 tests/qemu-iotests/034|   1 +
 tests/qemu-iotests/035|   1 +
 tests/qemu-iotests/036|   1 +
 tests/qemu-iotests/037|   1 +
 tests/qemu-iotests/038|   1 +
 tests/qemu-iotests/039|   1 +
 tests/qemu-iotests/040|   1 +
 tests/qemu-iotests/041|   1 +
 tests/qemu-iotests/042|   1 +
 tests/qemu-iotests/043|   1 +
 tests/qemu-iotests/044|   1 +
 tests/qemu-iotests/045|   1 +
 tests/qemu-iotests/046|   1 +
 tests/qemu-iotests/047|   1 +
 tests/qemu-iotests/048|   1 +
 tests/qemu-iotests/049|   1 +
 tests/qemu-iotests/050|   1 +
 tests/qemu-iotests/051|   1 +
 tests/qemu-iotests/052|   1 +
 tests/qemu-iotests/053|   1 +
 tests/qemu-iotests/054|   1 +
 tests/qemu-iotests/055|   1 +
 tests/qemu-iotests/056|   1 +
 tests/qemu-iotests/057|   1 +
 tests/qemu-iotests/058|   1 +
 tests/qemu-iotests/059|   1 +
 tests/qemu-iotests/060|   1 +
 tests/qemu-iotests/061|   1 +
 tests/qemu-iotests/062|   1 +
 tests/qemu-iotests/063|   1 +
 tests/qemu-iotests/064|   1 +
 tests/qemu-iotests/065|   1 +
 tests/qemu-iotests/066|   1 +
 tests/qemu-iotests/067|   1 +
 tests/qemu-iotests/068|   1 +
 tests/qemu-iotests/069|   1 +
 tests/qemu-iotests/070|   1 +
 tests/qemu-iotests/071|   1 +
 tests/qemu-iotests/072|   1 +
 tests/qemu-iotests/073|   1 +
 tests/qemu-iotests/074|   1 +
 tests/qemu-iotests/075|   1 +
 tests/qemu-iotests/076|   1 +
 tests/qemu-iotests/077|   1 +
 tests/qemu-iotests/078|   1 +
 tests/qemu-iotests/079|   1 +
 tests/qemu-iotests/080|   

Re: [PULL 00/51] Block layer patches

[no-reply]

Patchew URL: https://patchew.org/QEMU/20200515124521.335403-1-kw...@redhat.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing 
commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

Not run: 259
Failures: 229
Failed 1 of 119 iotests
make: *** [check-tests/check-block.sh] Error 1
make: *** Waiting for unfinished jobs
  TESTcheck-qtest-aarch64: tests/qtest/test-hmp
  TESTcheck-qtest-aarch64: tests/qtest/qos-test
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', 
'--label', 'com.qemu.instance.uuid=50c29e1822dc4e0e9a4c063f9410f2d5', '-u', 
'1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', 
'-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 
'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', 
'/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', 
'/var/tmp/patchew-tester-tmp-cfid8wn3/src/docker-src.2020-05-15-16.14.21.2026:/var/tmp/qemu:z,ro',
 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit 
status 2.
filter=--filter=label=com.qemu.instance.uuid=50c29e1822dc4e0e9a4c063f9410f2d5
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-cfid8wn3/src'
make: *** [docker-run-test-quick@centos7] Error 2

real15m41.654s
user0m9.390s


The full log is available at
http://patchew.org/logs/20200515124521.335403-1-kw...@redhat.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [PATCH v2 0/5] fix migration with bitmaps and mirror

[Vladimir Sementsov-Ogievskiy]

15.05.2020 20:51, Eric Blake wrote:

On 5/15/20 6:15 AM, Vladimir Sementsov-Ogievskiy wrote:


Max is trying to tackle the node-name issue:
https://lists.gnu.org/archive/html/qemu-devel/2020-05/msg03358.html

And trying to apply that patch after staging this series hits a conflict in 
mnigration/block-dirty-bitmap.c.  Which one should go in first?



My patches are needed to fix migration for the pre-blockdev configuration with 
mirror-top filter.

We ofcrouse need them in Virtuozzo, but it's not hard to keep the in 
downstream-only.. And it will be not simple to use new command from Max in 
pre-blockdev libvirt configuration, with auto-generated node-names.


Carrying a downstream fork forever is more work on you.  If the patch is easy 
enough to maintain, incorporating it upstream is best all around, even if 
libvirt has moved on to the point of no longer caring since it no longer uses 
pre-blockdev.


I hope not forever, when Rhel moves to node-names, we will do it too (hmm, I 
don't know, may be future already came, and Rhel8 libvirt is node-name 
oriented?) Still, yes it's always better to reduce the downstream overhead





How much we care about pre-blockdev libvirt now in upstream Qemu?

If we don't care, than these series are only for downstreams, and we don't need 
to apply them upstream..


Eventually, we may want to deprecate pre-blockdev, but I don't think we are 
there yet, and even when it does happen, it will be two more releases with it 
being deprecated before it is gone, so we might as well make it work correctly 
in the meantime.


Agree. Better to fix old behavior first, and then do proper deprecation if 
needed.





On the other hand, Max have to resend anyway, to handle old code, which uses 
device name instead of node-name. And if we don't want to drop now the code 
which can use device name (needed for old libvirt), why not to apply the 
series, which just make old code better?



In other words: do we still support pre-blockdev libvirt (and any other 
pre-blockdev users)?

If we support, than, as I said somewhere, I need to resend these series as I 
have updated version in our downstream. And I think, I can rebase Max's patch 
by myself and send together with this all, if no objections.



I'm going to resend the series today, let's look at it.



Sounds reasonable.



--
Best regards,
Vladimir

Re: [PATCH v2 0/5] fix migration with bitmaps and mirror

[Eric Blake]

On 5/15/20 6:15 AM, Vladimir Sementsov-Ogievskiy wrote:


Max is trying to tackle the node-name issue:
https://lists.gnu.org/archive/html/qemu-devel/2020-05/msg03358.html

And trying to apply that patch after staging this series hits a 
conflict in mnigration/block-dirty-bitmap.c.  Which one should go in 
first?




My patches are needed to fix migration for the pre-blockdev 
configuration with mirror-top filter.


We ofcrouse need them in Virtuozzo, but it's not hard to keep the in 
downstream-only.. And it will be not simple to use new command from 
Max in pre-blockdev libvirt configuration, with auto-generated 
node-names.


Carrying a downstream fork forever is more work on you.  If the patch is 
easy enough to maintain, incorporating it upstream is best all around, 
even if libvirt has moved on to the point of no longer caring since it 
no longer uses pre-blockdev.




How much we care about pre-blockdev libvirt now in upstream Qemu?

If we don't care, than these series are only for downstreams, and we 
don't need to apply them upstream..


Eventually, we may want to deprecate pre-blockdev, but I don't think we 
are there yet, and even when it does happen, it will be two more 
releases with it being deprecated before it is gone, so we might as well 
make it work correctly in the meantime.




On the other hand, Max have to resend anyway, to handle old code, 
which uses device name instead of node-name. And if we don't want to 
drop now the code which can use device name (needed for old libvirt), 
why not to apply the series, which just make old code better?




In other words: do we still support pre-blockdev libvirt (and any 
other pre-blockdev users)?


If we support, than, as I said somewhere, I need to resend these 
series as I have updated version in our downstream. And I think, I can 
rebase Max's patch by myself and send together with this all, if no 
objections.




I'm going to resend the series today, let's look at it.



Sounds reasonable.

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [PATCH v2 4/5] sysemu/block-backend: Document blk_read()/blk_pwrite()

[Eric Blake]

On 5/15/20 4:19 AM, Philippe Mathieu-Daudé wrote:

The blk_read()/blk_pwrite() return value is not obvious,


s/read/pread/ here and in the subject


document it.

Signed-off-by: Philippe Mathieu-Daudé 
---



+/**
+ * blk_pread:
+ *
+ * @blk: the block backend where the buffer content is going to be read from
+ * @offset: position in bytes to read at
+ * @buf: the data buffer
+ * @bytes: number of bytes to read
+ *
+ * Returns: the number of bytes read on success, or a negative errno otherwise.
+ */
  int blk_pread(BlockBackend *blk, int64_t offset, void *buf, int bytes);


"the number of bytes read" is ambiguous - it sounds too much like the 
read() syscall where short reads are successful.  But blk_pread() never 
does short reads, on success, the result is exactly 'bytes'.



+
+/**
+ * blk_pwrite:
+ *
+ * @blk: the block backend where the buffer content is going to be written to
+ * @offset: position in bytes to write at
+ * @buf: the data buffer
+ * @bytes: number of bytes of @buf to write
+ * @flags: request flags
+ *
+ * Returns: the number of bytes consumed on success,


Ditto - we don't support short writes, so on success, it is always 'bytes'.


+ *  or a negative errno otherwise.
+ */
  int blk_pwrite(BlockBackend *blk, int64_t offset, const void *buf, int bytes,
 BdrvRequestFlags flags);
  int64_t blk_getlength(BlockBackend *blk);



--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [PATCH v2 0/1] qemu-nbd: Close inherited stderr

[Eric Blake]

On 5/15/20 1:36 AM, Raphael Pour wrote:

From e5749541494abcdcaa37d752172741e1bc38e984 Mon Sep 17 00:00:00 2001

From: Raphael Pour 
Date: Fri, 15 May 2020 08:30:50 +0200
Subject: [PATCH] qemu-nbd: Close inherited stderr

Close inherited stderr of the parent if fork_process is false.
Otherwise no one will close it. (introduced by e6df58a5)

Signed-off-by: Raphael Pour 
---
  qemu-nbd.c | 6 +-
  1 file changed, 5 insertions(+), 1 deletion(-)


Thanks. It might have been easier posting this as a standalone v3 rather 
than buried in reply to v2, but I have now queued it to go in through my 
NBD tree (I plan on doing a pull request Monday).


I've also taken the liberty to amend the commit message as follows, to 
give more context into why the fix is needed:


qemu-nbd: Close inherited stderr

Close inherited stderr of the parent if fork_process is false.
Otherwise no one will close it. (introduced by e6df58a5)

This only affected 'qemu-nbd -c /dev/nbd0'.

Signed-off-by: Raphael Pour 
Message-Id: 
Reviewed-by: Eric Blake 
[eblake: Enhance commit message]
Signed-off-by: Eric Blake 

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [PATCH v6 04/14] block/amend: separate amend and create options for qemu-img

[Eric Blake]

On 5/15/20 1:22 AM, Max Reitz wrote:




+    QCOW_COMMON_OPTIONS,
+    { /* end of list */ }


...the intended usage is to use the macro name followed by a comma, so
including a trailing comma in the macro itself would lead to a syntax
error.


But why is that the indended usage?  Is there something in our coding
style that forbids macros that don’t allow a separator to be placed
after them?


If we have more than one such macro, it is easier to write and indent 
(especially when using your editor's ability to decipher enough syntax 
to suggest how to indent):


myarray = {
  COMMON_ELEMENTS,
  MORE_ELEMENTS,
  { /* end of list */ }
};

than it is:

myarray = {
  COMMON_ELEMENTS
  MORE_ELEMENTS
  { /* end of list */ }
};

which in turn implies that it is better to NOT stick a trailing comma in 
the macro itself.  Similarly, for macros intended to replace statements, 
we tend to avoid the trailing ; in the macro itself, because it is 
easier to read:


{
  code;
  MACRO();
  more code;
}

than it is:

{
  code;
  MACRO()
  more code;
}

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [PATCH v2 5/5] vhost: add device started check in migration set log

[Dima Stepanov]

On Thu, May 14, 2020 at 03:34:24PM +0800, Jason Wang wrote:
> 
> On 2020/5/13 下午5:47, Dima Stepanov wrote:
> >>> case CHR_EVENT_CLOSED:
> >>> /* a close event may happen during a read/write, but vhost
> >>>  * code assumes the vhost_dev remains setup, so delay the
> >>>  * stop & clear to idle.
> >>>  * FIXME: better handle failure in vhost code, remove bh
> >>>  */
> >>> if (s->watch) {
> >>> AioContext *ctx = qemu_get_current_aio_context();
> >>>
> >>> g_source_remove(s->watch);
> >>> s->watch = 0;
> >>> qemu_chr_fe_set_handlers(>chr, NULL, NULL, NULL, NULL,
> >>>  NULL, NULL, false);
> >>>
> >>> aio_bh_schedule_oneshot(ctx, chr_closed_bh, opaque);
> >>> }
> >>> break;
> >>>
> >>>I think it's time we dropped the FIXME and moved the handling to common
> >>>code. Jason? Marc-André?
> >>I agree. Just to confirm, do you prefer bh or doing changes like what is
> >>done in this series? It looks to me bh can have more easier codes.
> >Could it be a good idea just to make disconnect in the char device but
> >postphone clean up in the vhost-user-blk (or any other vhost-user
> >device) itself? So we are moving the postphone logic and decision from
> >the char device to vhost-user device. One of the idea i have is as
> >follows:
> >   - Put ourself in the INITIALIZATION state
> >   - Start these vhost-user "handshake" commands
> >   - If we got a disconnect error, perform disconnect, but don't clean up
> > device (it will be clean up on the roll back). I can be done by
> > checking the state in vhost_user_..._disconnect routine or smth like it
> 
> 
> Any issue you saw just using the aio bh as Michael posted above.
> 
> Then we don't need to deal with the silent vhost_dev_stop() and we will have
> codes that is much more easier to understand.
I've implemented this solution inside
hw/block/vhost-user-blk.c:vhost_user_blk_event() in the similar way by
using the s->connected field. Looks good and more correct fix ). I have
two questions here before i'll rework the fixes:
1. Is it okay to make the similar fix inside vhost_user_blk_event() or
we are looking for more generic vhost-user solution? What do you think?
2. For migration we require an additional information that for the
vhost-user device it isn't an error, because i'm trigerring the
following assert error:
  Core was generated by `x86_64-softmmu/qemu-system-x86_64 -nodefaults 
-no-user-config -M q35,sata=false'.
  Program terminated with signal SIGABRT, Aborted.
  #0  0x7fb56e729428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
  [Current thread is 1 (Thread 0x7fb486ef5700 (LWP 527734))]

  (gdb) bt
  #0  0x7fb56e729428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
  #1  0x7fb56e72b02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
  #2  0x5648ea376ee6 in vhost_log_global_start
  (listener=0x5648ece4eb08) at ./hw/virtio/vhost.c:857
  #3  0x5648ea2dde7e in memory_global_dirty_log_start ()
  at ./memory.c:2611
  #4  0x5648ea2e68e7 in ram_init_bitmaps (rs=0x7fb4740008c0)
  at ./migration/ram.c:2305
  #5  0x5648ea2e698b in ram_init_all (rsp=0x5648eb1f0f20 )
  at ./migration/ram.c:2323
  #6  0x5648ea2e6cc5 in ram_save_setup (f=0x5648ec609e00,
  opaque=0x5648eb1f0f20 )
  at ./migration/ram.c:2436
  #7  0x5648ea67b7d3 in qemu_savevm_state_setup (f=0x5648ec609e00) at
  migration/savevm.c:1176
  #8  0x5648ea674511 in migration_thread (opaque=0x5648ec031ff0) at
  migration/migration.c:3416
  #9  0x5648ea85d65d in qemu_thread_start (args=0x5648ec6057f0) at
  util/qemu-thread-posix.c:519
  #10 0x7fb56eac56ba in start_thread () from
  /lib/x86_64-linux-gnu/libpthread.so.0
  #11 0x7fb56e7fb41d in clone () from /lib/x86_64-linux-gnu/libc.so.6
  (gdb) frame 2
  #2  0x5648ea376ee6 in vhost_log_global_start
 (listener=0x5648ece4eb08) at ./hw/virtio/vhost.c:857
  857 abort();
  (gdb) list
  852 {
  853 int r;
  854
  855 r = vhost_migration_log(listener, true);
  856 if (r < 0) {
  857 abort();
  858 }
  859 }
  860
  861 static void vhost_log_global_stop(MemoryListener *listener)
Since bh postphone the clean up, we can't use the ->started field.
Do we have any mechanism to get the device type/state in the common
vhost_migration_log() routine? So for example for the vhost-user/disconnect
device we will be able to return 0. Or should we implement it and introduce
it in this patch set?

Thanks, Dima.

> 
> Thank
> 
> 
> >   - vhost-user command returns error back to the _start() routine
> >   - Rollback in one place in the start() routine, by calling this
> > postphoned clean up for the disconnect
> >
> 

Re: [RFC v4 6/6] hmp: add x-debug-virtio commands

[Laurent Vivier]

On 15/05/2020 17:45, Dr. David Alan Gilbert wrote:
> * Laurent Vivier (lviv...@redhat.com) wrote:
>> On 13/05/2020 12:51, Dr. David Alan Gilbert wrote:
>>> * Laurent Vivier (lviv...@redhat.com) wrote:
 This patch implements HMP version of the virtio QMP commands

 Signed-off-by: Laurent Vivier 
>>>
>>> Reviewed-by: Dr. David Alan Gilbert 
>>>
>>> With a thought below
>>>
 ---
  Makefile|   2 +-
  Makefile.target |   7 +-
  docs/system/monitor.rst |   2 +
  hmp-commands-virtio.hx  | 160 +
  hmp-commands.hx |  10 +++
  hw/virtio/virtio.c  | 193 +++-
  include/monitor/hmp.h   |   4 +
  monitor/misc.c  |  17 
  8 files changed, 391 insertions(+), 4 deletions(-)
  create mode 100644 hmp-commands-virtio.hx

>> ...
 diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
 index 66dc2cef1b39..c3d6b783417e 100644
 --- a/hw/virtio/virtio.c
 +++ b/hw/virtio/virtio.c
>> ...
 @@ -4033,6 +4092,92 @@ VirtioStatus *qmp_x_debug_virtio_status(const char* 
 path, Error **errp)
  return status;
  }
  
 +#define DUMP_FEATURES(type, field)
  \
 +do {  
  \
 +type##FeatureList *list = features->device->u.field.data; 
  \
 +if (list) {   
  \
 +monitor_printf(mon, "");  
  \
 +while (list) {
  \
 +monitor_printf(mon, "%s", 
 type##Feature_str(list->value)); \
 +list = list->next;
  \
 +if (list != NULL) {   
  \
 +monitor_printf(mon, ", ");
  \
 +} 
  \
 +} 
  \
 +monitor_printf(mon, "\n");
  \
 +} 
  \
 +} while (0)
>>>
>>> It feels like you should be able to have an array of Feature_str's
>>> indexed by VIRTIO_DEVICE_FEATURE_KIND_ enum, so that when a new
>>> VIRTIO_DEVICE_FEATURE_KIND is added you don't need to fix this up.
>>
>> I don't understand what you mean here.
> 
> Instead of the switch below, I'm thinking you could have something like:
> 
> if (features->device->type < something_MAX) {
> features_str = anarray[features->device->type];
> 
> 
> monitor_printf(mon, "%s", features_str(list->value));
> 
> }
> 
> with 'anarray' somewhere more central, so we don't have to keep
> these switch structures and macros spread around.

OK, I tried that, but in fact we need to know the type of the list to be
able to scan it (the "type##FeatureList": VirtoSerialFeatureList,
VirtioBlkFeatureList, ...), except if we introduce a generic feature
list node (for "next " and "value"). But it becomes more complicated,
because we also need to generate the "anarray" somewhere.

[Note: I've changed the legacy enum to a flat enum as proposed by Eric
in v3]

Thanks,
Laurent

Re: [RFC v4 6/6] hmp: add x-debug-virtio commands

[Dr. David Alan Gilbert]

* Laurent Vivier (lviv...@redhat.com) wrote:
> On 13/05/2020 12:51, Dr. David Alan Gilbert wrote:
> > * Laurent Vivier (lviv...@redhat.com) wrote:
> >> This patch implements HMP version of the virtio QMP commands
> >>
> >> Signed-off-by: Laurent Vivier 
> > 
> > Reviewed-by: Dr. David Alan Gilbert 
> > 
> > With a thought below
> > 
> >> ---
> >>  Makefile|   2 +-
> >>  Makefile.target |   7 +-
> >>  docs/system/monitor.rst |   2 +
> >>  hmp-commands-virtio.hx  | 160 +
> >>  hmp-commands.hx |  10 +++
> >>  hw/virtio/virtio.c  | 193 +++-
> >>  include/monitor/hmp.h   |   4 +
> >>  monitor/misc.c  |  17 
> >>  8 files changed, 391 insertions(+), 4 deletions(-)
> >>  create mode 100644 hmp-commands-virtio.hx
> >>
> ...
> >> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> >> index 66dc2cef1b39..c3d6b783417e 100644
> >> --- a/hw/virtio/virtio.c
> >> +++ b/hw/virtio/virtio.c
> ...
> >> @@ -4033,6 +4092,92 @@ VirtioStatus *qmp_x_debug_virtio_status(const char* 
> >> path, Error **errp)
> >>  return status;
> >>  }
> >>  
> >> +#define DUMP_FEATURES(type, field)
> >>  \
> >> +do {  
> >>  \
> >> +type##FeatureList *list = features->device->u.field.data; 
> >>  \
> >> +if (list) {   
> >>  \
> >> +monitor_printf(mon, "");  
> >>  \
> >> +while (list) {
> >>  \
> >> +monitor_printf(mon, "%s", 
> >> type##Feature_str(list->value)); \
> >> +list = list->next;
> >>  \
> >> +if (list != NULL) {   
> >>  \
> >> +monitor_printf(mon, ", ");
> >>  \
> >> +} 
> >>  \
> >> +} 
> >>  \
> >> +monitor_printf(mon, "\n");
> >>  \
> >> +} 
> >>  \
> >> +} while (0)
> > 
> > It feels like you should be able to have an array of Feature_str's
> > indexed by VIRTIO_DEVICE_FEATURE_KIND_ enum, so that when a new
> > VIRTIO_DEVICE_FEATURE_KIND is added you don't need to fix this up.
> 
> I don't understand what you mean here.

Instead of the switch below, I'm thinking you could have something like:

if (features->device->type < something_MAX) {
features_str = anarray[features->device->type];


monitor_printf(mon, "%s", features_str(list->value));

}

with 'anarray' somewhere more central, so we don't have to keep
these switch structures and macros spread around.

Dave

> >> +
> >> +static void hmp_virtio_dump_features(Monitor *mon,
> >> + VirtioStatusFeatures *features)
> >> +{
> >> +VirtioTransportFeatureList *transport_list = features->transport;
> >> +while (transport_list) {
> >> +monitor_printf(mon, "%s",
> >> +   VirtioTransportFeature_str(transport_list->value));
> >> +transport_list = transport_list->next;
> >> +if (transport_list != NULL) {
> >> +monitor_printf(mon, ", ");
> >> +}
> >> +}
> >> +monitor_printf(mon, "\n");
> >> +switch (features->device->type) {
> >> +case VIRTIO_DEVICE_FEATURES_KIND_VIRTIO_SERIAL:
> >> +DUMP_FEATURES(VirtioSerial, virtio_serial);
> >> +break;
> >> +case VIRTIO_DEVICE_FEATURES_KIND_VIRTIO_BLK:
> >> +DUMP_FEATURES(VirtioBlk, virtio_blk);
> >> +break;
> >> +case VIRTIO_DEVICE_FEATURES_KIND_VIRTIO_GPU:
> >> +DUMP_FEATURES(VirtioGpu, virtio_gpu);
> >> +break;
> >> +case VIRTIO_DEVICE_FEATURES_KIND_VIRTIO_NET:
> >> +DUMP_FEATURES(VirtioNet, virtio_net);
> >> +break;
> >> +case VIRTIO_DEVICE_FEATURES_KIND_VIRTIO_SCSI:
> >> +DUMP_FEATURES(VirtioScsi, virtio_scsi);
> >> +break;
> >> +case VIRTIO_DEVICE_FEATURES_KIND_VIRTIO_BALLOON:
> >> +DUMP_FEATURES(VirtioBalloon, virtio_balloon);
> >> +break;
> >> +case VIRTIO_DEVICE_FEATURES_KIND_VIRTIO_IOMMU:
> >> +DUMP_FEATURES(VirtioIommu, virtio_iommu);
> >> +break;
> >> +default:
> >> +g_assert_not_reached();
> >> +}
> >> +if (features->unknown) {
> >> +monitor_printf(mon, "
> >> unknown(0x%016"PRIx64")\n", \
> >> +   features->unknown);
> >> +}
> >> +}
> ...
> 
> Thanks,
> Laurent
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, 

Re: [RFC v4 6/6] hmp: add x-debug-virtio commands

[Laurent Vivier]

On 13/05/2020 12:51, Dr. David Alan Gilbert wrote:
> * Laurent Vivier (lviv...@redhat.com) wrote:
>> This patch implements HMP version of the virtio QMP commands
>>
>> Signed-off-by: Laurent Vivier 
> 
> Reviewed-by: Dr. David Alan Gilbert 
> 
> With a thought below
> 
>> ---
>>  Makefile|   2 +-
>>  Makefile.target |   7 +-
>>  docs/system/monitor.rst |   2 +
>>  hmp-commands-virtio.hx  | 160 +
>>  hmp-commands.hx |  10 +++
>>  hw/virtio/virtio.c  | 193 +++-
>>  include/monitor/hmp.h   |   4 +
>>  monitor/misc.c  |  17 
>>  8 files changed, 391 insertions(+), 4 deletions(-)
>>  create mode 100644 hmp-commands-virtio.hx
>>
...
>> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
>> index 66dc2cef1b39..c3d6b783417e 100644
>> --- a/hw/virtio/virtio.c
>> +++ b/hw/virtio/virtio.c
...
>> @@ -4033,6 +4092,92 @@ VirtioStatus *qmp_x_debug_virtio_status(const char* 
>> path, Error **errp)
>>  return status;
>>  }
>>  
>> +#define DUMP_FEATURES(type, field) \
>> +do {   \
>> +type##FeatureList *list = features->device->u.field.data;  \
>> +if (list) {\
>> +monitor_printf(mon, "");   \
>> +while (list) { \
>> +monitor_printf(mon, "%s", type##Feature_str(list->value)); \
>> +list = list->next; \
>> +if (list != NULL) {\
>> +monitor_printf(mon, ", "); \
>> +}  \
>> +}  \
>> +monitor_printf(mon, "\n"); \
>> +}  \
>> +} while (0)
> 
> It feels like you should be able to have an array of Feature_str's
> indexed by VIRTIO_DEVICE_FEATURE_KIND_ enum, so that when a new
> VIRTIO_DEVICE_FEATURE_KIND is added you don't need to fix this up.

I don't understand what you mean here.

>> +
>> +static void hmp_virtio_dump_features(Monitor *mon,
>> + VirtioStatusFeatures *features)
>> +{
>> +VirtioTransportFeatureList *transport_list = features->transport;
>> +while (transport_list) {
>> +monitor_printf(mon, "%s",
>> +   VirtioTransportFeature_str(transport_list->value));
>> +transport_list = transport_list->next;
>> +if (transport_list != NULL) {
>> +monitor_printf(mon, ", ");
>> +}
>> +}
>> +monitor_printf(mon, "\n");
>> +switch (features->device->type) {
>> +case VIRTIO_DEVICE_FEATURES_KIND_VIRTIO_SERIAL:
>> +DUMP_FEATURES(VirtioSerial, virtio_serial);
>> +break;
>> +case VIRTIO_DEVICE_FEATURES_KIND_VIRTIO_BLK:
>> +DUMP_FEATURES(VirtioBlk, virtio_blk);
>> +break;
>> +case VIRTIO_DEVICE_FEATURES_KIND_VIRTIO_GPU:
>> +DUMP_FEATURES(VirtioGpu, virtio_gpu);
>> +break;
>> +case VIRTIO_DEVICE_FEATURES_KIND_VIRTIO_NET:
>> +DUMP_FEATURES(VirtioNet, virtio_net);
>> +break;
>> +case VIRTIO_DEVICE_FEATURES_KIND_VIRTIO_SCSI:
>> +DUMP_FEATURES(VirtioScsi, virtio_scsi);
>> +break;
>> +case VIRTIO_DEVICE_FEATURES_KIND_VIRTIO_BALLOON:
>> +DUMP_FEATURES(VirtioBalloon, virtio_balloon);
>> +break;
>> +case VIRTIO_DEVICE_FEATURES_KIND_VIRTIO_IOMMU:
>> +DUMP_FEATURES(VirtioIommu, virtio_iommu);
>> +break;
>> +default:
>> +g_assert_not_reached();
>> +}
>> +if (features->unknown) {
>> +monitor_printf(mon, "
>> unknown(0x%016"PRIx64")\n", \
>> +   features->unknown);
>> +}
>> +}
...

Thanks,
Laurent

[PATCH v6 16/16] acpi: q35: drop _SB.PCI0.ISA.LPCD opregion.

[Gerd Hoffmann]

Seems to be unused.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Igor Mammedov 
---
 hw/i386/acpi-build.c | 11 ---
 1 file changed, 11 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index c1e63cce5e8e..1afb47b09ee9 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1417,7 +1417,6 @@ static void build_q35_isa_bridge(Aml *table)
 {
 Aml *dev;
 Aml *scope;
-Aml *field;
 
 scope =  aml_scope("_SB.PCI0");
 dev = aml_device("ISA");
@@ -1427,16 +1426,6 @@ static void build_q35_isa_bridge(Aml *table)
 aml_append(dev, aml_operation_region("PIRQ", AML_PCI_CONFIG,
  aml_int(0x60), 0x0C));
 
-aml_append(dev, aml_operation_region("LPCD", AML_PCI_CONFIG,
- aml_int(0x80), 0x02));
-field = aml_field("LPCD", AML_ANY_ACC, AML_NOLOCK, AML_PRESERVE);
-aml_append(field, aml_named_field("COMA", 3));
-aml_append(field, aml_reserved_field(1));
-aml_append(field, aml_named_field("COMB", 3));
-aml_append(field, aml_reserved_field(1));
-aml_append(field, aml_named_field("LPTD", 2));
-aml_append(dev, field);
-
 aml_append(scope, dev);
 aml_append(table, scope);
 }
-- 
2.18.4

[PATCH v6 08/16] acpi: move aml builder code for floppy device

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Igor Mammedov 
---
 hw/block/fdc.c   | 83 
 hw/i386/acpi-build.c | 83 
 stubs/cmos.c |  7 
 stubs/Makefile.objs  |  1 +
 4 files changed, 91 insertions(+), 83 deletions(-)
 create mode 100644 stubs/cmos.c

diff --git a/hw/block/fdc.c b/hw/block/fdc.c
index c5fb9d6ece77..b4d2eaf66dcd 100644
--- a/hw/block/fdc.c
+++ b/hw/block/fdc.c
@@ -32,6 +32,8 @@
 #include "qapi/error.h"
 #include "qemu/error-report.h"
 #include "qemu/timer.h"
+#include "hw/i386/pc.h"
+#include "hw/acpi/aml-build.h"
 #include "hw/irq.h"
 #include "hw/isa/isa.h"
 #include "hw/qdev-properties.h"
@@ -2765,6 +2767,85 @@ void isa_fdc_get_drive_max_chs(FloppyDriveType type,
 (*maxc)--;
 }
 
+static Aml *build_fdinfo_aml(int idx, FloppyDriveType type)
+{
+Aml *dev, *fdi;
+uint8_t maxc, maxh, maxs;
+
+isa_fdc_get_drive_max_chs(type, , , );
+
+dev = aml_device("FLP%c", 'A' + idx);
+
+aml_append(dev, aml_name_decl("_ADR", aml_int(idx)));
+
+fdi = aml_package(16);
+aml_append(fdi, aml_int(idx));  /* Drive Number */
+aml_append(fdi,
+aml_int(cmos_get_fd_drive_type(type)));  /* Device Type */
+/*
+ * the values below are the limits of the drive, and are thus independent
+ * of the inserted media
+ */
+aml_append(fdi, aml_int(maxc));  /* Maximum Cylinder Number */
+aml_append(fdi, aml_int(maxs));  /* Maximum Sector Number */
+aml_append(fdi, aml_int(maxh));  /* Maximum Head Number */
+/*
+ * SeaBIOS returns the below values for int 0x13 func 0x08 regardless of
+ * the drive type, so shall we
+ */
+aml_append(fdi, aml_int(0xAF));  /* disk_specify_1 */
+aml_append(fdi, aml_int(0x02));  /* disk_specify_2 */
+aml_append(fdi, aml_int(0x25));  /* disk_motor_wait */
+aml_append(fdi, aml_int(0x02));  /* disk_sector_siz */
+aml_append(fdi, aml_int(0x12));  /* disk_eot */
+aml_append(fdi, aml_int(0x1B));  /* disk_rw_gap */
+aml_append(fdi, aml_int(0xFF));  /* disk_dtl */
+aml_append(fdi, aml_int(0x6C));  /* disk_formt_gap */
+aml_append(fdi, aml_int(0xF6));  /* disk_fill */
+aml_append(fdi, aml_int(0x0F));  /* disk_head_sttl */
+aml_append(fdi, aml_int(0x08));  /* disk_motor_strt */
+
+aml_append(dev, aml_name_decl("_FDI", fdi));
+return dev;
+}
+
+static void fdc_isa_build_aml(ISADevice *isadev, Aml *scope)
+{
+Aml *dev;
+Aml *crs;
+int i;
+
+#define ACPI_FDE_MAX_FD 4
+uint32_t fde_buf[5] = {
+0, 0, 0, 0, /* presence of floppy drives #0 - #3 */
+cpu_to_le32(2)  /* tape presence (2 == never present) */
+};
+
+crs = aml_resource_template();
+aml_append(crs, aml_io(AML_DECODE16, 0x03F2, 0x03F2, 0x00, 0x04));
+aml_append(crs, aml_io(AML_DECODE16, 0x03F7, 0x03F7, 0x00, 0x01));
+aml_append(crs, aml_irq_no_flags(6));
+aml_append(crs,
+aml_dma(AML_COMPATIBILITY, AML_NOTBUSMASTER, AML_TRANSFER8, 2));
+
+dev = aml_device("FDC0");
+aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0700")));
+aml_append(dev, aml_name_decl("_CRS", crs));
+
+for (i = 0; i < MIN(MAX_FD, ACPI_FDE_MAX_FD); i++) {
+FloppyDriveType type = isa_fdc_get_drive_type(isadev, i);
+
+if (type < FLOPPY_DRIVE_TYPE_NONE) {
+fde_buf[i] = cpu_to_le32(1);  /* drive present */
+aml_append(dev, build_fdinfo_aml(i, type));
+}
+}
+aml_append(dev, aml_name_decl("_FDE",
+   aml_buffer(sizeof(fde_buf), (uint8_t *)fde_buf)));
+
+aml_append(scope, dev);
+}
+
 static const VMStateDescription vmstate_isa_fdc ={
 .name = "fdc",
 .version_id = 2,
@@ -2798,11 +2879,13 @@ static Property isa_fdc_properties[] = {
 static void isabus_fdc_class_init(ObjectClass *klass, void *data)
 {
 DeviceClass *dc = DEVICE_CLASS(klass);
+ISADeviceClass *isa = ISA_DEVICE_CLASS(klass);
 
 dc->realize = isabus_fdc_realize;
 dc->fw_name = "fdc";
 dc->reset = fdctrl_external_reset_isa;
 dc->vmsd = _isa_fdc;
+isa->build_aml = fdc_isa_build_aml;
 device_class_set_props(dc, isa_fdc_properties);
 set_bit(DEVICE_CATEGORY_STORAGE, dc->categories);
 }
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 443db94deb5b..775936e28b9a 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1058,85 +1058,6 @@ static void build_hpet_aml(Aml *table)
 aml_append(table, scope);
 }
 
-static Aml *build_fdinfo_aml(int idx, FloppyDriveType type)
-{
-Aml *dev, *fdi;
-uint8_t maxc, maxh, maxs;
-
-isa_fdc_get_drive_max_chs(type, , , );
-
-dev = aml_device("FLP%c", 'A' + idx);
-
-aml_append(dev, aml_name_decl("_ADR", aml_int(idx)));
-
-fdi = aml_package(16);
-aml_append(fdi, aml_int(idx));  /* Drive Number */
-aml_append(fdi,
-aml_int(cmos_get_fd_drive_type(type)));  /* Device Type */
-/*
- * the values below are the 

Re: [RFC v3 4/6] qmp: add QMP command x-debug-virtio-queue-status

[Laurent Vivier]

On 08/05/2020 04:57, Jason Wang wrote:
> 
> On 2020/5/7 下午7:49, Laurent Vivier wrote:
>> This new command shows internal status of a VirtQueue.
>> (vrings and indexes).
>>
>> Signed-off-by: Laurent Vivier 
> 
> 
> It looks to me that packed virtqueue is not supported. It's better to
> add them in the future.

I agree, it's why the series still remains an "RFC".

...
>> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
>> index 59bf6ef651a6..57552bf05014 100644
>> --- a/hw/virtio/virtio.c
>> +++ b/hw/virtio/virtio.c
>> @@ -3877,6 +3877,41 @@ static VirtIODevice *virtio_device_find(const
>> char *path)
>>   return NULL;
>>   }
>>   +VirtQueueStatus *qmp_x_debug_virtio_queue_status(const char *path,
>> + uint16_t queue,
>> Error **errp)
>> +{
>> +    VirtIODevice *vdev;
>> +    VirtQueueStatus *status;
>> +
>> +    vdev = virtio_device_find(path);
>> +    if (vdev == NULL) {
>> +    error_setg(errp, "Path %s is not a VirtIO device", path);
>> +    return NULL;
>> +    }
>> +
>> +    if (queue >= VIRTIO_QUEUE_MAX || !virtio_queue_get_num(vdev,
>> queue)) {
>> +    error_setg(errp, "Invalid virtqueue number %d", queue);
>> +    return NULL;
>> +    }
>> +
>> +    status = g_new0(VirtQueueStatus, 1);
>> +    status->queue_index = vdev->vq[queue].queue_index;
>> +    status->inuse = vdev->vq[queue].inuse;
>> +    status->vring_num = vdev->vq[queue].vring.num;
>> +    status->vring_num_default = vdev->vq[queue].vring.num_default;
>> +    status->vring_align = vdev->vq[queue].vring.align;
>> +    status->vring_desc = vdev->vq[queue].vring.desc;
>> +    status->vring_avail = vdev->vq[queue].vring.avail;
>> +    status->vring_used = vdev->vq[queue].vring.used;
>> +    status->last_avail_idx = vdev->vq[queue].last_avail_idx;
> 
> 
> This might not be correct when vhost is used.
> 
> We may consider to sync last_avail_idx from vhost backends here?

Yes, but I don't know how to do that. Where can I find the information?

Thanks,
Laurent

[PATCH v6 11/16] acpi: move aml builder code for i8042 (kbd+mouse) device

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Igor Mammedov 
---
 hw/i386/acpi-build.c | 39 ---
 hw/input/pckbd.c | 31 +++
 2 files changed, 31 insertions(+), 39 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 775936e28b9a..829e20664fff 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1058,42 +1058,6 @@ static void build_hpet_aml(Aml *table)
 aml_append(table, scope);
 }
 
-static Aml *build_kbd_device_aml(void)
-{
-Aml *dev;
-Aml *crs;
-
-dev = aml_device("KBD");
-aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0303")));
-
-aml_append(dev, aml_name_decl("_STA", aml_int(0xf)));
-
-crs = aml_resource_template();
-aml_append(crs, aml_io(AML_DECODE16, 0x0060, 0x0060, 0x01, 0x01));
-aml_append(crs, aml_io(AML_DECODE16, 0x0064, 0x0064, 0x01, 0x01));
-aml_append(crs, aml_irq_no_flags(1));
-aml_append(dev, aml_name_decl("_CRS", crs));
-
-return dev;
-}
-
-static Aml *build_mouse_device_aml(void)
-{
-Aml *dev;
-Aml *crs;
-
-dev = aml_device("MOU");
-aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0F13")));
-
-aml_append(dev, aml_name_decl("_STA", aml_int(0xf)));
-
-crs = aml_resource_template();
-aml_append(crs, aml_irq_no_flags(12));
-aml_append(dev, aml_name_decl("_CRS", crs));
-
-return dev;
-}
-
 static void build_isa_devices_aml(Aml *table)
 {
 bool ambiguous;
@@ -1101,9 +1065,6 @@ static void build_isa_devices_aml(Aml *table)
 Aml *scope = aml_scope("_SB.PCI0.ISA");
 Object *obj = object_resolve_path_type("", TYPE_ISA_BUS, );
 
-aml_append(scope, build_kbd_device_aml());
-aml_append(scope, build_mouse_device_aml());
-
 if (ambiguous) {
 error_report("Multiple ISA busses, unable to define IPMI ACPI data");
 } else if (!obj) {
diff --git a/hw/input/pckbd.c b/hw/input/pckbd.c
index 60a41303203a..29d633ca9478 100644
--- a/hw/input/pckbd.c
+++ b/hw/input/pckbd.c
@@ -26,6 +26,7 @@
 #include "qemu/log.h"
 #include "hw/isa/isa.h"
 #include "migration/vmstate.h"
+#include "hw/acpi/aml-build.h"
 #include "hw/input/ps2.h"
 #include "hw/irq.h"
 #include "hw/input/i8042.h"
@@ -561,12 +562,42 @@ static void i8042_realizefn(DeviceState *dev, Error 
**errp)
 qemu_register_reset(kbd_reset, s);
 }
 
+static void i8042_build_aml(ISADevice *isadev, Aml *scope)
+{
+Aml *kbd;
+Aml *mou;
+Aml *crs;
+
+crs = aml_resource_template();
+aml_append(crs, aml_io(AML_DECODE16, 0x0060, 0x0060, 0x01, 0x01));
+aml_append(crs, aml_io(AML_DECODE16, 0x0064, 0x0064, 0x01, 0x01));
+aml_append(crs, aml_irq_no_flags(1));
+
+kbd = aml_device("KBD");
+aml_append(kbd, aml_name_decl("_HID", aml_eisaid("PNP0303")));
+aml_append(kbd, aml_name_decl("_STA", aml_int(0xf)));
+aml_append(kbd, aml_name_decl("_CRS", crs));
+
+crs = aml_resource_template();
+aml_append(crs, aml_irq_no_flags(12));
+
+mou = aml_device("MOU");
+aml_append(mou, aml_name_decl("_HID", aml_eisaid("PNP0F13")));
+aml_append(mou, aml_name_decl("_STA", aml_int(0xf)));
+aml_append(mou, aml_name_decl("_CRS", crs));
+
+aml_append(scope, kbd);
+aml_append(scope, mou);
+}
+
 static void i8042_class_initfn(ObjectClass *klass, void *data)
 {
 DeviceClass *dc = DEVICE_CLASS(klass);
+ISADeviceClass *isa = ISA_DEVICE_CLASS(klass);
 
 dc->realize = i8042_realizefn;
 dc->vmsd = _kbd_isa;
+isa->build_aml = i8042_build_aml;
 set_bit(DEVICE_CATEGORY_INPUT, dc->categories);
 }
 
-- 
2.18.4

[PATCH v6 09/16] floppy: make isa_fdc_get_drive_max_chs static

[Gerd Hoffmann]

acpi aml generator needs this, but it is in floppy code now
so we can make the function static.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Igor Mammedov 
Reviewed-by: Philippe Mathieu-Daudé 
Acked-by: John Snow 
---
 include/hw/block/fdc.h | 2 --
 hw/block/fdc.c | 4 ++--
 2 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/include/hw/block/fdc.h b/include/hw/block/fdc.h
index c15ff4c62315..5d71cf972268 100644
--- a/include/hw/block/fdc.h
+++ b/include/hw/block/fdc.h
@@ -16,7 +16,5 @@ void sun4m_fdctrl_init(qemu_irq irq, hwaddr io_base,
DriveInfo **fds, qemu_irq *fdc_tc);
 
 FloppyDriveType isa_fdc_get_drive_type(ISADevice *fdc, int i);
-void isa_fdc_get_drive_max_chs(FloppyDriveType type,
-   uint8_t *maxc, uint8_t *maxh, uint8_t *maxs);
 
 #endif
diff --git a/hw/block/fdc.c b/hw/block/fdc.c
index b4d2eaf66dcd..8024c822cea3 100644
--- a/hw/block/fdc.c
+++ b/hw/block/fdc.c
@@ -2744,8 +2744,8 @@ FloppyDriveType isa_fdc_get_drive_type(ISADevice *fdc, 
int i)
 return isa->state.drives[i].drive;
 }
 
-void isa_fdc_get_drive_max_chs(FloppyDriveType type,
-   uint8_t *maxc, uint8_t *maxh, uint8_t *maxs)
+static void isa_fdc_get_drive_max_chs(FloppyDriveType type, uint8_t *maxc,
+  uint8_t *maxh, uint8_t *maxs)
 {
 const FDFormat *fdf;
 
-- 
2.18.4

[PATCH v6 02/16] acpi: move aml builder code for rtc device

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Igor Mammedov 
Reviewed-by: Philippe Mathieu-Daudé 
---
 hw/i386/acpi-build.c | 17 -
 hw/rtc/mc146818rtc.c | 22 ++
 2 files changed, 22 insertions(+), 17 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 2e15f6848e7e..0bfa2dd23fcc 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1137,22 +1137,6 @@ static Aml *build_fdc_device_aml(ISADevice *fdc)
 return dev;
 }
 
-static Aml *build_rtc_device_aml(void)
-{
-Aml *dev;
-Aml *crs;
-
-dev = aml_device("RTC");
-aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0B00")));
-crs = aml_resource_template();
-aml_append(crs, aml_io(AML_DECODE16, 0x0070, 0x0070, 0x10, 0x02));
-aml_append(crs, aml_irq_no_flags(8));
-aml_append(crs, aml_io(AML_DECODE16, 0x0072, 0x0072, 0x02, 0x06));
-aml_append(dev, aml_name_decl("_CRS", crs));
-
-return dev;
-}
-
 static Aml *build_kbd_device_aml(void)
 {
 Aml *dev;
@@ -1278,7 +1262,6 @@ static void build_isa_devices_aml(Aml *table)
 Aml *scope = aml_scope("_SB.PCI0.ISA");
 Object *obj = object_resolve_path_type("", TYPE_ISA_BUS, );
 
-aml_append(scope, build_rtc_device_aml());
 aml_append(scope, build_kbd_device_aml());
 aml_append(scope, build_mouse_device_aml());
 if (fdc) {
diff --git a/hw/rtc/mc146818rtc.c b/hw/rtc/mc146818rtc.c
index 9c30cbdcd76f..fe05a4488e43 100644
--- a/hw/rtc/mc146818rtc.c
+++ b/hw/rtc/mc146818rtc.c
@@ -27,6 +27,7 @@
 #include "qemu/cutils.h"
 #include "qemu/module.h"
 #include "qemu/bcd.h"
+#include "hw/acpi/aml-build.h"
 #include "hw/irq.h"
 #include "hw/qdev-properties.h"
 #include "qemu/timer.h"
@@ -1007,13 +1008,34 @@ static void rtc_resetdev(DeviceState *d)
 }
 }
 
+static void rtc_build_aml(ISADevice *isadev, Aml *scope)
+{
+Aml *dev;
+Aml *crs;
+
+crs = aml_resource_template();
+aml_append(crs, aml_io(AML_DECODE16, RTC_ISA_BASE, RTC_ISA_BASE,
+   0x10, 0x02));
+aml_append(crs, aml_irq_no_flags(RTC_ISA_IRQ));
+aml_append(crs, aml_io(AML_DECODE16, RTC_ISA_BASE + 2, RTC_ISA_BASE + 2,
+   0x02, 0x06));
+
+dev = aml_device("RTC");
+aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0B00")));
+aml_append(dev, aml_name_decl("_CRS", crs));
+
+aml_append(scope, dev);
+}
+
 static void rtc_class_initfn(ObjectClass *klass, void *data)
 {
 DeviceClass *dc = DEVICE_CLASS(klass);
+ISADeviceClass *isa = ISA_DEVICE_CLASS(klass);
 
 dc->realize = rtc_realizefn;
 dc->reset = rtc_resetdev;
 dc->vmsd = _rtc;
+isa->build_aml = rtc_build_aml;
 device_class_set_props(dc, mc146818rtc_properties);
 }
 
-- 
2.18.4

[PATCH v6 14/16] acpi: drop serial/parallel enable bits from dsdt

[Gerd Hoffmann]

The _STA methods for COM+LPT used to reference them,
but that isn't the case any more.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Igor Mammedov 
---
 hw/i386/acpi-build.c | 23 ---
 1 file changed, 23 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 1922868f3401..765409a90eb6 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1437,15 +1437,6 @@ static void build_q35_isa_bridge(Aml *table)
 aml_append(field, aml_named_field("LPTD", 2));
 aml_append(dev, field);
 
-aml_append(dev, aml_operation_region("LPCE", AML_PCI_CONFIG,
- aml_int(0x82), 0x02));
-/* enable bits */
-field = aml_field("LPCE", AML_ANY_ACC, AML_NOLOCK, AML_PRESERVE);
-aml_append(field, aml_named_field("CAEN", 1));
-aml_append(field, aml_named_field("CBEN", 1));
-aml_append(field, aml_named_field("LPEN", 1));
-aml_append(dev, field);
-
 aml_append(scope, dev);
 aml_append(table, scope);
 }
@@ -1469,7 +1460,6 @@ static void build_piix4_isa_bridge(Aml *table)
 {
 Aml *dev;
 Aml *scope;
-Aml *field;
 
 scope =  aml_scope("_SB.PCI0");
 dev = aml_device("ISA");
@@ -1478,19 +1468,6 @@ static void build_piix4_isa_bridge(Aml *table)
 /* PIIX PCI to ISA irq remapping */
 aml_append(dev, aml_operation_region("P40C", AML_PCI_CONFIG,
  aml_int(0x60), 0x04));
-/* enable bits */
-field = aml_field("^PX13.P13C", AML_ANY_ACC, AML_NOLOCK, AML_PRESERVE);
-/* Offset(0x5f),, 7, */
-aml_append(field, aml_reserved_field(0x2f8));
-aml_append(field, aml_reserved_field(7));
-aml_append(field, aml_named_field("LPEN", 1));
-/* Offset(0x67),, 3, */
-aml_append(field, aml_reserved_field(0x38));
-aml_append(field, aml_reserved_field(3));
-aml_append(field, aml_named_field("CAEN", 1));
-aml_append(field, aml_reserved_field(3));
-aml_append(field, aml_named_field("CBEN", 1));
-aml_append(dev, field);
 
 aml_append(scope, dev);
 aml_append(table, scope);
-- 
2.18.4

[PATCH v6 12/16] acpi: factor out fw_cfg_add_acpi_dsdt()

[Gerd Hoffmann]

Add helper function to add fw_cfg device,
also move code to hw/i386/fw_cfg.c.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Igor Mammedov 
---
 hw/i386/fw_cfg.h |  1 +
 hw/i386/acpi-build.c | 24 +---
 hw/i386/fw_cfg.c | 28 
 3 files changed, 30 insertions(+), 23 deletions(-)

diff --git a/hw/i386/fw_cfg.h b/hw/i386/fw_cfg.h
index 9e742787792b..275f15c1c5e8 100644
--- a/hw/i386/fw_cfg.h
+++ b/hw/i386/fw_cfg.h
@@ -25,5 +25,6 @@ FWCfgState *fw_cfg_arch_create(MachineState *ms,
uint16_t apic_id_limit);
 void fw_cfg_build_smbios(MachineState *ms, FWCfgState *fw_cfg);
 void fw_cfg_build_feature_control(MachineState *ms, FWCfgState *fw_cfg);
+void fw_cfg_add_acpi_dsdt(Aml *scope, FWCfgState *fw_cfg);
 
 #endif
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 829e20664fff..cb3913d2ee76 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1880,30 +1880,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
 
 /* create fw_cfg node, unconditionally */
 {
-/* when using port i/o, the 8-bit data register *always* overlaps
- * with half of the 16-bit control register. Hence, the total size
- * of the i/o region used is FW_CFG_CTL_SIZE; when using DMA, the
- * DMA control register is located at FW_CFG_DMA_IO_BASE + 4 */
-uint8_t io_size = object_property_get_bool(OBJECT(x86ms->fw_cfg),
-   "dma_enabled", NULL) ?
-  ROUND_UP(FW_CFG_CTL_SIZE, 4) + sizeof(dma_addr_t) :
-  FW_CFG_CTL_SIZE;
-
 scope = aml_scope("\\_SB.PCI0");
-dev = aml_device("FWCF");
-
-aml_append(dev, aml_name_decl("_HID", aml_string("QEMU0002")));
-
-/* device present, functioning, decoding, not shown in UI */
-aml_append(dev, aml_name_decl("_STA", aml_int(0xB)));
-
-crs = aml_resource_template();
-aml_append(crs,
-aml_io(AML_DECODE16, FW_CFG_IO_BASE, FW_CFG_IO_BASE, 0x01, io_size)
-);
-aml_append(dev, aml_name_decl("_CRS", crs));
-
-aml_append(scope, dev);
+fw_cfg_add_acpi_dsdt(scope, x86ms->fw_cfg);
 aml_append(dsdt, scope);
 }
 
diff --git a/hw/i386/fw_cfg.c b/hw/i386/fw_cfg.c
index da60ada59462..c55abfb01abb 100644
--- a/hw/i386/fw_cfg.c
+++ b/hw/i386/fw_cfg.c
@@ -15,6 +15,7 @@
 #include "qemu/osdep.h"
 #include "sysemu/numa.h"
 #include "hw/acpi/acpi.h"
+#include "hw/acpi/aml-build.h"
 #include "hw/firmware/smbios.h"
 #include "hw/i386/fw_cfg.h"
 #include "hw/timer/hpet.h"
@@ -179,3 +180,30 @@ void fw_cfg_build_feature_control(MachineState *ms, 
FWCfgState *fw_cfg)
 *val = cpu_to_le64(feature_control_bits | FEATURE_CONTROL_LOCKED);
 fw_cfg_add_file(fw_cfg, "etc/msr_feature_control", val, sizeof(*val));
 }
+
+void fw_cfg_add_acpi_dsdt(Aml *scope, FWCfgState *fw_cfg)
+{
+/*
+ * when using port i/o, the 8-bit data register *always* overlaps
+ * with half of the 16-bit control register. Hence, the total size
+ * of the i/o region used is FW_CFG_CTL_SIZE; when using DMA, the
+ * DMA control register is located at FW_CFG_DMA_IO_BASE + 4
+ */
+Object *obj = OBJECT(fw_cfg);
+uint8_t io_size = object_property_get_bool(obj, "dma_enabled", NULL) ?
+ROUND_UP(FW_CFG_CTL_SIZE, 4) + sizeof(dma_addr_t) :
+FW_CFG_CTL_SIZE;
+Aml *dev = aml_device("FWCF");
+Aml *crs = aml_resource_template();
+
+aml_append(dev, aml_name_decl("_HID", aml_string("QEMU0002")));
+
+/* device present, functioning, decoding, not shown in UI */
+aml_append(dev, aml_name_decl("_STA", aml_int(0xB)));
+
+aml_append(crs,
+aml_io(AML_DECODE16, FW_CFG_IO_BASE, FW_CFG_IO_BASE, 0x01, io_size));
+
+aml_append(dev, aml_name_decl("_CRS", crs));
+aml_append(scope, dev);
+}
-- 
2.18.4

[PATCH v6 06/16] acpi: parallel: don't use _STA method

[Gerd Hoffmann]

The _STA method dates back to the days where we had a static DSDT.  The
device is listed in the DSDT table unconditionally and the _STA method
checks a bit in the isa bridge pci config space to figure whenever a
given is isa device is present or not, then evaluates to 0x0f (present)
or 0x00 (absent).

These days the DSDT is generated by qemu anyway, so if a device is not
present we can simply drop it from the DSDT instead.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Igor Mammedov 
---
 hw/i386/acpi-build.c | 29 -
 1 file changed, 8 insertions(+), 21 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 0e6a5151f4c3..2188a2b99d18 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1173,39 +1173,26 @@ static Aml *build_mouse_device_aml(void)
 return dev;
 }
 
-static Aml *build_lpt_device_aml(void)
+static void build_lpt_device_aml(Aml *scope)
 {
 Aml *dev;
 Aml *crs;
-Aml *method;
-Aml *if_ctx;
-Aml *else_ctx;
-Aml *zero = aml_int(0);
-Aml *is_present = aml_local(0);
+
+if (!memory_region_present(get_system_io(), 0x0378)) {
+return;
+}
 
 dev = aml_device("LPT");
 aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0400")));
 
-method = aml_method("_STA", 0, AML_NOTSERIALIZED);
-aml_append(method, aml_store(aml_name("LPEN"), is_present));
-if_ctx = aml_if(aml_equal(is_present, zero));
-{
-aml_append(if_ctx, aml_return(aml_int(0x00)));
-}
-aml_append(method, if_ctx);
-else_ctx = aml_else();
-{
-aml_append(else_ctx, aml_return(aml_int(0x0f)));
-}
-aml_append(method, else_ctx);
-aml_append(dev, method);
+aml_append(dev, aml_name_decl("_STA", aml_int(0xf)));
 
 crs = aml_resource_template();
 aml_append(crs, aml_io(AML_DECODE16, 0x0378, 0x0378, 0x08, 0x08));
 aml_append(crs, aml_irq_no_flags(7));
 aml_append(dev, aml_name_decl("_CRS", crs));
 
-return dev;
+aml_append(scope, dev);
 }
 
 static void build_isa_devices_aml(Aml *table)
@@ -1221,7 +1208,7 @@ static void build_isa_devices_aml(Aml *table)
 if (fdc) {
 aml_append(scope, build_fdc_device_aml(fdc));
 }
-aml_append(scope, build_lpt_device_aml());
+build_lpt_device_aml(scope);
 
 if (ambiguous) {
 error_report("Multiple ISA busses, unable to define IPMI ACPI data");
-- 
2.18.4

[PATCH v6 15/16] acpi: drop build_piix4_pm()

[Gerd Hoffmann]

The _SB.PCI0.PX13.P13C opregion (holds isa device enable bits)
is not used any more, remove it from DSDT.

Signed-off-by: Gerd Hoffmann 
---
 hw/i386/acpi-build.c | 16 
 1 file changed, 16 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 765409a90eb6..c1e63cce5e8e 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1441,21 +1441,6 @@ static void build_q35_isa_bridge(Aml *table)
 aml_append(table, scope);
 }
 
-static void build_piix4_pm(Aml *table)
-{
-Aml *dev;
-Aml *scope;
-
-scope =  aml_scope("_SB.PCI0");
-dev = aml_device("PX13");
-aml_append(dev, aml_name_decl("_ADR", aml_int(0x00010003)));
-
-aml_append(dev, aml_operation_region("P13C", AML_PCI_CONFIG,
- aml_int(0x00), 0xff));
-aml_append(scope, dev);
-aml_append(table, scope);
-}
-
 static void build_piix4_isa_bridge(Aml *table)
 {
 Aml *dev;
@@ -1607,7 +1592,6 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
 aml_append(dsdt, sb_scope);
 
 build_hpet_aml(dsdt);
-build_piix4_pm(dsdt);
 build_piix4_isa_bridge(dsdt);
 build_isa_devices_aml(dsdt);
 build_piix4_pci_hotplug(dsdt);
-- 
2.18.4

[PATCH v6 07/16] acpi: move aml builder code for parallel device

[Gerd Hoffmann]

Also adds support for multiple LPT devices.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Igor Mammedov 
Reviewed-by: Philippe Mathieu-Daudé 
---
 hw/char/parallel.c   | 22 ++
 hw/i386/acpi-build.c | 23 ---
 2 files changed, 22 insertions(+), 23 deletions(-)

diff --git a/hw/char/parallel.c b/hw/char/parallel.c
index 8dd67d13759b..c0f34bf924ec 100644
--- a/hw/char/parallel.c
+++ b/hw/char/parallel.c
@@ -28,6 +28,7 @@
 #include "qemu/module.h"
 #include "chardev/char-parallel.h"
 #include "chardev/char-fe.h"
+#include "hw/acpi/aml-build.h"
 #include "hw/irq.h"
 #include "hw/isa/isa.h"
 #include "hw/qdev-properties.h"
@@ -568,6 +569,25 @@ static void parallel_isa_realizefn(DeviceState *dev, Error 
**errp)
  s, "parallel");
 }
 
+static void parallel_isa_build_aml(ISADevice *isadev, Aml *scope)
+{
+ISAParallelState *isa = ISA_PARALLEL(isadev);
+Aml *dev;
+Aml *crs;
+
+crs = aml_resource_template();
+aml_append(crs, aml_io(AML_DECODE16, isa->iobase, isa->iobase, 0x08, 
0x08));
+aml_append(crs, aml_irq_no_flags(isa->isairq));
+
+dev = aml_device("LPT%d", isa->index + 1);
+aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0400")));
+aml_append(dev, aml_name_decl("_UID", aml_int(isa->index + 1)));
+aml_append(dev, aml_name_decl("_STA", aml_int(0xf)));
+aml_append(dev, aml_name_decl("_CRS", crs));
+
+aml_append(scope, dev);
+}
+
 /* Memory mapped interface */
 static uint64_t parallel_mm_readfn(void *opaque, hwaddr addr, unsigned size)
 {
@@ -624,9 +644,11 @@ static Property parallel_isa_properties[] = {
 static void parallel_isa_class_initfn(ObjectClass *klass, void *data)
 {
 DeviceClass *dc = DEVICE_CLASS(klass);
+ISADeviceClass *isa = ISA_DEVICE_CLASS(klass);
 
 dc->realize = parallel_isa_realizefn;
 dc->vmsd = _parallel_isa;
+isa->build_aml = parallel_isa_build_aml;
 device_class_set_props(dc, parallel_isa_properties);
 set_bit(DEVICE_CATEGORY_INPUT, dc->categories);
 }
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 2188a2b99d18..443db94deb5b 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1173,28 +1173,6 @@ static Aml *build_mouse_device_aml(void)
 return dev;
 }
 
-static void build_lpt_device_aml(Aml *scope)
-{
-Aml *dev;
-Aml *crs;
-
-if (!memory_region_present(get_system_io(), 0x0378)) {
-return;
-}
-
-dev = aml_device("LPT");
-aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0400")));
-
-aml_append(dev, aml_name_decl("_STA", aml_int(0xf)));
-
-crs = aml_resource_template();
-aml_append(crs, aml_io(AML_DECODE16, 0x0378, 0x0378, 0x08, 0x08));
-aml_append(crs, aml_irq_no_flags(7));
-aml_append(dev, aml_name_decl("_CRS", crs));
-
-aml_append(scope, dev);
-}
-
 static void build_isa_devices_aml(Aml *table)
 {
 ISADevice *fdc = pc_find_fdc0();
@@ -1208,7 +1186,6 @@ static void build_isa_devices_aml(Aml *table)
 if (fdc) {
 aml_append(scope, build_fdc_device_aml(fdc));
 }
-build_lpt_device_aml(scope);
 
 if (ambiguous) {
 error_report("Multiple ISA busses, unable to define IPMI ACPI data");
-- 
2.18.4

[PATCH v6 13/16] acpi: simplify build_isa_devices_aml()

[Gerd Hoffmann]

x86 machines can have a single ISA bus only.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Igor Mammedov 
Reviewed-by: Philippe Mathieu-Daudé 
---
 hw/i386/acpi-build.c | 15 +--
 1 file changed, 5 insertions(+), 10 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index cb3913d2ee76..1922868f3401 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1061,19 +1061,14 @@ static void build_hpet_aml(Aml *table)
 static void build_isa_devices_aml(Aml *table)
 {
 bool ambiguous;
-
-Aml *scope = aml_scope("_SB.PCI0.ISA");
 Object *obj = object_resolve_path_type("", TYPE_ISA_BUS, );
+Aml *scope;
 
-if (ambiguous) {
-error_report("Multiple ISA busses, unable to define IPMI ACPI data");
-} else if (!obj) {
-error_report("No ISA bus, unable to define IPMI ACPI data");
-} else {
-build_acpi_ipmi_devices(scope, BUS(obj), "\\_SB.PCI0.ISA");
-isa_build_aml(ISA_BUS(obj), scope);
-}
+assert(obj && !ambiguous);
 
+scope = aml_scope("_SB.PCI0.ISA");
+build_acpi_ipmi_devices(scope, BUS(obj), "\\_SB.PCI0.ISA");
+isa_build_aml(ISA_BUS(obj), scope);
 aml_append(table, scope);
 }
 
-- 
2.18.4

[PATCH v6 00/16] acpi: i386 tweaks

[Gerd Hoffmann]

First batch of microvm patches, some generic acpi stuff.
Split the acpi-build.c monster, specifically split the
pc and q35 and pci bits into a separate file which we
can skip building at some point in the future.

v2 changes: leave acpi-build.c largely as-is, move useful
bits to other places to allow them being reused, specifically:

 * move isa device generator functions to individual isa devices.
 * move fw_cfg generator function to fw_cfg.c

v3 changes: fix rtc, support multiple lpt devices.

v4 changes:
 * drop merged patches.
 * split rtc crs change to separata patch.
 * added two cleanup patches.
 * picked up ack & review tags.

v5 changes:
 * add comment for rtc crs update.
 * add even more cleanup patches.
 * picked up ack & review tags.

v6 changes:
 * floppy: move cmos_get_fd_drive_type.
 * picked up ack & review tags.

take care,
  Gerd

Gerd Hoffmann (16):
  qtest: allow DSDT acpi table changes
  acpi: move aml builder code for rtc device
  acpi: rtc: use a single crs range
  acpi: serial: don't use _STA method
  acpi: move aml builder code for serial device
  acpi: parallel: don't use _STA method
  acpi: move aml builder code for parallel device
  acpi: move aml builder code for floppy device
  floppy: make isa_fdc_get_drive_max_chs static
  floppy: move cmos_get_fd_drive_type() from pc
  acpi: move aml builder code for i8042 (kbd+mouse) device
  acpi: factor out fw_cfg_add_acpi_dsdt()
  acpi: simplify build_isa_devices_aml()
  acpi: drop serial/parallel enable bits from dsdt
  acpi: drop build_piix4_pm()
  acpi: q35: drop _SB.PCI0.ISA.LPCD opregion.

 hw/i386/fw_cfg.h|   1 +
 include/hw/block/fdc.h  |   3 +-
 include/hw/i386/pc.h|   1 -
 tests/qtest/bios-tables-test-allowed-diff.h |  17 ++
 hw/block/fdc.c  | 111 ++-
 hw/char/parallel.c  |  22 ++
 hw/char/serial-isa.c|  22 ++
 hw/i386/acpi-build.c| 312 +---
 hw/i386/fw_cfg.c|  28 ++
 hw/i386/pc.c|  25 --
 hw/input/pckbd.c|  31 ++
 hw/rtc/mc146818rtc.c|  24 ++
 stubs/cmos.c|   7 +
 stubs/Makefile.objs |   1 +
 14 files changed, 269 insertions(+), 336 deletions(-)
 create mode 100644 stubs/cmos.c

-- 
2.18.4

[PATCH v6 10/16] floppy: move cmos_get_fd_drive_type() from pc

[Gerd Hoffmann]

---
 include/hw/block/fdc.h |  1 +
 include/hw/i386/pc.h   |  1 -
 hw/block/fdc.c | 26 +-
 hw/i386/pc.c   | 25 -
 4 files changed, 26 insertions(+), 27 deletions(-)

diff --git a/include/hw/block/fdc.h b/include/hw/block/fdc.h
index 5d71cf972268..479cebc0a330 100644
--- a/include/hw/block/fdc.h
+++ b/include/hw/block/fdc.h
@@ -16,5 +16,6 @@ void sun4m_fdctrl_init(qemu_irq irq, hwaddr io_base,
DriveInfo **fds, qemu_irq *fdc_tc);
 
 FloppyDriveType isa_fdc_get_drive_type(ISADevice *fdc, int i);
+int cmos_get_fd_drive_type(FloppyDriveType fd0);
 
 #endif
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index 8d764f965cd3..5e3b19ab78fc 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -176,7 +176,6 @@ typedef void (*cpu_set_smm_t)(int smm, void *arg);
 void pc_i8259_create(ISABus *isa_bus, qemu_irq *i8259_irqs);
 
 ISADevice *pc_find_fdc0(void);
-int cmos_get_fd_drive_type(FloppyDriveType fd0);
 
 /* port92.c */
 #define PORT92_A20_LINE "a20"
diff --git a/hw/block/fdc.c b/hw/block/fdc.c
index 8024c822cea3..ea0fb8ee15b9 100644
--- a/hw/block/fdc.c
+++ b/hw/block/fdc.c
@@ -32,7 +32,6 @@
 #include "qapi/error.h"
 #include "qemu/error-report.h"
 #include "qemu/timer.h"
-#include "hw/i386/pc.h"
 #include "hw/acpi/aml-build.h"
 #include "hw/irq.h"
 #include "hw/isa/isa.h"
@@ -2809,6 +2808,31 @@ static Aml *build_fdinfo_aml(int idx, FloppyDriveType 
type)
 return dev;
 }
 
+int cmos_get_fd_drive_type(FloppyDriveType fd0)
+{
+int val;
+
+switch (fd0) {
+case FLOPPY_DRIVE_TYPE_144:
+/* 1.44 Mb 3"5 drive */
+val = 4;
+break;
+case FLOPPY_DRIVE_TYPE_288:
+/* 2.88 Mb 3"5 drive */
+val = 5;
+break;
+case FLOPPY_DRIVE_TYPE_120:
+/* 1.2 Mb 5"5 drive */
+val = 2;
+break;
+case FLOPPY_DRIVE_TYPE_NONE:
+default:
+val = 0;
+break;
+}
+return val;
+}
+
 static void fdc_isa_build_aml(ISADevice *isadev, Aml *scope)
 {
 Aml *dev;
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 2128f3d6fe8b..c5db7be6d8b1 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -385,31 +385,6 @@ static uint64_t ioportF0_read(void *opaque, hwaddr addr, 
unsigned size)
 
 #define REG_EQUIPMENT_BYTE  0x14
 
-int cmos_get_fd_drive_type(FloppyDriveType fd0)
-{
-int val;
-
-switch (fd0) {
-case FLOPPY_DRIVE_TYPE_144:
-/* 1.44 Mb 3"5 drive */
-val = 4;
-break;
-case FLOPPY_DRIVE_TYPE_288:
-/* 2.88 Mb 3"5 drive */
-val = 5;
-break;
-case FLOPPY_DRIVE_TYPE_120:
-/* 1.2 Mb 5"5 drive */
-val = 2;
-break;
-case FLOPPY_DRIVE_TYPE_NONE:
-default:
-val = 0;
-break;
-}
-return val;
-}
-
 static void cmos_init_hd(ISADevice *s, int type_ofs, int info_ofs,
  int16_t cylinders, int8_t heads, int8_t sectors)
 {
-- 
2.18.4

[PATCH v6 05/16] acpi: move aml builder code for serial device

[Gerd Hoffmann]

The code uses the isa_serial_io array to figure what the device uid is.
Side effect is that acpi antries are not limited to port 1+2 any more,
we'll also get entries for ports 3+4.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Igor Mammedov 
Reviewed-by: Philippe Mathieu-Daudé 
---
 hw/char/serial-isa.c | 22 ++
 hw/i386/acpi-build.c | 32 
 2 files changed, 22 insertions(+), 32 deletions(-)

diff --git a/hw/char/serial-isa.c b/hw/char/serial-isa.c
index f9b6eed7833d..165e320e65f0 100644
--- a/hw/char/serial-isa.c
+++ b/hw/char/serial-isa.c
@@ -27,6 +27,7 @@
 #include "qapi/error.h"
 #include "qemu/module.h"
 #include "sysemu/sysemu.h"
+#include "hw/acpi/aml-build.h"
 #include "hw/char/serial.h"
 #include "hw/isa/isa.h"
 #include "hw/qdev-properties.h"
@@ -81,6 +82,25 @@ static void serial_isa_realizefn(DeviceState *dev, Error 
**errp)
 isa_register_ioport(isadev, >io, isa->iobase);
 }
 
+static void serial_isa_build_aml(ISADevice *isadev, Aml *scope)
+{
+ISASerialState *isa = ISA_SERIAL(isadev);
+Aml *dev;
+Aml *crs;
+
+crs = aml_resource_template();
+aml_append(crs, aml_io(AML_DECODE16, isa->iobase, isa->iobase, 0x00, 
0x08));
+aml_append(crs, aml_irq_no_flags(isa->isairq));
+
+dev = aml_device("COM%d", isa->index + 1);
+aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0501")));
+aml_append(dev, aml_name_decl("_UID", aml_int(isa->index + 1)));
+aml_append(dev, aml_name_decl("_STA", aml_int(0xf)));
+aml_append(dev, aml_name_decl("_CRS", crs));
+
+aml_append(scope, dev);
+}
+
 static const VMStateDescription vmstate_isa_serial = {
 .name = "serial",
 .version_id = 3,
@@ -103,9 +123,11 @@ static Property serial_isa_properties[] = {
 static void serial_isa_class_initfn(ObjectClass *klass, void *data)
 {
 DeviceClass *dc = DEVICE_CLASS(klass);
+ISADeviceClass *isa = ISA_DEVICE_CLASS(klass);
 
 dc->realize = serial_isa_realizefn;
 dc->vmsd = _isa_serial;
+isa->build_aml = serial_isa_build_aml;
 device_class_set_props(dc, serial_isa_properties);
 set_bit(DEVICE_CATEGORY_INPUT, dc->categories);
 }
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 3a82730a0d19..0e6a5151f4c3 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1208,36 +1208,6 @@ static Aml *build_lpt_device_aml(void)
 return dev;
 }
 
-static void build_com_device_aml(Aml *scope, uint8_t uid)
-{
-Aml *dev;
-Aml *crs;
-uint8_t irq = 4;
-uint16_t io_port = 0x03F8;
-
-assert(uid == 1 || uid == 2);
-if (uid == 2) {
-irq = 3;
-io_port = 0x02F8;
-}
-if (!memory_region_present(get_system_io(), io_port)) {
-return;
-}
-
-dev = aml_device("COM%d", uid);
-aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0501")));
-aml_append(dev, aml_name_decl("_UID", aml_int(uid)));
-
-aml_append(dev, aml_name_decl("_STA", aml_int(0xf)));
-
-crs = aml_resource_template();
-aml_append(crs, aml_io(AML_DECODE16, io_port, io_port, 0x00, 0x08));
-aml_append(crs, aml_irq_no_flags(irq));
-aml_append(dev, aml_name_decl("_CRS", crs));
-
-aml_append(scope, dev);
-}
-
 static void build_isa_devices_aml(Aml *table)
 {
 ISADevice *fdc = pc_find_fdc0();
@@ -1252,8 +1222,6 @@ static void build_isa_devices_aml(Aml *table)
 aml_append(scope, build_fdc_device_aml(fdc));
 }
 aml_append(scope, build_lpt_device_aml());
-build_com_device_aml(scope, 1);
-build_com_device_aml(scope, 2);
 
 if (ambiguous) {
 error_report("Multiple ISA busses, unable to define IPMI ACPI data");
-- 
2.18.4

[PATCH v6 03/16] acpi: rtc: use a single crs range

[Gerd Hoffmann]

Use a single io range for _CRS instead of two,
following what real hardware does.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Igor Mammedov 
---
 hw/rtc/mc146818rtc.c | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/hw/rtc/mc146818rtc.c b/hw/rtc/mc146818rtc.c
index fe05a4488e43..1e9fa0f33fed 100644
--- a/hw/rtc/mc146818rtc.c
+++ b/hw/rtc/mc146818rtc.c
@@ -1013,12 +1013,14 @@ static void rtc_build_aml(ISADevice *isadev, Aml *scope)
 Aml *dev;
 Aml *crs;
 
+/*
+ * Reserving 8 io ports here, following what physical hardware
+ * does, even though qemu only responds to the first two ports.
+ */
 crs = aml_resource_template();
 aml_append(crs, aml_io(AML_DECODE16, RTC_ISA_BASE, RTC_ISA_BASE,
-   0x10, 0x02));
+   0x01, 0x08));
 aml_append(crs, aml_irq_no_flags(RTC_ISA_IRQ));
-aml_append(crs, aml_io(AML_DECODE16, RTC_ISA_BASE + 2, RTC_ISA_BASE + 2,
-   0x02, 0x06));
 
 dev = aml_device("RTC");
 aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0B00")));
-- 
2.18.4

[PATCH v6 04/16] acpi: serial: don't use _STA method

[Gerd Hoffmann]

The _STA method dates back to the days where we had a static DSDT.  The
device is listed in the DSDT table unconditionally and the _STA method
checks a bit in the isa bridge pci config space to figure whenever a
given is isa device is present or not, then evaluates to 0x0f (present)
or 0x00 (absent).

These days the DSDT is generated by qemu anyway, so if a device is not
present we can simply drop it from the DSDT instead.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Igor Mammedov 
Reviewed-by: Philippe Mathieu-Daudé 
---
 hw/i386/acpi-build.c | 32 
 1 file changed, 8 insertions(+), 24 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 0bfa2dd23fcc..3a82730a0d19 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1208,50 +1208,34 @@ static Aml *build_lpt_device_aml(void)
 return dev;
 }
 
-static Aml *build_com_device_aml(uint8_t uid)
+static void build_com_device_aml(Aml *scope, uint8_t uid)
 {
 Aml *dev;
 Aml *crs;
-Aml *method;
-Aml *if_ctx;
-Aml *else_ctx;
-Aml *zero = aml_int(0);
-Aml *is_present = aml_local(0);
-const char *enabled_field = "CAEN";
 uint8_t irq = 4;
 uint16_t io_port = 0x03F8;
 
 assert(uid == 1 || uid == 2);
 if (uid == 2) {
-enabled_field = "CBEN";
 irq = 3;
 io_port = 0x02F8;
 }
+if (!memory_region_present(get_system_io(), io_port)) {
+return;
+}
 
 dev = aml_device("COM%d", uid);
 aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0501")));
 aml_append(dev, aml_name_decl("_UID", aml_int(uid)));
 
-method = aml_method("_STA", 0, AML_NOTSERIALIZED);
-aml_append(method, aml_store(aml_name("%s", enabled_field), is_present));
-if_ctx = aml_if(aml_equal(is_present, zero));
-{
-aml_append(if_ctx, aml_return(aml_int(0x00)));
-}
-aml_append(method, if_ctx);
-else_ctx = aml_else();
-{
-aml_append(else_ctx, aml_return(aml_int(0x0f)));
-}
-aml_append(method, else_ctx);
-aml_append(dev, method);
+aml_append(dev, aml_name_decl("_STA", aml_int(0xf)));
 
 crs = aml_resource_template();
 aml_append(crs, aml_io(AML_DECODE16, io_port, io_port, 0x00, 0x08));
 aml_append(crs, aml_irq_no_flags(irq));
 aml_append(dev, aml_name_decl("_CRS", crs));
 
-return dev;
+aml_append(scope, dev);
 }
 
 static void build_isa_devices_aml(Aml *table)
@@ -1268,8 +1252,8 @@ static void build_isa_devices_aml(Aml *table)
 aml_append(scope, build_fdc_device_aml(fdc));
 }
 aml_append(scope, build_lpt_device_aml());
-aml_append(scope, build_com_device_aml(1));
-aml_append(scope, build_com_device_aml(2));
+build_com_device_aml(scope, 1);
+build_com_device_aml(scope, 2);
 
 if (ambiguous) {
 error_report("Multiple ISA busses, unable to define IPMI ACPI data");
-- 
2.18.4

[PATCH v6 01/16] qtest: allow DSDT acpi table changes

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
---
 tests/qtest/bios-tables-test-allowed-diff.h | 17 +
 1 file changed, 17 insertions(+)

diff --git a/tests/qtest/bios-tables-test-allowed-diff.h 
b/tests/qtest/bios-tables-test-allowed-diff.h
index dfb8523c8bf4..6a052c50447a 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1 +1,18 @@
 /* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/pc/DSDT",
+"tests/data/acpi/pc/DSDT.acpihmat",
+"tests/data/acpi/pc/DSDT.bridge",
+"tests/data/acpi/pc/DSDT.cphp",
+"tests/data/acpi/pc/DSDT.dimmpxm",
+"tests/data/acpi/pc/DSDT.ipmikcs",
+"tests/data/acpi/pc/DSDT.memhp",
+"tests/data/acpi/pc/DSDT.numamem",
+"tests/data/acpi/q35/DSDT",
+"tests/data/acpi/q35/DSDT.acpihmat",
+"tests/data/acpi/q35/DSDT.bridge",
+"tests/data/acpi/q35/DSDT.cphp",
+"tests/data/acpi/q35/DSDT.dimmpxm",
+"tests/data/acpi/q35/DSDT.ipmibt",
+"tests/data/acpi/q35/DSDT.memhp",
+"tests/data/acpi/q35/DSDT.mmio64",
+"tests/data/acpi/q35/DSDT.numamem",
-- 
2.18.4

Re: [PATCH v5 08/15] acpi: move aml builder code for floppy device

[Gerd Hoffmann]

> > +#include "hw/i386/pc.h"
> 
> I'd rather not see this target-specific header used in the generic device
> model... The culprit seems cmos_get_fd_drive_type(). Is the value really PC
> specific?

Given that the same value is used in acpi tables probably not really.
cmos_get_fd_drive_type() hasn't any dependency on pc or cmos btw, we
can simply move over the function to floppy.

> 
> Hervé, do you know if such info & RTC nvram layout is used in other arch?
> 
> > +#include "hw/acpi/aml-build.h"
> >   #include "hw/irq.h"
> >   #include "hw/isa/isa.h"
> >   #include "hw/qdev-properties.h"
> > @@ -2765,6 +2767,85 @@ void isa_fdc_get_drive_max_chs(FloppyDriveType type,
> >   (*maxc)--;
> >   }
> > +static Aml *build_fdinfo_aml(int idx, FloppyDriveType type)
> > +{
> > +Aml *dev, *fdi;
> > +uint8_t maxc, maxh, maxs;
> > +
> > +isa_fdc_get_drive_max_chs(type, , , );
> > +
> > +dev = aml_device("FLP%c", 'A' + idx);
> > +
> > +aml_append(dev, aml_name_decl("_ADR", aml_int(idx)));
> > +
> > +fdi = aml_package(16);
> > +aml_append(fdi, aml_int(idx));  /* Drive Number */
> > +aml_append(fdi,
> > +aml_int(cmos_get_fd_drive_type(type)));  /* Device Type */
> > +/*
> > + * the values below are the limits of the drive, and are thus 
> > independent
> > + * of the inserted media
> > + */
> > +aml_append(fdi, aml_int(maxc));  /* Maximum Cylinder Number */
> > +aml_append(fdi, aml_int(maxs));  /* Maximum Sector Number */
> > +aml_append(fdi, aml_int(maxh));  /* Maximum Head Number */
> > +/*
> > + * SeaBIOS returns the below values for int 0x13 func 0x08 regardless 
> > of
> > + * the drive type, so shall we
> > + */
> > +aml_append(fdi, aml_int(0xAF));  /* disk_specify_1 */
> > +aml_append(fdi, aml_int(0x02));  /* disk_specify_2 */
> > +aml_append(fdi, aml_int(0x25));  /* disk_motor_wait */
> > +aml_append(fdi, aml_int(0x02));  /* disk_sector_siz */
> > +aml_append(fdi, aml_int(0x12));  /* disk_eot */
> > +aml_append(fdi, aml_int(0x1B));  /* disk_rw_gap */
> > +aml_append(fdi, aml_int(0xFF));  /* disk_dtl */
> > +aml_append(fdi, aml_int(0x6C));  /* disk_formt_gap */
> > +aml_append(fdi, aml_int(0xF6));  /* disk_fill */
> > +aml_append(fdi, aml_int(0x0F));  /* disk_head_sttl */
> > +aml_append(fdi, aml_int(0x08));  /* disk_motor_strt */
> > +
> > +aml_append(dev, aml_name_decl("_FDI", fdi));
> > +return dev;
> > +}
> > +
> > +static void fdc_isa_build_aml(ISADevice *isadev, Aml *scope)
> > +{
> > +Aml *dev;
> > +Aml *crs;
> > +int i;
> > +
> > +#define ACPI_FDE_MAX_FD 4
> > +uint32_t fde_buf[5] = {
> > +0, 0, 0, 0, /* presence of floppy drives #0 - #3 */
> > +cpu_to_le32(2)  /* tape presence (2 == never present) */
> > +};
> > +
> > +crs = aml_resource_template();
> > +aml_append(crs, aml_io(AML_DECODE16, 0x03F2, 0x03F2, 0x00, 0x04));
> > +aml_append(crs, aml_io(AML_DECODE16, 0x03F7, 0x03F7, 0x00, 0x01));
> > +aml_append(crs, aml_irq_no_flags(6));
> > +aml_append(crs,
> > +aml_dma(AML_COMPATIBILITY, AML_NOTBUSMASTER, AML_TRANSFER8, 2));
> > +
> > +dev = aml_device("FDC0");
> > +aml_append(dev, aml_name_decl("_HID", aml_eisaid("PNP0700")));
> > +aml_append(dev, aml_name_decl("_CRS", crs));
> > +
> > +for (i = 0; i < MIN(MAX_FD, ACPI_FDE_MAX_FD); i++) {
> > +FloppyDriveType type = isa_fdc_get_drive_type(isadev, i);
> > +
> > +if (type < FLOPPY_DRIVE_TYPE_NONE) {
> > +fde_buf[i] = cpu_to_le32(1);  /* drive present */
> > +aml_append(dev, build_fdinfo_aml(i, type));
> > +}
> > +}
> > +aml_append(dev, aml_name_decl("_FDE",
> > +   aml_buffer(sizeof(fde_buf), (uint8_t *)fde_buf)));
> > +
> > +aml_append(scope, dev);
> > +}
> > +
> >   static const VMStateDescription vmstate_isa_fdc ={
> >   .name = "fdc",
> >   .version_id = 2,
> > @@ -2798,11 +2879,13 @@ static Property isa_fdc_properties[] = {
> >   static void isabus_fdc_class_init(ObjectClass *klass, void *data)
> >   {
> >   DeviceClass *dc = DEVICE_CLASS(klass);
> > +ISADeviceClass *isa = ISA_DEVICE_CLASS(klass);
> >   dc->realize = isabus_fdc_realize;
> >   dc->fw_name = "fdc";
> >   dc->reset = fdctrl_external_reset_isa;
> >   dc->vmsd = _isa_fdc;
> > +isa->build_aml = fdc_isa_build_aml;
> >   device_class_set_props(dc, isa_fdc_properties);
> >   set_bit(DEVICE_CATEGORY_STORAGE, dc->categories);
> >   }
> > diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> > index 443db94deb5b..775936e28b9a 100644
> > --- a/hw/i386/acpi-build.c
> > +++ b/hw/i386/acpi-build.c
> > @@ -1058,85 +1058,6 @@ static void build_hpet_aml(Aml *table)
> >   aml_append(table, scope);
> >   }
> > -static Aml *build_fdinfo_aml(int idx, FloppyDriveType type)
> > -{
> > -Aml *dev, *fdi;
> > -uint8_t maxc, maxh, maxs;
> > -
> > -

Re: [PULL 00/51] Block layer patches

[Peter Maydell]

On Fri, 15 May 2020 at 13:45, Kevin Wolf  wrote:
>
> The following changes since commit 013a18edbbc59cdad019100c7d03c0494642b74c:
>
>   Merge remote-tracking branch 
> 'remotes/pmaydell/tags/pull-target-arm-20200514' into staging (2020-05-14 
> 16:17:55 +0100)
>
> are available in the Git repository at:
>
>   git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 7d8f21a650e562270f5ed5341134c9c2c39dc5e8:
>
>   iotests/030: Reduce run time by unthrottling job earlier (2020-05-15 
> 14:12:34 +0200)
>
> 
> Block layer patches:
>
> - Introduce real BdrvChildRole
> - blk/bdrv_make_empty() functions instead of calling callbacks directly
> - mirror: Make sure that source and target size match
> - block-copy: Fix uninitialized variable
> - block/replication: Avoid cancelling the job twice
> - ahci: Log lost IRQs
> - iotests: Run pylint and mypy in a testcase
> - iotests: log messages from notrun()

Hi; I'm afraid aarch64, ppc64 and aarch32 all failed on iotest 229:


PPC64:
  TESTiotest-qcow2: 229 [fail]
QEMU  --
"/home/pm215/qemu/build/all/tests/qemu-iotests/../../ppc64-softmmu/qemu-system-ppc64"
-nodefaults -display none -accel qtest
QEMU_IMG  -- "/home/pm215/qemu/build/all/tests/qemu-iotests/../../qemu-img"
QEMU_IO   --
"/home/pm215/qemu/build/all/tests/qemu-iotests/../../qemu-io"  --cache
writeback --aio threads -f qcow2
QEMU_NBD  -- "/home/pm215/qemu/build/all/tests/qemu-iotests/../../qemu-nbd"
IMGFMT-- qcow2 (compat=1.1)
IMGPROTO  -- file
PLATFORM  -- Linux/ppc64 gcc1-power7 3.10.0-862.14.4.el7.ppc64
TEST_DIR  -- /home/pm215/qemu/build/all/tests/qemu-iotests/scratch
SOCK_DIR  -- /tmp/tmp.6nPwQm0Y01
SOCKET_SCM_HELPER --
/home/pm215/qemu/build/all/tests/qemu-iotests/socket_scm_helper

--- /home/pm215/qemu/tests/qemu-iotests/229.out 2020-05-15
12:46:57.991595216 +
+++ /home/pm215/qemu/build/all/tests/qemu-iotests/229.out.bad
2020-05-15 12:55:31.022877791 +
@@ -21,5 +21,5 @@
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP},
"event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id":
"testdisk"}}
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP},
"event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id":
"testdisk"}}
-{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP},
"event": "BLOCK_JOB_CANCELLED", "data": {"device": "testdisk", "len":
2097152, "offset": 2097152, "speed": 0, "type": "mirror"}}
+{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP},
"event": "BLOCK_JOB_CANCELLED", "data": {"device": "testdisk", "len":
2097152, "offset": 1048576, "speed": 0, "type": "mirror"}}
 *** done

AArch64:
  TESTiotest-qcow2: 229 [fail]
QEMU  --
"/home/pm/qemu/build/all/tests/qemu-iotests/../../aarch64-softmmu/qemu-system-aarch64"
-nodefaults -dis
play none -machine virt -accel qtest
QEMU_IMG  -- "/home/pm/qemu/build/all/tests/qemu-iotests/../../qemu-img"
QEMU_IO   --
"/home/pm/qemu/build/all/tests/qemu-iotests/../../qemu-io"  --cache
writeback --aio threads -f qcow2
QEMU_NBD  -- "/home/pm/qemu/build/all/tests/qemu-iotests/../../qemu-nbd"
IMGFMT-- qcow2 (compat=1.1)
IMGPROTO  -- file
PLATFORM  -- Linux/aarch64 qemu-test 5.4.19
TEST_DIR  -- /home/pm/qemu/build/all/tests/qemu-iotests/scratch
SOCK_DIR  -- /tmp/tmp.xiNXu5VDz9
SOCKET_SCM_HELPER --
/home/pm/qemu/build/all/tests/qemu-iotests/socket_scm_helper

--- /home/pm/qemu/tests/qemu-iotests/229.out2020-05-15
12:46:56.086403523 +
+++ /home/pm/qemu/build/all/tests/qemu-iotests/229.out.bad
2020-05-15 12:58:25.074083672 +
@@ -21,5 +21,5 @@
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP},
"event": "JOB_STATUS_CHANGE", "data": {"status": "ru
nning", "id": "testdisk"}}
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP},
"event": "JOB_STATUS_CHANGE", "data": {"status": "ab
orting", "id": "testdisk"}}
-{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP},
"event": "BLOCK_JOB_CANCELLED", "data": {"device": "
testdisk", "len": 2097152, "offset": 2097152, "speed": 0, "type": "mirror"}}
+{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP},
"event": "BLOCK_JOB_CANCELLED", "data": {"device": "
testdisk", "len": 2097152, "offset": 1048576, "speed": 0, "type": "mirror"}}
 *** done

AArch32:

  TESTiotest-qcow2: 229 [fail]
QEMU  --
"/home/peter.maydell/qemu/build/all-a32/tests/qemu-iotests/../../aarch64-softmmu/qemu-system-aarch64"
-
nodefaults -display none -machine virt -accel qtest
QEMU_IMG  --
"/home/peter.maydell/qemu/build/all-a32/tests/qemu-iotests/../../qemu-img"
QEMU_IO   --
"/home/peter.maydell/qemu/build/all-a32/tests/qemu-iotests/../../qemu-io"
 --cache writeback --aio thre
ads -f qcow2
QEMU_NBD  --
"/home/peter.maydell/qemu/build/all-a32/tests/qemu-iotests/../../qemu-nbd"
IMGFMT

Re: [PATCH 3/5] block/nbd.c: Add yank feature

[Daniel P . Berrangé]

On Fri, May 15, 2020 at 03:03:30PM +0200, Lukas Straub wrote:
> On Fri, 15 May 2020 11:26:13 +0100
> Daniel P. Berrangé  wrote:
> 
> > On Fri, May 15, 2020 at 12:14:47PM +0200, Lukas Straub wrote:
> > > On Fri, 15 May 2020 11:04:13 +0100
> > > Daniel P. Berrangé  wrote:
> > >   
> > > > On Fri, May 15, 2020 at 11:48:18AM +0200, Lukas Straub wrote:  
> > > > > On Tue, 12 May 2020 09:54:58 +0100
> > > > > Daniel P. Berrangé  wrote:
> > > > > 
> > > > > > On Mon, May 11, 2020 at 07:05:24PM +0200, Lukas Straub wrote:
> > > > > > > On Mon, 11 May 2020 17:19:09 +0100
> > > > > > > "Dr. David Alan Gilbert"  wrote:
> > > > > > >   
> > > > > > > > * Lukas Straub (lukasstra...@web.de) wrote:  
> > > > > > > > > Add yank option, pass it to the socket-channel and register a 
> > > > > > > > > yank
> > > > > > > > > function which sets s->state = NBD_CLIENT_QUIT. This is the 
> > > > > > > > > same
> > > > > > > > > behaviour as if an error occured.
> > > > > > > > > 
> > > > > > > > > Signed-off-by: Lukas Straub 
> > > > > > > >   
> > > > > > > > > +static void nbd_yank(void *opaque)
> > > > > > > > > +{
> > > > > > > > > +BlockDriverState *bs = opaque;
> > > > > > > > > +BDRVNBDState *s = (BDRVNBDState *)bs->opaque;
> > > > > > > > > +
> > > > > > > > > +atomic_set(>state, NBD_CLIENT_QUIT);
> > > > > > > > 
> > > > > > > > I think I was expecting a shutdown on the socket here - why 
> > > > > > > > doesn't it
> > > > > > > > have one?  
> > > > > > > 
> > > > > > > For nbd, we register two yank functions: This one and we enable
> > > > > > > the yank feature on the qio channel (see function
> > > > > > > nbd_establish_connection below).  
> > > > > > 
> > > > > > As mentioned on the earlier patch, I don't want to see any yank
> > > > > > code in the QIOChannel object directly. This nbd_yank function
> > > > > > can simply call the qio_channel_shutdown() function directly
> > > > > > and avoid need for modifying the QIOChannel object with yank
> > > > > > support.
> > > > > 
> > > > > Hi,
> > > > > Looking at it again, the problem is not with registering the yank 
> > > > > functions, but with tracking the lifetime of it. Suppose we add 
> > > > > qio_channel_shutdown to the yank_nbd function. Then we need to 
> > > > > unregister it whenever the QIOChannel object is freed.
> > > > > 
> > > > > In the code that would lead to the following constructs in a lot of 
> > > > > places:
> > > > >  if (local_err) {
> > > > >  yank_unregister_function(s->yank_name, yank_nbd, bs);
> > > > >  object_unref(OBJECT(sioc));
> > > > >  error_propagate(errp, local_err);
> > > > >  return NULL;
> > > > >  }
> > > > 
> > > > The nbd patch here already has a yank_unregister_function() so I'm
> > > > not seeing anything changes in that respect. The "yank_nbd" function
> > > > should check that the I/O channel is non-NULL before calling the
> > > > qio_channel_shutdown method.  
> > > 
> > > Hmm, but if object_unref frees the object, it doesn't set the
> > > pointer to NULL does it?  
> > 
> > So set  "ioc = NULL" after calling object_unref. AFAICT, nbd already
> > does exactly this.
> 
> I see 3 options to do that in a thread-safe manner:
> 1. Introduce a mutex here.
> 2. Use atomics to check for NULL and increase the reference count at the same 
> time in yank_nbd so it isn't freed while calling qio_channel_shutdown on it. 
> (I'm unsure how to do that)
> 3. Do it like it is currently done (but with the new subclass). We get thread 
> safety for free trough the mutex in yank.c.

Oh, so the problem is that  the yank function can be invoked concurrently
with the object being unreffed.

In normal object finalizers, we just have to order things such that
yank_unregister_function() is called before object_unref(ioc) is
called.

The NBD code is slightly harder because we can close & re-open the
IO separately from the finalizer. eg in nbd_reconnect_attempt we
have

if (s->ioc) {
nbd_client_detach_aio_context(s->bs);
object_unref(OBJECT(s->sioc));
s->sioc = NULL;
object_unref(OBJECT(s->ioc));
s->ioc = NULL;
}

s->connect_status = nbd_client_connect(s->bs, _err);

If the io channel is not open, then we don't need a yank function
registered. So this code would changed to


if (s->ioc) {
nbd_client_detach_aio_context(s->bs);
yank_unregister_function(...);
object_unref(OBJECT(s->sioc));
s->sioc = NULL;
object_unref(OBJECT(s->ioc));
s->ioc = NULL;
}

s->connect_status = nbd_client_connect(s->bs, _err);

The locking in yank_unregister_function() should ensure that any
currently executing yank callback has completed before the
yank_unregister_function() call returns. Thus it should be safe
to unref the cannel.

nbd_client_connect() will call yank_register_function() once it
has successfully started a new connection, which your patch 

Re: [PATCH 3/5] block/nbd.c: Add yank feature

[Lukas Straub]

On Fri, 15 May 2020 11:26:13 +0100
Daniel P. Berrangé  wrote:

> On Fri, May 15, 2020 at 12:14:47PM +0200, Lukas Straub wrote:
> > On Fri, 15 May 2020 11:04:13 +0100
> > Daniel P. Berrangé  wrote:
> >   
> > > On Fri, May 15, 2020 at 11:48:18AM +0200, Lukas Straub wrote:  
> > > > On Tue, 12 May 2020 09:54:58 +0100
> > > > Daniel P. Berrangé  wrote:
> > > > 
> > > > > On Mon, May 11, 2020 at 07:05:24PM +0200, Lukas Straub wrote:
> > > > > > On Mon, 11 May 2020 17:19:09 +0100
> > > > > > "Dr. David Alan Gilbert"  wrote:
> > > > > >   
> > > > > > > * Lukas Straub (lukasstra...@web.de) wrote:  
> > > > > > > > Add yank option, pass it to the socket-channel and register a 
> > > > > > > > yank
> > > > > > > > function which sets s->state = NBD_CLIENT_QUIT. This is the same
> > > > > > > > behaviour as if an error occured.
> > > > > > > > 
> > > > > > > > Signed-off-by: Lukas Straub 
> > > > > > >   
> > > > > > > > +static void nbd_yank(void *opaque)
> > > > > > > > +{
> > > > > > > > +BlockDriverState *bs = opaque;
> > > > > > > > +BDRVNBDState *s = (BDRVNBDState *)bs->opaque;
> > > > > > > > +
> > > > > > > > +atomic_set(>state, NBD_CLIENT_QUIT);
> > > > > > > 
> > > > > > > I think I was expecting a shutdown on the socket here - why 
> > > > > > > doesn't it
> > > > > > > have one?  
> > > > > > 
> > > > > > For nbd, we register two yank functions: This one and we enable
> > > > > > the yank feature on the qio channel (see function
> > > > > > nbd_establish_connection below).  
> > > > > 
> > > > > As mentioned on the earlier patch, I don't want to see any yank
> > > > > code in the QIOChannel object directly. This nbd_yank function
> > > > > can simply call the qio_channel_shutdown() function directly
> > > > > and avoid need for modifying the QIOChannel object with yank
> > > > > support.
> > > > 
> > > > Hi,
> > > > Looking at it again, the problem is not with registering the yank 
> > > > functions, but with tracking the lifetime of it. Suppose we add 
> > > > qio_channel_shutdown to the yank_nbd function. Then we need to 
> > > > unregister it whenever the QIOChannel object is freed.
> > > > 
> > > > In the code that would lead to the following constructs in a lot of 
> > > > places:
> > > >  if (local_err) {
> > > >  yank_unregister_function(s->yank_name, yank_nbd, bs);
> > > >  object_unref(OBJECT(sioc));
> > > >  error_propagate(errp, local_err);
> > > >  return NULL;
> > > >  }
> > > 
> > > The nbd patch here already has a yank_unregister_function() so I'm
> > > not seeing anything changes in that respect. The "yank_nbd" function
> > > should check that the I/O channel is non-NULL before calling the
> > > qio_channel_shutdown method.  
> > 
> > Hmm, but if object_unref frees the object, it doesn't set the
> > pointer to NULL does it?  
> 
> So set  "ioc = NULL" after calling object_unref. AFAICT, nbd already
> does exactly this.

I see 3 options to do that in a thread-safe manner:
1. Introduce a mutex here.
2. Use atomics to check for NULL and increase the reference count at the same 
time in yank_nbd so it isn't freed while calling qio_channel_shutdown on it. 
(I'm unsure how to do that)
3. Do it like it is currently done (but with the new subclass). We get thread 
safety for free trough the mutex in yank.c.

I prefer option 3 :)

The subclass can live in yank.c. There'd have to be 2 yank functions again but 
a comment in yank_nbd should suffice.

Regards,
Lukas Straub


pgpnW2vkHaFaO.pgp
Description: OpenPGP digital signature

[PULL 49/51] iotests: log messages from notrun()

[Kevin Wolf]

From: John Snow 

Shift the logging initialization up to occur prior to validation checks,
so that notrun() messages still get printed to console.

(Also, remove the "debugging messages active" message, because we don't
need to see that hundreds of times per iotest suite run.)

Signed-off-by: John Snow 
Message-Id: <20200514201614.19941-2-js...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 tests/qemu-iotests/iotests.py | 11 +--
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 1d7f6fd7cf..f20d90f969 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -1168,18 +1168,17 @@ def execute_setup_common(supported_fmts: Sequence[str] 
= (),
 sys.stderr.write('Please run this test via the "check" script\n')
 sys.exit(os.EX_USAGE)
 
+debug = '-d' in sys.argv
+if debug:
+sys.argv.remove('-d')
+logging.basicConfig(level=(logging.DEBUG if debug else logging.WARN))
+
 _verify_image_format(supported_fmts, unsupported_fmts)
 _verify_protocol(supported_protocols, unsupported_protocols)
 _verify_platform(supported=supported_platforms)
 _verify_cache_mode(supported_cache_modes)
 _verify_aio_mode(supported_aio_modes)
 
-debug = '-d' in sys.argv
-if debug:
-sys.argv.remove('-d')
-logging.basicConfig(level=(logging.DEBUG if debug else logging.WARN))
-logger.debug("iotests debugging messages active")
-
 return debug
 
 def execute_test(*args, test_function=None, **kwargs):
-- 
2.25.4

[PULL 44/51] block: Drop child_file

[Kevin Wolf]

From: Max Reitz 

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-33-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 include/block/block_int.h |  1 -
 block.c   | 39 ++-
 tests/test-bdrv-drain.c   |  8 +++-
 3 files changed, 5 insertions(+), 43 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index e791c40496..7fbe3206b4 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -741,7 +741,6 @@ struct BdrvChildClass {
 };
 
 extern const BdrvChildClass child_of_bds;
-extern const BdrvChildClass child_file;
 
 struct BdrvChild {
 BlockDriverState *bs;
diff --git a/block.c b/block.c
index e79fe6e07e..6a24b23d51 100644
--- a/block.c
+++ b/block.c
@@ -80,13 +80,6 @@ static BlockDriverState *bdrv_open_inherit(const char 
*filename,
BdrvChildRole child_role,
Error **errp);
 
-/* TODO: Remove when no longer needed */
-static void bdrv_inherited_options(BdrvChildRole role, bool parent_is_format,
-   int *child_flags, QDict *child_options,
-   int parent_flags, QDict *parent_options);
-static void bdrv_child_cb_attach(BdrvChild *child);
-static void bdrv_child_cb_detach(BdrvChild *child);
-
 /* If non-zero, use only whitelisted block drivers */
 static int use_bdrv_whitelist;
 
@@ -1145,33 +1138,6 @@ static void bdrv_temp_snapshot_options(int *child_flags, 
QDict *child_options,
 *child_flags &= ~BDRV_O_NATIVE_AIO;
 }
 
-/*
- * Returns the options and flags that bs->file should get if a protocol driver
- * is expected, based on the given options and flags for the parent BDS
- */
-static void bdrv_protocol_options(BdrvChildRole role, bool parent_is_format,
-  int *child_flags, QDict *child_options,
-  int parent_flags, QDict *parent_options)
-{
-bdrv_inherited_options(BDRV_CHILD_IMAGE, true,
-   child_flags, child_options,
-   parent_flags, parent_options);
-}
-
-const BdrvChildClass child_file = {
-.parent_is_bds   = true,
-.get_parent_desc = bdrv_child_get_parent_desc,
-.inherit_options = bdrv_protocol_options,
-.drained_begin   = bdrv_child_cb_drained_begin,
-.drained_poll= bdrv_child_cb_drained_poll,
-.drained_end = bdrv_child_cb_drained_end,
-.attach  = bdrv_child_cb_attach,
-.detach  = bdrv_child_cb_detach,
-.inactivate  = bdrv_child_cb_inactivate,
-.can_set_aio_ctx = bdrv_child_cb_can_set_aio_ctx,
-.set_aio_ctx = bdrv_child_cb_set_aio_ctx,
-};
-
 static void bdrv_backing_attach(BdrvChild *c)
 {
 BlockDriverState *parent = c->opaque;
@@ -2444,9 +2410,8 @@ static void 
bdrv_default_perms_for_storage(BlockDriverState *bs, BdrvChild *c,
 {
 int flags;
 
-assert(child_class == _file ||
-   (child_class == _of_bds &&
-(role & (BDRV_CHILD_METADATA | BDRV_CHILD_DATA;
+assert(child_class == _of_bds &&
+   (role & (BDRV_CHILD_METADATA | BDRV_CHILD_DATA)));
 
 flags = bdrv_reopen_get_flags(reopen_queue, bs);
 
diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c
index 0da5a3a6a1..655fd0d085 100644
--- a/tests/test-bdrv-drain.c
+++ b/tests/test-bdrv-drain.c
@@ -93,12 +93,10 @@ static void bdrv_test_child_perm(BlockDriverState *bs, 
BdrvChild *c,
  uint64_t *nperm, uint64_t *nshared)
 {
 /*
- * bdrv_default_perms() accepts only these two, so disguise
- * detach_by_driver_cb_parent as one of them.
+ * bdrv_default_perms() accepts nothing else, so disguise
+ * detach_by_driver_cb_parent.
  */
-if (child_class != _file && child_class != _of_bds) {
-child_class = _of_bds;
-}
+child_class = _of_bds;
 
 bdrv_default_perms(bs, c, child_class, role, reopen_queue,
perm, shared, nperm, nshared);
-- 
2.25.4

[PULL 36/51] block: Drop child_backing

[Kevin Wolf]

From: Max Reitz 

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-25-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 include/block/block_int.h |  1 -
 block.c   | 60 ++-
 2 files changed, 3 insertions(+), 58 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 4b09fa1124..0781d43af5 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -742,7 +742,6 @@ struct BdrvChildClass {
 
 extern const BdrvChildClass child_of_bds;
 extern const BdrvChildClass child_file;
-extern const BdrvChildClass child_backing;
 
 struct BdrvChild {
 BlockDriverState *bs;
diff --git a/block.c b/block.c
index 131ae20ffc..d138a3c261 100644
--- a/block.c
+++ b/block.c
@@ -1215,15 +1215,6 @@ static void bdrv_backing_attach(BdrvChild *c)
 parent->backing_blocker);
 }
 
-/* XXX: Will be removed along with child_backing */
-static void bdrv_child_cb_attach_backing(BdrvChild *c)
-{
-if (!(c->role & BDRV_CHILD_COW)) {
-bdrv_backing_attach(c);
-}
-bdrv_child_cb_attach(c);
-}
-
 static void bdrv_backing_detach(BdrvChild *c)
 {
 BlockDriverState *parent = c->opaque;
@@ -1234,28 +1225,6 @@ static void bdrv_backing_detach(BdrvChild *c)
 parent->backing_blocker = NULL;
 }
 
-/* XXX: Will be removed along with child_backing */
-static void bdrv_child_cb_detach_backing(BdrvChild *c)
-{
-if (!(c->role & BDRV_CHILD_COW)) {
-bdrv_backing_detach(c);
-}
-bdrv_child_cb_detach(c);
-}
-
-/*
- * Returns the options and flags that bs->backing should get, based on the
- * given options and flags for the parent BDS
- */
-static void bdrv_backing_options(BdrvChildRole role, bool parent_is_format,
- int *child_flags, QDict *child_options,
- int parent_flags, QDict *parent_options)
-{
-bdrv_inherited_options(BDRV_CHILD_COW, true,
-   child_flags, child_options,
-   parent_flags, parent_options);
-}
-
 static int bdrv_backing_update_filename(BdrvChild *c, BlockDriverState *base,
 const char *filename, Error **errp)
 {
@@ -1283,21 +1252,6 @@ static int bdrv_backing_update_filename(BdrvChild *c, 
BlockDriverState *base,
 return ret;
 }
 
-const BdrvChildClass child_backing = {
-.parent_is_bds   = true,
-.get_parent_desc = bdrv_child_get_parent_desc,
-.attach  = bdrv_child_cb_attach_backing,
-.detach  = bdrv_child_cb_detach_backing,
-.inherit_options = bdrv_backing_options,
-.drained_begin   = bdrv_child_cb_drained_begin,
-.drained_poll= bdrv_child_cb_drained_poll,
-.drained_end = bdrv_child_cb_drained_end,
-.inactivate  = bdrv_child_cb_inactivate,
-.update_filename = bdrv_backing_update_filename,
-.can_set_aio_ctx = bdrv_child_cb_can_set_aio_ctx,
-.set_aio_ctx = bdrv_child_cb_set_aio_ctx,
-};
-
 /*
  * Returns the options and flags that a generic child of a BDS should
  * get, based on the given options and flags for the parent BDS.
@@ -2446,8 +2400,7 @@ static void bdrv_default_perms_for_cow(BlockDriverState 
*bs, BdrvChild *c,
uint64_t perm, uint64_t shared,
uint64_t *nperm, uint64_t *nshared)
 {
-assert(child_class == _backing ||
-   (child_class == _of_bds && (role & BDRV_CHILD_COW)));
+assert(child_class == _of_bds && (role & BDRV_CHILD_COW));
 
 /*
  * We want consistent read from backing files if the parent needs it.
@@ -2566,23 +2519,16 @@ void bdrv_format_default_perms(BlockDriverState *bs, 
BdrvChild *c,
uint64_t perm, uint64_t shared,
uint64_t *nperm, uint64_t *nshared)
 {
-bool backing = (child_class == _backing);
-
 if (child_class == _of_bds) {
 bdrv_default_perms(bs, c, child_class, role, reopen_queue,
perm, shared, nperm, nshared);
 return;
 }
 
-assert(child_class == _backing || child_class == _file);
+assert(child_class == _file);
 
-if (!backing) {
-bdrv_default_perms_for_storage(bs, c, child_class, role, reopen_queue,
-   perm, shared, nperm, nshared);
-} else {
-bdrv_default_perms_for_cow(bs, c, child_class, role, reopen_queue,
+bdrv_default_perms_for_storage(bs, c, child_class, role, reopen_queue,
perm, shared, nperm, nshared);
-}
 }
 
 void bdrv_default_perms(BlockDriverState *bs, BdrvChild *c,
-- 
2.25.4

[PULL 33/51] block: Switch child_format users to child_of_bds

[Kevin Wolf]

From: Max Reitz 

Both users (quorum and blkverify) use child_format for
not-really-filtered children, so the appropriate BdrvChildRole in both
cases is DATA.  (Note that this will cause bdrv_inherited_options() to
force-allow format probing.)

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-22-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block/blkverify.c | 4 ++--
 block/quorum.c| 6 --
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/block/blkverify.c b/block/blkverify.c
index ba4f6d7b7c..1684b7aa2e 100644
--- a/block/blkverify.c
+++ b/block/blkverify.c
@@ -134,8 +134,8 @@ static int blkverify_open(BlockDriverState *bs, QDict 
*options, int flags,
 
 /* Open the test file */
 s->test_file = bdrv_open_child(qemu_opt_get(opts, "x-image"), options,
-   "test", bs, _format, 0, false,
-   _err);
+   "test", bs, _of_bds, BDRV_CHILD_DATA,
+   false, _err);
 if (local_err) {
 ret = -EINVAL;
 error_propagate(errp, local_err);
diff --git a/block/quorum.c b/block/quorum.c
index d37b77a522..616ac3a927 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -977,7 +977,8 @@ static int quorum_open(BlockDriverState *bs, QDict 
*options, int flags,
 assert(ret < 32);
 
 s->children[i] = bdrv_open_child(NULL, options, indexstr, bs,
- _format, 0, false, _err);
+ _of_bds, BDRV_CHILD_DATA, false,
+ _err);
 if (local_err) {
 ret = -EINVAL;
 goto close_exit;
@@ -1053,7 +1054,8 @@ static void quorum_add_child(BlockDriverState *bs, 
BlockDriverState *child_bs,
 /* We can safely add the child now */
 bdrv_ref(child_bs);
 
-child = bdrv_attach_child(bs, child_bs, indexstr, _format, 0, errp);
+child = bdrv_attach_child(bs, child_bs, indexstr, _of_bds,
+  BDRV_CHILD_DATA, errp);
 if (child == NULL) {
 s->next_child_index--;
 goto out;
-- 
2.25.4

[PULL 34/51] block: Drop child_format

[Kevin Wolf]

From: Max Reitz 

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-23-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block_int.h |  1 -
 block.c   | 29 -
 2 files changed, 30 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 3a9dda9be7..4b09fa1124 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -742,7 +742,6 @@ struct BdrvChildClass {
 
 extern const BdrvChildClass child_of_bds;
 extern const BdrvChildClass child_file;
-extern const BdrvChildClass child_format;
 extern const BdrvChildClass child_backing;
 
 struct BdrvChild {
diff --git a/block.c b/block.c
index 088727fdbe..85b4f947ba 100644
--- a/block.c
+++ b/block.c
@@ -1172,35 +1172,6 @@ const BdrvChildClass child_file = {
 .set_aio_ctx = bdrv_child_cb_set_aio_ctx,
 };
 
-/*
- * Returns the options and flags that bs->file should get if the use of formats
- * (and not only protocols) is permitted for it, based on the given options and
- * flags for the parent BDS
- */
-static void bdrv_inherited_fmt_options(BdrvChildRole role,
-   bool parent_is_format,
-   int *child_flags, QDict *child_options,
-   int parent_flags, QDict *parent_options)
-{
-bdrv_inherited_options(BDRV_CHILD_DATA, false,
-   child_flags, child_options,
-   parent_flags, parent_options);
-}
-
-const BdrvChildClass child_format = {
-.parent_is_bds   = true,
-.get_parent_desc = bdrv_child_get_parent_desc,
-.inherit_options = bdrv_inherited_fmt_options,
-.drained_begin   = bdrv_child_cb_drained_begin,
-.drained_poll= bdrv_child_cb_drained_poll,
-.drained_end = bdrv_child_cb_drained_end,
-.attach  = bdrv_child_cb_attach,
-.detach  = bdrv_child_cb_detach,
-.inactivate  = bdrv_child_cb_inactivate,
-.can_set_aio_ctx = bdrv_child_cb_can_set_aio_ctx,
-.set_aio_ctx = bdrv_child_cb_set_aio_ctx,
-};
-
 static void bdrv_backing_attach(BdrvChild *c)
 {
 BlockDriverState *parent = c->opaque;
-- 
2.25.4

[PULL 38/51] block: Make filter drivers use child_of_bds

[Kevin Wolf]

From: Max Reitz 

Note that some filters have secondary children, namely blkverify (the
image to be verified) and blklogwrites (the log).  This patch does not
touch those children.

Note that for blkverify, the filtered child should not be format-probed.
While there is nothing enforcing this here, in practice, it will not be:
blkverify implements .bdrv_file_open.  The block layer ensures (and in
fact, asserts) that BDRV_O_PROTOCOL is set for every BDS whose driver
implements .bdrv_file_open.  This flag will then be bequeathed to
blkverify's children, and they will thus (by default) not be probed
either.

("By default" refers to the fact that blkverify's other child (the
non-filtered one) will have BDRV_O_PROTOCOL force-unset, because that is
what happens for all non-filtered children of non-format drivers.)

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-27-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 block/blkdebug.c| 4 +++-
 block/blklogwrites.c| 3 ++-
 block/blkreplay.c   | 5 +++--
 block/blkverify.c   | 4 +++-
 block/copy-on-read.c| 5 +++--
 block/filter-compress.c | 5 +++--
 block/replication.c | 3 ++-
 block/throttle.c| 5 +++--
 8 files changed, 22 insertions(+), 12 deletions(-)

diff --git a/block/blkdebug.c b/block/blkdebug.c
index 8dd8ed6055..b31fa40b0e 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -497,7 +497,9 @@ static int blkdebug_open(BlockDriverState *bs, QDict 
*options, int flags,
 
 /* Open the image file */
 bs->file = bdrv_open_child(qemu_opt_get(opts, "x-image"), options, "image",
-   bs, _file, 0, false, _err);
+   bs, _of_bds,
+   BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+   false, _err);
 if (local_err) {
 ret = -EINVAL;
 error_propagate(errp, local_err);
diff --git a/block/blklogwrites.c b/block/blklogwrites.c
index 4faf912ef1..78b0c49460 100644
--- a/block/blklogwrites.c
+++ b/block/blklogwrites.c
@@ -157,7 +157,8 @@ static int blk_log_writes_open(BlockDriverState *bs, QDict 
*options, int flags,
 }
 
 /* Open the file */
-bs->file = bdrv_open_child(NULL, options, "file", bs, _file, 0, 
false,
+bs->file = bdrv_open_child(NULL, options, "file", bs, _of_bds,
+   BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, false,
_err);
 if (local_err) {
 ret = -EINVAL;
diff --git a/block/blkreplay.c b/block/blkreplay.c
index 9b2814fc58..20d6139baa 100644
--- a/block/blkreplay.c
+++ b/block/blkreplay.c
@@ -27,8 +27,9 @@ static int blkreplay_open(BlockDriverState *bs, QDict 
*options, int flags,
 int ret;
 
 /* Open the image file */
-bs->file = bdrv_open_child(NULL, options, "image",
-   bs, _file, 0, false, _err);
+bs->file = bdrv_open_child(NULL, options, "image", bs, _of_bds,
+   BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+   false, _err);
 if (local_err) {
 ret = -EINVAL;
 error_propagate(errp, local_err);
diff --git a/block/blkverify.c b/block/blkverify.c
index 1684b7aa2e..5c3b29244a 100644
--- a/block/blkverify.c
+++ b/block/blkverify.c
@@ -125,7 +125,9 @@ static int blkverify_open(BlockDriverState *bs, QDict 
*options, int flags,
 
 /* Open the raw file */
 bs->file = bdrv_open_child(qemu_opt_get(opts, "x-raw"), options, "raw",
-   bs, _file, 0, false, _err);
+   bs, _of_bds,
+   BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+   false, _err);
 if (local_err) {
 ret = -EINVAL;
 error_propagate(errp, local_err);
diff --git a/block/copy-on-read.c b/block/copy-on-read.c
index a2d92ac394..c857ea0da7 100644
--- a/block/copy-on-read.c
+++ b/block/copy-on-read.c
@@ -28,8 +28,9 @@
 static int cor_open(BlockDriverState *bs, QDict *options, int flags,
 Error **errp)
 {
-bs->file = bdrv_open_child(NULL, options, "file", bs, _file, 0, 
false,
-   errp);
+bs->file = bdrv_open_child(NULL, options, "file", bs, _of_bds,
+   BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+   false, errp);
 if (!bs->file) {
 return -EINVAL;
 }
diff --git a/block/filter-compress.c b/block/filter-compress.c
index 4dc5f9fb8c..9edd937645 100644
--- a/block/filter-compress.c
+++ b/block/filter-compress.c
@@ -30,8 +30,9 @@
 static int compress_open(BlockDriverState *bs, QDict *options, int flags,
  Error **errp)
 {
-bs->file = bdrv_open_child(NULL, options, "file", bs, _file, 0, 
false,
-   errp);
+bs->file = bdrv_open_child(NULL, options, "file", bs, _of_bds,
+   

[PULL 43/51] block: Drop bdrv_format_default_perms()

[Kevin Wolf]

From: Max Reitz 

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-32-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block_int.h | 11 ---
 block.c   | 19 ---
 2 files changed, 30 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 6fc5f0d333..e791c40496 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1262,17 +1262,6 @@ int bdrv_child_try_set_perm(BdrvChild *c, uint64_t perm, 
uint64_t shared,
  */
 int bdrv_child_refresh_perms(BlockDriverState *bs, BdrvChild *c, Error **errp);
 
-/* Default implementation for BlockDriver.bdrv_child_perm() that can be used by
- * (non-raw) image formats: Like above for bs->backing, but for bs->file it
- * requires WRITE | RESIZE for read-write images, always requires
- * CONSISTENT_READ and doesn't share WRITE. */
-void bdrv_format_default_perms(BlockDriverState *bs, BdrvChild *c,
-   const BdrvChildClass *child_class,
-   BdrvChildRole child_role,
-   BlockReopenQueue *reopen_queue,
-   uint64_t perm, uint64_t shared,
-   uint64_t *nperm, uint64_t *nshared);
-
 bool bdrv_recurse_can_replace(BlockDriverState *bs,
   BlockDriverState *to_replace);
 
diff --git a/block.c b/block.c
index 54bc1c3b2d..e79fe6e07e 100644
--- a/block.c
+++ b/block.c
@@ -2517,25 +2517,6 @@ static void 
bdrv_default_perms_for_storage(BlockDriverState *bs, BdrvChild *c,
 *nshared = shared;
 }
 
-void bdrv_format_default_perms(BlockDriverState *bs, BdrvChild *c,
-   const BdrvChildClass *child_class,
-   BdrvChildRole role,
-   BlockReopenQueue *reopen_queue,
-   uint64_t perm, uint64_t shared,
-   uint64_t *nperm, uint64_t *nshared)
-{
-if (child_class == _of_bds) {
-bdrv_default_perms(bs, c, child_class, role, reopen_queue,
-   perm, shared, nperm, nshared);
-return;
-}
-
-assert(child_class == _file);
-
-bdrv_default_perms_for_storage(bs, c, child_class, role, reopen_queue,
-   perm, shared, nperm, nshared);
-}
-
 void bdrv_default_perms(BlockDriverState *bs, BdrvChild *c,
 const BdrvChildClass *child_class, BdrvChildRole role,
 BlockReopenQueue *reopen_queue,
-- 
2.25.4

[PULL 42/51] block: Make bdrv_filter_default_perms() static

[Kevin Wolf]

From: Max Reitz 

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-31-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block_int.h | 10 --
 block.c   | 17 +++--
 2 files changed, 11 insertions(+), 16 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 0781d43af5..6fc5f0d333 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1262,16 +1262,6 @@ int bdrv_child_try_set_perm(BdrvChild *c, uint64_t perm, 
uint64_t shared,
  */
 int bdrv_child_refresh_perms(BlockDriverState *bs, BdrvChild *c, Error **errp);
 
-/* Default implementation for BlockDriver.bdrv_child_perm() that can be used by
- * block filters: Forward CONSISTENT_READ, WRITE, WRITE_UNCHANGED and RESIZE to
- * all children */
-void bdrv_filter_default_perms(BlockDriverState *bs, BdrvChild *c,
-   const BdrvChildClass *child_class,
-   BdrvChildRole child_role,
-   BlockReopenQueue *reopen_queue,
-   uint64_t perm, uint64_t shared,
-   uint64_t *nperm, uint64_t *nshared);
-
 /* Default implementation for BlockDriver.bdrv_child_perm() that can be used by
  * (non-raw) image formats: Like above for bs->backing, but for bs->file it
  * requires WRITE | RESIZE for read-write images, always requires
diff --git a/block.c b/block.c
index fb94adcca4..54bc1c3b2d 100644
--- a/block.c
+++ b/block.c
@@ -2382,12 +2382,17 @@ int bdrv_child_refresh_perms(BlockDriverState *bs, 
BdrvChild *c, Error **errp)
 return bdrv_child_try_set_perm(c, perms, shared, errp);
 }
 
-void bdrv_filter_default_perms(BlockDriverState *bs, BdrvChild *c,
-   const BdrvChildClass *child_class,
-   BdrvChildRole role,
-   BlockReopenQueue *reopen_queue,
-   uint64_t perm, uint64_t shared,
-   uint64_t *nperm, uint64_t *nshared)
+/*
+ * Default implementation for .bdrv_child_perm() for block filters:
+ * Forward CONSISTENT_READ, WRITE, WRITE_UNCHANGED, and RESIZE to the
+ * filtered child.
+ */
+static void bdrv_filter_default_perms(BlockDriverState *bs, BdrvChild *c,
+  const BdrvChildClass *child_class,
+  BdrvChildRole role,
+  BlockReopenQueue *reopen_queue,
+  uint64_t perm, uint64_t shared,
+  uint64_t *nperm, uint64_t *nshared)
 {
 *nperm = perm & DEFAULT_PERM_PASSTHROUGH;
 *nshared = (shared & DEFAULT_PERM_PASSTHROUGH) | DEFAULT_PERM_UNCHANGED;
-- 
2.25.4

[PULL 39/51] block: Use child_of_bds in remaining places

[Kevin Wolf]

From: Max Reitz 

Replace child_file by child_of_bds in all remaining places (excluding
tests).

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-28-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 block.c  |  3 ++-
 block/backup-top.c   |  4 ++--
 block/blklogwrites.c |  4 ++--
 block/raw-format.c   | 15 +--
 4 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/block.c b/block.c
index d138a3c261..fb94adcca4 100644
--- a/block.c
+++ b/block.c
@@ -3406,7 +3406,8 @@ static BlockDriverState *bdrv_open_inherit(const char 
*filename,
 BlockDriverState *file_bs;
 
 file_bs = bdrv_open_child_bs(filename, options, "file", bs,
- _file, 0, true, _err);
+ _of_bds, BDRV_CHILD_IMAGE,
+ true, _err);
 if (local_err) {
 goto fail;
 }
diff --git a/block/backup-top.c b/block/backup-top.c
index f059617095..8af2c5fe9b 100644
--- a/block/backup-top.c
+++ b/block/backup-top.c
@@ -215,8 +215,8 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState 
*source,
  source->supported_zero_flags);
 
 bdrv_ref(target);
-state->target = bdrv_attach_child(top, target, "target", _file, 0,
-  errp);
+state->target = bdrv_attach_child(top, target, "target", _of_bds,
+  BDRV_CHILD_DATA, errp);
 if (!state->target) {
 bdrv_unref(target);
 bdrv_unref(top);
diff --git a/block/blklogwrites.c b/block/blklogwrites.c
index 78b0c49460..3a57b273fc 100644
--- a/block/blklogwrites.c
+++ b/block/blklogwrites.c
@@ -167,8 +167,8 @@ static int blk_log_writes_open(BlockDriverState *bs, QDict 
*options, int flags,
 }
 
 /* Open the log file */
-s->log_file = bdrv_open_child(NULL, options, "log", bs, _file, 0,
-  false, _err);
+s->log_file = bdrv_open_child(NULL, options, "log", bs, _of_bds,
+  BDRV_CHILD_METADATA, false, _err);
 if (local_err) {
 ret = -EINVAL;
 error_propagate(errp, local_err);
diff --git a/block/raw-format.c b/block/raw-format.c
index 824fe70686..bfb4d7ddb7 100644
--- a/block/raw-format.c
+++ b/block/raw-format.c
@@ -441,6 +441,7 @@ static int raw_open(BlockDriverState *bs, QDict *options, 
int flags,
 BDRVRawState *s = bs->opaque;
 bool has_size;
 uint64_t offset, size;
+BdrvChildRole file_role;
 int ret;
 
 ret = raw_read_options(options, , _size, , errp);
@@ -448,8 +449,18 @@ static int raw_open(BlockDriverState *bs, QDict *options, 
int flags,
 return ret;
 }
 
-bs->file = bdrv_open_child(NULL, options, "file", bs, _file, 0,
-   false, errp);
+/*
+ * Without offset and a size limit, this driver behaves very much
+ * like a filter.  With any such limit, it does not.
+ */
+if (offset || has_size) {
+file_role = BDRV_CHILD_DATA | BDRV_CHILD_PRIMARY;
+} else {
+file_role = BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY;
+}
+
+bs->file = bdrv_open_child(NULL, options, "file", bs, _of_bds,
+   file_role, false, errp);
 if (!bs->file) {
 return -EINVAL;
 }
-- 
2.25.4

[PULL 37/51] block: Make format drivers use child_of_bds

[Kevin Wolf]

From: Max Reitz 

Commonly, they need to pass the BDRV_CHILD_IMAGE set as the
BdrvChildRole; but there are exceptions for drivers with external data
files (qcow2 and vmdk).

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-26-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block/bochs.c |  4 ++--
 block/cloop.c |  4 ++--
 block/crypto.c|  4 ++--
 block/dmg.c   |  4 ++--
 block/parallels.c |  4 ++--
 block/qcow.c  |  4 ++--
 block/qcow2.c | 19 +--
 block/qed.c   |  4 ++--
 block/vdi.c   |  4 ++--
 block/vhdx.c  |  4 ++--
 block/vmdk.c  | 20 +---
 block/vpc.c   |  4 ++--
 12 files changed, 50 insertions(+), 29 deletions(-)

diff --git a/block/bochs.c b/block/bochs.c
index b013e73063..62c3f42548 100644
--- a/block/bochs.c
+++ b/block/bochs.c
@@ -110,8 +110,8 @@ static int bochs_open(BlockDriverState *bs, QDict *options, 
int flags,
 return ret;
 }
 
-bs->file = bdrv_open_child(NULL, options, "file", bs, _file, 0,
-   false, errp);
+bs->file = bdrv_open_child(NULL, options, "file", bs, _of_bds,
+   BDRV_CHILD_IMAGE, false, errp);
 if (!bs->file) {
 return -EINVAL;
 }
diff --git a/block/cloop.c b/block/cloop.c
index 3ed9fa63cc..d374a8427d 100644
--- a/block/cloop.c
+++ b/block/cloop.c
@@ -71,8 +71,8 @@ static int cloop_open(BlockDriverState *bs, QDict *options, 
int flags,
 return ret;
 }
 
-bs->file = bdrv_open_child(NULL, options, "file", bs, _file, 0,
-   false, errp);
+bs->file = bdrv_open_child(NULL, options, "file", bs, _of_bds,
+   BDRV_CHILD_IMAGE, false, errp);
 if (!bs->file) {
 return -EINVAL;
 }
diff --git a/block/crypto.c b/block/crypto.c
index 8b516bfee2..457f3a06a5 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -218,8 +218,8 @@ static int block_crypto_open_generic(QCryptoBlockFormat 
format,
 unsigned int cflags = 0;
 QDict *cryptoopts = NULL;
 
-bs->file = bdrv_open_child(NULL, options, "file", bs, _file, 0,
-   false, errp);
+bs->file = bdrv_open_child(NULL, options, "file", bs, _of_bds,
+   BDRV_CHILD_IMAGE, false, errp);
 if (!bs->file) {
 return -EINVAL;
 }
diff --git a/block/dmg.c b/block/dmg.c
index af8188638c..bc64194577 100644
--- a/block/dmg.c
+++ b/block/dmg.c
@@ -439,8 +439,8 @@ static int dmg_open(BlockDriverState *bs, QDict *options, 
int flags,
 return ret;
 }
 
-bs->file = bdrv_open_child(NULL, options, "file", bs, _file, 0,
-   false, errp);
+bs->file = bdrv_open_child(NULL, options, "file", bs, _of_bds,
+   BDRV_CHILD_IMAGE, false, errp);
 if (!bs->file) {
 return -EINVAL;
 }
diff --git a/block/parallels.c b/block/parallels.c
index 9855ac1162..168d6c531a 100644
--- a/block/parallels.c
+++ b/block/parallels.c
@@ -739,8 +739,8 @@ static int parallels_open(BlockDriverState *bs, QDict 
*options, int flags,
 Error *local_err = NULL;
 char *buf;
 
-bs->file = bdrv_open_child(NULL, options, "file", bs, _file, 0,
-   false, errp);
+bs->file = bdrv_open_child(NULL, options, "file", bs, _of_bds,
+   BDRV_CHILD_IMAGE, false, errp);
 if (!bs->file) {
 return -EINVAL;
 }
diff --git a/block/qcow.c b/block/qcow.c
index 13583f0339..5347037720 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -130,8 +130,8 @@ static int qcow_open(BlockDriverState *bs, QDict *options, 
int flags,
 qdict_extract_subqdict(options, , "encrypt.");
 encryptfmt = qdict_get_try_str(encryptopts, "format");
 
-bs->file = bdrv_open_child(NULL, options, "file", bs, _file, 0,
-   false, errp);
+bs->file = bdrv_open_child(NULL, options, "file", bs, _of_bds,
+   BDRV_CHILD_IMAGE, false, errp);
 if (!bs->file) {
 ret = -EINVAL;
 goto fail;
diff --git a/block/qcow2.c b/block/qcow2.c
index 86335d9403..8d7230dca8 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1590,8 +1590,9 @@ static int coroutine_fn qcow2_do_open(BlockDriverState 
*bs, QDict *options,
 }
 
 /* Open external data file */
-s->data_file = bdrv_open_child(NULL, options, "data-file", bs, _file,
-   0, true, _err);
+s->data_file = bdrv_open_child(NULL, options, "data-file", bs,
+   _of_bds, BDRV_CHILD_DATA,
+   true, _err);
 if (local_err) {
 error_propagate(errp, local_err);
 ret = -EINVAL;
@@ -1601,8 +1602,8 @@ static int coroutine_fn qcow2_do_open(BlockDriverState 
*bs, QDict *options,
 if (s->incompatible_features & QCOW2_INCOMPAT_DATA_FILE) {
 if (!s->data_file && 

[PULL 41/51] block: Use bdrv_default_perms()

[Kevin Wolf]

From: Max Reitz 

bdrv_default_perms() can decide which permission profile to use based on
the BdrvChildRole, so block drivers do not need to select it explicitly.

The blkverify driver now no longer shares the WRITE permission for the
image to verify.  We thus have to adjust two places in
test-block-iothread not to take it.  (Note that in theory, blkverify
should behave like quorum in this regard and share neither WRITE nor
RESIZE for both of its children.  In practice, it does not really
matter, because blkverify is used only for debugging, so we might as
well keep its permissions rather liberal.)

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-30-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block/backup-top.c  |  4 ++--
 block/blkdebug.c|  4 ++--
 block/blklogwrites.c|  9 ++---
 block/blkreplay.c   |  2 +-
 block/blkverify.c   |  2 +-
 block/bochs.c   |  2 +-
 block/cloop.c   |  2 +-
 block/crypto.c  |  2 +-
 block/dmg.c |  2 +-
 block/filter-compress.c |  2 +-
 block/parallels.c   |  2 +-
 block/qcow.c|  2 +-
 block/qcow2.c   |  2 +-
 block/qed.c |  2 +-
 block/raw-format.c  |  2 +-
 block/throttle.c|  2 +-
 block/vdi.c |  2 +-
 block/vhdx.c|  2 +-
 block/vmdk.c|  2 +-
 block/vpc.c |  2 +-
 tests/test-bdrv-drain.c | 10 +-
 tests/test-bdrv-graph-mod.c |  2 +-
 tests/test-block-iothread.c | 17 ++---
 23 files changed, 43 insertions(+), 37 deletions(-)

diff --git a/block/backup-top.c b/block/backup-top.c
index 8af2c5fe9b..f0efec18b5 100644
--- a/block/backup-top.c
+++ b/block/backup-top.c
@@ -156,8 +156,8 @@ static void backup_top_child_perm(BlockDriverState *bs, 
BdrvChild *c,
 *nperm = BLK_PERM_WRITE;
 } else {
 /* Source child */
-bdrv_filter_default_perms(bs, c, child_class, role, reopen_queue,
-  perm, shared, nperm, nshared);
+bdrv_default_perms(bs, c, child_class, role, reopen_queue,
+   perm, shared, nperm, nshared);
 
 if (perm & BLK_PERM_WRITE) {
 *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
diff --git a/block/blkdebug.c b/block/blkdebug.c
index b31fa40b0e..a925d8295e 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -1003,8 +1003,8 @@ static void blkdebug_child_perm(BlockDriverState *bs, 
BdrvChild *c,
 {
 BDRVBlkdebugState *s = bs->opaque;
 
-bdrv_filter_default_perms(bs, c, child_class, role, reopen_queue,
-  perm, shared, nperm, nshared);
+bdrv_default_perms(bs, c, child_class, role, reopen_queue,
+   perm, shared, nperm, nshared);
 
 *nperm |= s->take_child_perms;
 *nshared &= ~s->unshare_child_perms;
diff --git a/block/blklogwrites.c b/block/blklogwrites.c
index 3a57b273fc..8684fb1c74 100644
--- a/block/blklogwrites.c
+++ b/block/blklogwrites.c
@@ -295,13 +295,8 @@ static void blk_log_writes_child_perm(BlockDriverState 
*bs, BdrvChild *c,
 return;
 }
 
-if (!strcmp(c->name, "log")) {
-bdrv_format_default_perms(bs, c, child_class, role, ro_q, perm, shrd,
-  nperm, nshrd);
-} else {
-bdrv_filter_default_perms(bs, c, child_class, role, ro_q, perm, shrd,
-  nperm, nshrd);
-}
+bdrv_default_perms(bs, c, child_class, role, ro_q, perm, shrd,
+   nperm, nshrd);
 }
 
 static void blk_log_writes_refresh_limits(BlockDriverState *bs, Error **errp)
diff --git a/block/blkreplay.c b/block/blkreplay.c
index 20d6139baa..30a0f5d57a 100644
--- a/block/blkreplay.c
+++ b/block/blkreplay.c
@@ -139,7 +139,7 @@ static BlockDriver bdrv_blkreplay = {
 .is_filter  = true,
 
 .bdrv_open  = blkreplay_open,
-.bdrv_child_perm= bdrv_filter_default_perms,
+.bdrv_child_perm= bdrv_default_perms,
 .bdrv_getlength = blkreplay_getlength,
 
 .bdrv_co_preadv = blkreplay_co_preadv,
diff --git a/block/blkverify.c b/block/blkverify.c
index 5c3b29244a..2f261de24b 100644
--- a/block/blkverify.c
+++ b/block/blkverify.c
@@ -319,7 +319,7 @@ static BlockDriver bdrv_blkverify = {
 .bdrv_parse_filename  = blkverify_parse_filename,
 .bdrv_file_open   = blkverify_open,
 .bdrv_close   = blkverify_close,
-.bdrv_child_perm  = bdrv_filter_default_perms,
+.bdrv_child_perm  = bdrv_default_perms,
 .bdrv_getlength   = blkverify_getlength,
 .bdrv_refresh_filename= blkverify_refresh_filename,
 .bdrv_dirname = blkverify_dirname,
diff --git a/block/bochs.c b/block/bochs.c
index 62c3f42548..2f010ab40a 100644

[PULL 35/51] block: Make backing files child_of_bds children

[Kevin Wolf]

From: Max Reitz 

Make all parents of backing files pass the appropriate BdrvChildRole.
By doing so, we can switch their BdrvChildClass over to the generic
child_of_bds, which will do the right thing when given a correct
BdrvChildRole.

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-24-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 block.c | 26 --
 block/backup-top.c  |  2 +-
 block/vvfat.c   |  3 ++-
 tests/test-bdrv-drain.c | 13 +++--
 4 files changed, 30 insertions(+), 14 deletions(-)

diff --git a/block.c b/block.c
index 85b4f947ba..131ae20ffc 100644
--- a/block.c
+++ b/block.c
@@ -2941,6 +2941,20 @@ static bool 
bdrv_inherits_from_recursive(BlockDriverState *child,
 return child != NULL;
 }
 
+/*
+ * Return the BdrvChildRole for @bs's backing child.  bs->backing is
+ * mostly used for COW backing children (role = COW), but also for
+ * filtered children (role = FILTERED | PRIMARY).
+ */
+static BdrvChildRole bdrv_backing_role(BlockDriverState *bs)
+{
+if (bs->drv && bs->drv->is_filter) {
+return BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY;
+} else {
+return BDRV_CHILD_COW;
+}
+}
+
 /*
  * Sets the backing file link of a BDS. A new reference is created; callers
  * which don't need their own reference any more must call bdrv_unref().
@@ -2968,8 +2982,8 @@ void bdrv_set_backing_hd(BlockDriverState *bs, 
BlockDriverState *backing_hd,
 goto out;
 }
 
-bs->backing = bdrv_attach_child(bs, backing_hd, "backing", _backing,
-0, errp);
+bs->backing = bdrv_attach_child(bs, backing_hd, "backing", _of_bds,
+bdrv_backing_role(bs), errp);
 /* If backing_hd was already part of bs's backing chain, and
  * inherits_from pointed recursively to bs then let's update it to
  * point directly to bs (else it will become NULL). */
@@ -3066,7 +3080,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict 
*parent_options,
 }
 
 backing_hd = bdrv_open_inherit(backing_filename, reference, options, 0, bs,
-   _backing, 0, errp);
+   _of_bds, bdrv_backing_role(bs), errp);
 if (!backing_hd) {
 bs->open_flags |= BDRV_O_NO_BACKING;
 error_prepend(errp, "Could not open backing file: ");
@@ -3895,8 +3909,8 @@ int bdrv_reopen_multiple(BlockReopenQueue *bs_queue, 
Error **errp)
 if (state->replace_backing_bs && state->new_backing_bs) {
 uint64_t nperm, nshared;
 bdrv_child_perm(state->bs, state->new_backing_bs,
-NULL, _backing, 0, bs_queue,
-state->perm, state->shared_perm,
+NULL, _of_bds, bdrv_backing_role(state->bs),
+bs_queue, state->perm, state->shared_perm,
 , );
 ret = bdrv_check_update_perm(state->new_backing_bs, NULL,
  nperm, nshared, NULL, NULL, errp);
@@ -6852,7 +6866,7 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 drv->bdrv_gather_child_options(bs, opts, backing_overridden);
 } else {
 QLIST_FOREACH(child, >children, next) {
-if (child->klass == _backing && !backing_overridden) {
+if (child == bs->backing && !backing_overridden) {
 /* We can skip the backing BDS if it has not been overridden */
 continue;
 }
diff --git a/block/backup-top.c b/block/backup-top.c
index e2b4d2acd3..f059617095 100644
--- a/block/backup-top.c
+++ b/block/backup-top.c
@@ -143,7 +143,7 @@ static void backup_top_child_perm(BlockDriverState *bs, 
BdrvChild *c,
 return;
 }
 
-if (child_class == _file) {
+if (!(role & BDRV_CHILD_FILTERED)) {
 /*
  * Target child
  *
diff --git a/block/vvfat.c b/block/vvfat.c
index b4c8417dbd..e8848a0497 100644
--- a/block/vvfat.c
+++ b/block/vvfat.c
@@ -3219,7 +3219,8 @@ static void vvfat_child_perm(BlockDriverState *bs, 
BdrvChild *c,
 {
 BDRVVVFATState *s = bs->opaque;
 
-assert(c == s->qcow || child_class == _backing);
+assert(c == s->qcow ||
+   (child_class == _of_bds && (role & BDRV_CHILD_COW)));
 
 if (c == s->qcow) {
 /* This is a private node, nobody should try to attach to it */
diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c
index b3d7960bd0..15393a0140 100644
--- a/tests/test-bdrv-drain.c
+++ b/tests/test-bdrv-drain.c
@@ -96,7 +96,7 @@ static void bdrv_test_child_perm(BlockDriverState *bs, 
BdrvChild *c,
  * bdrv_format_default_perms() accepts only these two, so disguise
  * detach_by_driver_cb_parent as one of them.
  */
-if (child_class != _file && child_class != _backing) {
+if (child_class != _file && child_class != _of_bds) {
 child_class = _file;
  

[PULL 20/51] block: Pass parent_is_format to .inherit_options()

[Kevin Wolf]

From: Max Reitz 

We plan to unify the generic .inherit_options() functions.  The
resulting common function will need to decide whether to force-enable
format probing, force-disable it, or leave it as-is.  To make this
decision, it will need to know whether the parent node is a format node
or not (because we never want format probing if the parent is a format
node already (except for the backing chain)).

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-9-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block_int.h |  2 +-
 block.c   | 37 +++--
 block/block-backend.c |  2 +-
 block/vvfat.c |  2 +-
 4 files changed, 30 insertions(+), 13 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 8c1160a577..6245d8a18d 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -677,7 +677,7 @@ struct BdrvChildClass {
  * non-BDS parents. */
 bool parent_is_bds;
 
-void (*inherit_options)(BdrvChildRole role,
+void (*inherit_options)(BdrvChildRole role, bool parent_is_format,
 int *child_flags, QDict *child_options,
 int parent_flags, QDict *parent_options);
 
diff --git a/block.c b/block.c
index 40db0b78b9..b78e2dc141 100644
--- a/block.c
+++ b/block.c
@@ -1154,7 +1154,7 @@ static void bdrv_temp_snapshot_options(int *child_flags, 
QDict *child_options,
  * Returns the options and flags that bs->file should get if a protocol driver
  * is expected, based on the given options and flags for the parent BDS
  */
-static void bdrv_inherited_options(BdrvChildRole role,
+static void bdrv_inherited_options(BdrvChildRole role, bool parent_is_format,
int *child_flags, QDict *child_options,
int parent_flags, QDict *parent_options)
 {
@@ -1205,10 +1205,12 @@ const BdrvChildClass child_file = {
  * flags for the parent BDS
  */
 static void bdrv_inherited_fmt_options(BdrvChildRole role,
+   bool parent_is_format,
int *child_flags, QDict *child_options,
int parent_flags, QDict *parent_options)
 {
-child_file.inherit_options(role, child_flags, child_options,
+child_file.inherit_options(role, parent_is_format,
+   child_flags, child_options,
parent_flags, parent_options);
 
 *child_flags &= ~(BDRV_O_PROTOCOL | BDRV_O_NO_IO);
@@ -1289,7 +1291,7 @@ static void bdrv_backing_detach(BdrvChild *c)
  * Returns the options and flags that bs->backing should get, based on the
  * given options and flags for the parent BDS
  */
-static void bdrv_backing_options(BdrvChildRole role,
+static void bdrv_backing_options(BdrvChildRole role, bool parent_is_format,
  int *child_flags, QDict *child_options,
  int parent_flags, QDict *parent_options)
 {
@@ -3173,8 +3175,22 @@ static BlockDriverState *bdrv_open_inherit(const char 
*filename,
 bs->explicit_options = qdict_clone_shallow(options);
 
 if (child_class) {
+bool parent_is_format;
+
+if (parent->drv) {
+parent_is_format = parent->drv->is_format;
+} else {
+/*
+ * parent->drv is not set yet because this node is opened for
+ * (potential) format probing.  That means that @parent is going
+ * to be a format node.
+ */
+parent_is_format = true;
+}
+
 bs->inherits_from = parent;
-child_class->inherit_options(child_role, , options,
+child_class->inherit_options(child_role, parent_is_format,
+ , options,
  parent->open_flags, parent->options);
 }
 
@@ -3203,7 +3219,7 @@ static BlockDriverState *bdrv_open_inherit(const char 
*filename,
flags, options);
 /* Let bdrv_backing_options() override "read-only" */
 qdict_del(options, BDRV_OPT_READ_ONLY);
-bdrv_backing_options(0, , options, flags, options);
+bdrv_backing_options(0, true, , options, flags, options);
 }
 
 bs->open_flags = flags;
@@ -3488,6 +3504,7 @@ static BlockReopenQueue 
*bdrv_reopen_queue_child(BlockReopenQueue *bs_queue,
  QDict *options,
  const BdrvChildClass *klass,
  BdrvChildRole role,
+ bool parent_is_format,
  QDict *parent_options,
  int parent_flags,
  bool 

[PULL 32/51] raw-format: Split raw_read_options()

[Kevin Wolf]

From: Max Reitz 

Split raw_read_options() into one function that actually just reads the
options, and another that applies them.  This will allow us to detect
whether the user has specified any options before attaching the file
child (so we can decide on its role based on the options).

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-21-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block/raw-format.c | 110 ++---
 1 file changed, 65 insertions(+), 45 deletions(-)

diff --git a/block/raw-format.c b/block/raw-format.c
index 4b8d4ce8be..824fe70686 100644
--- a/block/raw-format.c
+++ b/block/raw-format.c
@@ -71,20 +71,13 @@ static QemuOptsList raw_create_opts = {
 }
 };
 
-static int raw_read_options(QDict *options, BlockDriverState *bs,
-BDRVRawState *s, Error **errp)
+static int raw_read_options(QDict *options, uint64_t *offset, bool *has_size,
+uint64_t *size, Error **errp)
 {
 Error *local_err = NULL;
 QemuOpts *opts = NULL;
-int64_t real_size = 0;
 int ret;
 
-real_size = bdrv_getlength(bs->file->bs);
-if (real_size < 0) {
-error_setg_errno(errp, -real_size, "Could not get image size");
-return real_size;
-}
-
 opts = qemu_opts_create(_runtime_opts, NULL, 0, _abort);
 qemu_opts_absorb_qdict(opts, options, _err);
 if (local_err) {
@@ -93,64 +86,84 @@ static int raw_read_options(QDict *options, 
BlockDriverState *bs,
 goto end;
 }
 
-s->offset = qemu_opt_get_size(opts, "offset", 0);
-if (s->offset > real_size) {
-error_setg(errp, "Offset (%" PRIu64 ") cannot be greater than "
-"size of the containing file (%" PRId64 ")",
-s->offset, real_size);
-ret = -EINVAL;
-goto end;
-}
+*offset = qemu_opt_get_size(opts, "offset", 0);
+*has_size = qemu_opt_find(opts, "size");
+*size = qemu_opt_get_size(opts, "size", 0);
 
-if (qemu_opt_find(opts, "size") != NULL) {
-s->size = qemu_opt_get_size(opts, "size", 0);
-s->has_size = true;
-} else {
-s->has_size = false;
-s->size = real_size - s->offset;
+ret = 0;
+end:
+qemu_opts_del(opts);
+return ret;
+}
+
+static int raw_apply_options(BlockDriverState *bs, BDRVRawState *s,
+ uint64_t offset, bool has_size, uint64_t size,
+ Error **errp)
+{
+int64_t real_size = 0;
+
+real_size = bdrv_getlength(bs->file->bs);
+if (real_size < 0) {
+error_setg_errno(errp, -real_size, "Could not get image size");
+return real_size;
 }
 
 /* Check size and offset */
-if ((real_size - s->offset) < s->size) {
+if (offset > real_size) {
+error_setg(errp, "Offset (%" PRIu64 ") cannot be greater than "
+   "size of the containing file (%" PRId64 ")",
+   s->offset, real_size);
+return -EINVAL;
+}
+
+if (has_size && (real_size - offset) < size) {
 error_setg(errp, "The sum of offset (%" PRIu64 ") and size "
-"(%" PRIu64 ") has to be smaller or equal to the "
-" actual size of the containing file (%" PRId64 ")",
-s->offset, s->size, real_size);
-ret = -EINVAL;
-goto end;
+   "(%" PRIu64 ") has to be smaller or equal to the "
+   " actual size of the containing file (%" PRId64 ")",
+   s->offset, s->size, real_size);
+return -EINVAL;
 }
 
 /* Make sure size is multiple of BDRV_SECTOR_SIZE to prevent rounding
  * up and leaking out of the specified area. */
-if (s->has_size && !QEMU_IS_ALIGNED(s->size, BDRV_SECTOR_SIZE)) {
+if (has_size && !QEMU_IS_ALIGNED(size, BDRV_SECTOR_SIZE)) {
 error_setg(errp, "Specified size is not multiple of %llu",
-BDRV_SECTOR_SIZE);
-ret = -EINVAL;
-goto end;
+   BDRV_SECTOR_SIZE);
+return -EINVAL;
 }
 
-ret = 0;
-
-end:
+s->offset = offset;
+s->has_size = has_size;
+s->size = has_size ? size : real_size - offset;
 
-qemu_opts_del(opts);
-
-return ret;
+return 0;
 }
 
 static int raw_reopen_prepare(BDRVReopenState *reopen_state,
   BlockReopenQueue *queue, Error **errp)
 {
+bool has_size;
+uint64_t offset, size;
+int ret;
+
 assert(reopen_state != NULL);
 assert(reopen_state->bs != NULL);
 
 reopen_state->opaque = g_new0(BDRVRawState, 1);
 
-return raw_read_options(
-reopen_state->options,
-reopen_state->bs,
-reopen_state->opaque,
-errp);
+ret = raw_read_options(reopen_state->options, , _size, ,
+   errp);
+if (ret < 0) {
+return ret;
+}
+
+ret = raw_apply_options(reopen_state->bs, reopen_state->opaque,
+offset, has_size, size, 

[PULL 28/51] block: Pull out bdrv_default_perms_for_cow()

[Kevin Wolf]

From: Max Reitz 

Right now, bdrv_format_default_perms() is used by format parents
(generally). We want to switch to a model where most parents use a
single BdrvChildClass, which then decides the permissions based on the
child role. To do so, we have to split bdrv_format_default_perms() into
separate functions for each such role.

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-17-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 block.c | 62 +
 1 file changed, 40 insertions(+), 22 deletions(-)

diff --git a/block.c b/block.c
index b3e7ae70c7..b1a471 100644
--- a/block.c
+++ b/block.c
@@ -2468,6 +2468,44 @@ void bdrv_filter_default_perms(BlockDriverState *bs, 
BdrvChild *c,
 *nshared = (shared & DEFAULT_PERM_PASSTHROUGH) | DEFAULT_PERM_UNCHANGED;
 }
 
+static void bdrv_default_perms_for_cow(BlockDriverState *bs, BdrvChild *c,
+   const BdrvChildClass *child_class,
+   BdrvChildRole role,
+   BlockReopenQueue *reopen_queue,
+   uint64_t perm, uint64_t shared,
+   uint64_t *nperm, uint64_t *nshared)
+{
+assert(child_class == _backing ||
+   (child_class == _of_bds && (role & BDRV_CHILD_COW)));
+
+/*
+ * We want consistent read from backing files if the parent needs it.
+ * No other operations are performed on backing files.
+ */
+perm &= BLK_PERM_CONSISTENT_READ;
+
+/*
+ * If the parent can deal with changing data, we're okay with a
+ * writable and resizable backing file.
+ * TODO Require !(perm & BLK_PERM_CONSISTENT_READ), too?
+ */
+if (shared & BLK_PERM_WRITE) {
+shared = BLK_PERM_WRITE | BLK_PERM_RESIZE;
+} else {
+shared = 0;
+}
+
+shared |= BLK_PERM_CONSISTENT_READ | BLK_PERM_GRAPH_MOD |
+  BLK_PERM_WRITE_UNCHANGED;
+
+if (bs->open_flags & BDRV_O_INACTIVE) {
+shared |= BLK_PERM_WRITE | BLK_PERM_RESIZE;
+}
+
+*nperm = perm;
+*nshared = shared;
+}
+
 void bdrv_format_default_perms(BlockDriverState *bs, BdrvChild *c,
const BdrvChildClass *child_class,
BdrvChildRole role,
@@ -2505,28 +2543,8 @@ void bdrv_format_default_perms(BlockDriverState *bs, 
BdrvChild *c,
 *nperm = perm;
 *nshared = shared;
 } else {
-/* We want consistent read from backing files if the parent needs it.
- * No other operations are performed on backing files. */
-perm &= BLK_PERM_CONSISTENT_READ;
-
-/* If the parent can deal with changing data, we're okay with a
- * writable and resizable backing file. */
-/* TODO Require !(perm & BLK_PERM_CONSISTENT_READ), too? */
-if (shared & BLK_PERM_WRITE) {
-shared = BLK_PERM_WRITE | BLK_PERM_RESIZE;
-} else {
-shared = 0;
-}
-
-shared |= BLK_PERM_CONSISTENT_READ | BLK_PERM_GRAPH_MOD |
-  BLK_PERM_WRITE_UNCHANGED;
-
-if (bs->open_flags & BDRV_O_INACTIVE) {
-shared |= BLK_PERM_WRITE | BLK_PERM_RESIZE;
-}
-
-*nperm = perm;
-*nshared = shared;
+bdrv_default_perms_for_cow(bs, c, child_class, role, reopen_queue,
+   perm, shared, nperm, nshared);
 }
 }
 
-- 
2.25.4

[PULL 21/51] block: Rename bdrv_inherited_options()

[Kevin Wolf]

From: Max Reitz 

The other two .inherit_options implementations specify exactly for what
case they are used in their name, so do it for this one as well.

(The actual intention behind this patch is to follow it up with a
generic bdrv_inherited_options() that works for all three cases.)

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-10-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/block.c b/block.c
index b78e2dc141..a17c29f54b 100644
--- a/block.c
+++ b/block.c
@@ -1154,9 +1154,9 @@ static void bdrv_temp_snapshot_options(int *child_flags, 
QDict *child_options,
  * Returns the options and flags that bs->file should get if a protocol driver
  * is expected, based on the given options and flags for the parent BDS
  */
-static void bdrv_inherited_options(BdrvChildRole role, bool parent_is_format,
-   int *child_flags, QDict *child_options,
-   int parent_flags, QDict *parent_options)
+static void bdrv_protocol_options(BdrvChildRole role, bool parent_is_format,
+  int *child_flags, QDict *child_options,
+  int parent_flags, QDict *parent_options)
 {
 int flags = parent_flags;
 
@@ -1188,7 +1188,7 @@ static void bdrv_inherited_options(BdrvChildRole role, 
bool parent_is_format,
 const BdrvChildClass child_file = {
 .parent_is_bds   = true,
 .get_parent_desc = bdrv_child_get_parent_desc,
-.inherit_options = bdrv_inherited_options,
+.inherit_options = bdrv_protocol_options,
 .drained_begin   = bdrv_child_cb_drained_begin,
 .drained_poll= bdrv_child_cb_drained_poll,
 .drained_end = bdrv_child_cb_drained_end,
-- 
2.25.4

[PULL 25/51] block: Unify bdrv_child_cb_detach()

[Kevin Wolf]

From: Max Reitz 

Make bdrv_child_cb_detach() call bdrv_backing_detach() for children with
a COW role (and drop the reverse call from bdrv_backing_detach()), so it
can be used for any child (with a proper role set).

Because so far no child has a proper role set, we need a temporary new
callback for child_backing.detach that ensures bdrv_backing_detach() is
called for all COW children that do not have their role set yet.

(Also, move bdrv_child_cb_detach() down to group it with
bdrv_inherited_options() and bdrv_child_cb_attach().)

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-14-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 block.c | 27 ---
 1 file changed, 20 insertions(+), 7 deletions(-)

diff --git a/block.c b/block.c
index 755704a54c..f63417c06d 100644
--- a/block.c
+++ b/block.c
@@ -85,6 +85,7 @@ static void bdrv_inherited_options(BdrvChildRole role, bool 
parent_is_format,
int *child_flags, QDict *child_options,
int parent_flags, QDict *parent_options);
 static void bdrv_child_cb_attach(BdrvChild *child);
+static void bdrv_child_cb_detach(BdrvChild *child);
 
 /* If non-zero, use only whitelisted block drivers */
 static int use_bdrv_whitelist;
@@ -1100,12 +1101,6 @@ static void bdrv_child_cb_drained_end(BdrvChild *child,
 bdrv_drained_end_no_poll(bs, drained_end_counter);
 }
 
-static void bdrv_child_cb_detach(BdrvChild *child)
-{
-BlockDriverState *bs = child->opaque;
-bdrv_unapply_subtree_drain(child, bs);
-}
-
 static int bdrv_child_cb_inactivate(BdrvChild *child)
 {
 BlockDriverState *bs = child->opaque;
@@ -1266,7 +1261,14 @@ static void bdrv_backing_detach(BdrvChild *c)
 bdrv_op_unblock_all(c->bs, parent->backing_blocker);
 error_free(parent->backing_blocker);
 parent->backing_blocker = NULL;
+}
 
+/* XXX: Will be removed along with child_backing */
+static void bdrv_child_cb_detach_backing(BdrvChild *c)
+{
+if (!(c->role & BDRV_CHILD_COW)) {
+bdrv_backing_detach(c);
+}
 bdrv_child_cb_detach(c);
 }
 
@@ -1314,7 +1316,7 @@ const BdrvChildClass child_backing = {
 .parent_is_bds   = true,
 .get_parent_desc = bdrv_child_get_parent_desc,
 .attach  = bdrv_child_cb_attach_backing,
-.detach  = bdrv_backing_detach,
+.detach  = bdrv_child_cb_detach_backing,
 .inherit_options = bdrv_backing_options,
 .drained_begin   = bdrv_child_cb_drained_begin,
 .drained_poll= bdrv_child_cb_drained_poll,
@@ -1416,6 +1418,17 @@ static void bdrv_child_cb_attach(BdrvChild *child)
 bdrv_apply_subtree_drain(child, bs);
 }
 
+static void bdrv_child_cb_detach(BdrvChild *child)
+{
+BlockDriverState *bs = child->opaque;
+
+if (child->role & BDRV_CHILD_COW) {
+bdrv_backing_detach(child);
+}
+
+bdrv_unapply_subtree_drain(child, bs);
+}
+
 static int bdrv_open_flags(BlockDriverState *bs, int flags)
 {
 int open_flags = flags;
-- 
2.25.4

[PULL 29/51] block: Pull out bdrv_default_perms_for_storage()

[Kevin Wolf]

From: Max Reitz 

Right now, bdrv_format_default_perms() is used by format parents
(generally). We want to switch to a model where most parents use a
single BdrvChildClass, which then decides the permissions based on the
child role. To do so, we have to split bdrv_format_default_perms() into
separate functions for each such role.

Note that bdrv_default_perms_for_storage() currently handles all DATA |
METADATA children.  A follow-up patch is going to split it further into
one function for each case.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-18-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block.c | 71 +
 1 file changed, 46 insertions(+), 25 deletions(-)

diff --git a/block.c b/block.c
index b1a471..5d17aa1cc3 100644
--- a/block.c
+++ b/block.c
@@ -2506,6 +2506,50 @@ static void bdrv_default_perms_for_cow(BlockDriverState 
*bs, BdrvChild *c,
 *nshared = shared;
 }
 
+static void bdrv_default_perms_for_storage(BlockDriverState *bs, BdrvChild *c,
+   const BdrvChildClass *child_class,
+   BdrvChildRole role,
+   BlockReopenQueue *reopen_queue,
+   uint64_t perm, uint64_t shared,
+   uint64_t *nperm, uint64_t *nshared)
+{
+int flags;
+
+assert(child_class == _file ||
+   (child_class == _of_bds &&
+(role & (BDRV_CHILD_METADATA | BDRV_CHILD_DATA;
+
+flags = bdrv_reopen_get_flags(reopen_queue, bs);
+
+/*
+ * Apart from the modifications below, the same permissions are
+ * forwarded and left alone as for filters
+ */
+bdrv_filter_default_perms(bs, c, child_class, role, reopen_queue,
+  perm, shared, , );
+
+/* Format drivers may touch metadata even if the guest doesn't write */
+if (bdrv_is_writable_after_reopen(bs, reopen_queue)) {
+perm |= BLK_PERM_WRITE | BLK_PERM_RESIZE;
+}
+
+/*
+ * bs->file always needs to be consistent because of the metadata. We
+ * can never allow other users to resize or write to it.
+ */
+if (!(flags & BDRV_O_NO_IO)) {
+perm |= BLK_PERM_CONSISTENT_READ;
+}
+shared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
+
+if (bs->open_flags & BDRV_O_INACTIVE) {
+shared |= BLK_PERM_WRITE | BLK_PERM_RESIZE;
+}
+
+*nperm = perm;
+*nshared = shared;
+}
+
 void bdrv_format_default_perms(BlockDriverState *bs, BdrvChild *c,
const BdrvChildClass *child_class,
BdrvChildRole role,
@@ -2517,31 +2561,8 @@ void bdrv_format_default_perms(BlockDriverState *bs, 
BdrvChild *c,
 assert(child_class == _backing || child_class == _file);
 
 if (!backing) {
-int flags = bdrv_reopen_get_flags(reopen_queue, bs);
-
-/* Apart from the modifications below, the same permissions are
- * forwarded and left alone as for filters */
-bdrv_filter_default_perms(bs, c, child_class, role, reopen_queue,
-  perm, shared, , );
-
-/* Format drivers may touch metadata even if the guest doesn't write */
-if (bdrv_is_writable_after_reopen(bs, reopen_queue)) {
-perm |= BLK_PERM_WRITE | BLK_PERM_RESIZE;
-}
-
-/* bs->file always needs to be consistent because of the metadata. We
- * can never allow other users to resize or write to it. */
-if (!(flags & BDRV_O_NO_IO)) {
-perm |= BLK_PERM_CONSISTENT_READ;
-}
-shared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
-
-if (bs->open_flags & BDRV_O_INACTIVE) {
-shared |= BLK_PERM_WRITE | BLK_PERM_RESIZE;
-}
-
-*nperm = perm;
-*nshared = shared;
+bdrv_default_perms_for_storage(bs, c, child_class, role, reopen_queue,
+   perm, shared, nperm, nshared);
 } else {
 bdrv_default_perms_for_cow(bs, c, child_class, role, reopen_queue,
perm, shared, nperm, nshared);
-- 
2.25.4

[PULL 26/51] block: Add child_of_bds

[Kevin Wolf]

From: Max Reitz 

Any current user of child_file, child_format, and child_backing can and
should use this generic BdrvChildClass instead, as it can handle all of
these cases.  However, to be able to do so, the users must pass the
appropriate BdrvChildRole when the child is created/attached.  (The
following commits will take care of that.)

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-15-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 include/block/block_int.h |  1 +
 block.c   | 24 
 2 files changed, 25 insertions(+)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 6245d8a18d..54df821d61 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -740,6 +740,7 @@ struct BdrvChildClass {
 void (*set_aio_ctx)(BdrvChild *child, AioContext *ctx, GSList **ignore);
 };
 
+extern const BdrvChildClass child_of_bds;
 extern const BdrvChildClass child_file;
 extern const BdrvChildClass child_format;
 extern const BdrvChildClass child_backing;
diff --git a/block.c b/block.c
index f63417c06d..be9bfa9d46 100644
--- a/block.c
+++ b/block.c
@@ -1429,6 +1429,30 @@ static void bdrv_child_cb_detach(BdrvChild *child)
 bdrv_unapply_subtree_drain(child, bs);
 }
 
+static int bdrv_child_cb_update_filename(BdrvChild *c, BlockDriverState *base,
+ const char *filename, Error **errp)
+{
+if (c->role & BDRV_CHILD_COW) {
+return bdrv_backing_update_filename(c, base, filename, errp);
+}
+return 0;
+}
+
+const BdrvChildClass child_of_bds = {
+.parent_is_bds   = true,
+.get_parent_desc = bdrv_child_get_parent_desc,
+.inherit_options = bdrv_inherited_options,
+.drained_begin   = bdrv_child_cb_drained_begin,
+.drained_poll= bdrv_child_cb_drained_poll,
+.drained_end = bdrv_child_cb_drained_end,
+.attach  = bdrv_child_cb_attach,
+.detach  = bdrv_child_cb_detach,
+.inactivate  = bdrv_child_cb_inactivate,
+.can_set_aio_ctx = bdrv_child_cb_can_set_aio_ctx,
+.set_aio_ctx = bdrv_child_cb_set_aio_ctx,
+.update_filename = bdrv_child_cb_update_filename,
+};
+
 static int bdrv_open_flags(BlockDriverState *bs, int flags)
 {
 int open_flags = flags;
-- 
2.25.4

[PULL 51/51] iotests/030: Reduce run time by unthrottling job earlier

[Kevin Wolf]

test_overlapping_3() throttles its active commit job so it can be sure
the job is still busy when it checks that you can't start a conflicting
streaming job.

However, it only sets the commit job back to full speed when it is
ready, which takes a few seconds while it's throttled. We can already
reset the limit after having checked that block-stream returns an error
and save these seconds.

Signed-off-by: Kevin Wolf 
Message-Id: <20200513100025.33543-1-kw...@redhat.com>
Reviewed-by: Alberto Garcia 
Signed-off-by: Kevin Wolf 
---
 tests/qemu-iotests/030 | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030
index 104e3cee1b..1cdd7e2999 100755
--- a/tests/qemu-iotests/030
+++ b/tests/qemu-iotests/030
@@ -354,14 +354,14 @@ class TestParallelOps(iotests.QMPTestCase):
 self.assert_qmp(result, 'error/desc',
 "Node 'node5' is busy: block device is in use by block job: 
commit")
 
+result = self.vm.qmp('block-job-set-speed', device='commit-drive0', 
speed=0)
+self.assert_qmp(result, 'return', {})
+
 event = self.vm.event_wait(name='BLOCK_JOB_READY')
 self.assert_qmp(event, 'data/device', 'commit-drive0')
 self.assert_qmp(event, 'data/type', 'commit')
 self.assert_qmp_absent(event, 'data/error')
 
-result = self.vm.qmp('block-job-set-speed', device='commit-drive0', 
speed=0)
-self.assert_qmp(result, 'return', {})
-
 result = self.vm.qmp('block-job-complete', device='commit-drive0')
 self.assert_qmp(result, 'return', {})
 
-- 
2.25.4

[PULL 17/51] block: Add BdrvChildRole to BdrvChild

[Kevin Wolf]

From: Max Reitz 

For now, it is always set to 0.  Later patches in this series will
ensure that all callers pass an appropriate combination of flags.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-6-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block.h   |  2 ++
 include/block/block_int.h   |  2 ++
 block.c | 11 ---
 block/backup-top.c  |  3 ++-
 block/blkdebug.c|  2 +-
 block/blklogwrites.c|  6 +++---
 block/blkreplay.c   |  2 +-
 block/blkverify.c   |  4 ++--
 block/block-backend.c   |  4 ++--
 block/bochs.c   |  2 +-
 block/cloop.c   |  2 +-
 block/copy-on-read.c|  2 +-
 block/crypto.c  |  2 +-
 block/dmg.c |  2 +-
 block/filter-compress.c |  2 +-
 block/parallels.c   |  2 +-
 block/qcow.c|  2 +-
 block/qcow2.c   |  6 +++---
 block/qed.c |  2 +-
 block/quorum.c  |  4 ++--
 block/raw-format.c  |  2 +-
 block/replication.c |  2 +-
 block/throttle.c|  2 +-
 block/vdi.c |  2 +-
 block/vhdx.c|  2 +-
 block/vmdk.c|  4 ++--
 block/vpc.c |  2 +-
 block/vvfat.c   |  2 +-
 blockjob.c  |  5 +++--
 tests/test-bdrv-drain.c | 20 +++-
 tests/test-bdrv-graph-mod.c |  4 ++--
 31 files changed, 62 insertions(+), 49 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index 35f1a1cecf..25e299605e 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -353,6 +353,7 @@ BdrvChild *bdrv_open_child(const char *filename,
QDict *options, const char *bdref_key,
BlockDriverState* parent,
const BdrvChildClass *child_class,
+   BdrvChildRole child_role,
bool allow_none, Error **errp);
 BlockDriverState *bdrv_open_blockdev_ref(BlockdevRef *ref, Error **errp);
 void bdrv_set_backing_hd(BlockDriverState *bs, BlockDriverState *backing_hd,
@@ -598,6 +599,7 @@ BdrvChild *bdrv_attach_child(BlockDriverState *parent_bs,
  BlockDriverState *child_bs,
  const char *child_name,
  const BdrvChildClass *child_class,
+ BdrvChildRole child_role,
  Error **errp);
 
 bool bdrv_op_is_blocked(BlockDriverState *bs, BlockOpType op, Error **errp);
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 9a78b981e4..1c6641c17a 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -746,6 +746,7 @@ struct BdrvChild {
 BlockDriverState *bs;
 char *name;
 const BdrvChildClass *klass;
+BdrvChildRole role;
 void *opaque;
 
 /**
@@ -1233,6 +1234,7 @@ BlockJob *backup_job_create(const char *job_id, 
BlockDriverState *bs,
 BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs,
   const char *child_name,
   const BdrvChildClass *child_class,
+  BdrvChildRole child_role,
   AioContext *ctx,
   uint64_t perm, uint64_t shared_perm,
   void *opaque, Error **errp);
diff --git a/block.c b/block.c
index 0ce9b61c97..14810e0426 100644
--- a/block.c
+++ b/block.c
@@ -2571,6 +2571,7 @@ static void bdrv_replace_child(BdrvChild *child, 
BlockDriverState *new_bs)
 BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs,
   const char *child_name,
   const BdrvChildClass *child_class,
+  BdrvChildRole child_role,
   AioContext *ctx,
   uint64_t perm, uint64_t shared_perm,
   void *opaque, Error **errp)
@@ -2592,6 +2593,7 @@ BdrvChild *bdrv_root_attach_child(BlockDriverState 
*child_bs,
 .bs = NULL,
 .name   = g_strdup(child_name),
 .klass  = child_class,
+.role   = child_role,
 .perm   = perm,
 .shared_perm= shared_perm,
 .opaque = opaque,
@@ -2644,6 +2646,7 @@ BdrvChild *bdrv_attach_child(BlockDriverState *parent_bs,
  BlockDriverState *child_bs,
  const char *child_name,
  const BdrvChildClass *child_class,
+ BdrvChildRole child_role,
  Error **errp)
 {
 BdrvChild *child;
@@ -2656,7 +2659,7 @@ BdrvChild *bdrv_attach_child(BlockDriverState *parent_bs,
 perm, 

[PULL 50/51] hw/ide/ahci: Log lost IRQs

[Kevin Wolf]

From: Philippe Mathieu-Daudé 

One might find interesting to look at AHCI IRQs.

Signed-off-by: Philippe Mathieu-Daudé 
Message-Id: <20200504094858.5975-1-f4...@amsat.org>
Reviewed-by: John Snow 
Acked-by: John Snow 
Signed-off-by: Kevin Wolf 
---
 hw/ide/ahci.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/ide/ahci.c b/hw/ide/ahci.c
index 13d91e109a..fc82cbd5f1 100644
--- a/hw/ide/ahci.c
+++ b/hw/ide/ahci.c
@@ -1509,6 +1509,7 @@ static void ahci_cmd_done(IDEDMA *dma)
 
 static void ahci_irq_set(void *opaque, int n, int level)
 {
+qemu_log_mask(LOG_UNIMP, "ahci: IRQ#%d level:%d\n", n, level);
 }
 
 static const IDEDMAOps ahci_dma_ops = {
-- 
2.25.4

[PULL 48/51] block/block-copy: Simplify block_copy_do_copy()

[Kevin Wolf]

From: Philippe Mathieu-Daudé 

block_copy_do_copy() is static, only used in block_copy_task_entry
with the error_is_read argument set. No need to check for it,
simplify.

Signed-off-by: Philippe Mathieu-Daudé 
Message-Id: <20200507121129.29760-3-phi...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 block/block-copy.c | 12 +++-
 1 file changed, 3 insertions(+), 9 deletions(-)

diff --git a/block/block-copy.c b/block/block-copy.c
index 7c7f7acbae..bb8d0569f2 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -343,9 +343,7 @@ static int coroutine_fn block_copy_do_copy(BlockCopyState 
*s,
 ~BDRV_REQ_WRITE_COMPRESSED);
 if (ret < 0) {
 trace_block_copy_write_zeroes_fail(s, offset, ret);
-if (error_is_read) {
-*error_is_read = false;
-}
+*error_is_read = false;
 }
 return ret;
 }
@@ -393,9 +391,7 @@ static int coroutine_fn block_copy_do_copy(BlockCopyState 
*s,
 ret = bdrv_co_pread(s->source, offset, nbytes, bounce_buffer, 0);
 if (ret < 0) {
 trace_block_copy_read_fail(s, offset, ret);
-if (error_is_read) {
-*error_is_read = true;
-}
+*error_is_read = true;
 goto out;
 }
 
@@ -403,9 +399,7 @@ static int coroutine_fn block_copy_do_copy(BlockCopyState 
*s,
  s->write_flags);
 if (ret < 0) {
 trace_block_copy_write_fail(s, offset, ret);
-if (error_is_read) {
-*error_is_read = false;
-}
+*error_is_read = false;
 goto out;
 }
 
-- 
2.25.4

[PULL 15/51] block: Rename BdrvChildRole to BdrvChildClass

[Kevin Wolf]

From: Max Reitz 

This structure nearly only contains parent callbacks for child state
changes.  It cannot really reflect a child's role, because different
roles may overlap (as we will see when real roles are introduced), and
because parents can have custom callbacks even when the child fulfills a
standard role.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
Message-Id: <20200513110544.176672-4-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block.h   |   6 +-
 include/block/block_int.h   |  22 +++---
 block.c | 142 ++--
 block/backup-top.c  |   8 +-
 block/blkdebug.c|   4 +-
 block/blklogwrites.c|   8 +-
 block/block-backend.c   |   6 +-
 block/commit.c  |   2 +-
 block/copy-on-read.c|   2 +-
 block/io.c  |  22 +++---
 block/mirror.c  |   2 +-
 block/quorum.c  |   2 +-
 block/replication.c |   2 +-
 block/vvfat.c   |   6 +-
 blockjob.c  |   2 +-
 tests/test-bdrv-drain.c |  36 -
 tests/test-bdrv-graph-mod.c |   2 +-
 17 files changed, 141 insertions(+), 133 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index 8f3eb70df4..bc42e507bb 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -13,7 +13,7 @@
 /* block.c */
 typedef struct BlockDriver BlockDriver;
 typedef struct BdrvChild BdrvChild;
-typedef struct BdrvChildRole BdrvChildRole;
+typedef struct BdrvChildClass BdrvChildClass;
 
 typedef struct BlockDriverInfo {
 /* in bytes, 0 if irrelevant */
@@ -296,7 +296,7 @@ int bdrv_parse_discard_flags(const char *mode, int *flags);
 BdrvChild *bdrv_open_child(const char *filename,
QDict *options, const char *bdref_key,
BlockDriverState* parent,
-   const BdrvChildRole *child_role,
+   const BdrvChildClass *child_class,
bool allow_none, Error **errp);
 BlockDriverState *bdrv_open_blockdev_ref(BlockdevRef *ref, Error **errp);
 void bdrv_set_backing_hd(BlockDriverState *bs, BlockDriverState *backing_hd,
@@ -541,7 +541,7 @@ void bdrv_unref_child(BlockDriverState *parent, BdrvChild 
*child);
 BdrvChild *bdrv_attach_child(BlockDriverState *parent_bs,
  BlockDriverState *child_bs,
  const char *child_name,
- const BdrvChildRole *child_role,
+ const BdrvChildClass *child_class,
  Error **errp);
 
 bool bdrv_op_is_blocked(BlockDriverState *bs, BlockOpType op, Error **errp);
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 1c24df53fd..9a78b981e4 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -556,14 +556,14 @@ struct BlockDriver {
  * the parents in @parent_perm and @parent_shared.
  *
  * If @c is NULL, return the permissions for attaching a new child for the
- * given @role.
+ * given @child_class.
  *
  * If @reopen_queue is non-NULL, don't return the currently needed
  * permissions, but those that will be needed after applying the
  * @reopen_queue.
  */
  void (*bdrv_child_perm)(BlockDriverState *bs, BdrvChild *c,
- const BdrvChildRole *role,
+ const BdrvChildClass *child_class,
  BlockReopenQueue *reopen_queue,
  uint64_t parent_perm, uint64_t parent_shared,
  uint64_t *nperm, uint64_t *nshared);
@@ -665,7 +665,7 @@ typedef struct BdrvAioNotifier {
 QLIST_ENTRY(BdrvAioNotifier) list;
 } BdrvAioNotifier;
 
-struct BdrvChildRole {
+struct BdrvChildClass {
 /* If true, bdrv_replace_node() doesn't change the node this BdrvChild
  * points to. */
 bool stay_at_node;
@@ -738,14 +738,14 @@ struct BdrvChildRole {
 void (*set_aio_ctx)(BdrvChild *child, AioContext *ctx, GSList **ignore);
 };
 
-extern const BdrvChildRole child_file;
-extern const BdrvChildRole child_format;
-extern const BdrvChildRole child_backing;
+extern const BdrvChildClass child_file;
+extern const BdrvChildClass child_format;
+extern const BdrvChildClass child_backing;
 
 struct BdrvChild {
 BlockDriverState *bs;
 char *name;
-const BdrvChildRole *role;
+const BdrvChildClass *klass;
 void *opaque;
 
 /**
@@ -772,7 +772,7 @@ struct BdrvChild {
 
 /*
  * How many times the parent of this child has been drained
- * (through role->drained_*).
+ * (through klass->drained_*).
  * Usually, this is equal to bs->quiesce_counter (potentially
  * reduced by bdrv_drain_all_count).  It may differ while the
  * child is entering or leaving a drained section.
@@ -1232,7 +1232,7 @@ BlockJob 

[PULL 46/51] block: Drop @child_class from bdrv_child_perm()

[Kevin Wolf]

From: Max Reitz 

Implementations should decide the necessary permissions based on @role.

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-35-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 include/block/block_int.h   |  4 +---
 block.c | 39 +++--
 block/backup-top.c  |  3 +--
 block/blkdebug.c|  3 +--
 block/blklogwrites.c|  3 +--
 block/commit.c  |  1 -
 block/copy-on-read.c|  1 -
 block/mirror.c  |  1 -
 block/quorum.c  |  1 -
 block/replication.c |  1 -
 block/vvfat.c   |  4 +---
 tests/test-bdrv-drain.c | 19 +-
 tests/test-bdrv-graph-mod.c |  1 -
 13 files changed, 22 insertions(+), 59 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 7fbe3206b4..5e4f4c348c 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -563,7 +563,6 @@ struct BlockDriver {
  * @reopen_queue.
  */
  void (*bdrv_child_perm)(BlockDriverState *bs, BdrvChild *c,
- const BdrvChildClass *child_class,
  BdrvChildRole role,
  BlockReopenQueue *reopen_queue,
  uint64_t parent_perm, uint64_t parent_shared,
@@ -1270,8 +1269,7 @@ bool bdrv_recurse_can_replace(BlockDriverState *bs,
  * child_of_bds child class and set an appropriate BdrvChildRole.
  */
 void bdrv_default_perms(BlockDriverState *bs, BdrvChild *c,
-const BdrvChildClass *child_class, BdrvChildRole role,
-BlockReopenQueue *reopen_queue,
+BdrvChildRole role, BlockReopenQueue *reopen_queue,
 uint64_t perm, uint64_t shared,
 uint64_t *nperm, uint64_t *nshared);
 
diff --git a/block.c b/block.c
index 6a24b23d51..8416376c9b 100644
--- a/block.c
+++ b/block.c
@@ -1947,13 +1947,13 @@ bool bdrv_is_writable(BlockDriverState *bs)
 }
 
 static void bdrv_child_perm(BlockDriverState *bs, BlockDriverState *child_bs,
-BdrvChild *c, const BdrvChildClass *child_class,
-BdrvChildRole role, BlockReopenQueue *reopen_queue,
+BdrvChild *c, BdrvChildRole role,
+BlockReopenQueue *reopen_queue,
 uint64_t parent_perm, uint64_t parent_shared,
 uint64_t *nperm, uint64_t *nshared)
 {
 assert(bs->drv && bs->drv->bdrv_child_perm);
-bs->drv->bdrv_child_perm(bs, c, child_class, role, reopen_queue,
+bs->drv->bdrv_child_perm(bs, c, role, reopen_queue,
  parent_perm, parent_shared,
  nperm, nshared);
 /* TODO Take force_share from reopen_queue */
@@ -2047,7 +2047,7 @@ static int bdrv_check_perm(BlockDriverState *bs, 
BlockReopenQueue *q,
 uint64_t cur_perm, cur_shared;
 bool child_tighten_restr;
 
-bdrv_child_perm(bs, c->bs, c, c->klass, c->role, q,
+bdrv_child_perm(bs, c->bs, c, c->role, q,
 cumulative_perms, cumulative_shared_perms,
 _perm, _shared);
 ret = bdrv_child_check_perm(c, q, cur_perm, cur_shared, 
ignore_children,
@@ -2114,7 +2114,7 @@ static void bdrv_set_perm(BlockDriverState *bs, uint64_t 
cumulative_perms,
 /* Update all children */
 QLIST_FOREACH(c, >children, next) {
 uint64_t cur_perm, cur_shared;
-bdrv_child_perm(bs, c->bs, c, c->klass, c->role, NULL,
+bdrv_child_perm(bs, c->bs, c, c->role, NULL,
 cumulative_perms, cumulative_shared_perms,
 _perm, _shared);
 bdrv_child_set_perm(c, cur_perm, cur_shared);
@@ -2342,7 +2342,7 @@ int bdrv_child_refresh_perms(BlockDriverState *bs, 
BdrvChild *c, Error **errp)
 uint64_t perms, shared;
 
 bdrv_get_cumulative_perm(bs, _perms, _shared);
-bdrv_child_perm(bs, c->bs, c, c->klass, c->role, NULL,
+bdrv_child_perm(bs, c->bs, c, c->role, NULL,
 parent_perms, parent_shared, , );
 
 return bdrv_child_try_set_perm(c, perms, shared, errp);
@@ -2354,7 +2354,6 @@ int bdrv_child_refresh_perms(BlockDriverState *bs, 
BdrvChild *c, Error **errp)
  * filtered child.
  */
 static void bdrv_filter_default_perms(BlockDriverState *bs, BdrvChild *c,
-  const BdrvChildClass *child_class,
   BdrvChildRole role,
   BlockReopenQueue *reopen_queue,
   uint64_t perm, uint64_t shared,
@@ -2365,13 +2364,12 @@ static void bdrv_filter_default_perms(BlockDriverState 
*bs, BdrvChild *c,
 }
 
 static void bdrv_default_perms_for_cow(BlockDriverState *bs, BdrvChild *c,
-

[PULL 47/51] block/block-copy: Fix uninitialized variable in block_copy_task_entry

[Kevin Wolf]

From: Philippe Mathieu-Daudé 

Fix when building with -Os:

CC  block/block-copy.o
  block/block-copy.c: In function ‘block_copy_task_entry’:
  block/block-copy.c:428:38: error: ‘error_is_read’ may be used uninitialized 
in this function [-Werror=maybe-uninitialized]
428 | t->call_state->error_is_read = error_is_read;
| ~^~~

Signed-off-by: Philippe Mathieu-Daudé 
Message-Id: <20200507121129.29760-2-phi...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 block/block-copy.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/block-copy.c b/block/block-copy.c
index 4713c8f2a3..7c7f7acbae 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -418,7 +418,7 @@ out:
 static coroutine_fn int block_copy_task_entry(AioTask *task)
 {
 BlockCopyTask *t = container_of(task, BlockCopyTask, task);
-bool error_is_read;
+bool error_is_read = false;
 int ret;
 
 ret = block_copy_do_copy(t->s, t->offset, t->bytes, t->zeroes,
-- 
2.25.4

[PULL 40/51] tests: Use child_of_bds instead of child_file

[Kevin Wolf]

From: Max Reitz 

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-29-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 tests/test-bdrv-drain.c | 29 +
 tests/test-bdrv-graph-mod.c |  6 --
 2 files changed, 21 insertions(+), 14 deletions(-)

diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c
index 15393a0140..91567ca97d 100644
--- a/tests/test-bdrv-drain.c
+++ b/tests/test-bdrv-drain.c
@@ -97,7 +97,7 @@ static void bdrv_test_child_perm(BlockDriverState *bs, 
BdrvChild *c,
  * detach_by_driver_cb_parent as one of them.
  */
 if (child_class != _file && child_class != _of_bds) {
-child_class = _file;
+child_class = _of_bds;
 }
 
 bdrv_format_default_perms(bs, c, child_class, role, reopen_queue,
@@ -1203,7 +1203,8 @@ static void do_test_delete_by_drain(bool 
detach_instead_of_delete,
 
 null_bs = bdrv_open("null-co://", NULL, NULL, BDRV_O_RDWR | 
BDRV_O_PROTOCOL,
 _abort);
-bdrv_attach_child(bs, null_bs, "null-child", _file, 0, _abort);
+bdrv_attach_child(bs, null_bs, "null-child", _of_bds,
+  BDRV_CHILD_DATA, _abort);
 
 /* This child will be the one to pass to requests through to, and
  * it will stall until a drain occurs */
@@ -1211,14 +1212,17 @@ static void do_test_delete_by_drain(bool 
detach_instead_of_delete,
 _abort);
 child_bs->total_sectors = 65536 >> BDRV_SECTOR_BITS;
 /* Takes our reference to child_bs */
-tts->wait_child = bdrv_attach_child(bs, child_bs, "wait-child", 
_file,
-0, _abort);
+tts->wait_child = bdrv_attach_child(bs, child_bs, "wait-child",
+_of_bds,
+BDRV_CHILD_DATA | BDRV_CHILD_PRIMARY,
+_abort);
 
 /* This child is just there to be deleted
  * (for detach_instead_of_delete == true) */
 null_bs = bdrv_open("null-co://", NULL, NULL, BDRV_O_RDWR | 
BDRV_O_PROTOCOL,
 _abort);
-bdrv_attach_child(bs, null_bs, "null-child", _file, 0, _abort);
+bdrv_attach_child(bs, null_bs, "null-child", _of_bds, 
BDRV_CHILD_DATA,
+  _abort);
 
 blk = blk_new(qemu_get_aio_context(), BLK_PERM_ALL, BLK_PERM_ALL);
 blk_insert_bs(blk, bs, _abort);
@@ -1315,7 +1319,8 @@ static void detach_indirect_bh(void *opaque)
 
 bdrv_ref(data->c);
 data->child_c = bdrv_attach_child(data->parent_b, data->c, "PB-C",
-  _file, 0, _abort);
+  _of_bds, BDRV_CHILD_DATA,
+  _abort);
 }
 
 static void detach_by_parent_aio_cb(void *opaque, int ret)
@@ -1332,7 +1337,7 @@ static void detach_by_driver_cb_drained_begin(BdrvChild 
*child)
 {
 aio_bh_schedule_oneshot(qemu_get_current_aio_context(),
 detach_indirect_bh, _by_parent_data);
-child_file.drained_begin(child);
+child_of_bds.drained_begin(child);
 }
 
 static BdrvChildClass detach_by_driver_cb_class;
@@ -1367,7 +1372,7 @@ static void test_detach_indirect(bool by_parent_cb)
 QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, NULL, 0);
 
 if (!by_parent_cb) {
-detach_by_driver_cb_class = child_file;
+detach_by_driver_cb_class = child_of_bds;
 detach_by_driver_cb_class.drained_begin =
 detach_by_driver_cb_drained_begin;
 }
@@ -1397,15 +1402,15 @@ static void test_detach_indirect(bool by_parent_cb)
 /* Set child relationships */
 bdrv_ref(b);
 bdrv_ref(a);
-child_b = bdrv_attach_child(parent_b, b, "PB-B", _file, 0,
-_abort);
+child_b = bdrv_attach_child(parent_b, b, "PB-B", _of_bds,
+BDRV_CHILD_DATA, _abort);
 child_a = bdrv_attach_child(parent_b, a, "PB-A", _of_bds,
 BDRV_CHILD_COW, _abort);
 
 bdrv_ref(a);
 bdrv_attach_child(parent_a, a, "PA-A",
-  by_parent_cb ? _file : _by_driver_cb_class,
-  0, _abort);
+  by_parent_cb ? _of_bds : 
_by_driver_cb_class,
+  BDRV_CHILD_DATA, _abort);
 
 g_assert_cmpint(parent_a->refcnt, ==, 1);
 g_assert_cmpint(parent_b->refcnt, ==, 1);
diff --git a/tests/test-bdrv-graph-mod.c b/tests/test-bdrv-graph-mod.c
index 3707e2533c..6ae91ff171 100644
--- a/tests/test-bdrv-graph-mod.c
+++ b/tests/test-bdrv-graph-mod.c
@@ -112,7 +112,8 @@ static void test_update_perm_tree(void)
 
 blk_insert_bs(root, bs, _abort);
 
-bdrv_attach_child(filter, bs, "child", _file, 0, _abort);
+bdrv_attach_child(filter, bs, "child", _of_bds,
+  BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, _abort);
 
 bdrv_append(filter, bs, _err);
 
@@ -178,7 +179,8 @@ static void 

[PULL 45/51] block: Pass BdrvChildRole in remaining cases

[Kevin Wolf]

From: Max Reitz 

These calls have no real use for the child role yet, but it will not
harm to give one.

Notably, the bdrv_root_attach_child() call in blockjob.c is left
unmodified because there is not much the generic BlockJob object wants
from its children.

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-34-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 block/block-backend.c | 11 +++
 block/vvfat.c |  4 +++-
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/block/block-backend.c b/block/block-backend.c
index f2e81af27d..6936b25c83 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -424,8 +424,9 @@ BlockBackend *blk_new_open(const char *filename, const char 
*reference,
 return NULL;
 }
 
-blk->root = bdrv_root_attach_child(bs, "root", _root, 0, blk->ctx,
-   perm, BLK_PERM_ALL, blk, errp);
+blk->root = bdrv_root_attach_child(bs, "root", _root,
+   BDRV_CHILD_FILTERED | 
BDRV_CHILD_PRIMARY,
+   blk->ctx, perm, BLK_PERM_ALL, blk, 
errp);
 if (!blk->root) {
 blk_unref(blk);
 return NULL;
@@ -835,8 +836,10 @@ int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, 
Error **errp)
 {
 ThrottleGroupMember *tgm = >public.throttle_group_member;
 bdrv_ref(bs);
-blk->root = bdrv_root_attach_child(bs, "root", _root, 0, blk->ctx,
-   blk->perm, blk->shared_perm, blk, errp);
+blk->root = bdrv_root_attach_child(bs, "root", _root,
+   BDRV_CHILD_FILTERED | 
BDRV_CHILD_PRIMARY,
+   blk->ctx, blk->perm, blk->shared_perm,
+   blk, errp);
 if (blk->root == NULL) {
 return -EPERM;
 }
diff --git a/block/vvfat.c b/block/vvfat.c
index e8848a0497..089abe1e29 100644
--- a/block/vvfat.c
+++ b/block/vvfat.c
@@ -3184,7 +3184,9 @@ static int enable_write_target(BlockDriverState *bs, 
Error **errp)
 options = qdict_new();
 qdict_put_str(options, "write-target.driver", "qcow");
 s->qcow = bdrv_open_child(s->qcow_filename, options, "write-target", bs,
-  _vvfat_qcow, 0, false, errp);
+  _vvfat_qcow,
+  BDRV_CHILD_DATA | BDRV_CHILD_METADATA,
+  false, errp);
 qobject_unref(options);
 if (!s->qcow) {
 ret = -EINVAL;
-- 
2.25.4

[PULL 24/51] block: Unify bdrv_child_cb_attach()

[Kevin Wolf]

From: Max Reitz 

Make bdrv_child_cb_attach() call bdrv_backing_attach() for children with
a COW role (and drop the reverse call from bdrv_backing_attach()), so it
can be used for any child (with a proper role set).

Because so far no child has a proper role set, we need a temporary new
callback for child_backing.attach that ensures bdrv_backing_attach() is
called for all COW children that do not have their role set yet.

(Also, move bdrv_child_cb_attach() down to group it with
bdrv_inherited_options().)

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-13-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 block.c | 27 ---
 1 file changed, 20 insertions(+), 7 deletions(-)

diff --git a/block.c b/block.c
index b3b978a092..755704a54c 100644
--- a/block.c
+++ b/block.c
@@ -84,6 +84,7 @@ static BlockDriverState *bdrv_open_inherit(const char 
*filename,
 static void bdrv_inherited_options(BdrvChildRole role, bool parent_is_format,
int *child_flags, QDict *child_options,
int parent_flags, QDict *parent_options);
+static void bdrv_child_cb_attach(BdrvChild *child);
 
 /* If non-zero, use only whitelisted block drivers */
 static int use_bdrv_whitelist;
@@ -1099,12 +1100,6 @@ static void bdrv_child_cb_drained_end(BdrvChild *child,
 bdrv_drained_end_no_poll(bs, drained_end_counter);
 }
 
-static void bdrv_child_cb_attach(BdrvChild *child)
-{
-BlockDriverState *bs = child->opaque;
-bdrv_apply_subtree_drain(child, bs);
-}
-
 static void bdrv_child_cb_detach(BdrvChild *child)
 {
 BlockDriverState *bs = child->opaque;
@@ -1252,7 +1247,14 @@ static void bdrv_backing_attach(BdrvChild *c)
 parent->backing_blocker);
 bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_BACKUP_TARGET,
 parent->backing_blocker);
+}
 
+/* XXX: Will be removed along with child_backing */
+static void bdrv_child_cb_attach_backing(BdrvChild *c)
+{
+if (!(c->role & BDRV_CHILD_COW)) {
+bdrv_backing_attach(c);
+}
 bdrv_child_cb_attach(c);
 }
 
@@ -1311,7 +1313,7 @@ static int bdrv_backing_update_filename(BdrvChild *c, 
BlockDriverState *base,
 const BdrvChildClass child_backing = {
 .parent_is_bds   = true,
 .get_parent_desc = bdrv_child_get_parent_desc,
-.attach  = bdrv_backing_attach,
+.attach  = bdrv_child_cb_attach_backing,
 .detach  = bdrv_backing_detach,
 .inherit_options = bdrv_backing_options,
 .drained_begin   = bdrv_child_cb_drained_begin,
@@ -1403,6 +1405,17 @@ static void bdrv_inherited_options(BdrvChildRole role, 
bool parent_is_format,
 *child_flags = flags;
 }
 
+static void bdrv_child_cb_attach(BdrvChild *child)
+{
+BlockDriverState *bs = child->opaque;
+
+if (child->role & BDRV_CHILD_COW) {
+bdrv_backing_attach(child);
+}
+
+bdrv_apply_subtree_drain(child, bs);
+}
+
 static int bdrv_open_flags(BlockDriverState *bs, int flags)
 {
 int open_flags = flags;
-- 
2.25.4

[PULL 18/51] block: Pass BdrvChildRole to bdrv_child_perm()

[Kevin Wolf]

From: Max Reitz 

For now, all callers pass 0 and no callee evaluates this value.  Later
patches will change both.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-7-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block_int.h   |  5 -
 block.c | 22 --
 block/backup-top.c  |  3 ++-
 block/blkdebug.c|  5 +++--
 block/blklogwrites.c|  9 +
 block/commit.c  |  1 +
 block/copy-on-read.c|  1 +
 block/mirror.c  |  1 +
 block/quorum.c  |  1 +
 block/replication.c |  1 +
 block/vvfat.c   |  1 +
 tests/test-bdrv-drain.c |  5 +++--
 tests/test-bdrv-graph-mod.c |  1 +
 13 files changed, 36 insertions(+), 20 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 1c6641c17a..dc4bc486d6 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -556,7 +556,7 @@ struct BlockDriver {
  * the parents in @parent_perm and @parent_shared.
  *
  * If @c is NULL, return the permissions for attaching a new child for the
- * given @child_class.
+ * given @child_class and @role.
  *
  * If @reopen_queue is non-NULL, don't return the currently needed
  * permissions, but those that will be needed after applying the
@@ -564,6 +564,7 @@ struct BlockDriver {
  */
  void (*bdrv_child_perm)(BlockDriverState *bs, BdrvChild *c,
  const BdrvChildClass *child_class,
+ BdrvChildRole role,
  BlockReopenQueue *reopen_queue,
  uint64_t parent_perm, uint64_t parent_shared,
  uint64_t *nperm, uint64_t *nshared);
@@ -1266,6 +1267,7 @@ int bdrv_child_refresh_perms(BlockDriverState *bs, 
BdrvChild *c, Error **errp);
  * all children */
 void bdrv_filter_default_perms(BlockDriverState *bs, BdrvChild *c,
const BdrvChildClass *child_class,
+   BdrvChildRole child_role,
BlockReopenQueue *reopen_queue,
uint64_t perm, uint64_t shared,
uint64_t *nperm, uint64_t *nshared);
@@ -1276,6 +1278,7 @@ void bdrv_filter_default_perms(BlockDriverState *bs, 
BdrvChild *c,
  * CONSISTENT_READ and doesn't share WRITE. */
 void bdrv_format_default_perms(BlockDriverState *bs, BdrvChild *c,
const BdrvChildClass *child_class,
+   BdrvChildRole child_role,
BlockReopenQueue *reopen_queue,
uint64_t perm, uint64_t shared,
uint64_t *nperm, uint64_t *nshared);
diff --git a/block.c b/block.c
index 14810e0426..5d9a331f5b 100644
--- a/block.c
+++ b/block.c
@@ -1954,12 +1954,12 @@ bool bdrv_is_writable(BlockDriverState *bs)
 
 static void bdrv_child_perm(BlockDriverState *bs, BlockDriverState *child_bs,
 BdrvChild *c, const BdrvChildClass *child_class,
-BlockReopenQueue *reopen_queue,
+BdrvChildRole role, BlockReopenQueue *reopen_queue,
 uint64_t parent_perm, uint64_t parent_shared,
 uint64_t *nperm, uint64_t *nshared)
 {
 assert(bs->drv && bs->drv->bdrv_child_perm);
-bs->drv->bdrv_child_perm(bs, c, child_class, reopen_queue,
+bs->drv->bdrv_child_perm(bs, c, child_class, role, reopen_queue,
  parent_perm, parent_shared,
  nperm, nshared);
 /* TODO Take force_share from reopen_queue */
@@ -2053,7 +2053,7 @@ static int bdrv_check_perm(BlockDriverState *bs, 
BlockReopenQueue *q,
 uint64_t cur_perm, cur_shared;
 bool child_tighten_restr;
 
-bdrv_child_perm(bs, c->bs, c, c->klass, q,
+bdrv_child_perm(bs, c->bs, c, c->klass, c->role, q,
 cumulative_perms, cumulative_shared_perms,
 _perm, _shared);
 ret = bdrv_child_check_perm(c, q, cur_perm, cur_shared, 
ignore_children,
@@ -2120,7 +2120,7 @@ static void bdrv_set_perm(BlockDriverState *bs, uint64_t 
cumulative_perms,
 /* Update all children */
 QLIST_FOREACH(c, >children, next) {
 uint64_t cur_perm, cur_shared;
-bdrv_child_perm(bs, c->bs, c, c->klass, NULL,
+bdrv_child_perm(bs, c->bs, c, c->klass, c->role, NULL,
 cumulative_perms, cumulative_shared_perms,
 _perm, _shared);
 bdrv_child_set_perm(c, cur_perm, cur_shared);
@@ -2348,14 +2348,15 @@ int bdrv_child_refresh_perms(BlockDriverState *bs, 
BdrvChild *c, Error **errp)
 uint64_t perms, shared;
 
 bdrv_get_cumulative_perm(bs, _perms, _shared);
-bdrv_child_perm(bs, 

[PULL 22/51] block: Add generic bdrv_inherited_options()

[Kevin Wolf]

From: Max Reitz 

After the series this patch belongs to, we want to have a common
BdrvChildClass that encompasses all of child_file, child_format, and
child_backing.  Such a single class needs a single .inherit_options()
implementation, and this patch introduces it.

The next patch will show how the existing implementations can fall back
to it just by passing appropriate BdrvChildRole and parent_is_format
values.

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-11-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 block.c | 81 +
 1 file changed, 81 insertions(+)

diff --git a/block.c b/block.c
index a17c29f54b..9b0e13d537 100644
--- a/block.c
+++ b/block.c
@@ -1356,6 +1356,87 @@ const BdrvChildClass child_backing = {
 .set_aio_ctx = bdrv_child_cb_set_aio_ctx,
 };
 
+/*
+ * Returns the options and flags that a generic child of a BDS should
+ * get, based on the given options and flags for the parent BDS.
+ */
+static void __attribute__((unused))
+bdrv_inherited_options(BdrvChildRole role, bool parent_is_format,
+   int *child_flags, QDict *child_options,
+   int parent_flags, QDict *parent_options)
+{
+int flags = parent_flags;
+
+/*
+ * First, decide whether to set, clear, or leave BDRV_O_PROTOCOL.
+ * Generally, the question to answer is: Should this child be
+ * format-probed by default?
+ */
+
+/*
+ * Pure and non-filtered data children of non-format nodes should
+ * be probed by default (even when the node itself has BDRV_O_PROTOCOL
+ * set).  This only affects a very limited set of drivers (namely
+ * quorum and blkverify when this comment was written).
+ * Force-clear BDRV_O_PROTOCOL then.
+ */
+if (!parent_is_format &&
+(role & BDRV_CHILD_DATA) &&
+!(role & (BDRV_CHILD_METADATA | BDRV_CHILD_FILTERED)))
+{
+flags &= ~BDRV_O_PROTOCOL;
+}
+
+/*
+ * All children of format nodes (except for COW children) and all
+ * metadata children in general should never be format-probed.
+ * Force-set BDRV_O_PROTOCOL then.
+ */
+if ((parent_is_format && !(role & BDRV_CHILD_COW)) ||
+(role & BDRV_CHILD_METADATA))
+{
+flags |= BDRV_O_PROTOCOL;
+}
+
+/*
+ * If the cache mode isn't explicitly set, inherit direct and no-flush from
+ * the parent.
+ */
+qdict_copy_default(child_options, parent_options, BDRV_OPT_CACHE_DIRECT);
+qdict_copy_default(child_options, parent_options, BDRV_OPT_CACHE_NO_FLUSH);
+qdict_copy_default(child_options, parent_options, BDRV_OPT_FORCE_SHARE);
+
+if (role & BDRV_CHILD_COW) {
+/* backing files are opened read-only by default */
+qdict_set_default_str(child_options, BDRV_OPT_READ_ONLY, "on");
+qdict_set_default_str(child_options, BDRV_OPT_AUTO_READ_ONLY, "off");
+} else {
+/* Inherit the read-only option from the parent if it's not set */
+qdict_copy_default(child_options, parent_options, BDRV_OPT_READ_ONLY);
+qdict_copy_default(child_options, parent_options,
+   BDRV_OPT_AUTO_READ_ONLY);
+}
+
+/*
+ * bdrv_co_pdiscard() respects unmap policy for the parent, so we
+ * can default to enable it on lower layers regardless of the
+ * parent option.
+ */
+qdict_set_default_str(child_options, BDRV_OPT_DISCARD, "unmap");
+
+/* Clear flags that only apply to the top layer */
+flags &= ~(BDRV_O_SNAPSHOT | BDRV_O_NO_BACKING | BDRV_O_COPY_ON_READ);
+
+if (role & BDRV_CHILD_METADATA) {
+flags &= ~BDRV_O_NO_IO;
+}
+if (role & BDRV_CHILD_COW) {
+flags &= ~BDRV_O_TEMPORARY;
+}
+
+*child_flags = flags;
+}
+
 static int bdrv_open_flags(BlockDriverState *bs, int flags)
 {
 int open_flags = flags;
-- 
2.25.4

[PULL 23/51] block: Use bdrv_inherited_options()

[Kevin Wolf]

From: Max Reitz 

Let child_file's, child_format's, and child_backing's .inherit_options()
implementations fall back to bdrv_inherited_options() to show that it
would really work for all of these cases, if only the parents passed the
appropriate BdrvChildRole and parent_is_format values.

(Also, make bdrv_open_inherit(), the only place to explicitly call
bdrv_backing_options(), call bdrv_inherited_options() instead.)

This patch should incur only two visible changes, both for child_format
children, both of which are effectively bug fixes:

First, they no longer have discard=unmap set by default.  This reason it
was set is because bdrv_inherited_fmt_options() fell through to
bdrv_protocol_options(), and that set it because "format drivers take
care to send flushes and respect unmap policy".  None of the drivers
that use child_format for their children (quorum and blkverify) are
format drivers, though, so this reasoning does not apply here.

Second, they no longer have BDRV_O_NO_IO force-cleared.  child_format
was used solely for children that do not store any metadata and as such
will not be accessed by their parents as long as those parents do not
receive I/O themselves.  Thus, such children should inherit
BDRV_O_NO_IO.

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-12-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 block.c | 71 +++--
 1 file changed, 19 insertions(+), 52 deletions(-)

diff --git a/block.c b/block.c
index 9b0e13d537..b3b978a092 100644
--- a/block.c
+++ b/block.c
@@ -80,6 +80,11 @@ static BlockDriverState *bdrv_open_inherit(const char 
*filename,
BdrvChildRole child_role,
Error **errp);
 
+/* TODO: Remove when no longer needed */
+static void bdrv_inherited_options(BdrvChildRole role, bool parent_is_format,
+   int *child_flags, QDict *child_options,
+   int parent_flags, QDict *parent_options);
+
 /* If non-zero, use only whitelisted block drivers */
 static int use_bdrv_whitelist;
 
@@ -1158,31 +1163,9 @@ static void bdrv_protocol_options(BdrvChildRole role, 
bool parent_is_format,
   int *child_flags, QDict *child_options,
   int parent_flags, QDict *parent_options)
 {
-int flags = parent_flags;
-
-/* Enable protocol handling, disable format probing for bs->file */
-flags |= BDRV_O_PROTOCOL;
-
-/* If the cache mode isn't explicitly set, inherit direct and no-flush from
- * the parent. */
-qdict_copy_default(child_options, parent_options, BDRV_OPT_CACHE_DIRECT);
-qdict_copy_default(child_options, parent_options, BDRV_OPT_CACHE_NO_FLUSH);
-qdict_copy_default(child_options, parent_options, BDRV_OPT_FORCE_SHARE);
-
-/* Inherit the read-only option from the parent if it's not set */
-qdict_copy_default(child_options, parent_options, BDRV_OPT_READ_ONLY);
-qdict_copy_default(child_options, parent_options, BDRV_OPT_AUTO_READ_ONLY);
-
-/* Our block drivers take care to send flushes and respect unmap policy,
- * so we can default to enable both on lower layers regardless of the
- * corresponding parent options. */
-qdict_set_default_str(child_options, BDRV_OPT_DISCARD, "unmap");
-
-/* Clear flags that only apply to the top layer */
-flags &= ~(BDRV_O_SNAPSHOT | BDRV_O_NO_BACKING | BDRV_O_COPY_ON_READ |
-   BDRV_O_NO_IO);
-
-*child_flags = flags;
+bdrv_inherited_options(BDRV_CHILD_IMAGE, true,
+   child_flags, child_options,
+   parent_flags, parent_options);
 }
 
 const BdrvChildClass child_file = {
@@ -1209,11 +1192,9 @@ static void bdrv_inherited_fmt_options(BdrvChildRole 
role,
int *child_flags, QDict *child_options,
int parent_flags, QDict *parent_options)
 {
-child_file.inherit_options(role, parent_is_format,
-   child_flags, child_options,
-   parent_flags, parent_options);
-
-*child_flags &= ~(BDRV_O_PROTOCOL | BDRV_O_NO_IO);
+bdrv_inherited_options(BDRV_CHILD_DATA, false,
+   child_flags, child_options,
+   parent_flags, parent_options);
 }
 
 const BdrvChildClass child_format = {
@@ -1295,23 +1276,9 @@ static void bdrv_backing_options(BdrvChildRole role, 
bool parent_is_format,
  int *child_flags, QDict *child_options,
  int parent_flags, QDict *parent_options)
 {
-int flags = parent_flags;
-
-/* The cache mode is inherited unmodified for backing files; except WCE,
- * which is only applied on the top level (BlockBackend) */
-qdict_copy_default(child_options, parent_options, 

[PULL 16/51] block: Add BdrvChildRole and BdrvChildRoleBits

[Kevin Wolf]

From: Max Reitz 

This mask will supplement BdrvChildClass when it comes to what role (or
combination of roles) a child takes for its parent.  It consists of
BdrvChildRoleBits values (which is an enum).

Because empty enums are not allowed, let us just start with it filled.

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-5-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 include/block/block.h | 56 +++
 1 file changed, 56 insertions(+)

diff --git a/include/block/block.h b/include/block/block.h
index bc42e507bb..35f1a1cecf 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -268,6 +268,62 @@ enum {
 DEFAULT_PERM_UNCHANGED  = BLK_PERM_ALL & ~DEFAULT_PERM_PASSTHROUGH,
 };
 
+/*
+ * Flags that parent nodes assign to child nodes to specify what kind of
+ * role(s) they take.
+ *
+ * At least one of DATA, METADATA, FILTERED, or COW must be set for
+ * every child.
+ */
+enum BdrvChildRoleBits {
+/*
+ * This child stores data.
+ * Any node may have an arbitrary number of such children.
+ */
+BDRV_CHILD_DATA = (1 << 0),
+
+/*
+ * This child stores metadata.
+ * Any node may have an arbitrary number of metadata-storing
+ * children.
+ */
+BDRV_CHILD_METADATA = (1 << 1),
+
+/*
+ * A child that always presents exactly the same visible data as
+ * the parent, e.g. by virtue of the parent forwarding all reads
+ * and writes.
+ * This flag is mutually exclusive with DATA, METADATA, and COW.
+ * Any node may have at most one filtered child at a time.
+ */
+BDRV_CHILD_FILTERED = (1 << 2),
+
+/*
+ * Child from which to read all data that isn’t allocated in the
+ * parent (i.e., the backing child); such data is copied to the
+ * parent through COW (and optionally COR).
+ * This field is mutually exclusive with DATA, METADATA, and
+ * FILTERED.
+ * Any node may have at most one such backing child at a time.
+ */
+BDRV_CHILD_COW  = (1 << 3),
+
+/*
+ * The primary child.  For most drivers, this is the child whose
+ * filename applies best to the parent node.
+ * Any node may have at most one primary child at a time.
+ */
+BDRV_CHILD_PRIMARY  = (1 << 4),
+
+/* Useful combination of flags */
+BDRV_CHILD_IMAGE= BDRV_CHILD_DATA
+  | BDRV_CHILD_METADATA
+  | BDRV_CHILD_PRIMARY,
+};
+
+/* Mask of BdrvChildRoleBits values */
+typedef unsigned int BdrvChildRole;
+
 char *bdrv_perm_names(uint64_t perm);
 uint64_t bdrv_qapi_perm_to_blk_perm(BlockPermission qapi_perm);
 
-- 
2.25.4

[PULL 19/51] block: Pass BdrvChildRole to .inherit_options()

[Kevin Wolf]

From: Max Reitz 

For now, all callers (effectively) pass 0 and no callee evaluates thie
value.  Later patches will change both.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-8-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block_int.h |  3 ++-
 block.c   | 40 +++
 block/block-backend.c |  3 ++-
 block/vvfat.c |  3 ++-
 4 files changed, 30 insertions(+), 19 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index dc4bc486d6..8c1160a577 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -677,7 +677,8 @@ struct BdrvChildClass {
  * non-BDS parents. */
 bool parent_is_bds;
 
-void (*inherit_options)(int *child_flags, QDict *child_options,
+void (*inherit_options)(BdrvChildRole role,
+int *child_flags, QDict *child_options,
 int parent_flags, QDict *parent_options);
 
 void (*change_media)(BdrvChild *child, bool load);
diff --git a/block.c b/block.c
index 5d9a331f5b..40db0b78b9 100644
--- a/block.c
+++ b/block.c
@@ -77,6 +77,7 @@ static BlockDriverState *bdrv_open_inherit(const char 
*filename,
QDict *options, int flags,
BlockDriverState *parent,
const BdrvChildClass *child_class,
+   BdrvChildRole child_role,
Error **errp);
 
 /* If non-zero, use only whitelisted block drivers */
@@ -1153,7 +1154,8 @@ static void bdrv_temp_snapshot_options(int *child_flags, 
QDict *child_options,
  * Returns the options and flags that bs->file should get if a protocol driver
  * is expected, based on the given options and flags for the parent BDS
  */
-static void bdrv_inherited_options(int *child_flags, QDict *child_options,
+static void bdrv_inherited_options(BdrvChildRole role,
+   int *child_flags, QDict *child_options,
int parent_flags, QDict *parent_options)
 {
 int flags = parent_flags;
@@ -1202,10 +1204,11 @@ const BdrvChildClass child_file = {
  * (and not only protocols) is permitted for it, based on the given options and
  * flags for the parent BDS
  */
-static void bdrv_inherited_fmt_options(int *child_flags, QDict *child_options,
+static void bdrv_inherited_fmt_options(BdrvChildRole role,
+   int *child_flags, QDict *child_options,
int parent_flags, QDict *parent_options)
 {
-child_file.inherit_options(child_flags, child_options,
+child_file.inherit_options(role, child_flags, child_options,
parent_flags, parent_options);
 
 *child_flags &= ~(BDRV_O_PROTOCOL | BDRV_O_NO_IO);
@@ -1286,7 +1289,8 @@ static void bdrv_backing_detach(BdrvChild *c)
  * Returns the options and flags that bs->backing should get, based on the
  * given options and flags for the parent BDS
  */
-static void bdrv_backing_options(int *child_flags, QDict *child_options,
+static void bdrv_backing_options(BdrvChildRole role,
+ int *child_flags, QDict *child_options,
  int parent_flags, QDict *parent_options)
 {
 int flags = parent_flags;
@@ -2876,7 +2880,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict 
*parent_options,
 }
 
 backing_hd = bdrv_open_inherit(backing_filename, reference, options, 0, bs,
-   _backing, errp);
+   _backing, 0, errp);
 if (!backing_hd) {
 bs->open_flags |= BDRV_O_NO_BACKING;
 error_prepend(errp, "Could not open backing file: ");
@@ -2911,7 +2915,7 @@ free_exit:
 static BlockDriverState *
 bdrv_open_child_bs(const char *filename, QDict *options, const char *bdref_key,
BlockDriverState *parent, const BdrvChildClass *child_class,
-   bool allow_none, Error **errp)
+   BdrvChildRole child_role, bool allow_none, Error **errp)
 {
 BlockDriverState *bs = NULL;
 QDict *image_options;
@@ -2942,7 +2946,7 @@ bdrv_open_child_bs(const char *filename, QDict *options, 
const char *bdref_key,
 }
 
 bs = bdrv_open_inherit(filename, reference, image_options, 0,
-   parent, child_class, errp);
+   parent, child_class, child_role, errp);
 if (!bs) {
 goto done;
 }
@@ -2976,7 +2980,7 @@ BdrvChild *bdrv_open_child(const char *filename,
 BlockDriverState *bs;
 
 bs = bdrv_open_child_bs(filename, options, bdref_key, parent, child_class,
-allow_none, errp);
+child_role, allow_none, errp);
 if (bs == NULL) {
 return 

[PULL 30/51] block: Relax *perms_for_storage for data children

[Kevin Wolf]

From: Max Reitz 

We can be less restrictive about pure data children than those with
metadata on them, so let bdrv_default_perms_for_storage() handle
metadata children differently from pure data children.

As explained in the code, the restrictions on metadata children are
strictly stricter than those for pure data children, so in theory we
just have to distinguish between pure-data and all other storage
children (pure metadata or data+metadata).  In practice, that is not
obvious, though, so we have two independent code paths for metadata and
for data children, and data+metadata children will go through both
(without the path for data children doing anything meaningful).

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-19-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 block.c | 58 +++--
 1 file changed, 48 insertions(+), 10 deletions(-)

diff --git a/block.c b/block.c
index 5d17aa1cc3..5ff6cbd796 100644
--- a/block.c
+++ b/block.c
@@ -2528,19 +2528,57 @@ static void 
bdrv_default_perms_for_storage(BlockDriverState *bs, BdrvChild *c,
 bdrv_filter_default_perms(bs, c, child_class, role, reopen_queue,
   perm, shared, , );
 
-/* Format drivers may touch metadata even if the guest doesn't write */
-if (bdrv_is_writable_after_reopen(bs, reopen_queue)) {
-perm |= BLK_PERM_WRITE | BLK_PERM_RESIZE;
+if (role & BDRV_CHILD_METADATA) {
+/* Format drivers may touch metadata even if the guest doesn't write */
+if (bdrv_is_writable_after_reopen(bs, reopen_queue)) {
+perm |= BLK_PERM_WRITE | BLK_PERM_RESIZE;
+}
+
+/*
+ * bs->file always needs to be consistent because of the
+ * metadata. We can never allow other users to resize or write
+ * to it.
+ */
+if (!(flags & BDRV_O_NO_IO)) {
+perm |= BLK_PERM_CONSISTENT_READ;
+}
+shared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
 }
 
-/*
- * bs->file always needs to be consistent because of the metadata. We
- * can never allow other users to resize or write to it.
- */
-if (!(flags & BDRV_O_NO_IO)) {
-perm |= BLK_PERM_CONSISTENT_READ;
+if (role & BDRV_CHILD_DATA) {
+/*
+ * Technically, everything in this block is a subset of the
+ * BDRV_CHILD_METADATA path taken above, and so this could
+ * be an "else if" branch.  However, that is not obvious, and
+ * this function is not performance critical, therefore we let
+ * this be an independent "if".
+ */
+
+/*
+ * We cannot allow other users to resize the file because the
+ * format driver might have some assumptions about the size
+ * (e.g. because it is stored in metadata, or because the file
+ * is split into fixed-size data files).
+ */
+shared &= ~BLK_PERM_RESIZE;
+
+/*
+ * WRITE_UNCHANGED often cannot be performed as such on the
+ * data file.  For example, the qcow2 driver may still need to
+ * write copied clusters on copy-on-read.
+ */
+if (perm & BLK_PERM_WRITE_UNCHANGED) {
+perm |= BLK_PERM_WRITE;
+}
+
+/*
+ * If the data file is written to, the format driver may
+ * expect to be able to resize it by writing beyond the EOF.
+ */
+if (perm & BLK_PERM_WRITE) {
+perm |= BLK_PERM_RESIZE;
+}
 }
-shared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
 
 if (bs->open_flags & BDRV_O_INACTIVE) {
 shared |= BLK_PERM_WRITE | BLK_PERM_RESIZE;
-- 
2.25.4

[PULL 11/51] replication: Avoid blk_make_empty() on read-only child

[Kevin Wolf]

This is just a bandaid to keep tests/test-replication working after
bdrv_make_empty() starts to assert that we're not trying to call it on a
read-only child.

For the real solution in the future, replication should not steal the
BdrvChild from its backing file (this is never correct to do!), but
instead have its own child node references, with the appropriate
permissions.

Signed-off-by: Kevin Wolf 
---
 block/replication.c | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/block/replication.c b/block/replication.c
index c03980a192..eb480a8e08 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -343,9 +343,18 @@ static void secondary_do_checkpoint(BDRVReplicationState 
*s, Error **errp)
 return;
 }
 
-ret = s->hidden_disk->bs->drv->bdrv_make_empty(s->hidden_disk->bs);
+BlockBackend *blk = blk_new(qemu_get_current_aio_context(),
+BLK_PERM_WRITE, BLK_PERM_ALL);
+blk_insert_bs(blk, s->hidden_disk->bs, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+blk_unref(blk);
+return;
+}
+
+ret = blk_make_empty(blk, errp);
+blk_unref(blk);
 if (ret < 0) {
-error_setg(errp, "Cannot make hidden disk empty");
 return;
 }
 }
-- 
2.25.4

[PULL 31/51] block: Add bdrv_default_perms()

[Kevin Wolf]

From: Max Reitz 

This callback can be used by BDSs that use child_of_bds with the
appropriate BdrvChildRole for their children.

Also, make bdrv_format_default_perms() use it for child_of_bds children
(just a temporary solution until we can drop bdrv_format_default_perms()
altogether).

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-20-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 include/block/block_int.h | 11 +++
 block.c   | 32 
 2 files changed, 43 insertions(+)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 54df821d61..3a9dda9be7 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1288,6 +1288,17 @@ void bdrv_format_default_perms(BlockDriverState *bs, 
BdrvChild *c,
 bool bdrv_recurse_can_replace(BlockDriverState *bs,
   BlockDriverState *to_replace);
 
+/*
+ * Default implementation for BlockDriver.bdrv_child_perm() that can
+ * be used by block filters and image formats, as long as they use the
+ * child_of_bds child class and set an appropriate BdrvChildRole.
+ */
+void bdrv_default_perms(BlockDriverState *bs, BdrvChild *c,
+const BdrvChildClass *child_class, BdrvChildRole role,
+BlockReopenQueue *reopen_queue,
+uint64_t perm, uint64_t shared,
+uint64_t *nperm, uint64_t *nshared);
+
 /*
  * Default implementation for drivers to pass bdrv_co_block_status() to
  * their file.
diff --git a/block.c b/block.c
index 5ff6cbd796..088727fdbe 100644
--- a/block.c
+++ b/block.c
@@ -2596,6 +2596,13 @@ void bdrv_format_default_perms(BlockDriverState *bs, 
BdrvChild *c,
uint64_t *nperm, uint64_t *nshared)
 {
 bool backing = (child_class == _backing);
+
+if (child_class == _of_bds) {
+bdrv_default_perms(bs, c, child_class, role, reopen_queue,
+   perm, shared, nperm, nshared);
+return;
+}
+
 assert(child_class == _backing || child_class == _file);
 
 if (!backing) {
@@ -2607,6 +2614,31 @@ void bdrv_format_default_perms(BlockDriverState *bs, 
BdrvChild *c,
 }
 }
 
+void bdrv_default_perms(BlockDriverState *bs, BdrvChild *c,
+const BdrvChildClass *child_class, BdrvChildRole role,
+BlockReopenQueue *reopen_queue,
+uint64_t perm, uint64_t shared,
+uint64_t *nperm, uint64_t *nshared)
+{
+assert(child_class == _of_bds);
+
+if (role & BDRV_CHILD_FILTERED) {
+assert(!(role & (BDRV_CHILD_DATA | BDRV_CHILD_METADATA |
+ BDRV_CHILD_COW)));
+bdrv_filter_default_perms(bs, c, child_class, role, reopen_queue,
+  perm, shared, nperm, nshared);
+} else if (role & BDRV_CHILD_COW) {
+assert(!(role & (BDRV_CHILD_DATA | BDRV_CHILD_METADATA)));
+bdrv_default_perms_for_cow(bs, c, child_class, role, reopen_queue,
+   perm, shared, nperm, nshared);
+} else if (role & (BDRV_CHILD_METADATA | BDRV_CHILD_DATA)) {
+bdrv_default_perms_for_storage(bs, c, child_class, role, reopen_queue,
+   perm, shared, nperm, nshared);
+} else {
+g_assert_not_reached();
+}
+}
+
 uint64_t bdrv_qapi_perm_to_blk_perm(BlockPermission qapi_perm)
 {
 static const uint64_t permissions[] = {
-- 
2.25.4

[PULL 27/51] block: Distinguish paths in *_format_default_perms

[Kevin Wolf]

From: Max Reitz 

bdrv_format_default_perms() has one code path for backing files, and one
for storage files.  We want to pull them out into their own functions,
so make sure they are completely distinct before so the next patches
will be a bit cleaner.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Message-Id: <20200513110544.176672-16-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block.c | 19 +--
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/block.c b/block.c
index be9bfa9d46..b3e7ae70c7 100644
--- a/block.c
+++ b/block.c
@@ -2497,6 +2497,13 @@ void bdrv_format_default_perms(BlockDriverState *bs, 
BdrvChild *c,
 perm |= BLK_PERM_CONSISTENT_READ;
 }
 shared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
+
+if (bs->open_flags & BDRV_O_INACTIVE) {
+shared |= BLK_PERM_WRITE | BLK_PERM_RESIZE;
+}
+
+*nperm = perm;
+*nshared = shared;
 } else {
 /* We want consistent read from backing files if the parent needs it.
  * No other operations are performed on backing files. */
@@ -2513,14 +2520,14 @@ void bdrv_format_default_perms(BlockDriverState *bs, 
BdrvChild *c,
 
 shared |= BLK_PERM_CONSISTENT_READ | BLK_PERM_GRAPH_MOD |
   BLK_PERM_WRITE_UNCHANGED;
-}
 
-if (bs->open_flags & BDRV_O_INACTIVE) {
-shared |= BLK_PERM_WRITE | BLK_PERM_RESIZE;
-}
+if (bs->open_flags & BDRV_O_INACTIVE) {
+shared |= BLK_PERM_WRITE | BLK_PERM_RESIZE;
+}
 
-*nperm = perm;
-*nshared = shared;
+*nperm = perm;
+*nshared = shared;
+}
 }
 
 uint64_t bdrv_qapi_perm_to_blk_perm(BlockPermission qapi_perm)
-- 
2.25.4

[PULL 10/51] block: Use blk_make_empty() after commits

[Kevin Wolf]

From: Max Reitz 

bdrv_commit() already has a BlockBackend pointing to the BDS that we
want to empty, it just has the wrong permissions.

qemu-img commit has no BlockBackend pointing to the old backing file
yet, but introducing one is simple.

After this commit, bdrv_make_empty() is the only remaining caller of
BlockDriver.bdrv_make_empty().

Signed-off-by: Max Reitz 
Message-Id: <20200429141126.85159-5-mre...@redhat.com>
Reviewed-by: Eric Blake 
[kwolf: Fixed up reference output for 098]
Signed-off-by: Kevin Wolf 
---
 block/commit.c | 16 +---
 qemu-img.c | 19 ++-
 tests/qemu-iotests/098.out |  8 
 3 files changed, 27 insertions(+), 16 deletions(-)

diff --git a/block/commit.c b/block/commit.c
index 87f6096d90..ba60fb7955 100644
--- a/block/commit.c
+++ b/block/commit.c
@@ -414,7 +414,9 @@ int bdrv_commit(BlockDriverState *bs)
 }
 
 ctx = bdrv_get_aio_context(bs);
-src = blk_new(ctx, BLK_PERM_CONSISTENT_READ, BLK_PERM_ALL);
+/* WRITE_UNCHANGED is required for bdrv_make_empty() */
+src = blk_new(ctx, BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGED,
+  BLK_PERM_ALL);
 backing = blk_new(ctx, BLK_PERM_WRITE | BLK_PERM_RESIZE, BLK_PERM_ALL);
 
 ret = blk_insert_bs(src, bs, _err);
@@ -492,14 +494,14 @@ int bdrv_commit(BlockDriverState *bs)
 }
 }
 
-if (drv->bdrv_make_empty) {
-ret = drv->bdrv_make_empty(bs);
-if (ret < 0) {
-goto ro_cleanup;
-}
-blk_flush(src);
+ret = blk_make_empty(src, NULL);
+/* Ignore -ENOTSUP */
+if (ret < 0 && ret != -ENOTSUP) {
+goto ro_cleanup;
 }
 
+blk_flush(src);
+
 /*
  * Make sure all data we wrote to the backing device is actually
  * stable on disk.
diff --git a/qemu-img.c b/qemu-img.c
index 6a4327aaba..f1795c788c 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -1104,11 +1104,20 @@ static int img_commit(int argc, char **argv)
 goto unref_backing;
 }
 
-if (!drop && bs->drv->bdrv_make_empty) {
-ret = bs->drv->bdrv_make_empty(bs);
-if (ret) {
-error_setg_errno(_err, -ret, "Could not empty %s",
- filename);
+if (!drop) {
+BlockBackend *old_backing_blk;
+
+old_backing_blk = blk_new_with_bs(bs, BLK_PERM_WRITE, BLK_PERM_ALL,
+  _err);
+if (!old_backing_blk) {
+goto unref_backing;
+}
+ret = blk_make_empty(old_backing_blk, _err);
+blk_unref(old_backing_blk);
+if (ret == -ENOTSUP) {
+error_free(local_err);
+local_err = NULL;
+} else if (ret < 0) {
 goto unref_backing;
 }
 }
diff --git a/tests/qemu-iotests/098.out b/tests/qemu-iotests/098.out
index 7634d0e8b0..23cf371f53 100644
--- a/tests/qemu-iotests/098.out
+++ b/tests/qemu-iotests/098.out
@@ -6,7 +6,7 @@ Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 
backing_file=TEST_DIR/t.IMGFMT.base
 wrote 65536/65536 bytes at offset 0
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-qemu-img: Could not empty blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: 
Input/output error
+qemu-img: Failed to empty blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: 
Input/output error
 No errors were found on the image.
 
 === empty_image_prepare ===
@@ -15,7 +15,7 @@ Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 
backing_file=TEST_DIR/t.IMGFMT.base
 wrote 65536/65536 bytes at offset 0
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-qemu-img: Could not empty blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: 
Input/output error
+qemu-img: Failed to empty blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: 
Input/output error
 Leaked cluster 4 refcount=1 reference=0
 Leaked cluster 5 refcount=1 reference=0
 Repairing cluster 4 refcount=1 reference=0
@@ -28,7 +28,7 @@ Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 
backing_file=TEST_DIR/t.IMGFMT.base
 wrote 65536/65536 bytes at offset 0
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-qemu-img: Could not empty blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: 
Input/output error
+qemu-img: Failed to empty blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: 
Input/output error
 ERROR cluster 0 refcount=0 reference=1
 ERROR cluster 1 refcount=0 reference=1
 ERROR cluster 3 refcount=0 reference=1
@@ -42,7 +42,7 @@ Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 
backing_file=TEST_DIR/t.IMGFMT.base
 wrote 65536/65536 bytes at offset 0
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-qemu-img: Could not empty 

[PULL 13/51] block: Mark commit, mirror, blkreplay as filters

[Kevin Wolf]

From: Max Reitz 

The commit, mirror, and blkreplay block nodes are filters, so they should
be marked as such.

Signed-off-by: Max Reitz 
Message-Id: <20200513110544.176672-2-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block/blkreplay.c | 1 +
 block/commit.c| 2 ++
 block/mirror.c| 2 ++
 3 files changed, 5 insertions(+)

diff --git a/block/blkreplay.c b/block/blkreplay.c
index c96ac8f4bc..131c9e8477 100644
--- a/block/blkreplay.c
+++ b/block/blkreplay.c
@@ -135,6 +135,7 @@ static int blkreplay_snapshot_goto(BlockDriverState *bs,
 static BlockDriver bdrv_blkreplay = {
 .format_name= "blkreplay",
 .instance_size  = 0,
+.is_filter  = true,
 
 .bdrv_open  = blkreplay_open,
 .bdrv_child_perm= bdrv_filter_default_perms,
diff --git a/block/commit.c b/block/commit.c
index ba60fb7955..b0a8a793cd 100644
--- a/block/commit.c
+++ b/block/commit.c
@@ -240,6 +240,8 @@ static BlockDriver bdrv_commit_top = {
 .bdrv_co_block_status   = bdrv_co_block_status_from_backing,
 .bdrv_refresh_filename  = bdrv_commit_top_refresh_filename,
 .bdrv_child_perm= bdrv_commit_top_child_perm,
+
+.is_filter  = true,
 };
 
 void commit_start(const char *job_id, BlockDriverState *bs,
diff --git a/block/mirror.c b/block/mirror.c
index 201ffa26f9..55e992670a 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -1530,6 +1530,8 @@ static BlockDriver bdrv_mirror_top = {
 .bdrv_co_block_status   = bdrv_co_block_status_from_backing,
 .bdrv_refresh_filename  = bdrv_mirror_top_refresh_filename,
 .bdrv_child_perm= bdrv_mirror_top_child_perm,
+
+.is_filter  = true,
 };
 
 static BlockJob *mirror_start_job(
-- 
2.25.4

[PULL 14/51] block: Add BlockDriver.is_format

[Kevin Wolf]

From: Max Reitz 

We want to unify child_format and child_file at some point.  One of the
important things that set format drivers apart from other drivers is
that they do not expect other format nodes under them (except in the
backing chain), i.e. we must not probe formats inside of formats.  That
means we need something on which to distinguish format drivers from
others, and hence this flag.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
Message-Id: <20200513110544.176672-3-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 include/block/block_int.h | 7 +++
 block/bochs.c | 1 +
 block/cloop.c | 1 +
 block/crypto.c| 2 ++
 block/dmg.c   | 1 +
 block/parallels.c | 1 +
 block/qcow.c  | 1 +
 block/qcow2.c | 1 +
 block/qed.c   | 1 +
 block/raw-format.c| 1 +
 block/vdi.c   | 1 +
 block/vhdx.c  | 1 +
 block/vmdk.c  | 1 +
 block/vpc.c   | 1 +
 14 files changed, 21 insertions(+)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 7ba8c89036..1c24df53fd 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -95,6 +95,13 @@ struct BlockDriver {
  * must implement them and return -ENOTSUP.
  */
 bool is_filter;
+/*
+ * Set to true if the BlockDriver is a format driver.  Format nodes
+ * generally do not expect their children to be other format nodes
+ * (except for backing files), and so format probing is disabled
+ * on those children.
+ */
+bool is_format;
 /*
  * Return true if @to_replace can be replaced by a BDS with the
  * same data as @bs without it affecting @bs's behavior (that is,
diff --git a/block/bochs.c b/block/bochs.c
index 32bb83b268..e7bbeaa1c4 100644
--- a/block/bochs.c
+++ b/block/bochs.c
@@ -301,6 +301,7 @@ static BlockDriver bdrv_bochs = {
 .bdrv_refresh_limits = bochs_refresh_limits,
 .bdrv_co_preadv = bochs_co_preadv,
 .bdrv_close= bochs_close,
+.is_format  = true,
 };
 
 static void bdrv_bochs_init(void)
diff --git a/block/cloop.c b/block/cloop.c
index 4de94876d4..f90f1a4b4c 100644
--- a/block/cloop.c
+++ b/block/cloop.c
@@ -297,6 +297,7 @@ static BlockDriver bdrv_cloop = {
 .bdrv_refresh_limits = cloop_refresh_limits,
 .bdrv_co_preadv = cloop_co_preadv,
 .bdrv_close = cloop_close,
+.is_format  = true,
 };
 
 static void bdrv_cloop_init(void)
diff --git a/block/crypto.c b/block/crypto.c
index 6b21d6bf6c..bdb2b27475 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -771,6 +771,8 @@ static BlockDriver bdrv_crypto_luks = {
 .bdrv_get_info  = block_crypto_get_info_luks,
 .bdrv_get_specific_info = block_crypto_get_specific_info_luks,
 
+.is_format  = true,
+
 .strong_runtime_opts = block_crypto_strong_runtime_opts,
 };
 
diff --git a/block/dmg.c b/block/dmg.c
index 4a045f2b3e..ef3c6e771d 100644
--- a/block/dmg.c
+++ b/block/dmg.c
@@ -753,6 +753,7 @@ static BlockDriver bdrv_dmg = {
 .bdrv_child_perm = bdrv_format_default_perms,
 .bdrv_co_preadv = dmg_co_preadv,
 .bdrv_close = dmg_close,
+.is_format  = true,
 };
 
 static void bdrv_dmg_init(void)
diff --git a/block/parallels.c b/block/parallels.c
index e7717c508e..bd5f6ffa09 100644
--- a/block/parallels.c
+++ b/block/parallels.c
@@ -918,6 +918,7 @@ static BlockDriver bdrv_parallels = {
 .bdrv_co_flush_to_os  = parallels_co_flush_to_os,
 .bdrv_co_readv  = parallels_co_readv,
 .bdrv_co_writev = parallels_co_writev,
+.is_format  = true,
 .supports_backing = true,
 .bdrv_co_create  = parallels_co_create,
 .bdrv_co_create_opts = parallels_co_create_opts,
diff --git a/block/qcow.c b/block/qcow.c
index b0475b73a5..6a72dea049 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -1185,6 +1185,7 @@ static BlockDriver bdrv_qcow = {
 .bdrv_co_create = qcow_co_create,
 .bdrv_co_create_opts= qcow_co_create_opts,
 .bdrv_has_zero_init = bdrv_has_zero_init_1,
+.is_format  = true,
 .supports_backing   = true,
 .bdrv_refresh_limits= qcow_refresh_limits,
 
diff --git a/block/qcow2.c b/block/qcow2.c
index ad9ab4fafa..76bec61ee9 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -5767,6 +5767,7 @@ BlockDriver bdrv_qcow2 = {
 .bdrv_save_vmstate= qcow2_save_vmstate,
 .bdrv_load_vmstate= qcow2_load_vmstate,
 
+.is_format  = true,
 .supports_backing   = true,
 .bdrv_change_backing_file   = qcow2_change_backing_file,
 
diff --git a/block/qed.c b/block/qed.c
index 5da9726518..337eb6dbb6 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -1665,6 +1665,7 @@ static BlockDriver bdrv_qed = {
 .format_name  = "qed",
 .instance_size= sizeof(BDRVQEDState),
 .create_opts  = _create_opts,
+.is_format

[PULL 12/51] block: Use bdrv_make_empty() where possible

[Kevin Wolf]

From: Max Reitz 

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Kevin Wolf 
Message-Id: <20200429141126.85159-3-mre...@redhat.com>
Signed-off-by: Kevin Wolf 
---
 block/replication.c | 3 +--
 block/vvfat.c   | 4 +---
 2 files changed, 2 insertions(+), 5 deletions(-)

diff --git a/block/replication.c b/block/replication.c
index eb480a8e08..9a9f36e524 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -331,9 +331,8 @@ static void secondary_do_checkpoint(BDRVReplicationState 
*s, Error **errp)
 return;
 }
 
-ret = s->active_disk->bs->drv->bdrv_make_empty(s->active_disk->bs);
+ret = bdrv_make_empty(s->active_disk, errp);
 if (ret < 0) {
-error_setg(errp, "Cannot make active disk empty");
 return;
 }
 
diff --git a/block/vvfat.c b/block/vvfat.c
index 6d5c090dec..34c121c07a 100644
--- a/block/vvfat.c
+++ b/block/vvfat.c
@@ -2960,9 +2960,7 @@ static int do_commit(BDRVVVFATState* s)
 return ret;
 }
 
-if (s->qcow->bs->drv && s->qcow->bs->drv->bdrv_make_empty) {
-s->qcow->bs->drv->bdrv_make_empty(s->qcow->bs);
-}
+bdrv_make_empty(s->qcow, NULL);
 
 memset(s->used_clusters, 0, sector2cluster(s, s->sector_count));
 
-- 
2.25.4

[PULL 07/51] iotests: Run pylint and mypy in a testcase

[Kevin Wolf]

We made sure that iotests.py passes pylint. It would be a shame if we
allowed new patches in that break this again, so let's just add a
meta-test case that runs pylint on it.

While we don't pass mypy --strict yet, we can already run it with a few
options that would be part of --strict to make sure that we won't
regress on these aspects at least until we can enable the full thing.

Signed-off-by: Kevin Wolf 
Message-Id: <20200511163529.349329-3-kw...@redhat.com>
Reviewed-by: Max Reitz 
Reviewed-by: John Snow 
Signed-off-by: Kevin Wolf 
---
 tests/qemu-iotests/297 | 44 ++
 tests/qemu-iotests/297.out |  3 +++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 48 insertions(+)
 create mode 100755 tests/qemu-iotests/297
 create mode 100644 tests/qemu-iotests/297.out

diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
new file mode 100755
index 00..5c5420712b
--- /dev/null
+++ b/tests/qemu-iotests/297
@@ -0,0 +1,44 @@
+#!/usr/bin/env bash
+#
+# Copyright (C) 2020 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+
+seq=$(basename $0)
+echo "QA output created by $seq"
+
+status=1   # failure is the default!
+
+# get standard environment
+. ./common.rc
+
+if ! type -p "pylint-3" > /dev/null; then
+_notrun "pylint-3 not found"
+fi
+if ! type -p "mypy" > /dev/null; then
+_notrun "mypy not found"
+fi
+
+pylint-3 --score=n iotests.py
+
+MYPYPATH=../../python/ mypy --warn-unused-configs --disallow-subclassing-any \
+--disallow-any-generics --disallow-incomplete-defs \
+--disallow-untyped-decorators --no-implicit-optional \
+--warn-redundant-casts --warn-unused-ignores \
+--no-implicit-reexport iotests.py
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/297.out b/tests/qemu-iotests/297.out
new file mode 100644
index 00..6acc843649
--- /dev/null
+++ b/tests/qemu-iotests/297.out
@@ -0,0 +1,3 @@
+QA output created by 297
+Success: no issues found in 1 source file
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 34175fd437..445c26f8d2 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -300,3 +300,4 @@
 289 rw quick
 290 rw auto quick
 292 rw auto quick
+297 meta
-- 
2.25.4

[PULL 09/51] block: Add blk_make_empty()

[Kevin Wolf]

From: Max Reitz 

Two callers of BlockDriver.bdrv_make_empty() remain that should not call
this method directly.  Both do not have access to a BdrvChild, but they
can use a BlockBackend, so we add this function that lets them use it.

Signed-off-by: Max Reitz 
Message-Id: <20200429141126.85159-4-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 include/sysemu/block-backend.h |  2 ++
 block/block-backend.c  | 10 ++
 2 files changed, 12 insertions(+)

diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
index 0917663d89..8203d7f6f9 100644
--- a/include/sysemu/block-backend.h
+++ b/include/sysemu/block-backend.h
@@ -266,4 +266,6 @@ int coroutine_fn blk_co_copy_range(BlockBackend *blk_in, 
int64_t off_in,
 
 const BdrvChild *blk_root(BlockBackend *blk);
 
+int blk_make_empty(BlockBackend *blk, Error **errp);
+
 #endif
diff --git a/block/block-backend.c b/block/block-backend.c
index f4944861fa..47bd56244d 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -2402,3 +2402,13 @@ const BdrvChild *blk_root(BlockBackend *blk)
 {
 return blk->root;
 }
+
+int blk_make_empty(BlockBackend *blk, Error **errp)
+{
+if (!blk_is_available(blk)) {
+error_setg(errp, "No medium inserted");
+return -ENOMEDIUM;
+}
+
+return bdrv_make_empty(blk->root, errp);
+}
-- 
2.25.4

[PULL 08/51] block: Add bdrv_make_empty()

[Kevin Wolf]

From: Max Reitz 

Right now, all users of bdrv_make_empty() call the BlockDriver method
directly.  That is not only bad style, it is also wrong, unless the
caller has a BdrvChild with a WRITE or WRITE_UNCHANGED permission.
(WRITE_UNCHANGED suffices, because callers generally use this function
to clear a node with a backing file after a commit operation.)

Introduce bdrv_make_empty() that verifies that it does.

Signed-off-by: Max Reitz 
Message-Id: <20200429141126.85159-2-mre...@redhat.com>
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 include/block/block.h |  1 +
 block.c   | 23 +++
 2 files changed, 24 insertions(+)

diff --git a/include/block/block.h b/include/block/block.h
index 4de8d8f8a6..8f3eb70df4 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -352,6 +352,7 @@ BlockMeasureInfo *bdrv_measure(BlockDriver *drv, QemuOpts 
*opts,
 void bdrv_get_geometry(BlockDriverState *bs, uint64_t *nb_sectors_ptr);
 void bdrv_refresh_limits(BlockDriverState *bs, Error **errp);
 int bdrv_commit(BlockDriverState *bs);
+int bdrv_make_empty(BdrvChild *c, Error **errp);
 int bdrv_change_backing_file(BlockDriverState *bs,
 const char *backing_file, const char *backing_fmt);
 void bdrv_register(BlockDriver *bdrv);
diff --git a/block.c b/block.c
index 0653ccb913..c6a5c144b7 100644
--- a/block.c
+++ b/block.c
@@ -6764,3 +6764,26 @@ void bdrv_del_child(BlockDriverState *parent_bs, 
BdrvChild *child, Error **errp)
 
 parent_bs->drv->bdrv_del_child(parent_bs, child, errp);
 }
+
+int bdrv_make_empty(BdrvChild *c, Error **errp)
+{
+BlockDriver *drv = c->bs->drv;
+int ret;
+
+assert(c->perm & (BLK_PERM_WRITE | BLK_PERM_WRITE_UNCHANGED));
+
+if (!drv->bdrv_make_empty) {
+error_setg(errp, "%s does not support emptying nodes",
+   drv->format_name);
+return -ENOTSUP;
+}
+
+ret = drv->bdrv_make_empty(c->bs);
+if (ret < 0) {
+error_setg_errno(errp, -ret, "Failed to empty %s",
+ c->bs->filename);
+return ret;
+}
+
+return 0;
+}
-- 
2.25.4

[PULL 05/51] block/replication.c: Avoid cancelling the job twice

[Kevin Wolf]

From: Lukas Straub 

If qemu in colo secondary mode is stopped, it crashes because
s->backup_job is canceled twice: First with job_cancel_sync_all()
in qemu_cleanup() and then in replication_stop().

Fix this by assigning NULL to s->backup_job when the job completes
so replication_stop() and replication_do_checkpoint() won't touch
the job.

Signed-off-by: Lukas Straub 
Message-Id: <20200511090801.7ed5d8f3@luklap>
Signed-off-by: Kevin Wolf 
---
 block/replication.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/block/replication.c b/block/replication.c
index 971f0fe266..c03980a192 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -398,6 +398,8 @@ static void backup_job_cleanup(BlockDriverState *bs)
 BDRVReplicationState *s = bs->opaque;
 BlockDriverState *top_bs;
 
+s->backup_job = NULL;
+
 top_bs = bdrv_lookup_bs(s->top_id, s->top_id, NULL);
 if (!top_bs) {
 return;
-- 
2.25.4

[PULL 04/51] iotests: Mirror with different source/target size

[Kevin Wolf]

This tests that the mirror job catches situations where the target node
has a different size than the source node. It must also forbid resize
operations when the job is already running.

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Message-Id: <20200511135825.219437-5-kw...@redhat.com>
Reviewed-by: Max Reitz 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
Signed-off-by: Kevin Wolf 
---
 tests/qemu-iotests/041 | 45 ++
 tests/qemu-iotests/041.out |  4 ++--
 2 files changed, 47 insertions(+), 2 deletions(-)

diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
index 1812dd8479..601c756117 100755
--- a/tests/qemu-iotests/041
+++ b/tests/qemu-iotests/041
@@ -240,6 +240,49 @@ class TestSingleBlockdev(TestSingleDrive):
  target=self.qmp_target)
 self.assert_qmp(result, 'error/class', 'GenericError')
 
+def do_test_resize(self, device, node):
+def pre_finalize():
+if device:
+result = self.vm.qmp('block_resize', device=device, size=65536)
+self.assert_qmp(result, 'error/class', 'GenericError')
+
+result = self.vm.qmp('block_resize', node_name=node, size=65536)
+self.assert_qmp(result, 'error/class', 'GenericError')
+
+result = self.vm.qmp(self.qmp_cmd, job_id='job0', device='drive0',
+ sync='full', target=self.qmp_target,
+ auto_finalize=False, auto_dismiss=False)
+self.assert_qmp(result, 'return', {})
+
+result = self.vm.run_job('job0', auto_finalize=False,
+ pre_finalize=pre_finalize)
+self.assertEqual(result, None)
+
+def test_source_resize(self):
+self.do_test_resize('drive0', 'top')
+
+def test_target_resize(self):
+self.do_test_resize(None, self.qmp_target)
+
+def do_test_target_size(self, size):
+result = self.vm.qmp('block_resize', node_name=self.qmp_target,
+ size=size)
+self.assert_qmp(result, 'return', {})
+
+result = self.vm.qmp(self.qmp_cmd, job_id='job0',
+ device='drive0', sync='full', auto_dismiss=False,
+ target=self.qmp_target)
+self.assert_qmp(result, 'return', {})
+
+result = self.vm.run_job('job0')
+self.assertEqual(result, 'Source and target image have different 
sizes')
+
+def test_small_target(self):
+self.do_test_target_size(self.image_len // 2)
+
+def test_large_target(self):
+self.do_test_target_size(self.image_len * 2)
+
 test_large_cluster = None
 test_image_not_found = None
 test_small_buffer2 = None
@@ -251,6 +294,8 @@ class TestSingleDriveZeroLength(TestSingleDrive):
 
 class TestSingleBlockdevZeroLength(TestSingleBlockdev):
 image_len = 0
+test_small_target = None
+test_large_target = None
 
 class TestSingleDriveUnalignedLength(TestSingleDrive):
 image_len = 1025 * 1024
diff --git a/tests/qemu-iotests/041.out b/tests/qemu-iotests/041.out
index 877b76fd31..53abe11d73 100644
--- a/tests/qemu-iotests/041.out
+++ b/tests/qemu-iotests/041.out
@@ -1,5 +1,5 @@
-..
+
 --
-Ran 94 tests
+Ran 104 tests
 
 OK
-- 
2.25.4

[PULL 06/51] iotests: Fix incomplete type declarations

[Kevin Wolf]

We need to fix only a few places so that iotests.py can pass
mypy --disallow-incomplete-defs, which seems to be a desirable option to
have enabled in the long run.

Signed-off-by: Kevin Wolf 
Message-Id: <20200511163529.349329-2-kw...@redhat.com>
Reviewed-by: Max Reitz 
Reviewed-by: John Snow 
Signed-off-by: Kevin Wolf 
---
 tests/qemu-iotests/iotests.py | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 6c0e781af7..1d7f6fd7cf 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -1040,7 +1040,7 @@ def _verify_cache_mode(supported_cache_modes: 
Sequence[str] = ()) -> None:
 if supported_cache_modes and (cachemode not in supported_cache_modes):
 notrun('not suitable for this cache mode: %s' % cachemode)
 
-def _verify_aio_mode(supported_aio_modes: Sequence[str] = ()):
+def _verify_aio_mode(supported_aio_modes: Sequence[str] = ()) -> None:
 if supported_aio_modes and (aiomode not in supported_aio_modes):
 notrun('not suitable for this aio mode: %s' % aiomode)
 
@@ -1087,7 +1087,8 @@ def skip_if_unsupported(required_formats=(), 
read_only=False):
 '''Skip Test Decorator
Runs the test if all the required formats are whitelisted'''
 def skip_test_decorator(func):
-def func_wrapper(test_case: QMPTestCase, *args, **kwargs):
+def func_wrapper(test_case: QMPTestCase, *args: List[Any],
+ **kwargs: Dict[str, Any]) -> None:
 if callable(required_formats):
 fmts = required_formats(test_case)
 else:
@@ -1097,9 +1098,8 @@ def skip_if_unsupported(required_formats=(), 
read_only=False):
 if usf_list:
 msg = f'{test_case}: formats {usf_list} are not whitelisted'
 test_case.case_skip(msg)
-return None
 else:
-return func(test_case, *args, **kwargs)
+func(test_case, *args, **kwargs)
 return func_wrapper
 return skip_test_decorator
 
-- 
2.25.4

[PULL 01/51] iotests/109: Don't mirror with mismatched size

[Kevin Wolf]

This patch makes the raw image the same size as the file in a different
format that is mirrored as raw to it to avoid errors when mirror starts
to enforce that source and target are the same size.

We check only that the first 512 bytes are zeroed (instead of 64k)
because some image formats create image files that are smaller than 64k,
so trying to read 64k would result in I/O errors. Apart from this, 512
is more appropriate anyway because the raw format driver protects
specifically the first 512 bytes.

Signed-off-by: Kevin Wolf 
Message-Id: <20200511135825.219437-2-kw...@redhat.com>
Reviewed-by: Max Reitz 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
Signed-off-by: Kevin Wolf 
---
 tests/qemu-iotests/109   | 10 ++---
 tests/qemu-iotests/109.out   | 74 +---
 tests/qemu-iotests/common.filter |  5 +++
 3 files changed, 41 insertions(+), 48 deletions(-)

diff --git a/tests/qemu-iotests/109 b/tests/qemu-iotests/109
index 5bc2e9b001..3ffeaf3c55 100755
--- a/tests/qemu-iotests/109
+++ b/tests/qemu-iotests/109
@@ -77,14 +77,14 @@ for fmt in qcow qcow2 qed vdi vmdk vpc; do
 echo "=== Writing a $fmt header into raw ==="
 echo
 
-_make_test_img 64M
 TEST_IMG="$TEST_IMG.src" IMGFMT=$fmt _make_test_img 64M
+_make_test_img $(du -b "$TEST_IMG.src" | cut -f1) | _filter_img_create_size
 
 # This first test should fail: The image format was probed, we may not
 # write an image header at the start of the image
 run_qemu "$TEST_IMG" "$TEST_IMG.src" "" "BLOCK_JOB_ERROR" |
 _filter_block_job_len
-$QEMU_IO -c 'read -P 0 0 64k' "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -c 'read -P 0 0 512' "$TEST_IMG" | _filter_qemu_io
 
 
 # When raw was explicitly specified, the same must succeed
@@ -103,12 +103,12 @@ for sample_img in empty.bochs iotest-dirtylog-10G-4M.vhdx 
parallels-v1 \
 
 # Can't use _use_sample_img because that isn't designed to be used multiple
 # times and it overwrites $TEST_IMG (both breaks cleanup)
-_make_test_img 64M
 bzcat "$SAMPLE_IMG_DIR/$sample_img.bz2" > "$TEST_IMG.src"
+_make_test_img $(du -b "$TEST_IMG.src" | cut -f1) | _filter_img_create_size
 
 run_qemu "$TEST_IMG" "$TEST_IMG.src" "" "BLOCK_JOB_ERROR" |
 _filter_block_job_offset | _filter_block_job_len
-$QEMU_IO -c 'read -P 0 0 64k' "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -c 'read -P 0 0 512' "$TEST_IMG" | _filter_qemu_io
 
 run_qemu "$TEST_IMG" "$TEST_IMG.src" "'format': 'raw'," "BLOCK_JOB_READY"
 $QEMU_IMG compare -f raw -F raw "$TEST_IMG" "$TEST_IMG.src"
@@ -119,8 +119,8 @@ echo "=== Write legitimate MBR into raw ==="
 echo
 
 for sample_img in grub_mbr.raw; do
-_make_test_img 64M
 bzcat "$SAMPLE_IMG_DIR/$sample_img.bz2" > "$TEST_IMG.src"
+_make_test_img $(du -b "$TEST_IMG.src" | cut -f1) | _filter_img_create_size
 
 run_qemu "$TEST_IMG" "$TEST_IMG.src" "" "BLOCK_JOB_READY"
 $QEMU_IMG compare -f raw -F raw "$TEST_IMG" "$TEST_IMG.src"
diff --git a/tests/qemu-iotests/109.out b/tests/qemu-iotests/109.out
index 884f65f18d..ad739df46c 100644
--- a/tests/qemu-iotests/109.out
+++ b/tests/qemu-iotests/109.out
@@ -2,8 +2,8 @@ QA output created by 109
 
 === Writing a qcow header into raw ===
 
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 Formatting 'TEST_DIR/t.raw.src', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=SIZE
 { 'execute': 'qmp_capabilities' }
 {"return": {}}
 {'execute':'drive-mirror', 'arguments':{ 'device': 'src', 'target': 
'TEST_DIR/t.IMGFMT', 'mode': 'existing', 'sync': 'full'}}
@@ -23,8 +23,8 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' 
and probing guessed
 {"execute":"quit"}
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": 
"SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}}
-read 65536/65536 bytes at offset 0
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 512/512 bytes at offset 0
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 { 'execute': 'qmp_capabilities' }
 {"return": {}}
 {'execute':'drive-mirror', 'arguments':{ 'device': 'src', 'target': 
'TEST_DIR/t.IMGFMT', 'format': 'IMGFMT', 'mode': 'existing', 'sync': 'full'}}
@@ -43,13 +43,12 @@ read 65536/65536 bytes at offset 0
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": 
"BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 1024, "offset": 1024, 
"speed": 0, "type": "mirror"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": 
"JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": 
"JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}}
-Warning: Image size mismatch!
 Images are identical.
 
 === Writing a qcow2 header into raw ===
 
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 Formatting 'TEST_DIR/t.raw.src', fmt=IMGFMT 

[PULL 03/51] mirror: Make sure that source and target size match

[Kevin Wolf]

If the target is shorter than the source, mirror would copy data until
it reaches the end of the target and then fail with an I/O error when
trying to write past the end.

If the target is longer than the source, the mirror job would complete
successfully, but the target wouldn't actually be an accurate copy of
the source image (it would contain some additional garbage at the end).

Fix this by checking that both images have the same size when the job
starts.

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Message-Id: <20200511135825.219437-4-kw...@redhat.com>
Reviewed-by: Max Reitz 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
Signed-off-by: Kevin Wolf 
---
 block/mirror.c | 21 -
 1 file changed, 12 insertions(+), 9 deletions(-)

diff --git a/block/mirror.c b/block/mirror.c
index aca95c9bc9..201ffa26f9 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -872,6 +872,7 @@ static int coroutine_fn mirror_run(Job *job, Error **errp)
 BlockDriverState *target_bs = blk_bs(s->target);
 bool need_drain = true;
 int64_t length;
+int64_t target_length;
 BlockDriverInfo bdi;
 char backing_filename[2]; /* we only need 2 characters because we are only
  checking for a NULL string */
@@ -887,24 +888,26 @@ static int coroutine_fn mirror_run(Job *job, Error **errp)
 goto immediate_exit;
 }
 
+target_length = blk_getlength(s->target);
+if (target_length < 0) {
+ret = target_length;
+goto immediate_exit;
+}
+
 /* Active commit must resize the base image if its size differs from the
  * active layer. */
 if (s->base == blk_bs(s->target)) {
-int64_t base_length;
-
-base_length = blk_getlength(s->target);
-if (base_length < 0) {
-ret = base_length;
-goto immediate_exit;
-}
-
-if (s->bdev_length > base_length) {
+if (s->bdev_length > target_length) {
 ret = blk_truncate(s->target, s->bdev_length, false,
PREALLOC_MODE_OFF, 0, NULL);
 if (ret < 0) {
 goto immediate_exit;
 }
 }
+} else if (s->bdev_length != target_length) {
+error_setg(errp, "Source and target image have different sizes");
+ret = -EINVAL;
+goto immediate_exit;
 }
 
 if (s->bdev_length == 0) {
-- 
2.25.4

[PULL 02/51] iotests/229: Use blkdebug to inject an error

[Kevin Wolf]

229 relies on the mirror running into an I/O error when the target is
smaller than the source. After changing mirror to catch this condition
while starting the job, this test case won't get a job that is paused
for an I/O error any more. Use blkdebug instead to inject an error.

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Message-Id: <20200511135825.219437-3-kw...@redhat.com>
Reviewed-by: Max Reitz 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
Signed-off-by: Kevin Wolf 
---
 tests/qemu-iotests/229 | 15 +++
 tests/qemu-iotests/229.out |  6 +++---
 2 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/tests/qemu-iotests/229 b/tests/qemu-iotests/229
index 866168b236..a5d4e5d4f2 100755
--- a/tests/qemu-iotests/229
+++ b/tests/qemu-iotests/229
@@ -33,6 +33,7 @@ _cleanup()
 _cleanup_test_img
 _rm_test_img "$TEST_IMG"
 _rm_test_img "$DEST_IMG"
+rm -f "$TEST_DIR/blkdebug.conf"
 }
 trap "_cleanup; exit \$status" 0 1 2 3 15
 
@@ -49,11 +50,10 @@ _supported_os Linux
 
 DEST_IMG="$TEST_DIR/d.$IMGFMT"
 TEST_IMG="$TEST_DIR/b.$IMGFMT"
+BLKDEBUG_CONF="$TEST_DIR/blkdebug.conf"
 
 _make_test_img 2M
-
-# destination for mirror will be too small, causing error
-TEST_IMG=$DEST_IMG _make_test_img 1M
+TEST_IMG=$DEST_IMG _make_test_img 2M
 
 $QEMU_IO -c 'write 0 2M' "$TEST_IMG" | _filter_qemu_io
 
@@ -67,11 +67,18 @@ echo
 echo '=== Starting drive-mirror, causing error & stop  ==='
 echo
 
+cat > "$BLKDEBUG_CONF" <

[PULL 00/51] Block layer patches

[Kevin Wolf]

The following changes since commit 013a18edbbc59cdad019100c7d03c0494642b74c:

  Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20200514' 
into staging (2020-05-14 16:17:55 +0100)

are available in the Git repository at:

  git://repo.or.cz/qemu/kevin.git tags/for-upstream

for you to fetch changes up to 7d8f21a650e562270f5ed5341134c9c2c39dc5e8:

  iotests/030: Reduce run time by unthrottling job earlier (2020-05-15 14:12:34 
+0200)


Block layer patches:

- Introduce real BdrvChildRole
- blk/bdrv_make_empty() functions instead of calling callbacks directly
- mirror: Make sure that source and target size match
- block-copy: Fix uninitialized variable
- block/replication: Avoid cancelling the job twice
- ahci: Log lost IRQs
- iotests: Run pylint and mypy in a testcase
- iotests: log messages from notrun()


John Snow (1):
  iotests: log messages from notrun()

Kevin Wolf (8):
  iotests/109: Don't mirror with mismatched size
  iotests/229: Use blkdebug to inject an error
  mirror: Make sure that source and target size match
  iotests: Mirror with different source/target size
  iotests: Fix incomplete type declarations
  iotests: Run pylint and mypy in a testcase
  replication: Avoid blk_make_empty() on read-only child
  iotests/030: Reduce run time by unthrottling job earlier

Lukas Straub (1):
  block/replication.c: Avoid cancelling the job twice

Max Reitz (38):
  block: Add bdrv_make_empty()
  block: Add blk_make_empty()
  block: Use blk_make_empty() after commits
  block: Use bdrv_make_empty() where possible
  block: Mark commit, mirror, blkreplay as filters
  block: Add BlockDriver.is_format
  block: Rename BdrvChildRole to BdrvChildClass
  block: Add BdrvChildRole and BdrvChildRoleBits
  block: Add BdrvChildRole to BdrvChild
  block: Pass BdrvChildRole to bdrv_child_perm()
  block: Pass BdrvChildRole to .inherit_options()
  block: Pass parent_is_format to .inherit_options()
  block: Rename bdrv_inherited_options()
  block: Add generic bdrv_inherited_options()
  block: Use bdrv_inherited_options()
  block: Unify bdrv_child_cb_attach()
  block: Unify bdrv_child_cb_detach()
  block: Add child_of_bds
  block: Distinguish paths in *_format_default_perms
  block: Pull out bdrv_default_perms_for_cow()
  block: Pull out bdrv_default_perms_for_storage()
  block: Relax *perms_for_storage for data children
  block: Add bdrv_default_perms()
  raw-format: Split raw_read_options()
  block: Switch child_format users to child_of_bds
  block: Drop child_format
  block: Make backing files child_of_bds children
  block: Drop child_backing
  block: Make format drivers use child_of_bds
  block: Make filter drivers use child_of_bds
  block: Use child_of_bds in remaining places
  tests: Use child_of_bds instead of child_file
  block: Use bdrv_default_perms()
  block: Make bdrv_filter_default_perms() static
  block: Drop bdrv_format_default_perms()
  block: Drop child_file
  block: Pass BdrvChildRole in remaining cases
  block: Drop @child_class from bdrv_child_perm()

Philippe Mathieu-Daudé (3):
  block/block-copy: Fix uninitialized variable in block_copy_task_entry
  block/block-copy: Simplify block_copy_do_copy()
  hw/ide/ahci: Log lost IRQs

 include/block/block.h|  65 -
 include/block/block_int.h|  57 ++--
 include/sysemu/block-backend.h   |   2 +
 block.c  | 601 ---
 block/backup-top.c   |  11 +-
 block/blkdebug.c |  10 +-
 block/blklogwrites.c |  16 +-
 block/blkreplay.c|   8 +-
 block/blkverify.c|  10 +-
 block/block-backend.c|  30 +-
 block/block-copy.c   |  14 +-
 block/bochs.c|   7 +-
 block/cloop.c|   7 +-
 block/commit.c   |  20 +-
 block/copy-on-read.c |   7 +-
 block/crypto.c   |   8 +-
 block/dmg.c  |   7 +-
 block/filter-compress.c  |   7 +-
 block/io.c   |  22 +-
 block/mirror.c   |  25 +-
 block/parallels.c|   7 +-
 block/qcow.c |   7 +-
 block/qcow2.c|  20 +-
 block/qed.c  |   7 +-
 block/quorum.c   |   8 +-
 block/raw-format.c   | 128 +
 block/replication.c  |  23 +-
 block/throttle.c |   7 +-
 block/vdi.c  |   7 +-
 block/vhdx.c |   7 +-
 block/vmdk.c |  23 +-
 block/vpc.c  |   7 +-
 block/vvfat.c|  

[PATCH v3 5/7] iotests: 194: test also migration of dirty bitmap

[Vladimir Sementsov-Ogievskiy]

Test that dirty bitmap migration works when we deal with mirror.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/194 | 14 ++
 tests/qemu-iotests/194.out |  6 ++
 2 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194
index 8b1f720af4..3fad7c6c1a 100755
--- a/tests/qemu-iotests/194
+++ b/tests/qemu-iotests/194
@@ -42,6 +42,8 @@ with iotests.FilePath('source.img') as source_img_path, \
 .add_incoming('unix:{0}'.format(migration_sock_path))
 .launch())
 
+source_vm.qmp_log('block-dirty-bitmap-add', node='drive0', name='bitmap0')
+
 iotests.log('Launching NBD server on destination...')
 iotests.log(dest_vm.qmp('nbd-server-start', addr={'type': 'unix', 'data': 
{'path': nbd_sock_path}}))
 iotests.log(dest_vm.qmp('nbd-server-add', device='drive0', writable=True))
@@ -61,12 +63,14 @@ with iotests.FilePath('source.img') as source_img_path, \
 filters=[iotests.filter_qmp_event])
 
 iotests.log('Starting migration...')
-source_vm.qmp('migrate-set-capabilities',
-  capabilities=[{'capability': 'events', 'state': True}])
-dest_vm.qmp('migrate-set-capabilities',
-capabilities=[{'capability': 'events', 'state': True}])
+capabilities = [{'capability': 'events', 'state': True},
+{'capability': 'dirty-bitmaps', 'state': True}]
+source_vm.qmp('migrate-set-capabilities', capabilities=capabilities)
+dest_vm.qmp('migrate-set-capabilities', capabilities=capabilities)
 iotests.log(source_vm.qmp('migrate', 
uri='unix:{0}'.format(migration_sock_path)))
 
+source_vm.qmp_log('migrate-start-postcopy')
+
 while True:
 event1 = source_vm.event_wait('MIGRATION')
 iotests.log(event1, filters=[iotests.filter_qmp_event])
@@ -82,3 +86,5 @@ with iotests.FilePath('source.img') as source_img_path, \
 iotests.log('Stopping the NBD server on destination...')
 iotests.log(dest_vm.qmp('nbd-server-stop'))
 break
+
+iotests.log(source_vm.qmp('query-block')['return'][0]['dirty-bitmaps'])
diff --git a/tests/qemu-iotests/194.out b/tests/qemu-iotests/194.out
index 71857853fb..dd60dcc14f 100644
--- a/tests/qemu-iotests/194.out
+++ b/tests/qemu-iotests/194.out
@@ -1,4 +1,6 @@
 Launching VMs...
+{"execute": "block-dirty-bitmap-add", "arguments": {"name": "bitmap0", "node": 
"drive0"}}
+{"return": {}}
 Launching NBD server on destination...
 {"return": {}}
 {"return": {}}
@@ -8,11 +10,15 @@ Waiting for `drive-mirror` to complete...
 {"data": {"device": "mirror-job0", "len": 1073741824, "offset": 1073741824, 
"speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_READY", "timestamp": 
{"microseconds": "USECS", "seconds": "SECS"}}
 Starting migration...
 {"return": {}}
+{"execute": "migrate-start-postcopy", "arguments": {}}
+{"return": {}}
 {"data": {"status": "setup"}, "event": "MIGRATION", "timestamp": 
{"microseconds": "USECS", "seconds": "SECS"}}
 {"data": {"status": "active"}, "event": "MIGRATION", "timestamp": 
{"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"status": "postcopy-active"}, "event": "MIGRATION", "timestamp": 
{"microseconds": "USECS", "seconds": "SECS"}}
 {"data": {"status": "completed"}, "event": "MIGRATION", "timestamp": 
{"microseconds": "USECS", "seconds": "SECS"}}
 Gracefully ending the `drive-mirror` job on source...
 {"return": {}}
 {"data": {"device": "mirror-job0", "len": 1073741824, "offset": 1073741824, 
"speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": 
{"microseconds": "USECS", "seconds": "SECS"}}
 Stopping the NBD server on destination...
 {"return": {}}
+[{"busy": false, "count": 0, "granularity": 65536, "name": "bitmap0", 
"persistent": false, "recording": true, "status": "active"}]
-- 
2.21.0

[PATCH v3 6/7] migration/block-dirty-bitmap: add_bitmaps_to_list: check disk name once

[Vladimir Sementsov-Ogievskiy]

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Andrey Shinkevich 
---
 migration/block-dirty-bitmap.c | 17 +++--
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
index 5d3a7d2b07..e0e081ce60 100644
--- a/migration/block-dirty-bitmap.c
+++ b/migration/block-dirty-bitmap.c
@@ -274,17 +274,22 @@ static int add_bitmaps_to_list(BlockDriverState *bs, 
const char *bs_name)
 DirtyBitmapMigBitmapState *dbms;
 Error *local_err = NULL;
 
+bitmap = bdrv_dirty_bitmap_first(bs);
+if (!bitmap) {
+return 0;
+}
+
+if (!bs_name || strcmp(bs_name, "") == 0) {
+error_report("Found bitmap '%s' in unnamed node %p. It can't "
+ "be migrated", bdrv_dirty_bitmap_name(bitmap), bs);
+return -1;
+}
+
 FOR_EACH_DIRTY_BITMAP(bs, bitmap) {
 if (!bdrv_dirty_bitmap_name(bitmap)) {
 continue;
 }
 
-if (!bs_name || strcmp(bs_name, "") == 0) {
-error_report("Found bitmap '%s' in unnamed node %p. It can't "
- "be migrated", bdrv_dirty_bitmap_name(bitmap), bs);
-return -1;
-}
-
 if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_DEFAULT, _err)) {
 error_report_err(local_err);
 return -1;
-- 
2.21.0

[PATCH v3 3/7] block/dirty-bitmap: add bdrv_has_named_bitmaps helper

[Vladimir Sementsov-Ogievskiy]

To be used for bitmap migration in further commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Andrey Shinkevich 
Reviewed-by: Eric Blake 
---
 include/block/dirty-bitmap.h |  1 +
 block/dirty-bitmap.c | 13 +
 2 files changed, 14 insertions(+)

diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 8a10029418..eee70df873 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -94,6 +94,7 @@ int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap);
 void bdrv_dirty_bitmap_truncate(BlockDriverState *bs, int64_t bytes);
 bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap);
 bool bdrv_has_readonly_bitmaps(BlockDriverState *bs);
+bool bdrv_has_named_bitmaps(BlockDriverState *bs);
 bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap);
 bool bdrv_dirty_bitmap_get_persistence(BdrvDirtyBitmap *bitmap);
 bool bdrv_dirty_bitmap_inconsistent(const BdrvDirtyBitmap *bitmap);
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 063793e316..c788ca7c69 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -809,6 +809,19 @@ bool bdrv_has_readonly_bitmaps(BlockDriverState *bs)
 return false;
 }
 
+bool bdrv_has_named_bitmaps(BlockDriverState *bs)
+{
+BdrvDirtyBitmap *bm;
+
+QLIST_FOREACH(bm, >dirty_bitmaps, list) {
+if (bdrv_dirty_bitmap_name(bm)) {
+return true;
+}
+}
+
+return false;
+}
+
 /* Called with BQL taken. */
 void bdrv_dirty_bitmap_set_persistence(BdrvDirtyBitmap *bitmap, bool 
persistent)
 {
-- 
2.21.0

[PATCH v3 7/7] migration/block-dirty-bitmap: forbid migration by generated node-name

[Vladimir Sementsov-Ogievskiy]

It actually never worked with libvirt, as auto-generated names are
different on source and destination.

It's unsafe and useless to migrate by auto-generated node-names, so
let's forbid it.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Andrey Shinkevich 
---
 migration/block-dirty-bitmap.c | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
index e0e081ce60..f5744c35e6 100644
--- a/migration/block-dirty-bitmap.c
+++ b/migration/block-dirty-bitmap.c
@@ -285,6 +285,13 @@ static int add_bitmaps_to_list(BlockDriverState *bs, const 
char *bs_name)
 return -1;
 }
 
+if (bs_name[0] == '#') {
+error_report("Found bitmap '%s' in a node with auto-generated "
+ "name: %s. It can't be migrated",
+ bdrv_dirty_bitmap_name(bitmap), bs_name);
+return -1;
+}
+
 FOR_EACH_DIRTY_BITMAP(bs, bitmap) {
 if (!bdrv_dirty_bitmap_name(bitmap)) {
 continue;
-- 
2.21.0