Re: [PATCH 02/22] machine: remove deprecated -machine enforce-config-section option

[Paolo Bonzini]

On 22/10/20 07:09, Thomas Huth wrote:
>> +#include "migration/misc.h"
> This new include does not make much sense to me, if all you did was removing
> code from this file. Why did you add this here?

Yeah, it's not needed anymore.  It's part of a different patch that I
threw away when I noticed the deprecation.

Paolo

> With this hunk removed:
> Reviewed-by: Thomas Huth 
> 

答复: 答复: [PATCH 1/1] Skip flatview_simplify() for cpu vendor zhaoxin

[FelixCui-oc]

hi paolo,


>So removing flatview_simplify() works because the higher area (0x1

>and above) remains the same.  I guess the simplest thing to do is to
>apply flatview_simplify() only to I/O areas, though we can also consider
>removing it completely.  I'm not sure in which case it would provide a
>noticeable improvement.


I agree with you very much. Both of these cases can solve this bug.

Thanks.


Best regards

Felixcui-oc


发件人: Paolo Bonzini 
发送时间: 2020年10月22日 11:30:37
收件人: FelixCui-oc; Alex Williamson
抄送: Richard Henderson; Eduardo Habkost; qemu-devel@nongnu.org; RockCui-oc; Tony 
W Wang-oc; CobeChen-oc
主题: Re: 答复: [PATCH 1/1] Skip flatview_simplify() for cpu vendor zhaoxin

On 22/10/20 05:02, FelixCui-oc wrote:
> In addition, before write pam registers, flatview_simplify() has merged
> a very large range.For example,
>
> this large range is 0xc-0xbfff. So even if EHCI is configured to
> not allocate buffers in low memory,
>
> this bug will still occur.Thanks.

So removing flatview_simplify() works because the higher area (0x1
and above) remains the same.  I guess the simplest thing to do is to
apply flatview_simplify() only to I/O areas, though we can also consider
removing it completely.  I'm not sure in which case it would provide a
noticeable improvement.

Paolo

Re: [PATCH v1 6/6] tests/acceptance: pick a random gdb port for reverse debugging

[Pavel Dovgalyuk]

On 21.10.2020 19:31, Alex Bennée wrote:

Currently the test randomly fails if you are using a shared machine
due to contention on the well known port 1234. We can ameliorate this
a bit by picking a random non-ephemeral port although it doesn't
totally avoid the problem. While we could use a totally unique socket
address for debugging it's impossible to probe for gdb support of the
feature which makes this a sub-optimal but less fiddly option.

Signed-off-by: Alex Bennée 


Reviewed-by: Pavel Dovgalyuk 


---
  tests/acceptance/reverse_debugging.py | 6 --
  1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/tests/acceptance/reverse_debugging.py 
b/tests/acceptance/reverse_debugging.py
index b72fdf6cdc..f2e8245471 100644
--- a/tests/acceptance/reverse_debugging.py
+++ b/tests/acceptance/reverse_debugging.py
@@ -16,6 +16,7 @@ from avocado.utils import gdb
  from avocado.utils import process
  from avocado.utils.path import find_command
  from boot_linux_console import LinuxKernelTest
+from random import randrange
  
  class ReverseDebugging(LinuxKernelTest):

  """
@@ -43,7 +44,8 @@ class ReverseDebugging(LinuxKernelTest):
  else:
  logger.info('replaying the execution...')
  mode = 'replay'
-vm.add_args('-s', '-S')
+self.port = randrange(2048, 49152)
+vm.add_args('-gdb', 'tcp::%d' % (self.port), '-S')
  vm.add_args('-icount', 'shift=%s,rr=%s,rrfile=%s,rrsnapshot=init' %
  (shift, mode, replay_path),
  '-net', 'none')
@@ -122,7 +124,7 @@ class ReverseDebugging(LinuxKernelTest):
  # replay and run debug commands
  vm = self.run_vm(False, shift, args, replay_path, image_path)
  logger.info('connecting to gdbstub')
-g = gdb.GDBRemote('127.0.0.1', 1234, False, False)
+g = gdb.GDBRemote('127.0.0.1', self.port, False, False)
  g.connect()
  r = g.cmd(b'qSupported')
  if b'qXfer:features:read+' in r:

Re: [PATCH v2 3/5] hw/misc: Add npcm7xx random number generator

[Thomas Huth]

On 21/10/2020 23.35, Havard Skinnemoen via wrote:
> The RNG module returns a byte of randomness when the Data Valid bit is
> set.
> 
> This implementation ignores the prescaler setting, and loads a new value
> into RNGD every time RNGCS is read while the RNG is enabled and random
> data is available.
> 
> A qtest featuring some simple randomness tests is included.
> 
> Reviewed-by: Tyrone Ting 
> Reviewed-by: Peter Maydell 
> Signed-off-by: Havard Skinnemoen 
> ---
[...]
> diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
> index f7b0e3ca9d..473935c3a8 100644
> --- a/tests/qtest/meson.build
> +++ b/tests/qtest/meson.build
> @@ -140,6 +140,7 @@ qtests_arm = \
> 'm25p80-test',
> 'npcm7xx_timer-test',
> 'npcm7xx_watchdog_timer-test',
> +   'npcm7xx_rng-test',
> 'test-arm-mptimer',
> 'boot-serial-test',
> 'hexloader-test']

Please fence this with CONFIG_NPCM7XX, too.

 Thanks,
  Thomas

Re: [PATCH v2 2/5] hw/timer: Adding watchdog for NPCM7XX Timer.

[Thomas Huth]

On 21/10/2020 23.35, Havard Skinnemoen via wrote:
> From: Hao Wu 
> 
> The watchdog is part of NPCM7XX's timer module. Its behavior is
> controlled by the WTCR register in the timer.
> 
> When enabled, the watchdog issues an interrupt signal after a pre-set
> amount of cycles, and issues a reset signal shortly after that.
> 
> Reviewed-by: Tyrone Ting 
> Signed-off-by: Hao Wu 
> Signed-off-by: Havard Skinnemoen 
> ---
[...]
> diff --git a/tests/qtest/npcm7xx_watchdog_timer-test.c 
> b/tests/qtest/npcm7xx_watchdog_timer-test.c
> new file mode 100644
> index 00..bfe0020ffc
> --- /dev/null
> +++ b/tests/qtest/npcm7xx_watchdog_timer-test.c
> @@ -0,0 +1,313 @@
> +/*
> + * QTests for Nuvoton NPCM7xx Timer Watchdog Modules.
> + *
> + * Copyright 2020 Google LLC
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License as published by the
> + * Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful, but 
> WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> + * for more details.
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qemu/timer.h"
> +
> +#include "libqtest-single.h"
[...]
> +/*
> + * Check a watchdog doesn't fire if corresponding flags (WTIE and WTRE) are 
> not
> + * set.
> + */
> +static void test_enabling_flags(gconstpointer watchdog)
> +{
> +const Watchdog *wd = watchdog;
> +QTestState *qts;
> +
> +/* Neither WTIE or WTRE is set, no interrupt or reset should happen */
> +qts = qtest_start("-machine quanta-gsj");
> +qtest_irq_intercept_in(qts, "/machine/soc/a9mpcore/gic");
> +watchdog_write_wtcr(wd, WTCLK(0) | WTE | WTIF | WTRF | WTR);
> +clock_step(watchdog_interrupt_steps(wd));
> +g_assert_true(watchdog_read_wtcr(wd) & WTIF);
> +g_assert_false(qtest_get_irq(qts, wd->irq));
> +clock_step(watchdog_calculate_steps(RESET_CYCLES, 
> watchdog_prescaler(wd)));
> +g_assert_true(watchdog_read_wtcr(wd) & WTIF);
> +g_assert_false(watchdog_read_wtcr(wd) & WTRF);
> +qtest_end();

You are mixing functions from libqtest.h and libqtest-single.h ... e.g. you
use qtest_get_irq() from libqtest.h instead of get_irq() from
libqtest-single.h, on the other hand, you use clock_step() from
libqtest-single.h instead of qtest_clock_step() from libqtest.h ... there is
no real problem with this, but the style looks a little bit strange.

For new tests, we prefer nowadays to avoid libqtest-single.h since it causes
trouble if you have to track multiple test states (e.g. when writing tests
that include migration tests). Could you maybe try to rewrite your code
without libqtest-single.h? I.e. use qtest_clock_step() instead of
clock_step() etc. ?

> diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
> index 28d4068718..f7b0e3ca9d 100644
> --- a/tests/qtest/meson.build
> +++ b/tests/qtest/meson.build
> @@ -139,6 +139,7 @@ qtests_arm = \
> 'microbit-test',
> 'm25p80-test',
> 'npcm7xx_timer-test',
> +   'npcm7xx_watchdog_timer-test',

Could you please fence this test with CONFIG_NPCM7XX ? ... since it won't
work if the quanta-gsj machine has not been compiled into the binary.

 Thanks,
  Thomas

[PATCH v7] sev: add sev-inject-launch-secret

[tobin]

From: Tobin Feldman-Fitzthum 

AMD SEV allows a guest owner to inject a secret blob
into the memory of a virtual machine. The secret is
encrypted with the SEV Transport Encryption Key and
integrity is guaranteed with the Transport Integrity
Key. Although QEMU facilitates the injection of the
launch secret, it cannot access the secret.

Signed-off-by: Tobin Feldman-Fitzthum 
Reviewed-by: Daniel P. Berrangé 
Reviewed-by: Brijesh Singh 
---
 include/monitor/monitor.h |  3 ++
 include/sysemu/sev.h  |  2 ++
 monitor/misc.c| 15 ++---
 qapi/misc-target.json | 18 +++
 target/i386/monitor.c |  7 +
 target/i386/sev-stub.c|  5 +++
 target/i386/sev.c | 65 +++
 target/i386/trace-events  |  1 +
 8 files changed, 112 insertions(+), 4 deletions(-)

diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 348bfad3d5..af3887bb71 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -4,6 +4,7 @@
 #include "block/block.h"
 #include "qapi/qapi-types-misc.h"
 #include "qemu/readline.h"
+#include "include/exec/hwaddr.h"
 
 typedef struct MonitorHMP MonitorHMP;
 typedef struct MonitorOptions MonitorOptions;
@@ -37,6 +38,8 @@ void monitor_flush(Monitor *mon);
 int monitor_set_cpu(Monitor *mon, int cpu_index);
 int monitor_get_cpu_index(Monitor *mon);
 
+void *gpa2hva(MemoryRegion **p_mr, hwaddr addr, uint64_t size, Error **errp);
+
 void monitor_read_command(MonitorHMP *mon, int show_prompt);
 int monitor_read_password(MonitorHMP *mon, ReadLineFunc *readline_func,
   void *opaque);
diff --git a/include/sysemu/sev.h b/include/sysemu/sev.h
index 98c1ec8d38..7ab6e3e31d 100644
--- a/include/sysemu/sev.h
+++ b/include/sysemu/sev.h
@@ -18,4 +18,6 @@
 
 void *sev_guest_init(const char *id);
 int sev_encrypt_data(void *handle, uint8_t *ptr, uint64_t len);
+int sev_inject_launch_secret(const char *hdr, const char *secret,
+ uint64_t gpa, Error **errp);
 #endif
diff --git a/monitor/misc.c b/monitor/misc.c
index 4a859fb24a..b6b2c1c60f 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -667,10 +667,10 @@ static void hmp_physical_memory_dump(Monitor *mon, const 
QDict *qdict)
 memory_dump(mon, count, format, size, addr, 1);
 }
 
-static void *gpa2hva(MemoryRegion **p_mr, hwaddr addr, Error **errp)
+void *gpa2hva(MemoryRegion **p_mr, hwaddr addr, uint64_t size, Error **errp)
 {
 MemoryRegionSection mrs = memory_region_find(get_system_memory(),
- addr, 1);
+ addr, size);
 
 if (!mrs.mr) {
 error_setg(errp, "No memory is mapped at address 0x%" HWADDR_PRIx, 
addr);
@@ -683,6 +683,13 @@ static void *gpa2hva(MemoryRegion **p_mr, hwaddr addr, 
Error **errp)
 return NULL;
 }
 
+if (mrs.size < size) {
+error_setg(errp, "Size of memory region at 0x%" HWADDR_PRIx
+   " exceeded.", addr);
+memory_region_unref(mrs.mr);
+return NULL;
+}
+
 *p_mr = mrs.mr;
 return qemu_map_ram_ptr(mrs.mr->ram_block, mrs.offset_within_region);
 }
@@ -694,7 +701,7 @@ static void hmp_gpa2hva(Monitor *mon, const QDict *qdict)
 MemoryRegion *mr = NULL;
 void *ptr;
 
-ptr = gpa2hva(&mr, addr, &local_err);
+ptr = gpa2hva(&mr, addr, 1, &local_err);
 if (local_err) {
 error_report_err(local_err);
 return;
@@ -770,7 +777,7 @@ static void hmp_gpa2hpa(Monitor *mon, const QDict *qdict)
 void *ptr;
 uint64_t physaddr;
 
-ptr = gpa2hva(&mr, addr, &local_err);
+ptr = gpa2hva(&mr, addr, 1, &local_err);
 if (local_err) {
 error_report_err(local_err);
 return;
diff --git a/qapi/misc-target.json b/qapi/misc-target.json
index 1e561fa97b..4486a543ae 100644
--- a/qapi/misc-target.json
+++ b/qapi/misc-target.json
@@ -201,6 +201,24 @@
 { 'command': 'query-sev-capabilities', 'returns': 'SevCapability',
   'if': 'defined(TARGET_I386)' }
 
+##
+# @sev-inject-launch-secret:
+#
+# This command injects a secret blob into memory of SEV guest.
+#
+# @packet-header: the launch secret packet header encoded in base64
+#
+# @secret: the launch secret data to be injected encoded in base64
+#
+# @gpa: the guest physical address where secret will be injected.
+#
+# Since: 5.2
+#
+##
+{ 'command': 'sev-inject-launch-secret',
+  'data': { 'packet-header': 'str', 'secret': 'str', 'gpa': 'uint64' },
+  'if': 'defined(TARGET_I386)' }
+
 ##
 # @dump-skeys:
 #
diff --git a/target/i386/monitor.c b/target/i386/monitor.c
index 7abae3c8df..f9d4951465 100644
--- a/target/i386/monitor.c
+++ b/target/i386/monitor.c
@@ -728,3 +728,10 @@ SevCapability *qmp_query_sev_capabilities(Error **errp)
 {
 return sev_get_capabilities(errp);
 }
+
+void qmp_sev_inject_launch_secret(const char *packet_hdr,
+  const char *secret, uint64_t gpa,
+  Error

Re: [PATCH v6] sev: add sev-inject-launch-secret

[Tobin Feldman-Fitzthum]

On 2020-10-22 00:16, to...@linux.ibm.com wrote:

From: Tobin Feldman-Fitzthum 

AMD SEV allows a guest owner to inject a secret blob
into the memory of a virtual machine. The secret is
encrypted with the SEV Transport Encryption Key and
integrity is guaranteed with the Transport Integrity
Key. Although QEMU facilitates the injection of the
launch secret, it cannot access the secret.

Signed-off-by: Tobin Feldman-Fitzthum 
Reviewed-by: Daniel P. Berrangé 
Reviewed-by: Brijesh Singh 
---
 include/monitor/monitor.h |  3 ++
 include/sysemu/sev.h  |  2 ++
 monitor/misc.c| 13 +---
 qapi/misc-target.json | 18 +++
 target/i386/monitor.c |  7 +
 target/i386/sev-stub.c|  5 +++
 target/i386/sev.c | 65 +++
 target/i386/trace-events  |  1 +
 8 files changed, 110 insertions(+), 4 deletions(-)

diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 348bfad3d5..af3887bb71 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -4,6 +4,7 @@
 #include "block/block.h"
 #include "qapi/qapi-types-misc.h"
 #include "qemu/readline.h"
+#include "include/exec/hwaddr.h"

 typedef struct MonitorHMP MonitorHMP;
 typedef struct MonitorOptions MonitorOptions;
@@ -37,6 +38,8 @@ void monitor_flush(Monitor *mon);
 int monitor_set_cpu(Monitor *mon, int cpu_index);
 int monitor_get_cpu_index(Monitor *mon);

+void *gpa2hva(MemoryRegion **p_mr, hwaddr addr, uint64_t size, Error 
**errp);

+
 void monitor_read_command(MonitorHMP *mon, int show_prompt);
 int monitor_read_password(MonitorHMP *mon, ReadLineFunc 
*readline_func,

   void *opaque);
diff --git a/include/sysemu/sev.h b/include/sysemu/sev.h
index 98c1ec8d38..7ab6e3e31d 100644
--- a/include/sysemu/sev.h
+++ b/include/sysemu/sev.h
@@ -18,4 +18,6 @@

 void *sev_guest_init(const char *id);
 int sev_encrypt_data(void *handle, uint8_t *ptr, uint64_t len);
+int sev_inject_launch_secret(const char *hdr, const char *secret,
+ uint64_t gpa, Error **errp);
 #endif
diff --git a/monitor/misc.c b/monitor/misc.c
index 4a859fb24a..dd148be5da 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -667,10 +667,10 @@ static void hmp_physical_memory_dump(Monitor
*mon, const QDict *qdict)
 memory_dump(mon, count, format, size, addr, 1);
 }

-static void *gpa2hva(MemoryRegion **p_mr, hwaddr addr, Error **errp)
+void *gpa2hva(MemoryRegion **p_mr, hwaddr addr, uint64_t size, Error 
**errp)

 {
 MemoryRegionSection mrs = memory_region_find(get_system_memory(),
- addr, 1);
+ addr, size);

 if (!mrs.mr) {
 error_setg(errp, "No memory is mapped at address 0x%"
HWADDR_PRIx, addr);
@@ -683,6 +683,11 @@ static void *gpa2hva(MemoryRegion **p_mr, hwaddr
addr, Error **errp)
 return NULL;
 }

+if (mrs.size < size) {
+error_setg(errp, "Size of memory region at 0x%" HWADDR_PRIx
+   " exceeded.", addr);
+}
+


Forgot to return :( Will update.


 *p_mr = mrs.mr;
 return qemu_map_ram_ptr(mrs.mr->ram_block, 
mrs.offset_within_region);

 }
@@ -694,7 +699,7 @@ static void hmp_gpa2hva(Monitor *mon, const QDict 
*qdict)

 MemoryRegion *mr = NULL;
 void *ptr;

-ptr = gpa2hva(&mr, addr, &local_err);
+ptr = gpa2hva(&mr, addr, 1, &local_err);
 if (local_err) {
 error_report_err(local_err);
 return;
@@ -770,7 +775,7 @@ static void hmp_gpa2hpa(Monitor *mon, const QDict 
*qdict)

 void *ptr;
 uint64_t physaddr;

-ptr = gpa2hva(&mr, addr, &local_err);
+ptr = gpa2hva(&mr, addr, 1, &local_err);
 if (local_err) {
 error_report_err(local_err);
 return;
diff --git a/qapi/misc-target.json b/qapi/misc-target.json
index 1e561fa97b..4486a543ae 100644
--- a/qapi/misc-target.json
+++ b/qapi/misc-target.json
@@ -201,6 +201,24 @@
 { 'command': 'query-sev-capabilities', 'returns': 'SevCapability',
   'if': 'defined(TARGET_I386)' }

+##
+# @sev-inject-launch-secret:
+#
+# This command injects a secret blob into memory of SEV guest.
+#
+# @packet-header: the launch secret packet header encoded in base64
+#
+# @secret: the launch secret data to be injected encoded in base64
+#
+# @gpa: the guest physical address where secret will be injected.
+#
+# Since: 5.2
+#
+##
+{ 'command': 'sev-inject-launch-secret',
+  'data': { 'packet-header': 'str', 'secret': 'str', 'gpa': 'uint64' 
},

+  'if': 'defined(TARGET_I386)' }
+
 ##
 # @dump-skeys:
 #
diff --git a/target/i386/monitor.c b/target/i386/monitor.c
index 7abae3c8df..f9d4951465 100644
--- a/target/i386/monitor.c
+++ b/target/i386/monitor.c
@@ -728,3 +728,10 @@ SevCapability *qmp_query_sev_capabilities(Error 
**errp)

 {
 return sev_get_capabilities(errp);
 }
+
+void qmp_sev_inject_launch_secret(const char *packet_hdr,
+  const char *secret, uint64_t gpa,
+ 

Re: [PATCH v6 6/6] migration-test: Only hide error if !QTEST_LOG

[Thomas Huth]

On 21/10/2020 23.27, Peter Xu wrote:
> The errors are very useful when debugging qtest failures, especially when
> QTEST_LOG=1 is set.  Let's allow override MigrateStart.hide_stderr when
> QTEST_LOG=1 is specified, because that means the user wants to be verbose.
> 
> Not very nice to introduce the first QTEST_LOG env access in migration-test.c,
> however it should be handy.  Without this patch, I was hacking error_report()
> when debugging such errors.  Let's make things easier.
> 
> Signed-off-by: Peter Xu 
> ---
>  tests/qtest/migration-test.c | 6 +-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
> index f410ec5996..f2142fbd3c 100644
> --- a/tests/qtest/migration-test.c
> +++ b/tests/qtest/migration-test.c
> @@ -464,6 +464,10 @@ static void migrate_postcopy_start(QTestState *from, 
> QTestState *to)
>  }
>  
>  typedef struct {
> +/*
> + * QTEST_LOG=1 may override this.  When QTEST_LOG=1, we always dump 
> errors
> + * unconditionally, because it means the user would like to be verbose.
> + */
>  bool hide_stderr;
>  bool use_shmem;
>  /* only launch the target process */
> @@ -557,7 +561,7 @@ static int test_migrate_start(QTestState **from, 
> QTestState **to,
>  
>  g_free(bootpath);
>  
> -if (args->hide_stderr) {
> +if (!getenv("QTEST_LOG") && args->hide_stderr) {
>  ignore_stderr = "2>/dev/null";
>  } else {
>  ignore_stderr = "";

Reviewed-by: Thomas Huth 

[PATCH 1/2] hw/riscv: sifive_u: Allow passing custom DTB

[Anup Patel]

Extend sifive_u machine to allow passing custom DTB using "-dtb"
command-line parameter. This will help users pass modified DTB
or Linux SiFive DTB to sifive_u machine.

Signed-off-by: Anup Patel 
---
 hw/riscv/sifive_u.c | 28 
 1 file changed, 20 insertions(+), 8 deletions(-)

diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
index 6ad975d692..554e38abf0 100644
--- a/hw/riscv/sifive_u.c
+++ b/hw/riscv/sifive_u.c
@@ -100,14 +100,25 @@ static void create_fdt(SiFiveUState *s, const struct 
MemmapEntry *memmap,
 int cpu;
 uint32_t *cells;
 char *nodename;
+const char *dtb_filename;
 char ethclk_names[] = "pclk\0hclk";
 uint32_t plic_phandle, prci_phandle, gpio_phandle, phandle = 1;
 uint32_t hfclk_phandle, rtcclk_phandle, phy_phandle;
 
-fdt = s->fdt = create_device_tree(&s->fdt_size);
-if (!fdt) {
-error_report("create_device_tree() failed");
-exit(1);
+dtb_filename = qemu_opt_get(qemu_get_machine_opts(), "dtb");
+if (dtb_filename) {
+fdt = s->fdt = load_device_tree(dtb_filename, &s->fdt_size);
+if (!fdt) {
+error_report("load_device_tree() failed");
+exit(1);
+}
+goto update_bootargs;
+} else {
+fdt = s->fdt = create_device_tree(&s->fdt_size);
+if (!fdt) {
+error_report("create_device_tree() failed");
+exit(1);
+}
 }
 
 qemu_fdt_setprop_string(fdt, "/", "model", "SiFive HiFive Unleashed A00");
@@ -390,13 +401,14 @@ static void create_fdt(SiFiveUState *s, const struct 
MemmapEntry *memmap,
 
 qemu_fdt_add_subnode(fdt, "/chosen");
 qemu_fdt_setprop_string(fdt, "/chosen", "stdout-path", nodename);
-if (cmdline) {
-qemu_fdt_setprop_string(fdt, "/chosen", "bootargs", cmdline);
-}
-
 qemu_fdt_setprop_string(fdt, "/aliases", "serial0", nodename);
 
 g_free(nodename);
+
+update_bootargs:
+if (cmdline) {
+qemu_fdt_setprop_string(fdt, "/chosen", "bootargs", cmdline);
+}
 }
 
 static void sifive_u_machine_reset(void *opaque, int n, int level)
-- 
2.25.1

[PATCH 2/2] hw/riscv: virt: Allow passing custom DTB

[Anup Patel]

Extend virt machine to allow passing custom DTB using "-dtb"
command-line parameter. This will help users pass modified DTB
to virt machine.

Signed-off-by: Anup Patel 
---
 hw/riscv/virt.c | 27 ---
 1 file changed, 20 insertions(+), 7 deletions(-)

diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 41bd2f38ba..d535119e37 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -181,6 +181,7 @@ static void create_fdt(RISCVVirtState *s, const struct 
MemmapEntry *memmap,
 {
 void *fdt;
 int i, cpu, socket;
+const char *dtb_filename;
 MachineState *mc = MACHINE(s);
 uint64_t addr, size;
 uint32_t *clint_cells, *plic_cells;
@@ -194,10 +195,20 @@ static void create_fdt(RISCVVirtState *s, const struct 
MemmapEntry *memmap,
 hwaddr flashsize = virt_memmap[VIRT_FLASH].size / 2;
 hwaddr flashbase = virt_memmap[VIRT_FLASH].base;
 
-fdt = s->fdt = create_device_tree(&s->fdt_size);
-if (!fdt) {
-error_report("create_device_tree() failed");
-exit(1);
+dtb_filename = qemu_opt_get(qemu_get_machine_opts(), "dtb");
+if (dtb_filename) {
+fdt = s->fdt = load_device_tree(dtb_filename, &s->fdt_size);
+if (!fdt) {
+error_report("load_device_tree() failed");
+exit(1);
+}
+goto update_bootargs;
+} else {
+fdt = s->fdt = create_device_tree(&s->fdt_size);
+if (!fdt) {
+error_report("create_device_tree() failed");
+exit(1);
+}
 }
 
 qemu_fdt_setprop_string(fdt, "/", "model", "riscv-virtio,qemu");
@@ -418,9 +429,6 @@ static void create_fdt(RISCVVirtState *s, const struct 
MemmapEntry *memmap,
 
 qemu_fdt_add_subnode(fdt, "/chosen");
 qemu_fdt_setprop_string(fdt, "/chosen", "stdout-path", name);
-if (cmdline) {
-qemu_fdt_setprop_string(fdt, "/chosen", "bootargs", cmdline);
-}
 g_free(name);
 
 name = g_strdup_printf("/soc/rtc@%lx", (long)memmap[VIRT_RTC].base);
@@ -441,6 +449,11 @@ static void create_fdt(RISCVVirtState *s, const struct 
MemmapEntry *memmap,
  2, flashbase + flashsize, 2, flashsize);
 qemu_fdt_setprop_cell(s->fdt, name, "bank-width", 4);
 g_free(name);
+
+update_bootargs:
+if (cmdline) {
+qemu_fdt_setprop_string(fdt, "/chosen", "bootargs", cmdline);
+}
 }
 
 static inline DeviceState *gpex_pcie_init(MemoryRegion *sys_mem,
-- 
2.25.1

Re: [PATCH v1 6/6] tests/acceptance: pick a random gdb port for reverse debugging

[Thomas Huth]

On 21/10/2020 18.31, Alex Bennée wrote:
> Currently the test randomly fails if you are using a shared machine
> due to contention on the well known port 1234. We can ameliorate this
> a bit by picking a random non-ephemeral port although it doesn't
> totally avoid the problem. While we could use a totally unique socket
> address for debugging it's impossible to probe for gdb support of the
> feature which makes this a sub-optimal but less fiddly option.
> 
> Signed-off-by: Alex Bennée 
> ---
>  tests/acceptance/reverse_debugging.py | 6 --
>  1 file changed, 4 insertions(+), 2 deletions(-)

Certainly better than before!

Reviewed-by: Thomas Huth 

[PULL 07/14] spice: move display_add_client() to QemuSpiceOps.

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Message-id: 20201019075224.14803-8-kra...@redhat.com
---
 include/ui/qemu-spice-module.h | 1 +
 include/ui/qemu-spice.h| 6 --
 monitor/qmp-cmds.c | 2 +-
 ui/spice-core.c| 3 ++-
 ui/spice-module.c  | 7 +++
 5 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/include/ui/qemu-spice-module.h b/include/ui/qemu-spice-module.h
index 1ea3a999ce8a..7422f6404ba0 100644
--- a/include/ui/qemu-spice-module.h
+++ b/include/ui/qemu-spice-module.h
@@ -29,6 +29,7 @@ struct QemuSpiceOps {
 int (*set_passwd)(const char *passwd,
   bool fail_if_connected, bool disconnect_if_connected);
 int (*set_pw_expire)(time_t expires);
+int (*display_add_client)(int csock, int skipauth, int tls);
 #ifdef CONFIG_SPICE
 int (*add_interface)(SpiceBaseInstance *sin);
 #endif
diff --git a/include/ui/qemu-spice.h b/include/ui/qemu-spice.h
index 921b7a38d023..2beb7929728c 100644
--- a/include/ui/qemu-spice.h
+++ b/include/ui/qemu-spice.h
@@ -28,7 +28,6 @@
 
 void qemu_spice_input_init(void);
 void qemu_spice_display_init(void);
-int qemu_spice_display_add_client(int csock, int skipauth, int tls);
 bool qemu_spice_have_display_interface(QemuConsole *con);
 int qemu_spice_add_display_interface(QXLInstance *qxlin, QemuConsole *con);
 int qemu_spice_migrate_info(const char *hostname, int port, int tls_port,
@@ -45,11 +44,6 @@ int qemu_spice_migrate_info(const char *hostname, int port, 
int tls_port,
 #include "qemu/error-report.h"
 
 #define spice_displays 0
-static inline int qemu_spice_display_add_client(int csock, int skipauth,
-int tls)
-{
-return -1;
-}
 
 #endif /* CONFIG_SPICE */
 
diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c
index 8ac59977e661..a08143b32335 100644
--- a/monitor/qmp-cmds.c
+++ b/monitor/qmp-cmds.c
@@ -340,7 +340,7 @@ void qmp_add_client(const char *protocol, const char 
*fdname,
 }
 skipauth = has_skipauth ? skipauth : false;
 tls = has_tls ? tls : false;
-if (qemu_spice_display_add_client(fd, skipauth, tls) < 0) {
+if (qemu_spice.display_add_client(fd, skipauth, tls) < 0) {
 error_setg(errp, "spice failed to add client");
 close(fd);
 }
diff --git a/ui/spice-core.c b/ui/spice-core.c
index 4fe543aba058..99457c226e6f 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -959,7 +959,7 @@ static int qemu_spice_set_pw_expire(time_t expires)
 return qemu_spice_set_ticket(false, false);
 }
 
-int qemu_spice_display_add_client(int csock, int skipauth, int tls)
+static int qemu_spice_display_add_client(int csock, int skipauth, int tls)
 {
 if (tls) {
 return spice_server_add_ssl_client(spice_server, csock, skipauth);
@@ -999,6 +999,7 @@ static struct QemuSpiceOps real_spice_ops = {
 .migrate_info = qemu_spice_migrate_info,
 .set_passwd   = qemu_spice_set_passwd,
 .set_pw_expire = qemu_spice_set_pw_expire,
+.display_add_client = qemu_spice_display_add_client,
 .add_interface = qemu_spice_add_interface,
 };
 
diff --git a/ui/spice-module.c b/ui/spice-module.c
index 299aeb479be5..8fbc99c03c55 100644
--- a/ui/spice-module.c
+++ b/ui/spice-module.c
@@ -52,10 +52,17 @@ static int qemu_spice_set_pw_expire_stub(time_t expires)
 return -1;
 }
 
+static int qemu_spice_display_add_client_stub(int csock, int skipauth,
+  int tls)
+{
+return -1;
+}
+
 struct QemuSpiceOps qemu_spice = {
 .init = qemu_spice_init_stub,
 .display_init = qemu_spice_display_init_stub,
 .migrate_info = qemu_spice_migrate_info_stub,
 .set_passwd   = qemu_spice_set_passwd_stub,
 .set_pw_expire = qemu_spice_set_pw_expire_stub,
+.display_add_client = qemu_spice_display_add_client_stub,
 };
-- 
2.27.0

[PULL 10/14] modules: dependencies infrastructure

[Gerd Hoffmann]

Allow modules depending on other modules.

module_load_file() gets the option to export symbols (by not adding the
G_MODULE_BIND_LOCAL flag).

module_load_one() will check the module dependency list to figure (a)
whenever are other modules must be loaded first, or (b) the module
should export the symbols.

The dependencies are specificed as static list in the source code for
now as I expect the list will stay small.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Message-id: 20201019075224.14803-11-kra...@redhat.com
---
 util/module.c | 32 +++-
 1 file changed, 27 insertions(+), 5 deletions(-)

diff --git a/util/module.c b/util/module.c
index f0ed05fbd06b..0c0f258923dc 100644
--- a/util/module.c
+++ b/util/module.c
@@ -110,7 +110,7 @@ void module_call_init(module_init_type type)
 }
 
 #ifdef CONFIG_MODULES
-static int module_load_file(const char *fname, bool mayfail)
+static int module_load_file(const char *fname, bool mayfail, bool 
export_symbols)
 {
 GModule *g_module;
 void (*sym)(void);
@@ -118,7 +118,7 @@ static int module_load_file(const char *fname, bool mayfail)
 int len = strlen(fname);
 int suf_len = strlen(dsosuf);
 ModuleEntry *e, *next;
-int ret;
+int ret, flags;
 
 if (len <= suf_len || strcmp(&fname[len - suf_len], dsosuf)) {
 /* wrong suffix */
@@ -132,7 +132,11 @@ static int module_load_file(const char *fname, bool 
mayfail)
 
 assert(QTAILQ_EMPTY(&dso_init_list));
 
-g_module = g_module_open(fname, G_MODULE_BIND_LAZY | G_MODULE_BIND_LOCAL);
+flags = G_MODULE_BIND_LAZY;
+if (!export_symbols) {
+flags |= G_MODULE_BIND_LOCAL;
+}
+g_module = g_module_open(fname, flags);
 if (!g_module) {
 if (!mayfail) {
 fprintf(stderr, "Failed to open module: %s\n",
@@ -167,6 +171,12 @@ static int module_load_file(const char *fname, bool 
mayfail)
 out:
 return ret;
 }
+
+static const struct {
+const char *name;
+const char *dep;
+} module_deps[] = {
+};
 #endif
 
 bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
@@ -182,7 +192,8 @@ bool module_load_one(const char *prefix, const char 
*lib_name, bool mayfail)
 char *dirs[5];
 char *module_name;
 int i = 0, n_dirs = 0;
-int ret;
+int ret, dep;
+bool export_symbols = false;
 static GHashTable *loaded_modules;
 
 if (!g_module_supported()) {
@@ -196,6 +207,17 @@ bool module_load_one(const char *prefix, const char 
*lib_name, bool mayfail)
 
 module_name = g_strdup_printf("%s%s", prefix, lib_name);
 
+for (dep = 0; dep < ARRAY_SIZE(module_deps); dep++) {
+if (strcmp(module_name, module_deps[dep].name) == 0) {
+/* we depend on another module */
+module_load_one("", module_deps[dep].dep, false);
+}
+if (strcmp(module_name, module_deps[dep].dep) == 0) {
+/* another module depends on us */
+export_symbols = true;
+}
+}
+
 if (!g_hash_table_add(loaded_modules, module_name)) {
 g_free(module_name);
 return true;
@@ -220,7 +242,7 @@ bool module_load_one(const char *prefix, const char 
*lib_name, bool mayfail)
 for (i = 0; i < n_dirs; i++) {
 fname = g_strdup_printf("%s/%s%s",
 dirs[i], module_name, CONFIG_HOST_DSOSUF);
-ret = module_load_file(fname, mayfail);
+ret = module_load_file(fname, mayfail, export_symbols);
 g_free(fname);
 fname = NULL;
 /* Try loading until loaded a module file */
-- 
2.27.0

[PULL 05/14] spice: move add_interface() to QemuSpiceOps.

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Message-id: 20201019075224.14803-6-kra...@redhat.com
---
 include/ui/qemu-spice-module.h | 7 +++
 include/ui/qemu-spice.h| 1 -
 audio/spiceaudio.c | 4 ++--
 chardev/spice.c| 2 +-
 ui/spice-core.c| 5 +++--
 ui/spice-input.c   | 6 +++---
 6 files changed, 16 insertions(+), 9 deletions(-)

diff --git a/include/ui/qemu-spice-module.h b/include/ui/qemu-spice-module.h
index dbe09035dade..f93acde5743c 100644
--- a/include/ui/qemu-spice-module.h
+++ b/include/ui/qemu-spice-module.h
@@ -18,10 +18,17 @@
 #ifndef QEMU_SPICE_MODULE_H
 #define QEMU_SPICE_MODULE_H
 
+#ifdef CONFIG_SPICE
+#include 
+#endif
+
 struct QemuSpiceOps {
 void (*init)(void);
 void (*display_init)(void);
 int (*migrate_info)(const char *h, int p, int t, const char *s);
+#ifdef CONFIG_SPICE
+int (*add_interface)(SpiceBaseInstance *sin);
+#endif
 };
 
 extern int using_spice;
diff --git a/include/ui/qemu-spice.h b/include/ui/qemu-spice.h
index a3fd1ea5ade8..6018577c5278 100644
--- a/include/ui/qemu-spice.h
+++ b/include/ui/qemu-spice.h
@@ -29,7 +29,6 @@
 void qemu_spice_input_init(void);
 void qemu_spice_display_init(void);
 int qemu_spice_display_add_client(int csock, int skipauth, int tls);
-int qemu_spice_add_interface(SpiceBaseInstance *sin);
 bool qemu_spice_have_display_interface(QemuConsole *con);
 int qemu_spice_add_display_interface(QXLInstance *qxlin, QemuConsole *con);
 int qemu_spice_set_passwd(const char *passwd,
diff --git a/audio/spiceaudio.c b/audio/spiceaudio.c
index ed6dff1dcc41..8967cca12925 100644
--- a/audio/spiceaudio.c
+++ b/audio/spiceaudio.c
@@ -106,7 +106,7 @@ static int line_out_init(HWVoiceOut *hw, struct audsettings 
*as,
 out->active = 0;
 
 out->sin.base.sif = &playback_sif.base;
-qemu_spice_add_interface (&out->sin.base);
+qemu_spice.add_interface(&out->sin.base);
 #if SPICE_INTERFACE_PLAYBACK_MAJOR > 1 || SPICE_INTERFACE_PLAYBACK_MINOR >= 3
 spice_server_set_playback_rate(&out->sin, settings.freq);
 #endif
@@ -215,7 +215,7 @@ static int line_in_init(HWVoiceIn *hw, struct audsettings 
*as, void *drv_opaque)
 in->active = 0;
 
 in->sin.base.sif = &record_sif.base;
-qemu_spice_add_interface (&in->sin.base);
+qemu_spice.add_interface(&in->sin.base);
 #if SPICE_INTERFACE_RECORD_MAJOR > 2 || SPICE_INTERFACE_RECORD_MINOR >= 3
 spice_server_set_record_rate(&in->sin, settings.freq);
 #endif
diff --git a/chardev/spice.c b/chardev/spice.c
index 7d1fb1771894..1104426e3a11 100644
--- a/chardev/spice.c
+++ b/chardev/spice.c
@@ -110,7 +110,7 @@ static void vmc_register_interface(SpiceChardev *scd)
 return;
 }
 scd->sin.base.sif = &vmc_interface.base;
-qemu_spice_add_interface(&scd->sin.base);
+qemu_spice.add_interface(&scd->sin.base);
 scd->active = true;
 trace_spice_vmc_register_interface(scd);
 }
diff --git a/ui/spice-core.c b/ui/spice-core.c
index 82d5dbda3e26..483d880a3362 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -800,7 +800,7 @@ static void qemu_spice_init(void)
 migration_state.notify = migration_state_notifier;
 add_migration_state_change_notifier(&migration_state);
 spice_migrate.base.sif = &migrate_interface.base;
-qemu_spice_add_interface(&spice_migrate.base);
+qemu_spice.add_interface(&spice_migrate.base);
 
 qemu_spice_input_init();
 
@@ -829,7 +829,7 @@ static void qemu_spice_init(void)
 #endif
 }
 
-int qemu_spice_add_interface(SpiceBaseInstance *sin)
+static int qemu_spice_add_interface(SpiceBaseInstance *sin)
 {
 if (!spice_server) {
 if (QTAILQ_FIRST(&qemu_spice_opts.head) != NULL) {
@@ -997,6 +997,7 @@ static struct QemuSpiceOps real_spice_ops = {
 .init = qemu_spice_init,
 .display_init = qemu_spice_display_init,
 .migrate_info = qemu_spice_migrate_info,
+.add_interface = qemu_spice_add_interface,
 };
 
 static void spice_register_config(void)
diff --git a/ui/spice-input.c b/ui/spice-input.c
index 21990fa9962b..bbd502564edf 100644
--- a/ui/spice-input.c
+++ b/ui/spice-input.c
@@ -231,7 +231,7 @@ static void mouse_mode_notifier(Notifier *notifier, void 
*data)
 }
 
 if (is_absolute) {
-qemu_spice_add_interface(&pointer->tablet.base);
+qemu_spice.add_interface(&pointer->tablet.base);
 } else {
 spice_server_remove_interface(&pointer->tablet.base);
 }
@@ -245,13 +245,13 @@ void qemu_spice_input_init(void)
 
 kbd = g_malloc0(sizeof(*kbd));
 kbd->sin.base.sif = &kbd_interface.base;
-qemu_spice_add_interface(&kbd->sin.base);
+qemu_spice.add_interface(&kbd->sin.base);
 qemu_add_led_event_handler(kbd_leds, kbd);
 
 pointer = g_malloc0(sizeof(*pointer));
 pointer->mouse.base.sif  = &mouse_interface.base;
 pointer->tablet.base.sif = &tablet_interface.base;
-qemu_spice_add_interface(&pointer->mouse.base);
+qemu_spice.add_interface(&pointer->mouse.base);
 
 

[PULL 14/14] opengl: build opengl helper code modular

[Gerd Hoffmann]

Removes opengl dependency from core qemu.  The number of shared
libraries for qemu-system-x86_64 goes down from 66 to 60 on my system.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Message-id: 20201019075224.14803-15-kra...@redhat.com
---
 util/module.c  | 7 +++
 ui/meson.build | 8 +++-
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/util/module.c b/util/module.c
index 21237dcc24df..fe3b82dd4d37 100644
--- a/util/module.c
+++ b/util/module.c
@@ -181,6 +181,13 @@ static const struct {
 { "hw-display-qxl", "ui-spice-core" },
 { "ui-spice-app",   "ui-spice-core" },
 { "ui-spice-app",   "chardev-spice" },
+
+#ifdef CONFIG_OPENGL
+{ "ui-egl-headless", "ui-opengl"},
+{ "ui-gtk",  "ui-opengl"},
+{ "ui-sdl",  "ui-opengl"},
+{ "ui-spice-core",   "ui-opengl"},
+#endif
 };
 #endif
 
diff --git a/ui/meson.build b/ui/meson.build
index 537e5e067358..5d4906c023b9 100644
--- a/ui/meson.build
+++ b/ui/meson.build
@@ -33,7 +33,6 @@ vnc_ss.add(zlib, png, jpeg)
 vnc_ss.add(when: sasl, if_true: files('vnc-auth-sasl.c'))
 softmmu_ss.add_all(when: vnc, if_true: vnc_ss)
 softmmu_ss.add(when: vnc, if_false: files('vnc-stubs.c'))
-softmmu_ss.add(when: [opengl, 'CONFIG_OPENGL'], if_true: files('shader.c', 
'console-gl.c', 'egl-helpers.c', 'egl-context.c'))
 specific_ss.add(when: ['CONFIG_SOFTMMU'], if_true: opengl)
 
 ui_modules = {}
@@ -44,6 +43,13 @@ if curses.found()
   ui_modules += {'curses' : curses_ss}
 endif
 
+if config_host.has_key('CONFIG_OPENGL')
+  opengl_ss = ss.source_set()
+  opengl_ss.add(when: [opengl, pixman, 'CONFIG_OPENGL'],
+   if_true: files('shader.c', 'console-gl.c', 'egl-helpers.c', 
'egl-context.c'))
+  ui_modules += {'opengl' : opengl_ss}
+endif
+
 if config_host.has_key('CONFIG_OPENGL_DMABUF')
   egl_headless_ss = ss.source_set()
   egl_headless_ss.add(when: [opengl, pixman, 'CONFIG_OPENGL_DMABUF'],
-- 
2.27.0

[PULL 12/14] spice: flip modules switch

[Gerd Hoffmann]

Build spice core code as module.  This removes libspice-server and a
handful of indirect dependencies from core qemu.  The number of shared
libraries for qemu-system-x86_64 goes down from 73 to 66 on my system.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Message-id: 20201019075224.14803-13-kra...@redhat.com
---
 ui/meson.build | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/ui/meson.build b/ui/meson.build
index e89e298643a3..509739709ef2 100644
--- a/ui/meson.build
+++ b/ui/meson.build
@@ -15,7 +15,6 @@ softmmu_ss.add(files(
 softmmu_ss.add([spice_headers, files('spice-module.c')])
 
 softmmu_ss.add(when: 'CONFIG_LINUX', if_true: files('input-linux.c'))
-softmmu_ss.add(when: [spice, 'CONFIG_SPICE'], if_true: files('spice-core.c', 
'spice-input.c', 'spice-display.c'))
 softmmu_ss.add(when: cocoa, if_true: files('cocoa.m'))
 
 vnc_ss = ss.source_set()
@@ -71,6 +70,16 @@ if sdl.found()
   ui_modules += {'sdl' : sdl_ss}
 endif
 
+if config_host.has_key('CONFIG_SPICE')
+  spice_core_ss = ss.source_set()
+  spice_core_ss.add(spice, pixman, files(
+'spice-core.c',
+'spice-input.c',
+'spice-display.c'
+  ))
+  ui_modules += {'spice-core' : spice_core_ss}
+endif
+
 if config_host.has_key('CONFIG_SPICE') and config_host.has_key('CONFIG_GIO')
   spice_ss = ss.source_set()
   spice_ss.add(spice, gio, pixman, files('spice-app.c'))
-- 
2.27.0

[PULL 13/14] opengl: build egl-headless display modular

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Message-id: 20201019075224.14803-14-kra...@redhat.com
---
 ui/meson.build | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/ui/meson.build b/ui/meson.build
index 509739709ef2..537e5e067358 100644
--- a/ui/meson.build
+++ b/ui/meson.build
@@ -34,7 +34,6 @@ vnc_ss.add(when: sasl, if_true: files('vnc-auth-sasl.c'))
 softmmu_ss.add_all(when: vnc, if_true: vnc_ss)
 softmmu_ss.add(when: vnc, if_false: files('vnc-stubs.c'))
 softmmu_ss.add(when: [opengl, 'CONFIG_OPENGL'], if_true: files('shader.c', 
'console-gl.c', 'egl-helpers.c', 'egl-context.c'))
-softmmu_ss.add(when: [opengl, 'CONFIG_OPENGL_DMABUF'], if_true: 
files('egl-headless.c'))
 specific_ss.add(when: ['CONFIG_SOFTMMU'], if_true: opengl)
 
 ui_modules = {}
@@ -45,6 +44,13 @@ if curses.found()
   ui_modules += {'curses' : curses_ss}
 endif
 
+if config_host.has_key('CONFIG_OPENGL_DMABUF')
+  egl_headless_ss = ss.source_set()
+  egl_headless_ss.add(when: [opengl, pixman, 'CONFIG_OPENGL_DMABUF'],
+  if_true: files('egl-headless.c'))
+  ui_modules += {'egl-headless' : egl_headless_ss}
+endif
+
 if config_host.has_key('CONFIG_GTK')
   softmmu_ss.add(when: 'CONFIG_WIN32', if_true: files('win32-kbd-hook.c'))
 
-- 
2.27.0

[PULL 06/14] spice: move auth functions to QemuSpiceOps.

[Gerd Hoffmann]

Move qemu_spice_set_passwd() and qemu_spice_set_pw_expire() functions to
QemuSpiceOps.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Message-id: 20201019075224.14803-7-kra...@redhat.com
---
 include/ui/qemu-spice-module.h |  3 +++
 include/ui/qemu-spice.h| 14 --
 monitor/qmp-cmds.c |  4 ++--
 ui/spice-core.c| 10 ++
 ui/spice-module.c  | 14 ++
 5 files changed, 25 insertions(+), 20 deletions(-)

diff --git a/include/ui/qemu-spice-module.h b/include/ui/qemu-spice-module.h
index f93acde5743c..1ea3a999ce8a 100644
--- a/include/ui/qemu-spice-module.h
+++ b/include/ui/qemu-spice-module.h
@@ -26,6 +26,9 @@ struct QemuSpiceOps {
 void (*init)(void);
 void (*display_init)(void);
 int (*migrate_info)(const char *h, int p, int t, const char *s);
+int (*set_passwd)(const char *passwd,
+  bool fail_if_connected, bool disconnect_if_connected);
+int (*set_pw_expire)(time_t expires);
 #ifdef CONFIG_SPICE
 int (*add_interface)(SpiceBaseInstance *sin);
 #endif
diff --git a/include/ui/qemu-spice.h b/include/ui/qemu-spice.h
index 6018577c5278..921b7a38d023 100644
--- a/include/ui/qemu-spice.h
+++ b/include/ui/qemu-spice.h
@@ -31,9 +31,6 @@ void qemu_spice_display_init(void);
 int qemu_spice_display_add_client(int csock, int skipauth, int tls);
 bool qemu_spice_have_display_interface(QemuConsole *con);
 int qemu_spice_add_display_interface(QXLInstance *qxlin, QemuConsole *con);
-int qemu_spice_set_passwd(const char *passwd,
-  bool fail_if_connected, bool 
disconnect_if_connected);
-int qemu_spice_set_pw_expire(time_t expires);
 int qemu_spice_migrate_info(const char *hostname, int port, int tls_port,
 const char *subject);
 
@@ -48,17 +45,6 @@ int qemu_spice_migrate_info(const char *hostname, int port, 
int tls_port,
 #include "qemu/error-report.h"
 
 #define spice_displays 0
-static inline int qemu_spice_set_passwd(const char *passwd,
-bool fail_if_connected,
-bool disconnect_if_connected)
-{
-return -1;
-}
-static inline int qemu_spice_set_pw_expire(time_t expires)
-{
-return -1;
-}
-
 static inline int qemu_spice_display_add_client(int csock, int skipauth,
 int tls)
 {
diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c
index 1abef70a8959..8ac59977e661 100644
--- a/monitor/qmp-cmds.c
+++ b/monitor/qmp-cmds.c
@@ -197,7 +197,7 @@ void qmp_set_password(const char *protocol, const char 
*password,
 if (!qemu_using_spice(errp)) {
 return;
 }
-rc = qemu_spice_set_passwd(password, fail_if_connected,
+rc = qemu_spice.set_passwd(password, fail_if_connected,
disconnect_if_connected);
 if (rc != 0) {
 error_setg(errp, QERR_SET_PASSWD_FAILED);
@@ -243,7 +243,7 @@ void qmp_expire_password(const char *protocol, const char 
*whenstr,
 if (!qemu_using_spice(errp)) {
 return;
 }
-rc = qemu_spice_set_pw_expire(when);
+rc = qemu_spice.set_pw_expire(when);
 if (rc != 0) {
 error_setg(errp, QERR_SET_PASSWD_FAILED);
 }
diff --git a/ui/spice-core.c b/ui/spice-core.c
index 483d880a3362..4fe543aba058 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -727,7 +727,7 @@ static void qemu_spice_init(void)
  tls_ciphers);
 }
 if (password) {
-qemu_spice_set_passwd(password, false, false);
+qemu_spice.set_passwd(password, false, false);
 }
 if (qemu_opt_get_bool(opts, "sasl", 0)) {
 if (spice_server_set_sasl(spice_server, 1) == -1) {
@@ -941,8 +941,8 @@ static int qemu_spice_set_ticket(bool fail_if_conn, bool 
disconnect_if_conn)
fail_if_conn, disconnect_if_conn);
 }
 
-int qemu_spice_set_passwd(const char *passwd,
-  bool fail_if_conn, bool disconnect_if_conn)
+static int qemu_spice_set_passwd(const char *passwd,
+ bool fail_if_conn, bool disconnect_if_conn)
 {
 if (strcmp(auth, "spice") != 0) {
 return -1;
@@ -953,7 +953,7 @@ int qemu_spice_set_passwd(const char *passwd,
 return qemu_spice_set_ticket(fail_if_conn, disconnect_if_conn);
 }
 
-int qemu_spice_set_pw_expire(time_t expires)
+static int qemu_spice_set_pw_expire(time_t expires)
 {
 auth_expires = expires;
 return qemu_spice_set_ticket(false, false);
@@ -997,6 +997,8 @@ static struct QemuSpiceOps real_spice_ops = {
 .init = qemu_spice_init,
 .display_init = qemu_spice_display_init,
 .migrate_info = qemu_spice_migrate_info,
+.set_passwd   = qemu_spice_set_passwd,
+.set_pw_expire = qemu_spice_set_pw_expire,
 .add_interface = qemu_spice_add_interface,
 };
 
diff --git a/ui/spice-module.c b/

[PULL 03/14] spice: move qemu_spice_init() to QemuSpiceOps.

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Message-id: 20201019075224.14803-4-kra...@redhat.com
---
 include/ui/qemu-spice-module.h | 1 +
 include/ui/qemu-spice.h| 5 -
 softmmu/vl.c   | 2 +-
 ui/spice-core.c| 3 ++-
 ui/spice-module.c  | 5 +
 5 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/include/ui/qemu-spice-module.h b/include/ui/qemu-spice-module.h
index 7a9963dd5810..b182bc4c21a6 100644
--- a/include/ui/qemu-spice-module.h
+++ b/include/ui/qemu-spice-module.h
@@ -19,6 +19,7 @@
 #define QEMU_SPICE_MODULE_H
 
 struct QemuSpiceOps {
+void (*init)(void);
 int (*migrate_info)(const char *h, int p, int t, const char *s);
 };
 
diff --git a/include/ui/qemu-spice.h b/include/ui/qemu-spice.h
index 3157016c2bb4..e6df0a8715b0 100644
--- a/include/ui/qemu-spice.h
+++ b/include/ui/qemu-spice.h
@@ -26,7 +26,6 @@
 #include 
 #include "qemu/config-file.h"
 
-void qemu_spice_init(void);
 void qemu_spice_input_init(void);
 void qemu_spice_display_init(void);
 int qemu_spice_display_add_client(int csock, int skipauth, int tls);
@@ -74,10 +73,6 @@ static inline void qemu_spice_display_init(void)
 abort();
 }
 
-static inline void qemu_spice_init(void)
-{
-}
-
 #endif /* CONFIG_SPICE */
 
 static inline bool qemu_using_spice(Error **errp)
diff --git a/softmmu/vl.c b/softmmu/vl.c
index cb476aa70bcc..65c34a6b8a2e 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -4151,7 +4151,7 @@ void qemu_init(int argc, char **argv, char **envp)
 /* spice needs the timers to be initialized by this point */
 /* spice must initialize before audio as it changes the default auiodev */
 /* spice must initialize before chardevs (for spicevmc and spiceport) */
-qemu_spice_init();
+qemu_spice.init();
 
 qemu_opts_foreach(qemu_find_opts("chardev"),
   chardev_init_func, NULL, &error_fatal);
diff --git a/ui/spice-core.c b/ui/spice-core.c
index b03d743cf9b9..6ef66eb387de 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -633,7 +633,7 @@ static void vm_change_state_handler(void *opaque, int 
running,
 }
 }
 
-void qemu_spice_init(void)
+static void qemu_spice_init(void)
 {
 QemuOpts *opts = QTAILQ_FIRST(&qemu_spice_opts.head);
 const char *password, *str, *x509_dir, *addr,
@@ -994,6 +994,7 @@ int qemu_spice_display_is_running(SimpleSpiceDisplay *ssd)
 }
 
 static struct QemuSpiceOps real_spice_ops = {
+.init = qemu_spice_init,
 .migrate_info = qemu_spice_migrate_info,
 };
 
diff --git a/ui/spice-module.c b/ui/spice-module.c
index f1939545a684..a30fa452ea93 100644
--- a/ui/spice-module.c
+++ b/ui/spice-module.c
@@ -22,6 +22,10 @@
 
 int using_spice;
 
+static void qemu_spice_init_stub(void)
+{
+}
+
 static int qemu_spice_migrate_info_stub(const char *h, int p, int t,
 const char *s)
 {
@@ -29,5 +33,6 @@ static int qemu_spice_migrate_info_stub(const char *h, int p, 
int t,
 }
 
 struct QemuSpiceOps qemu_spice = {
+.init = qemu_spice_init_stub,
 .migrate_info = qemu_spice_migrate_info_stub,
 };
-- 
2.27.0

[PULL 09/14] spice: load module when enabled on the cmdline

[Gerd Hoffmann]

In case the spice opts are not registered, try loading the spice module.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Message-id: 20201019075224.14803-10-kra...@redhat.com
---
 softmmu/vl.c | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index 8a06fe8bf77a..14fc527fc6e0 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -3705,7 +3705,11 @@ void qemu_init(int argc, char **argv, char **envp)
 break;
 }
 case QEMU_OPTION_spice:
-olist = qemu_find_opts("spice");
+olist = qemu_find_opts_err("spice", NULL);
+if (!olist) {
+ui_module_load_one("spice-core");
+olist = qemu_find_opts("spice");
+}
 if (!olist) {
 error_report("spice support is disabled");
 exit(1);
-- 
2.27.0

[PULL 01/14] spice: add module helpers

[Gerd Hoffmann]

Add new spice-module.c + qemu-spice-module.h files.  The code needed to
support modular spice will be there.  For starters this will be only the
using_spice variable, more will follow ...

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Message-id: 20201019075224.14803-2-kra...@redhat.com
---
 include/ui/qemu-spice-module.h | 23 +++
 include/ui/qemu-spice.h|  4 +---
 ui/spice-core.c|  1 -
 ui/spice-module.c  | 23 +++
 ui/meson.build |  1 +
 5 files changed, 48 insertions(+), 4 deletions(-)
 create mode 100644 include/ui/qemu-spice-module.h
 create mode 100644 ui/spice-module.c

diff --git a/include/ui/qemu-spice-module.h b/include/ui/qemu-spice-module.h
new file mode 100644
index ..1af0e659a109
--- /dev/null
+++ b/include/ui/qemu-spice-module.h
@@ -0,0 +1,23 @@
+/*
+ * Copyright (C) 2010 Red Hat, Inc.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 or
+ * (at your option) version 3 of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see .
+ */
+
+#ifndef QEMU_SPICE_MODULE_H
+#define QEMU_SPICE_MODULE_H
+
+extern int using_spice;
+
+#endif
diff --git a/include/ui/qemu-spice.h b/include/ui/qemu-spice.h
index 0e8ec3f0d776..ab523788b9a9 100644
--- a/include/ui/qemu-spice.h
+++ b/include/ui/qemu-spice.h
@@ -19,14 +19,13 @@
 #define QEMU_SPICE_H
 
 #include "qapi/error.h"
+#include "ui/qemu-spice-module.h"
 
 #ifdef CONFIG_SPICE
 
 #include 
 #include "qemu/config-file.h"
 
-extern int using_spice;
-
 void qemu_spice_init(void);
 void qemu_spice_input_init(void);
 void qemu_spice_display_init(void);
@@ -50,7 +49,6 @@ int qemu_spice_migrate_info(const char *hostname, int port, 
int tls_port,
 
 #include "qemu/error-report.h"
 
-#define using_spice 0
 #define spice_displays 0
 static inline int qemu_spice_set_passwd(const char *passwd,
 bool fail_if_connected,
diff --git a/ui/spice-core.c b/ui/spice-core.c
index 47700b220059..a7fa5743585f 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -48,7 +48,6 @@ static time_t auth_expires = TIME_MAX;
 static int spice_migration_completed;
 static int spice_display_is_running;
 static int spice_have_target_host;
-int using_spice = 0;
 
 static QemuThread me;
 
diff --git a/ui/spice-module.c b/ui/spice-module.c
new file mode 100644
index ..f86b0ac517dc
--- /dev/null
+++ b/ui/spice-module.c
@@ -0,0 +1,23 @@
+/*
+ * spice module support, also spice stubs.
+ *
+ * Copyright (C) 2010 Red Hat, Inc.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 or
+ * (at your option) version 3 of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "ui/qemu-spice-module.h"
+
+int using_spice;
diff --git a/ui/meson.build b/ui/meson.build
index ab4de98b381b..e89e298643a3 100644
--- a/ui/meson.build
+++ b/ui/meson.build
@@ -12,6 +12,7 @@ softmmu_ss.add(files(
   'keymaps.c',
   'qemu-pixman.c',
 ))
+softmmu_ss.add([spice_headers, files('spice-module.c')])
 
 softmmu_ss.add(when: 'CONFIG_LINUX', if_true: files('input-linux.c'))
 softmmu_ss.add(when: [spice, 'CONFIG_SPICE'], if_true: files('spice-core.c', 
'spice-input.c', 'spice-display.c'))
-- 
2.27.0

[PULL 00/14] Modules 20201022 patches

[Gerd Hoffmann]

The following changes since commit 4c41341af76cfc85b5a6c0f87de4838672ab9f89:

  Merge remote-tracking branch 'remotes/aperard/tags/pull-xen-20201020' into 
staging (2020-10-20 11:20:36 +0100)

are available in the Git repository at:

  git://git.kraxel.org/qemu tags/modules-20201022-pull-request

for you to fetch changes up to c8263659f1268a0f3502568d7663f722b2461935:

  opengl: build opengl helper code modular (2020-10-21 15:46:14 +0200)


modules: build spice and opengl as module.



Gerd Hoffmann (14):
  spice: add module helpers
  spice: add QemuSpiceOps, move migrate_info
  spice: move qemu_spice_init() to QemuSpiceOps.
  spice: move display_init() to QemuSpiceOps.
  spice: move add_interface() to QemuSpiceOps.
  spice: move auth functions to QemuSpiceOps.
  spice: move display_add_client() to QemuSpiceOps.
  spice: wire up monitor in QemuSpiceOps.
  spice: load module when enabled on the cmdline
  modules: dependencies infrastructure
  modules: add spice dependencies
  spice: flip modules switch
  opengl: build egl-headless display modular
  opengl: build opengl helper code modular

 include/ui/qemu-spice-module.h | 44 ++
 include/ui/qemu-spice.h| 42 +
 audio/spiceaudio.c |  4 +-
 chardev/spice.c|  2 +-
 monitor/misc.c |  2 +-
 monitor/qmp-cmds.c |  6 +--
 softmmu/vl.c   | 10 ++--
 ui/spice-core.c| 31 +
 ui/spice-input.c   |  6 +--
 ui/spice-module.c  | 85 ++
 util/module.c  | 44 --
 ui/meson.build | 28 +--
 12 files changed, 232 insertions(+), 72 deletions(-)
 create mode 100644 include/ui/qemu-spice-module.h
 create mode 100644 ui/spice-module.c

-- 
2.27.0

[PULL 11/14] modules: add spice dependencies

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Message-id: 20201019075224.14803-12-kra...@redhat.com
---
 util/module.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/util/module.c b/util/module.c
index 0c0f258923dc..21237dcc24df 100644
--- a/util/module.c
+++ b/util/module.c
@@ -176,6 +176,11 @@ static const struct {
 const char *name;
 const char *dep;
 } module_deps[] = {
+{ "audio-spice","ui-spice-core" },
+{ "chardev-spice",  "ui-spice-core" },
+{ "hw-display-qxl", "ui-spice-core" },
+{ "ui-spice-app",   "ui-spice-core" },
+{ "ui-spice-app",   "chardev-spice" },
 };
 #endif
 
-- 
2.27.0

[PULL 02/14] spice: add QemuSpiceOps, move migrate_info

[Gerd Hoffmann]

Add QemuSpiceOps struct.  This struct holds function pointers to the
spice functions.  It will be initialized with pointers to the stub
functions.  When spice gets initialized the function pointers will
be re-written to the real functions.

The spice stubs will move from qemu-spice.h to spice-module.c for that,
because they will be needed for both "CONFIG_SPICE=n" and "CONFIG_SPICE=y
but spice module not loaded" cases.

This patch adds the infrastructure and starts with moving
qemu_spice_migrate_info() to QemuSpiceOps.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Message-id: 20201019075224.14803-3-kra...@redhat.com
---
 include/ui/qemu-spice-module.h |  5 +
 include/ui/qemu-spice.h|  5 -
 monitor/misc.c |  2 +-
 ui/spice-core.c|  5 +
 ui/spice-module.c  | 10 ++
 5 files changed, 21 insertions(+), 6 deletions(-)

diff --git a/include/ui/qemu-spice-module.h b/include/ui/qemu-spice-module.h
index 1af0e659a109..7a9963dd5810 100644
--- a/include/ui/qemu-spice-module.h
+++ b/include/ui/qemu-spice-module.h
@@ -18,6 +18,11 @@
 #ifndef QEMU_SPICE_MODULE_H
 #define QEMU_SPICE_MODULE_H
 
+struct QemuSpiceOps {
+int (*migrate_info)(const char *h, int p, int t, const char *s);
+};
+
 extern int using_spice;
+extern struct QemuSpiceOps qemu_spice;
 
 #endif
diff --git a/include/ui/qemu-spice.h b/include/ui/qemu-spice.h
index ab523788b9a9..3157016c2bb4 100644
--- a/include/ui/qemu-spice.h
+++ b/include/ui/qemu-spice.h
@@ -60,11 +60,6 @@ static inline int qemu_spice_set_pw_expire(time_t expires)
 {
 return -1;
 }
-static inline int qemu_spice_migrate_info(const char *h, int p, int t,
-  const char *s)
-{
-return -1;
-}
 
 static inline int qemu_spice_display_add_client(int csock, int skipauth,
 int tls)
diff --git a/monitor/misc.c b/monitor/misc.c
index 4a859fb24a21..32e6a8c13d07 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -437,7 +437,7 @@ void qmp_client_migrate_info(const char *protocol, const 
char *hostname,
 return;
 }
 
-if (qemu_spice_migrate_info(hostname,
+if (qemu_spice.migrate_info(hostname,
 has_port ? port : -1,
 has_tls_port ? tls_port : -1,
 cert_subject)) {
diff --git a/ui/spice-core.c b/ui/spice-core.c
index a7fa5743585f..b03d743cf9b9 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -993,8 +993,13 @@ int qemu_spice_display_is_running(SimpleSpiceDisplay *ssd)
 return spice_display_is_running;
 }
 
+static struct QemuSpiceOps real_spice_ops = {
+.migrate_info = qemu_spice_migrate_info,
+};
+
 static void spice_register_config(void)
 {
+qemu_spice = real_spice_ops;
 qemu_add_opts(&qemu_spice_opts);
 }
 opts_init(spice_register_config);
diff --git a/ui/spice-module.c b/ui/spice-module.c
index f86b0ac517dc..f1939545a684 100644
--- a/ui/spice-module.c
+++ b/ui/spice-module.c
@@ -21,3 +21,13 @@
 #include "ui/qemu-spice-module.h"
 
 int using_spice;
+
+static int qemu_spice_migrate_info_stub(const char *h, int p, int t,
+const char *s)
+{
+return -1;
+}
+
+struct QemuSpiceOps qemu_spice = {
+.migrate_info = qemu_spice_migrate_info_stub,
+};
-- 
2.27.0

[PULL 08/14] spice: wire up monitor in QemuSpiceOps.

[Gerd Hoffmann]

Rename qmp_query_spice() to qmp_query_spice_real(), add to QemuSpiceOps.

Add new qmp_query_spice() function which calls the real function via
QemuSpiceOps if available, otherwise return SpiceInfo.enabled = false.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Message-id: 20201019075224.14803-9-kra...@redhat.com
---
 include/ui/qemu-spice-module.h |  3 +++
 ui/spice-core.c|  3 ++-
 ui/spice-module.c  | 17 +
 3 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/include/ui/qemu-spice-module.h b/include/ui/qemu-spice-module.h
index 7422f6404ba0..1f22d557ea2f 100644
--- a/include/ui/qemu-spice-module.h
+++ b/include/ui/qemu-spice-module.h
@@ -22,6 +22,8 @@
 #include 
 #endif
 
+typedef struct SpiceInfo SpiceInfo;
+
 struct QemuSpiceOps {
 void (*init)(void);
 void (*display_init)(void);
@@ -32,6 +34,7 @@ struct QemuSpiceOps {
 int (*display_add_client)(int csock, int skipauth, int tls);
 #ifdef CONFIG_SPICE
 int (*add_interface)(SpiceBaseInstance *sin);
+SpiceInfo* (*qmp_query)(Error **errp);
 #endif
 };
 
diff --git a/ui/spice-core.c b/ui/spice-core.c
index 99457c226e6f..eea52f538999 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -502,7 +502,7 @@ static QemuOptsList qemu_spice_opts = {
 },
 };
 
-SpiceInfo *qmp_query_spice(Error **errp)
+static SpiceInfo *qmp_query_spice_real(Error **errp)
 {
 QemuOpts *opts = QTAILQ_FIRST(&qemu_spice_opts.head);
 int port, tls_port;
@@ -1001,6 +1001,7 @@ static struct QemuSpiceOps real_spice_ops = {
 .set_pw_expire = qemu_spice_set_pw_expire,
 .display_add_client = qemu_spice_display_add_client,
 .add_interface = qemu_spice_add_interface,
+.qmp_query = qmp_query_spice_real,
 };
 
 static void spice_register_config(void)
diff --git a/ui/spice-module.c b/ui/spice-module.c
index 8fbc99c03c55..32223358722c 100644
--- a/ui/spice-module.c
+++ b/ui/spice-module.c
@@ -19,6 +19,9 @@
 
 #include "qemu/osdep.h"
 #include "qemu/error-report.h"
+#include "qapi/error.h"
+#include "qapi/qapi-types-ui.h"
+#include "qapi/qapi-commands-ui.h"
 #include "ui/qemu-spice-module.h"
 
 int using_spice;
@@ -66,3 +69,17 @@ struct QemuSpiceOps qemu_spice = {
 .set_pw_expire = qemu_spice_set_pw_expire_stub,
 .display_add_client = qemu_spice_display_add_client_stub,
 };
+
+#ifdef CONFIG_SPICE
+
+SpiceInfo *qmp_query_spice(Error **errp)
+{
+if (!qemu_spice.qmp_query) {
+SpiceInfo *info = g_new0(SpiceInfo, 1);
+info->enabled = false;
+return info;
+}
+return qemu_spice.qmp_query(errp);
+}
+
+#endif
-- 
2.27.0

[PULL 04/14] spice: move display_init() to QemuSpiceOps.

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Marc-André Lureau 
Message-id: 20201019075224.14803-5-kra...@redhat.com
---
 include/ui/qemu-spice-module.h | 1 +
 include/ui/qemu-spice.h| 7 ---
 softmmu/vl.c   | 2 +-
 ui/spice-core.c| 1 +
 ui/spice-module.c  | 9 +
 5 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/include/ui/qemu-spice-module.h b/include/ui/qemu-spice-module.h
index b182bc4c21a6..dbe09035dade 100644
--- a/include/ui/qemu-spice-module.h
+++ b/include/ui/qemu-spice-module.h
@@ -20,6 +20,7 @@
 
 struct QemuSpiceOps {
 void (*init)(void);
+void (*display_init)(void);
 int (*migrate_info)(const char *h, int p, int t, const char *s);
 };
 
diff --git a/include/ui/qemu-spice.h b/include/ui/qemu-spice.h
index e6df0a8715b0..a3fd1ea5ade8 100644
--- a/include/ui/qemu-spice.h
+++ b/include/ui/qemu-spice.h
@@ -66,13 +66,6 @@ static inline int qemu_spice_display_add_client(int csock, 
int skipauth,
 return -1;
 }
 
-static inline void qemu_spice_display_init(void)
-{
-/* This must never be called if CONFIG_SPICE is disabled */
-error_report("spice support is disabled");
-abort();
-}
-
 #endif /* CONFIG_SPICE */
 
 static inline bool qemu_using_spice(Error **errp)
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 65c34a6b8a2e..8a06fe8bf77a 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -4447,7 +4447,7 @@ void qemu_init(int argc, char **argv, char **envp)
 #endif
 
 if (using_spice) {
-qemu_spice_display_init();
+qemu_spice.display_init();
 }
 
 if (foreach_device_config(DEV_GDB, gdbserver_start) < 0) {
diff --git a/ui/spice-core.c b/ui/spice-core.c
index 6ef66eb387de..82d5dbda3e26 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -995,6 +995,7 @@ int qemu_spice_display_is_running(SimpleSpiceDisplay *ssd)
 
 static struct QemuSpiceOps real_spice_ops = {
 .init = qemu_spice_init,
+.display_init = qemu_spice_display_init,
 .migrate_info = qemu_spice_migrate_info,
 };
 
diff --git a/ui/spice-module.c b/ui/spice-module.c
index a30fa452ea93..56868aaffe9a 100644
--- a/ui/spice-module.c
+++ b/ui/spice-module.c
@@ -18,6 +18,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/error-report.h"
 #include "ui/qemu-spice-module.h"
 
 int using_spice;
@@ -26,6 +27,13 @@ static void qemu_spice_init_stub(void)
 {
 }
 
+static void qemu_spice_display_init_stub(void)
+{
+/* This must never be called if CONFIG_SPICE is disabled */
+error_report("spice support is disabled");
+abort();
+}
+
 static int qemu_spice_migrate_info_stub(const char *h, int p, int t,
 const char *s)
 {
@@ -34,5 +42,6 @@ static int qemu_spice_migrate_info_stub(const char *h, int p, 
int t,
 
 struct QemuSpiceOps qemu_spice = {
 .init = qemu_spice_init_stub,
+.display_init = qemu_spice_display_init_stub,
 .migrate_info = qemu_spice_migrate_info_stub,
 };
-- 
2.27.0

Re: [PATCH 03/22] machine: move UP defaults to class_base_init

[Thomas Huth]

On 21/10/2020 22.56, Paolo Bonzini wrote:
> Clean up vl.c, default min/max/default_cpus to uniprocessor
> directly in the QOM class initialization code.
> 
> Signed-off-by: Paolo Bonzini 
> ---
>  hw/core/machine.c | 6 +-
>  softmmu/vl.c  | 5 -
>  2 files changed, 5 insertions(+), 6 deletions(-)
> 
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index 80a918895a..d3a8450b1f 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -854,8 +854,12 @@ static void machine_class_init(ObjectClass *oc, void 
> *data)
>  
>  static void machine_class_base_init(ObjectClass *oc, void *data)
>  {
> +MachineClass *mc = MACHINE_CLASS(oc);
> +mc->max_cpus = mc->max_cpus ?: 1;
> +mc->min_cpus = mc->min_cpus ?: 1;
> +mc->default_cpus = mc->default_cpus ?: 1;
> +
>  if (!object_class_is_abstract(oc)) {
> -MachineClass *mc = MACHINE_CLASS(oc);
>  const char *cname = object_class_get_name(oc);
>  assert(g_str_has_suffix(cname, TYPE_MACHINE_SUFFIX));
>  mc->name = g_strndup(cname,
> diff --git a/softmmu/vl.c b/softmmu/vl.c
> index 42314e6ff9..75bc686397 100644
> --- a/softmmu/vl.c
> +++ b/softmmu/vl.c
> @@ -3966,11 +3966,6 @@ void qemu_init(int argc, char **argv, char **envp)
>  exit(0);
>  }
>  
> -/* machine_class: default to UP */
> -machine_class->max_cpus = machine_class->max_cpus ?: 1;
> -machine_class->min_cpus = machine_class->min_cpus ?: 1;
> -machine_class->default_cpus = machine_class->default_cpus ?: 1;
> -
>  /* default to machine_class->default_cpus */
>  current_machine->smp.cpus = machine_class->default_cpus;
>  current_machine->smp.max_cpus = machine_class->default_cpus;
> 

Reviewed-by: Thomas Huth 

Re: [PATCH 02/22] machine: remove deprecated -machine enforce-config-section option

[Thomas Huth]

On 21/10/2020 22.56, Paolo Bonzini wrote:
> Deprecated since 3.1 and complicates the initialization sequence,
> remove it.
> 
> Signed-off-by: Paolo Bonzini 
> ---
>  docs/system/deprecated.rst | 12 ++--
>  hw/core/machine.c  | 24 +---
>  include/hw/boards.h|  1 -
>  migration/migration.c  | 10 --
>  qemu-options.hx|  8 
>  5 files changed, 7 insertions(+), 48 deletions(-)
> 
> diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst
> index 895433c356..0ebce37a19 100644
> --- a/docs/system/deprecated.rst
> +++ b/docs/system/deprecated.rst
> @@ -21,12 +21,6 @@ deprecated.
>  System emulator command line arguments
>  --
>  
> -``-machine enforce-config-section=on|off`` (since 3.1)
> -''
> -
> -The ``enforce-config-section`` parameter is replaced by the
> -``-global migration.send-configuration={on|off}`` option.
> -
>  ``-usbdevice`` (since 2.10.0)
>  '
>  
> @@ -689,6 +683,12 @@ Support for invalid topologies is removed, the user must 
> ensure
>  topologies described with -smp include all possible cpus, i.e.
>  *sockets* * *cores* * *threads* = *maxcpus*.
>  
> +``-machine enforce-config-section=on|off`` (removed 5.2)
> +
> +
> +The ``enforce-config-section`` property was replaced by the
> +``-global migration.send-configuration={on|off}`` option.
> +
>  Block devices
>  -
>  
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index d740a7e963..80a918895a 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -26,6 +26,7 @@
>  #include "sysemu/qtest.h"
>  #include "hw/pci/pci.h"
>  #include "hw/mem/nvdimm.h"
> +#include "migration/misc.h"

This new include does not make much sense to me, if all you did was removing
code from this file. Why did you add this here?

With this hunk removed:
Reviewed-by: Thomas Huth 

Re: [PATCH 03/12] qom: Make object_class_property_add_uint*_ptr() get offset

[Markus Armbruster]

Eduardo Habkost  writes:

> On Wed, Oct 21, 2020 at 02:24:08PM +0200, Igor Mammedov wrote:
>> On Fri,  9 Oct 2020 12:01:13 -0400
>> Eduardo Habkost  wrote:
>> 
>> > The existing object_class_property_add_uint*_ptr() functions are
>> > not very useful, because they need a pointer to the property
>> > value, which can't really be provided before the object is
>> > created.
>> > 
>> > Replace the pointer parameter in those functions with a
>> > `ptrdiff_t offset` parameter.
>> > 
>> > Include a uint8 class property in check-qom-proplist unit tests,
>> > to ensure the feature is working.
>> 
>> 
>> Not sure I like approach, it's reinventing qdev pointer properties in QOM 
>> form.
>
> Yes, and that's on purpose.  If we want to eventually merge the
> two competing APIs into a single one, we need to make them
> converge.
>
>> I had an impression that Paolo wanted qdev pointer properties be gone
>> and replaced by something like link properties.
>
> This is completely unrelated to qdev pointer properties and link
> properties.  The properties that use object_property_add_uint*_ptr()
> today are not qdev pointer properties and will never be link
> properties.  They are just integer properties.
>
>> 
>> object_property_add_uintXX_ptr() were introduced as a quick hack,
>> when ACPI code generation was moved from Seabios, to avoid more
>> code shuffling in device models and adding more boiler plate in
>> form of custom setters/getters (the later didn't seem to bother
>> us everywhere else where we use object_[class_]property_add() ).
>> Then it spread little bit to another places.
>> 
>> I'd rather get rid of object_property_add_uintXX_ptr() API altogether
>> in favor of object_[class_]property_add() like it is used in other places
>> to handle intXX properties.
>> Adding helpers similar to object_property_add_bool() for intXX
>> could reduce boiler plate need for converting current instances of
>> _ptr(), and such helpers would also help with reducing boilerplate
>> for the rest of instances where object_[class_]property_add()
>> currently is used for dealing with integers.
>
> I find object_property_add_bool() terrible.  It requires too much
> boilerplate.  I actually have plans to introduce
> object*_property_add_bool_ptr() to simplify existing
> object_property_add_bool() callers.
>
> I don't love object*_property_add_*_ptr() either.  I consider the
> qdev property API better.  But we need a reasonable alternative,
> because the qdev API can't be used by non-device objects yet.

Emphasis on *yet*: we should be able to lift it up into QOM, shouldn't
we?

> I don't think object*_property_add() and
> object*_property_add_bool() are reasonable alternatives.

Re: [PATCH v1 4/6] gitlab: skip checkpatch.pl checks if no commit delta on branch

[Thomas Huth]

On 21/10/2020 18.31, Alex Bennée wrote:
> From: Daniel P. Berrangé 
> 
> If the current branch is synced to the current upstream git master,
> there are no commits that need checking. This causes checkpatch.pl
> to print an error that it found no commits. We need to avoid calling
> checkpatch.pl in this case.
> 
> Signed-off-by: Daniel P. Berrangé 
> Message-Id: <20201019143537.283094-2-berra...@redhat.com>
> Signed-off-by: Alex Bennée 
> ---
>  .gitlab-ci.d/check-patch.py | 8 
>  1 file changed, 8 insertions(+)
> 
> diff --git a/.gitlab-ci.d/check-patch.py b/.gitlab-ci.d/check-patch.py
> index 5a14a25b13..0ff30ee077 100755
> --- a/.gitlab-ci.d/check-patch.py
> +++ b/.gitlab-ci.d/check-patch.py
> @@ -33,8 +33,16 @@ ancestor = subprocess.check_output(["git", "merge-base",
>  
>  ancestor = ancestor.strip()
>  
> +log = subprocess.check_output(["git", "log", "--format=%H %s",
> +   ancestor + "..."],
> +  universal_newlines=True)
> +
>  subprocess.check_call(["git", "remote", "rm", "check-patch"])
>  
> +if log == "":
> +print("\nNo commits since %s, skipping checks\n" % ancestor)
> +sys.exit(0)
> +
>  errors = False
>  
>  print("\nChecking all commits since %s...\n" % ancestor)
> 

Acked-by: Thomas Huth 

Re: [PATCH 1/5] pc-dimm: Drop @errp argument of pc_dimm_plug()

[David Gibson]

On Mon, Oct 19, 2020 at 10:48:04AM +0200, Greg Kurz wrote:
> pc_dimm_plug() doesn't use it. It only aborts on error.
> 
> Drop @errp and adapt the callers accordingly.
> 
> Signed-off-by: Greg Kurz 

Reviewed-by: David Gibson 

ppc parts
Acked-by: David Gibson 

> ---
>  hw/arm/virt.c|9 +
>  hw/i386/pc.c |8 +---
>  hw/mem/pc-dimm.c |2 +-
>  hw/ppc/spapr.c   |5 +
>  include/hw/mem/pc-dimm.h |2 +-
>  5 files changed, 5 insertions(+), 21 deletions(-)
> 
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index e465a988d683..27dbeb549ef1 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -2261,12 +2261,8 @@ static void virt_memory_plug(HotplugHandler 
> *hotplug_dev,
>  VirtMachineState *vms = VIRT_MACHINE(hotplug_dev);
>  MachineState *ms = MACHINE(hotplug_dev);
>  bool is_nvdimm = object_dynamic_cast(OBJECT(dev), TYPE_NVDIMM);
> -Error *local_err = NULL;
>  
> -pc_dimm_plug(PC_DIMM(dev), MACHINE(vms), &local_err);
> -if (local_err) {
> -goto out;
> -}
> +pc_dimm_plug(PC_DIMM(dev), MACHINE(vms));
>  
>  if (is_nvdimm) {
>  nvdimm_plug(ms->nvdimms_state);
> @@ -2274,9 +2270,6 @@ static void virt_memory_plug(HotplugHandler 
> *hotplug_dev,
>  
>  hotplug_handler_plug(HOTPLUG_HANDLER(vms->acpi_dev),
>   dev, &error_abort);
> -
> -out:
> -error_propagate(errp, local_err);
>  }
>  
>  static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index e87be5d29a01..38b1be78e707 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -1265,24 +1265,18 @@ static void pc_memory_pre_plug(HotplugHandler 
> *hotplug_dev, DeviceState *dev,
>  static void pc_memory_plug(HotplugHandler *hotplug_dev,
> DeviceState *dev, Error **errp)
>  {
> -Error *local_err = NULL;
>  PCMachineState *pcms = PC_MACHINE(hotplug_dev);
>  X86MachineState *x86ms = X86_MACHINE(hotplug_dev);
>  MachineState *ms = MACHINE(hotplug_dev);
>  bool is_nvdimm = object_dynamic_cast(OBJECT(dev), TYPE_NVDIMM);
>  
> -pc_dimm_plug(PC_DIMM(dev), MACHINE(pcms), &local_err);
> -if (local_err) {
> -goto out;
> -}
> +pc_dimm_plug(PC_DIMM(dev), MACHINE(pcms));
>  
>  if (is_nvdimm) {
>  nvdimm_plug(ms->nvdimms_state);
>  }
>  
>  hotplug_handler_plug(x86ms->acpi_dev, dev, &error_abort);
> -out:
> -error_propagate(errp, local_err);
>  }
>  
>  static void pc_memory_unplug_request(HotplugHandler *hotplug_dev,
> diff --git a/hw/mem/pc-dimm.c b/hw/mem/pc-dimm.c
> index c30351070bb8..2ffc986734df 100644
> --- a/hw/mem/pc-dimm.c
> +++ b/hw/mem/pc-dimm.c
> @@ -64,7 +64,7 @@ void pc_dimm_pre_plug(PCDIMMDevice *dimm, MachineState 
> *machine,
> errp);
>  }
>  
> -void pc_dimm_plug(PCDIMMDevice *dimm, MachineState *machine, Error **errp)
> +void pc_dimm_plug(PCDIMMDevice *dimm, MachineState *machine)
>  {
>  PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm);
>  MemoryRegion *vmstate_mr = ddc->get_vmstate_memory_region(dimm,
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index ee716a12af73..4edd31b86915 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -3438,10 +3438,7 @@ static void spapr_memory_plug(HotplugHandler 
> *hotplug_dev, DeviceState *dev,
>  
>  size = memory_device_get_region_size(MEMORY_DEVICE(dev), &error_abort);
>  
> -pc_dimm_plug(dimm, MACHINE(ms), &local_err);
> -if (local_err) {
> -goto out;
> -}
> +pc_dimm_plug(dimm, MACHINE(ms));
>  
>  if (!is_nvdimm) {
>  addr = object_property_get_uint(OBJECT(dimm),
> diff --git a/include/hw/mem/pc-dimm.h b/include/hw/mem/pc-dimm.h
> index aec9527fdd96..3d3db82641f8 100644
> --- a/include/hw/mem/pc-dimm.h
> +++ b/include/hw/mem/pc-dimm.h
> @@ -72,6 +72,6 @@ struct PCDIMMDeviceClass {
>  
>  void pc_dimm_pre_plug(PCDIMMDevice *dimm, MachineState *machine,
>const uint64_t *legacy_align, Error **errp);
> -void pc_dimm_plug(PCDIMMDevice *dimm, MachineState *machine, Error **errp);
> +void pc_dimm_plug(PCDIMMDevice *dimm, MachineState *machine);
>  void pc_dimm_unplug(PCDIMMDevice *dimm, MachineState *machine);
>  #endif
> 
> 

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [PATCH 0/5] spapr: Error handling fixes and cleanups (round 3)

[David Gibson]

On Mon, Oct 19, 2020 at 10:47:52AM +0200, Greg Kurz wrote:
> Hi,
> 
> This is a followup to a previous cleanup for the sPAPR code:
> 
> https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg04860.html
> 
> The last two patches had to be dropped because they were wrongly assuming
> that object_property_get_uint() returning zero meant failure. This led to
> a discussion in which arose a consensus that most of the time (not to say
> always) object property getters should never fail actually, ie. failure
> is very likely the result of a programming error and QEMU should abort.
> 
> This series aims at demonstrating a revelant case I've found while auditing
> object property getters (this is patch 4 that I've isolated from a huge
> 50-patch series I haven't dared to post yet). The sPAPR memory hotplug code
> is tailored to support either regular PC DIMMs or NVDIMMs, which inherit
> from PC DIMMs. They expect to get some properties from the DIMM object,
> which happens to be set by default at the PC DIMM class level. It thus
> doesn't make sense to pass an error object and propagate it when getting
> them since this would lure the user into thinking they did something wrong.
> 
> Some preliminary cleanup is done on the way, especially dropping an unused
> @errp argument of pc_dimm_plug(). This affects several platforms other than
> sPAPR but I guess the patch is trivial enough to go through David's tree
> if it gets acks from the relevant maintainers.

Since this series mostly affects ppc, I've applied it to ppc-for-5.2.

It would be nice to have an acked-by from Igor or Michael for the
first patch, though.

> 
> ---
> 
> Greg Kurz (5):
>   pc-dimm: Drop @errp argument of pc_dimm_plug()
>   spapr: Use appropriate getter for PC_DIMM_ADDR_PROP
>   spapr: Use appropriate getter for PC_DIMM_SLOT_PROP
>   spapr: Pass &error_abort when getting some PC DIMM properties
>   spapr: Simplify error handling in spapr_memory_plug()
> 
> 
>  hw/arm/virt.c |9 +---
>  hw/i386/pc.c  |8 +--
>  hw/mem/pc-dimm.c  |2 +-
>  hw/ppc/spapr.c|   48 
> +++--
>  hw/ppc/spapr_nvdimm.c |5 +++-
>  include/hw/mem/pc-dimm.h  |2 +-
>  include/hw/ppc/spapr_nvdimm.h |2 +-
>  7 files changed, 25 insertions(+), 51 deletions(-)
> 

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [PATCH 2/5] spapr: Use appropriate getter for PC_DIMM_ADDR_PROP

[David Gibson]

On Mon, Oct 19, 2020 at 10:48:16AM +0200, Greg Kurz wrote:
> The PC_DIMM_ADDR_PROP property is defined as:
> 
> DEFINE_PROP_UINT64(PC_DIMM_ADDR_PROP, PCDIMMDevice, addr, 0),
> 
> Use object_property_get_uint() instead of object_property_get_int().
> 
> Signed-off-by: Greg Kurz 

Acked-by: David Gibson 

> ---
>  hw/ppc/spapr.c |4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 4edd31b86915..115fc52e3b06 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -3572,8 +3572,8 @@ static SpaprDimmState 
> *spapr_recover_pending_dimm_state(SpaprMachineState *ms,
>  uint64_t addr_start, addr;
>  int i;
>  
> -addr_start = object_property_get_int(OBJECT(dimm), PC_DIMM_ADDR_PROP,
> - &error_abort);
> +addr_start = object_property_get_uint(OBJECT(dimm), PC_DIMM_ADDR_PROP,
> +  &error_abort);
>  
>  addr = addr_start;
>  for (i = 0; i < nr_lmbs; i++) {
> 
> 

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [PATCH v2 0/5] spapr: Fix and cleanups for sPAPR CPU core

[David Gibson]

On Thu, Oct 15, 2020 at 11:18:18PM +0200, Greg Kurz wrote:
> While reading the code _again_ I spotted a memory leak and I realized
> that the realize/unrealize paths are uselessly complex and not really
> symmetrical.
> 
> This series fixes the leak and re-shuffles the code to make it cleaner.
> 
> Tested with 'make check', travis-ci and manual hotplug/unplug of CPU
> cores. Also tested error paths by simulating failures when creating
> interrupt presenters or when setting the vCPU id.
> 
> v2: - enforce symmetry between realize and unrealize
> - unrealize vCPUs with qdev_unrealize()
> - one loop to create/realize and to unrealize/delete vCPUs

Applied to ppc-for-5.2.

> 
> ---
> 
> Greg Kurz (5):
>   spapr: Fix leak of CPU machine specific data
>   spapr: Unrealize vCPUs with qdev_unrealize()
>   spapr: Drop spapr_delete_vcpu() unused argument
>   spapr: Make spapr_cpu_core_unrealize() idempotent
>   spapr: Simplify spapr_cpu_core_realize() and spapr_cpu_core_unrealize()
> 
> 
>  accel/tcg/user-exec-stub.c  |4 ++
>  hw/ppc/spapr_cpu_core.c |   69 
> ++-
>  target/ppc/translate_init.c.inc |2 +
>  3 files changed, 37 insertions(+), 38 deletions(-)
> 

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [PATCH 3/5] spapr: Use appropriate getter for PC_DIMM_SLOT_PROP

[David Gibson]

On Mon, Oct 19, 2020 at 10:48:27AM +0200, Greg Kurz wrote:
> The PC_DIMM_SLOT_PROP property is defined as:
> 
> DEFINE_PROP_INT32(PC_DIMM_SLOT_PROP, PCDIMMDevice, slot,
>   PC_DIMM_UNASSIGNED_SLOT),
> 
> Use object_property_get_int() instead of object_property_get_uint().
> Since spapr_memory_plug() only gets called if pc_dimm_pre_plug()
> succeeded, we expect to have a valid >= 0 slot number, either because
> the user passed a valid slot number or because pc_dimm_get_free_slot()
> picked one up for us.
> 
> Signed-off-by: Greg Kurz 

Acked-by: David Gibson 

> ---
>  hw/ppc/spapr.c |9 ++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 115fc52e3b06..1b173861152f 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -3433,7 +3433,8 @@ static void spapr_memory_plug(HotplugHandler 
> *hotplug_dev, DeviceState *dev,
>  Error *local_err = NULL;
>  SpaprMachineState *ms = SPAPR_MACHINE(hotplug_dev);
>  PCDIMMDevice *dimm = PC_DIMM(dev);
> -uint64_t size, addr, slot;
> +uint64_t size, addr;
> +int64_t slot;
>  bool is_nvdimm = object_dynamic_cast(OBJECT(dev), TYPE_NVDIMM);
>  
>  size = memory_device_get_region_size(MEMORY_DEVICE(dev), &error_abort);
> @@ -3450,11 +3451,13 @@ static void spapr_memory_plug(HotplugHandler 
> *hotplug_dev, DeviceState *dev,
> spapr_ovec_test(ms->ov5_cas, OV5_HP_EVT),
> &local_err);
>  } else {
> -slot = object_property_get_uint(OBJECT(dimm),
> -PC_DIMM_SLOT_PROP, &local_err);
> +slot = object_property_get_int(OBJECT(dimm),
> +   PC_DIMM_SLOT_PROP, &local_err);
>  if (local_err) {
>  goto out_unplug;
>  }
> +/* We should have valid slot number at this point */
> +g_assert(slot >= 0);
>  spapr_add_nvdimm(dev, slot, &local_err);
>  }
>  
> 
> 

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

[PATCH v6] sev: add sev-inject-launch-secret

[tobin]

From: Tobin Feldman-Fitzthum 

AMD SEV allows a guest owner to inject a secret blob
into the memory of a virtual machine. The secret is
encrypted with the SEV Transport Encryption Key and
integrity is guaranteed with the Transport Integrity
Key. Although QEMU facilitates the injection of the
launch secret, it cannot access the secret.

Signed-off-by: Tobin Feldman-Fitzthum 
Reviewed-by: Daniel P. Berrangé 
Reviewed-by: Brijesh Singh 
---
 include/monitor/monitor.h |  3 ++
 include/sysemu/sev.h  |  2 ++
 monitor/misc.c| 13 +---
 qapi/misc-target.json | 18 +++
 target/i386/monitor.c |  7 +
 target/i386/sev-stub.c|  5 +++
 target/i386/sev.c | 65 +++
 target/i386/trace-events  |  1 +
 8 files changed, 110 insertions(+), 4 deletions(-)

diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 348bfad3d5..af3887bb71 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -4,6 +4,7 @@
 #include "block/block.h"
 #include "qapi/qapi-types-misc.h"
 #include "qemu/readline.h"
+#include "include/exec/hwaddr.h"
 
 typedef struct MonitorHMP MonitorHMP;
 typedef struct MonitorOptions MonitorOptions;
@@ -37,6 +38,8 @@ void monitor_flush(Monitor *mon);
 int monitor_set_cpu(Monitor *mon, int cpu_index);
 int monitor_get_cpu_index(Monitor *mon);
 
+void *gpa2hva(MemoryRegion **p_mr, hwaddr addr, uint64_t size, Error **errp);
+
 void monitor_read_command(MonitorHMP *mon, int show_prompt);
 int monitor_read_password(MonitorHMP *mon, ReadLineFunc *readline_func,
   void *opaque);
diff --git a/include/sysemu/sev.h b/include/sysemu/sev.h
index 98c1ec8d38..7ab6e3e31d 100644
--- a/include/sysemu/sev.h
+++ b/include/sysemu/sev.h
@@ -18,4 +18,6 @@
 
 void *sev_guest_init(const char *id);
 int sev_encrypt_data(void *handle, uint8_t *ptr, uint64_t len);
+int sev_inject_launch_secret(const char *hdr, const char *secret,
+ uint64_t gpa, Error **errp);
 #endif
diff --git a/monitor/misc.c b/monitor/misc.c
index 4a859fb24a..dd148be5da 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -667,10 +667,10 @@ static void hmp_physical_memory_dump(Monitor *mon, const 
QDict *qdict)
 memory_dump(mon, count, format, size, addr, 1);
 }
 
-static void *gpa2hva(MemoryRegion **p_mr, hwaddr addr, Error **errp)
+void *gpa2hva(MemoryRegion **p_mr, hwaddr addr, uint64_t size, Error **errp)
 {
 MemoryRegionSection mrs = memory_region_find(get_system_memory(),
- addr, 1);
+ addr, size);
 
 if (!mrs.mr) {
 error_setg(errp, "No memory is mapped at address 0x%" HWADDR_PRIx, 
addr);
@@ -683,6 +683,11 @@ static void *gpa2hva(MemoryRegion **p_mr, hwaddr addr, 
Error **errp)
 return NULL;
 }
 
+if (mrs.size < size) {
+error_setg(errp, "Size of memory region at 0x%" HWADDR_PRIx
+   " exceeded.", addr);
+}
+
 *p_mr = mrs.mr;
 return qemu_map_ram_ptr(mrs.mr->ram_block, mrs.offset_within_region);
 }
@@ -694,7 +699,7 @@ static void hmp_gpa2hva(Monitor *mon, const QDict *qdict)
 MemoryRegion *mr = NULL;
 void *ptr;
 
-ptr = gpa2hva(&mr, addr, &local_err);
+ptr = gpa2hva(&mr, addr, 1, &local_err);
 if (local_err) {
 error_report_err(local_err);
 return;
@@ -770,7 +775,7 @@ static void hmp_gpa2hpa(Monitor *mon, const QDict *qdict)
 void *ptr;
 uint64_t physaddr;
 
-ptr = gpa2hva(&mr, addr, &local_err);
+ptr = gpa2hva(&mr, addr, 1, &local_err);
 if (local_err) {
 error_report_err(local_err);
 return;
diff --git a/qapi/misc-target.json b/qapi/misc-target.json
index 1e561fa97b..4486a543ae 100644
--- a/qapi/misc-target.json
+++ b/qapi/misc-target.json
@@ -201,6 +201,24 @@
 { 'command': 'query-sev-capabilities', 'returns': 'SevCapability',
   'if': 'defined(TARGET_I386)' }
 
+##
+# @sev-inject-launch-secret:
+#
+# This command injects a secret blob into memory of SEV guest.
+#
+# @packet-header: the launch secret packet header encoded in base64
+#
+# @secret: the launch secret data to be injected encoded in base64
+#
+# @gpa: the guest physical address where secret will be injected.
+#
+# Since: 5.2
+#
+##
+{ 'command': 'sev-inject-launch-secret',
+  'data': { 'packet-header': 'str', 'secret': 'str', 'gpa': 'uint64' },
+  'if': 'defined(TARGET_I386)' }
+
 ##
 # @dump-skeys:
 #
diff --git a/target/i386/monitor.c b/target/i386/monitor.c
index 7abae3c8df..f9d4951465 100644
--- a/target/i386/monitor.c
+++ b/target/i386/monitor.c
@@ -728,3 +728,10 @@ SevCapability *qmp_query_sev_capabilities(Error **errp)
 {
 return sev_get_capabilities(errp);
 }
+
+void qmp_sev_inject_launch_secret(const char *packet_hdr,
+  const char *secret, uint64_t gpa,
+  Error **errp)
+{
+sev_inject_launch_secret(packet_hdr, secret,

Re: 答复: [PATCH 1/1] Skip flatview_simplify() for cpu vendor zhaoxin

[Paolo Bonzini]

On 22/10/20 05:02, FelixCui-oc wrote:
> In addition, before write pam registers, flatview_simplify() has merged
> a very large range.For example,
> 
> this large range is 0xc-0xbfff. So even if EHCI is configured to
> not allocate buffers in low memory,
> 
> this bug will still occur.Thanks.

So removing flatview_simplify() works because the higher area (0x1
and above) remains the same.  I guess the simplest thing to do is to
apply flatview_simplify() only to I/O areas, though we can also consider
removing it completely.  I'm not sure in which case it would provide a
noticeable improvement.

Paolo

答复: [PATCH 1/1] Skip flatview_simplify() for cpu vendor zhaoxin

[FelixCui-oc]

hi  ,

>I assume it's the BIOS's driver and it's choosing a range in low memory,
>but still I'm not sure why its DMA is racing against the PAM update
>(which is done very early).  Felix, do you know the answer?


This bug is triggered by make_bios_readonly() in seabios. Make_bios_readonly() 
will write pam register.

And ehci_setup() is executed before make_bios_readonly(). After initializing 
EHCI in seabios, it will

continuously send dma cycles. So when write pam register, this bug appeared.


In addition, before write pam registers, flatview_simplify() has merged a very 
large range.For example,

this large range is 0xc-0xbfff. So even if EHCI is configured to not 
allocate buffers in low memory,

this bug will still occur.Thanks.


Best regards

Felixcui-oc


发件人: Paolo Bonzini 
发送时间: 2020年10月22日 3:50:15
收件人: Alex Williamson
抄送: FelixCui-oc; Richard Henderson; Eduardo Habkost; qemu-devel@nongnu.org; 
RockCui-oc; Tony W Wang-oc; CobeChen-oc
主题: Re: [PATCH 1/1] Skip flatview_simplify() for cpu vendor zhaoxin

On 21/10/20 20:49, Alex Williamson wrote:
> I can understand the general benefit of flatview_simplify(), but I
> wonder if the best short term solution is to skip operating on the x86
> PAM range, which I understand to be a small number of memory chunks
> below 1MB.

I'd rather remove flatview_simplify altogether, it probably triggers
relatively rarely.  Possibly do not let it operate on RAM/ROM regions,
only on I/O regions.

> I might also wonder why the EHCI controller on this
> platform is choosing that range for DMA.

I assume it's the BIOS's driver and it's choosing a range in low memory,
but still I'm not sure why its DMA is racing against the PAM update
(which is done very early).  Felix, do you know the answer?

Paolo

Re: [PATCH v3 1/2] hw/block/nvme: add dulbe support

[Keith Busch]

On Thu, Oct 22, 2020 at 12:17:35AM +0200, Klaus Jensen wrote:
> +for (int i = 1; i <= n->num_namespaces; i++) {

You can call me old-school, but I don't think C should have allowed
mixed declarations with code.

Re: [PATCH] WHPX: Fix WHPX build break

[Paolo Bonzini]

On 22/10/20 02:27, Sunil Muthuswamy wrote:
> With upstream commit#8a19980e3fc4, logic was introduced to only
> allow WHPX build on x64. But, the logic checks for the cpu family
> and not the cpu. On my fedora container build, the cpu family is
> x86 and the cpu is x86_64. Fixing the build break by checking for
> the cpu, instead of the cpu family.
> 
> Signed-off-by: Sunil Muthuswamy 
> ---
>  meson.build | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meson.build b/meson.build
> index 7627a0ae46..2d84e90495 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -198,7 +198,7 @@ else
>have_xen_pci_passthrough = false
>  endif
>  if not get_option('whpx').disabled() and targetos == 'windows'
> -  if get_option('whpx').enabled() and cpu != 'x86_64'
> +  if get_option('whpx').enabled() and host_machine.cpu() != 'x86_64'
>  error('WHPX requires 64-bit host')
>elif cc.has_header('WinHvPlatform.h', required: get_option('whpx')) and \
> cc.has_header('WinHvEmulation.h', required: get_option('whpx'))
> 

Queued, thanks.

Paolo

Re: [PATCH v3 2/2] hw/block/nvme: add the dataset management command

[Keith Busch]

On Thu, Oct 22, 2020 at 12:17:36AM +0200, Klaus Jensen wrote:
> +static void nvme_aio_discard_cb(void *opaque, int ret)
> +{
> +NvmeRequest *req = opaque;
> +int *discards = req->opaque;
> +
> +trace_pci_nvme_aio_discard_cb(nvme_cid(req));
> +
> +if (ret) {
> +req->status = NVME_INTERNAL_DEV_ERROR;
> +trace_pci_nvme_err_aio(nvme_cid(req), strerror(ret),
> +   req->status);
> +}
> +
> +if (discards && --(*discards) > 0) {
> +return;
> +}
> +
> +g_free(req->opaque);
> +req->opaque = NULL;
> +
> +nvme_enqueue_req_completion(nvme_cq(req), req);
> +}
> +
> +static uint16_t nvme_dsm(NvmeCtrl *n, NvmeRequest *req)
> +{
> +NvmeNamespace *ns = req->ns;
> +NvmeDsmCmd *dsm = (NvmeDsmCmd *) &req->cmd;
> +NvmeDsmRange *range = NULL;
> +int *discards = NULL;
> +
> +uint32_t attr = le32_to_cpu(dsm->attributes);
> +uint32_t nr = (le32_to_cpu(dsm->nr) & 0xff) + 1;
> +
> +uint16_t status = NVME_SUCCESS;
> +
> +trace_pci_nvme_dsm(nvme_cid(req), nvme_nsid(ns), nr, attr);
> +
> +if (attr & NVME_DSMGMT_AD) {
> +int64_t offset;
> +size_t len;
> +
> +range = g_new(NvmeDsmRange, nr);
> +
> +status = nvme_dma(n, (uint8_t *)range, nr * sizeof(NvmeDsmRange),
> +  DMA_DIRECTION_TO_DEVICE, req);
> +if (status) {
> +goto out;
> +}
> +
> +discards = g_new0(int, 1);
> +req->opaque = discards;

I think you need to initialize discards to 1 so that this function is
holding a reference on it while the asynchronous part is running.
Otherwise, the callback may see 'discards' at 0 and free it while we're
trying to discard the next range.

> +
> +for (int i = 0; i < nr; i++) {
> +uint64_t slba = le64_to_cpu(range[i].slba);
> +uint32_t nlb = le32_to_cpu(range[i].nlb);
> +
> +if (nvme_check_bounds(n, ns, slba, nlb)) {
> +trace_pci_nvme_err_invalid_lba_range(slba, nlb,
> + ns->id_ns.nsze);
> +continue;
> +}
> +
> +trace_pci_nvme_dsm_deallocate(nvme_cid(req), nvme_nsid(ns), slba,
> +  nlb);
> +
> +offset = nvme_l2b(ns, slba);
> +len = nvme_l2b(ns, nlb);
> +
> +while (len) {
> +size_t bytes = MIN(BDRV_REQUEST_MAX_BYTES, len);
> +
> +blk_aio_pdiscard(ns->blkconf.blk, offset, bytes,
> + nvme_aio_discard_cb, req);
> +
> +(*discards)++;

The increment ought to be before the aio call so that the _cb can't see
the value before it's accounted for. 

> +
> +offset += bytes;
> +len -= bytes;
> +}
> +}
> +
> +if (*discards) {
> +status = NVME_NO_COMPLETE;
> +} else {
> +g_free(discards);
> +req->opaque = NULL;
> +}
> +}
> +
> +out:
> +g_free(range);
> +
> +return status;
> +}
> +
>  static uint16_t nvme_flush(NvmeCtrl *n, NvmeRequest *req)
>  {
>  block_acct_start(blk_get_stats(req->ns->blkconf.blk), &req->acct, 0,
> @@ -1088,6 +1183,8 @@ static uint16_t nvme_io_cmd(NvmeCtrl *n, NvmeRequest 
> *req)
>  case NVME_CMD_WRITE:
>  case NVME_CMD_READ:
>  return nvme_rw(n, req);
> +case NVME_CMD_DSM:
> +return nvme_dsm(n, req);
>  default:
>  trace_pci_nvme_err_invalid_opc(req->cmd.opcode);
>  return NVME_INVALID_OPCODE | NVME_DNR;
> @@ -2810,7 +2907,7 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice 
> *pci_dev)
>  id->cqes = (0x4 << 4) | 0x4;
>  id->nn = cpu_to_le32(n->num_namespaces);
>  id->oncs = cpu_to_le16(NVME_ONCS_WRITE_ZEROES | NVME_ONCS_TIMESTAMP |
> -   NVME_ONCS_FEATURES);
> +   NVME_ONCS_FEATURES | NVME_ONCS_DSM);

I think we should set ID_NS.NDPG and NDPA since there really is a
preferred granularity and alignment.

>  
>  id->vwc = 0x1;
>  id->sgls = cpu_to_le32(NVME_CTRL_SGLS_SUPPORT_NO_ALIGN |
> -- 
> 2.28.0
> 

Re: [PATCH v3 1/2] hw/block/nvme: add dulbe support

[Keith Busch]

On Thu, Oct 22, 2020 at 12:17:35AM +0200, Klaus Jensen wrote:
> From: Klaus Jensen 
> 
> Add support for reporting the Deallocated or Unwritten Logical Block
> Error (DULBE).
> 
> Rely on the block status flags reported by the block layer and consider
> any block with the BDRV_BLOCK_ZERO flag to be deallocated.
> 
> Multiple factors affect when a Write Zeroes command result in
> deallocation of blocks.
> 
>   * the underlying file system block size
>   * the blockdev format
>   * the 'discard' and 'logical_block_size' parameters
> 
>  format | discard | wz (512b)  wz (4kb)  wz (64kb)
> ---
>   qcow2ignore   n  n y
>   qcow2unmapn  n y
>   raw  ignore   n  y y
>   raw  unmapn  y y
> 
> So, this works best with an image in raw format and 4kb LBAs, since
> holes can then be punched on a per-block basis (this assumes a file
> system with a 4kb block size, YMMV). A qcow2 image, uses a cluster size
> of 64kb by default and blocks will only be marked deallocated if a full
> cluster is zeroed or discarded. However, this *is* consistent with the
> spec since Write Zeroes "should" deallocate the block if the Deallocate
> attribute is set and "may" deallocate if the Deallocate attribute is not
> set. Thus, we always try to deallocate (the BDRV_REQ_MAY_UNMAP flag is
> always set).

This looks fine. The constraints sound reasonable to me.

Reviewed-by: Keith Busch 

[PATCH] WHPX: Fix WHPX build break

[Sunil Muthuswamy]

With upstream commit#8a19980e3fc4, logic was introduced to only
allow WHPX build on x64. But, the logic checks for the cpu family
and not the cpu. On my fedora container build, the cpu family is
x86 and the cpu is x86_64. Fixing the build break by checking for
the cpu, instead of the cpu family.

Signed-off-by: Sunil Muthuswamy 
---
 meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meson.build b/meson.build
index 7627a0ae46..2d84e90495 100644
--- a/meson.build
+++ b/meson.build
@@ -198,7 +198,7 @@ else
   have_xen_pci_passthrough = false
 endif
 if not get_option('whpx').disabled() and targetos == 'windows'
-  if get_option('whpx').enabled() and cpu != 'x86_64'
+  if get_option('whpx').enabled() and host_machine.cpu() != 'x86_64'
 error('WHPX requires 64-bit host')
   elif cc.has_header('WinHvPlatform.h', required: get_option('whpx')) and \
cc.has_header('WinHvEmulation.h', required: get_option('whpx'))
-- 
2.17.1

[Bug 1900918] Re: PXB devices

[bwidawsk]

*** This bug is a duplicate of bug 1900919 ***
https://bugs.launchpad.net/bugs/1900919

I accidentally double submitted this, and this one has the wrong description. 
Please close and use
#1900919 instead.

** This bug has been marked a duplicate of bug 1900919
   PXB selected as root bus incorrectly

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1900918

Title:
  PXB devices

Status in QEMU:
  New

Bug description:
  release: 4c41341af76cfc85b5a6c0f87de4838672ab9f89

  qdev_device_add() will search for the "closest" bus possible, and bail out 
early if that bus is a root bus. pxb devices are considered root buses and so 
if you either
  1. Add a PCI device on the QEMU command line *after* a pxb device, or
  2. Add an integrated PCI device (like a watchdog)

  #1: -device pxb-pcie,id=cxl.0,bus=pcie.0,bus_nr=52 -device 
ahci,id=sata0,addr=0x8
  #2: -watchdog i6300esb -device pxb-pcie,id=cxl.0,bus=pcie.0,bus_nr=52

  The PXB will get selected as the bus (instead of the real root bus)
  and this will cause an assertion failure with the message like "qemu-
  system-x86_64: -device ahci,id=sata0,addr=0x8: PCI: Only PCI/PCIe
  bridges can be plugged into pxb-pcie"

  I think this is relatively solvable in the code base by determining if
  a bus is an expander, and skipping it if so. However, I wonder if it
  makes more sense to just allow expanders to have endpoint devices.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1900918/+subscriptions

RE: [PATCH v7 05/11] hw/block/nvme: Support Zoned Namespace Command Set

[Dmitry Fomichev]

> -Original Message-
> From: Klaus Jensen 
> Sent: Wednesday, October 21, 2020 6:26 AM
> To: Dmitry Fomichev 
> Cc: Keith Busch ; Klaus Jensen
> ; Kevin Wolf ; Philippe
> Mathieu-Daudé ; Maxim Levitsky
> ; Fam Zheng ; Niklas Cassel
> ; Damien Le Moal ;
> qemu-bl...@nongnu.org; qemu-devel@nongnu.org; Alistair Francis
> ; Matias Bjorling 
> Subject: Re: [PATCH v7 05/11] hw/block/nvme: Support Zoned Namespace
> Command Set
> 
> On Oct 19 11:17, Dmitry Fomichev wrote:
> > +/*
> > + * Close or finish all the zones that are currently open.
> > + */
> > +static void nvme_zoned_clear_ns(NvmeNamespace *ns)
> > +{
> > +NvmeZone *zone;
> > +uint32_t set_state;
> > +int i;
> > +
> > +zone = ns->zone_array;
> > +for (i = 0; i < ns->num_zones; i++, zone++) {
> > +switch (nvme_get_zone_state(zone)) {
> > +case NVME_ZONE_STATE_IMPLICITLY_OPEN:
> > +QTAILQ_REMOVE(&ns->imp_open_zones, zone, entry);
> > +break;
> > +case NVME_ZONE_STATE_EXPLICITLY_OPEN:
> > +QTAILQ_REMOVE(&ns->exp_open_zones, zone, entry);
> > +break;
> > +case NVME_ZONE_STATE_CLOSED:
> > +/* fall through */
> > +default:
> > +continue;
> > +}
> > +
> > +if (zone->d.wp == zone->d.zslba) {
> > +set_state = NVME_ZONE_STATE_EMPTY;
> > +} else {
> > +set_state = NVME_ZONE_STATE_CLOSED;
> > +}
> > +
> > +switch (set_state) {
> > +case NVME_ZONE_STATE_CLOSED:
> > +trace_pci_nvme_clear_ns_close(nvme_get_zone_state(zone),
> > +  zone->d.zslba);
> > +QTAILQ_INSERT_TAIL(&ns->closed_zones, zone, entry);
> > +break;
> > +case NVME_ZONE_STATE_EMPTY:
> > +trace_pci_nvme_clear_ns_reset(nvme_get_zone_state(zone),
> > +  zone->d.zslba);
> > +break;
> > +case NVME_ZONE_STATE_FULL:
> > +trace_pci_nvme_clear_ns_full(nvme_get_zone_state(zone),
> > + zone->d.zslba);
> > +zone->d.wp = nvme_zone_wr_boundary(zone);
> > +QTAILQ_INSERT_TAIL(&ns->full_zones, zone, entry);
> > +}
> 
> No need for the switch here - just add to the closed list in the
> conditional.

The switch becomes handy later in the series, particularly after adding
descriptor extensions. For easier reviewing, it makes sense to add it from
the beginning even though it is rudimentary at this point.

> 
> The NVME_ZONE_STATE_FULL case is unreachable.

Indeed. This should be introduced in the next patch.

Now, I've looked at this code again and the active/open counting in this
function ends up to be not quite right, I am fixing it.

> 
> > +
> > +zone->w_ptr = zone->d.wp;
> > +nvme_set_zone_state(zone, set_state);
> > +}
> > +}

[Bug 1900919] [NEW] PXB selected as root bus incorrectly

[bwidawsk]

Public bug reported:

release: 4c41341af76cfc85b5a6c0f87de4838672ab9f89

qdev_device_add() will search for the "closest" bus possible, and bail out 
early if that bus is a root bus. pxb devices are considered root buses and so 
if you either
1. Add a PCI device on the QEMU command line *after* a pxb device, or
2. Add an integrated PCI device (like a watchdog)

#1: -device pxb-pcie,id=cxl.0,bus=pcie.0,bus_nr=52 -device 
ahci,id=sata0,addr=0x8
#2: -watchdog i6300esb -device pxb-pcie,id=cxl.0,bus=pcie.0,bus_nr=52

The PXB will get selected as the bus (instead of the real root bus) and
this will cause an assertion failure with the message like "qemu-system-
x86_64: -device ahci,id=sata0,addr=0x8: PCI: Only PCI/PCIe bridges can
be plugged into pxb-pcie"

I think this is relatively solvable in the code base by determining if a
bus is an expander, and skipping it if so. However, I wonder if it makes
more sense to just allow expanders to have endpoint devices.

** Affects: qemu
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1900919

Title:
  PXB selected as root bus incorrectly

Status in QEMU:
  New

Bug description:
  release: 4c41341af76cfc85b5a6c0f87de4838672ab9f89

  qdev_device_add() will search for the "closest" bus possible, and bail out 
early if that bus is a root bus. pxb devices are considered root buses and so 
if you either
  1. Add a PCI device on the QEMU command line *after* a pxb device, or
  2. Add an integrated PCI device (like a watchdog)

  #1: -device pxb-pcie,id=cxl.0,bus=pcie.0,bus_nr=52 -device 
ahci,id=sata0,addr=0x8
  #2: -watchdog i6300esb -device pxb-pcie,id=cxl.0,bus=pcie.0,bus_nr=52

  The PXB will get selected as the bus (instead of the real root bus)
  and this will cause an assertion failure with the message like "qemu-
  system-x86_64: -device ahci,id=sata0,addr=0x8: PCI: Only PCI/PCIe
  bridges can be plugged into pxb-pcie"

  I think this is relatively solvable in the code base by determining if
  a bus is an expander, and skipping it if so. However, I wonder if it
  makes more sense to just allow expanders to have endpoint devices.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1900919/+subscriptions

[PATCH v3 2/2] hw/block/nvme: add the dataset management command

[Klaus Jensen]

From: Klaus Jensen 

Add support for the Dataset Management command and the Deallocate
attribute. Deallocation results in discards being sent to the underlying
block device. Whether of not the blocks are actually deallocated is
affected by the same factors as Write Zeroes (see previous commit).

 format | discard | dsm (512b)  dsm (4kb)  dsm (64kb)
--
  qcow2ignore   n   n  n
  qcow2unmapn   n  y
  raw  ignore   n   n  n
  raw  unmapn   y  y

Again, a raw format and 4kb LBAs are preferable.

See NVM Express 1.3d, Section 6.7 ("Dataset Management command").

Signed-off-by: Klaus Jensen 
---
 hw/block/nvme.h |  2 +
 hw/block/nvme.c | 99 -
 2 files changed, 100 insertions(+), 1 deletion(-)

diff --git a/hw/block/nvme.h b/hw/block/nvme.h
index e080a2318a50..574333caa3f9 100644
--- a/hw/block/nvme.h
+++ b/hw/block/nvme.h
@@ -28,6 +28,7 @@ typedef struct NvmeRequest {
 struct NvmeNamespace*ns;
 BlockAIOCB  *aiocb;
 uint16_tstatus;
+void*opaque;
 NvmeCqe cqe;
 NvmeCmd cmd;
 BlockAcctCookie acct;
@@ -60,6 +61,7 @@ static inline const char *nvme_io_opc_str(uint8_t opc)
 case NVME_CMD_WRITE:return "NVME_NVM_CMD_WRITE";
 case NVME_CMD_READ: return "NVME_NVM_CMD_READ";
 case NVME_CMD_WRITE_ZEROES: return "NVME_NVM_CMD_WRITE_ZEROES";
+case NVME_CMD_DSM:  return "NVME_NVM_CMD_DSM";
 default:return "NVME_NVM_CMD_UNKNOWN";
 }
 }
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index 1758cfed965c..a6dd8ae8e220 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -959,6 +959,101 @@ static void nvme_rw_cb(void *opaque, int ret)
 nvme_enqueue_req_completion(nvme_cq(req), req);
 }
 
+static void nvme_aio_discard_cb(void *opaque, int ret)
+{
+NvmeRequest *req = opaque;
+int *discards = req->opaque;
+
+trace_pci_nvme_aio_discard_cb(nvme_cid(req));
+
+if (ret) {
+req->status = NVME_INTERNAL_DEV_ERROR;
+trace_pci_nvme_err_aio(nvme_cid(req), strerror(ret),
+   req->status);
+}
+
+if (discards && --(*discards) > 0) {
+return;
+}
+
+g_free(req->opaque);
+req->opaque = NULL;
+
+nvme_enqueue_req_completion(nvme_cq(req), req);
+}
+
+static uint16_t nvme_dsm(NvmeCtrl *n, NvmeRequest *req)
+{
+NvmeNamespace *ns = req->ns;
+NvmeDsmCmd *dsm = (NvmeDsmCmd *) &req->cmd;
+NvmeDsmRange *range = NULL;
+int *discards = NULL;
+
+uint32_t attr = le32_to_cpu(dsm->attributes);
+uint32_t nr = (le32_to_cpu(dsm->nr) & 0xff) + 1;
+
+uint16_t status = NVME_SUCCESS;
+
+trace_pci_nvme_dsm(nvme_cid(req), nvme_nsid(ns), nr, attr);
+
+if (attr & NVME_DSMGMT_AD) {
+int64_t offset;
+size_t len;
+
+range = g_new(NvmeDsmRange, nr);
+
+status = nvme_dma(n, (uint8_t *)range, nr * sizeof(NvmeDsmRange),
+  DMA_DIRECTION_TO_DEVICE, req);
+if (status) {
+goto out;
+}
+
+discards = g_new0(int, 1);
+req->opaque = discards;
+
+for (int i = 0; i < nr; i++) {
+uint64_t slba = le64_to_cpu(range[i].slba);
+uint32_t nlb = le32_to_cpu(range[i].nlb);
+
+if (nvme_check_bounds(n, ns, slba, nlb)) {
+trace_pci_nvme_err_invalid_lba_range(slba, nlb,
+ ns->id_ns.nsze);
+continue;
+}
+
+trace_pci_nvme_dsm_deallocate(nvme_cid(req), nvme_nsid(ns), slba,
+  nlb);
+
+offset = nvme_l2b(ns, slba);
+len = nvme_l2b(ns, nlb);
+
+while (len) {
+size_t bytes = MIN(BDRV_REQUEST_MAX_BYTES, len);
+
+blk_aio_pdiscard(ns->blkconf.blk, offset, bytes,
+ nvme_aio_discard_cb, req);
+
+(*discards)++;
+
+offset += bytes;
+len -= bytes;
+}
+}
+
+if (*discards) {
+status = NVME_NO_COMPLETE;
+} else {
+g_free(discards);
+req->opaque = NULL;
+}
+}
+
+out:
+g_free(range);
+
+return status;
+}
+
 static uint16_t nvme_flush(NvmeCtrl *n, NvmeRequest *req)
 {
 block_acct_start(blk_get_stats(req->ns->blkconf.blk), &req->acct, 0,
@@ -1088,6 +1183,8 @@ static uint16_t nvme_io_cmd(NvmeCtrl *n, NvmeRequest *req)
 case NVME_CMD_WRITE:
 case NVME_CMD_READ:
 return nvme_rw(n, req);
+case NVME_CMD_DSM:
+return nvme_dsm(n, req);
 default:
 trace_pci_nvme_err_invalid_opc(req->cmd.opcode);
 return NVME_INVALID_OPCODE

[Bug 1900918] [NEW] PXB devices

[bwidawsk]

Public bug reported:

release: 4c41341af76cfc85b5a6c0f87de4838672ab9f89

qdev_device_add() will search for the "closest" bus possible, and bail out 
early if that bus is a root bus. pxb devices are considered root buses and so 
if you either
1. Add a PCI device on the QEMU command line *after* a pxb device, or
2. Add an integrated PCI device (like a watchdog)

#1: -device pxb-pcie,id=cxl.0,bus=pcie.0,bus_nr=52 -device 
ahci,id=sata0,addr=0x8
#2: -watchdog i6300esb -device pxb-pcie,id=cxl.0,bus=pcie.0,bus_nr=52

The PXB will get selected as the bus (instead of the real root bus) and
this will cause an assertion failure with the message like "qemu-system-
x86_64: -device ahci,id=sata0,addr=0x8: PCI: Only PCI/PCIe bridges can
be plugged into pxb-pcie"

I think this is relatively solvable in the code base by determining if a
bus is an expander, and skipping it if so. However, I wonder if it makes
more sense to just allow expanders to have endpoint devices.

** Affects: qemu
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1900918

Title:
  PXB devices

Status in QEMU:
  New

Bug description:
  release: 4c41341af76cfc85b5a6c0f87de4838672ab9f89

  qdev_device_add() will search for the "closest" bus possible, and bail out 
early if that bus is a root bus. pxb devices are considered root buses and so 
if you either
  1. Add a PCI device on the QEMU command line *after* a pxb device, or
  2. Add an integrated PCI device (like a watchdog)

  #1: -device pxb-pcie,id=cxl.0,bus=pcie.0,bus_nr=52 -device 
ahci,id=sata0,addr=0x8
  #2: -watchdog i6300esb -device pxb-pcie,id=cxl.0,bus=pcie.0,bus_nr=52

  The PXB will get selected as the bus (instead of the real root bus)
  and this will cause an assertion failure with the message like "qemu-
  system-x86_64: -device ahci,id=sata0,addr=0x8: PCI: Only PCI/PCIe
  bridges can be plugged into pxb-pcie"

  I think this is relatively solvable in the code base by determining if
  a bus is an expander, and skipping it if so. However, I wonder if it
  makes more sense to just allow expanders to have endpoint devices.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1900918/+subscriptions

[PATCH v3 0/2] hw/block/nvme: dulbe and dsm support

[Klaus Jensen]

From: Klaus Jensen 

This adds support for the Deallocated or Unwritten Logical Block error
recovery feature as well as the Dataset Management command.

I wanted to add support for the NPDG and NPDA fields such that the host
could get a hint on how many blocks to request deallocation of for the
deallocation to actually happen, but I cannot find a realiable way to
get the actual block size of the underlying device. If it is an image on
a file system we could typically use the host page size, but if it is a
raw device, we might have 512 byte sectors that we can issue discards
on. And QEMU doesn't seem to provide this without root privileges at
least.

See the two patches for some gotchas.

I also integrated this into my zoned proposal. I'll spare you the v4, nobody
cares anyway. But I put it in my repo[1] for posterity.

  [1]: https://irrelevant.dk/g/pci-nvme.git/tag/?h=zoned-v4.

Klaus Jensen (2):
  hw/block/nvme: add dulbe support
  hw/block/nvme: add the dataset management command

 hw/block/nvme-ns.h|   4 +
 hw/block/nvme.h   |   2 +
 include/block/nvme.h  |   5 ++
 hw/block/nvme-ns.c|   8 +-
 hw/block/nvme.c   | 179 +-
 hw/block/trace-events |   4 +
 6 files changed, 197 insertions(+), 5 deletions(-)

-- 
2.28.0

[PATCH v3 1/2] hw/block/nvme: add dulbe support

[Klaus Jensen]

From: Klaus Jensen 

Add support for reporting the Deallocated or Unwritten Logical Block
Error (DULBE).

Rely on the block status flags reported by the block layer and consider
any block with the BDRV_BLOCK_ZERO flag to be deallocated.

Multiple factors affect when a Write Zeroes command result in
deallocation of blocks.

  * the underlying file system block size
  * the blockdev format
  * the 'discard' and 'logical_block_size' parameters

 format | discard | wz (512b)  wz (4kb)  wz (64kb)
---
  qcow2ignore   n  n y
  qcow2unmapn  n y
  raw  ignore   n  y y
  raw  unmapn  y y

So, this works best with an image in raw format and 4kb LBAs, since
holes can then be punched on a per-block basis (this assumes a file
system with a 4kb block size, YMMV). A qcow2 image, uses a cluster size
of 64kb by default and blocks will only be marked deallocated if a full
cluster is zeroed or discarded. However, this *is* consistent with the
spec since Write Zeroes "should" deallocate the block if the Deallocate
attribute is set and "may" deallocate if the Deallocate attribute is not
set. Thus, we always try to deallocate (the BDRV_REQ_MAY_UNMAP flag is
always set).

Signed-off-by: Klaus Jensen 
---
 hw/block/nvme-ns.h|  4 +++
 include/block/nvme.h  |  5 +++
 hw/block/nvme-ns.c|  8 +++--
 hw/block/nvme.c   | 80 ++-
 hw/block/trace-events |  4 +++
 5 files changed, 97 insertions(+), 4 deletions(-)

diff --git a/hw/block/nvme-ns.h b/hw/block/nvme-ns.h
index 83734f4606e1..44bf6271b744 100644
--- a/hw/block/nvme-ns.h
+++ b/hw/block/nvme-ns.h
@@ -31,6 +31,10 @@ typedef struct NvmeNamespace {
 NvmeIdNs id_ns;
 
 NvmeNamespaceParams params;
+
+struct {
+uint32_t err_rec;
+} features;
 } NvmeNamespace;
 
 static inline uint32_t nvme_nsid(NvmeNamespace *ns)
diff --git a/include/block/nvme.h b/include/block/nvme.h
index 8a46d9cf015f..966c3bb304bd 100644
--- a/include/block/nvme.h
+++ b/include/block/nvme.h
@@ -687,6 +687,7 @@ enum NvmeStatusCodes {
 NVME_E2E_REF_ERROR  = 0x0284,
 NVME_CMP_FAILURE= 0x0285,
 NVME_ACCESS_DENIED  = 0x0286,
+NVME_DULB   = 0x0287,
 NVME_MORE   = 0x2000,
 NVME_DNR= 0x4000,
 NVME_NO_COMPLETE= 0x,
@@ -903,6 +904,9 @@ enum NvmeIdCtrlLpa {
 #define NVME_AEC_NS_ATTR(aec)   ((aec >> 8) & 0x1)
 #define NVME_AEC_FW_ACTIVATION(aec) ((aec >> 9) & 0x1)
 
+#define NVME_ERR_REC_TLER(err_rec)  (err_rec & 0x)
+#define NVME_ERR_REC_DULBE(err_rec) (err_rec & 0x1)
+
 enum NvmeFeatureIds {
 NVME_ARBITRATION= 0x1,
 NVME_POWER_MANAGEMENT   = 0x2,
@@ -1023,6 +1027,7 @@ enum NvmeNsIdentifierType {
 
 
 #define NVME_ID_NS_NSFEAT_THIN(nsfeat)  ((nsfeat & 0x1))
+#define NVME_ID_NS_NSFEAT_DULBE(nsfeat) ((nsfeat >> 2) & 0x1)
 #define NVME_ID_NS_FLBAS_EXTENDED(flbas)((flbas >> 4) & 0x1)
 #define NVME_ID_NS_FLBAS_INDEX(flbas)   ((flbas & 0xf))
 #define NVME_ID_NS_MC_SEPARATE(mc)  ((mc >> 1) & 0x1)
diff --git a/hw/block/nvme-ns.c b/hw/block/nvme-ns.c
index 31c80cdf5b5f..f1cc734c60f5 100644
--- a/hw/block/nvme-ns.c
+++ b/hw/block/nvme-ns.c
@@ -33,9 +33,7 @@ static void nvme_ns_init(NvmeNamespace *ns)
 NvmeIdNs *id_ns = &ns->id_ns;
 int lba_index = NVME_ID_NS_FLBAS_INDEX(ns->id_ns.flbas);
 
-if (blk_get_flags(ns->blkconf.blk) & BDRV_O_UNMAP) {
-ns->id_ns.dlfeat = 0x9;
-}
+ns->id_ns.dlfeat = 0x9;
 
 id_ns->lbaf[lba_index].ds = 31 - clz32(ns->blkconf.logical_block_size);
 
@@ -44,6 +42,9 @@ static void nvme_ns_init(NvmeNamespace *ns)
 /* no thin provisioning */
 id_ns->ncap = id_ns->nsze;
 id_ns->nuse = id_ns->ncap;
+
+/* support DULBE */
+id_ns->nsfeat |= 0x4;
 }
 
 static int nvme_ns_init_blk(NvmeCtrl *n, NvmeNamespace *ns, Error **errp)
@@ -92,6 +93,7 @@ int nvme_ns_setup(NvmeCtrl *n, NvmeNamespace *ns, Error 
**errp)
 }
 
 nvme_ns_init(ns);
+
 if (nvme_register_namespace(n, ns, errp)) {
 return -1;
 }
diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index 2896bb49b9c0..1758cfed965c 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -105,6 +105,7 @@ static const bool nvme_feature_support[NVME_FID_MAX] = {
 
 static const uint32_t nvme_feature_cap[NVME_FID_MAX] = {
 [NVME_TEMPERATURE_THRESHOLD]= NVME_FEAT_CAP_CHANGE,
+[NVME_ERROR_RECOVERY]   = NVME_FEAT_CAP_CHANGE | NVME_FEAT_CAP_NS,
 [NVME_VOLATILE_WRITE_CACHE] = NVME_FEAT_CAP_CHANGE,
 [NVME_NUMBER_OF_QUEUES] = NVME_FEAT_CAP_CHANGE,
 [NVME_ASYNCHRONOUS_EVENT_CONF]  = NVME_FEAT_CAP_CHANGE,
@@ -878,6 +879,41 @@ static inline uint16_t nvme_check_bounds(NvmeCtrl *n, 
NvmeNamespace *ns,
 return NVME_SUCCESS;
 }
 
+static uint16_t nvme_check_dulbe(N

[PATCH v2 4/5] hw/arm/npcm7xx: Add EHCI and OHCI controllers

[Havard Skinnemoen via]

The NPCM730 and NPCM750 chips have a single USB host port shared between
a USB 2.0 EHCI host controller and a USB 1.1 OHCI host controller. This
adds support for both of them.

Testing notes:
  * With -device usb-kbd, qemu will automatically insert a full-speed
hub, and the keyboard becomes controlled by the OHCI controller.
  * With -device usb-kbd,bus=usb-bus.0,port=1, the keyboard is directly
attached to the port without any hubs, and the device becomes
controlled by the EHCI controller since it's high speed capable.
  * With -device usb-kbd,bus=usb-bus.0,port=1,usb_version=1, the
keyboard is directly attached to the port, but it only advertises
itself as full-speed capable, so it becomes controlled by the OHCI
controller.

In all cases, the keyboard device enumerates correctly.

Reviewed-by: Tyrone Ting 
Reviewed-by: Gerd Hoffmann 
Signed-off-by: Havard Skinnemoen 
---
 docs/system/arm/nuvoton.rst |  2 +-
 hw/usb/hcd-ehci.h   |  1 +
 include/hw/arm/npcm7xx.h|  4 
 hw/arm/npcm7xx.c| 27 +--
 hw/usb/hcd-ehci-sysbus.c| 19 +++
 5 files changed, 50 insertions(+), 3 deletions(-)

diff --git a/docs/system/arm/nuvoton.rst b/docs/system/arm/nuvoton.rst
index 4342434df4..99fc61c740 100644
--- a/docs/system/arm/nuvoton.rst
+++ b/docs/system/arm/nuvoton.rst
@@ -39,6 +39,7 @@ Supported devices
  * OTP controllers (no protection features)
  * Flash Interface Unit (FIU; no protection features)
  * Random Number Generator (RNG)
+ * USB host (USBH)
 
 Missing devices
 ---
@@ -54,7 +55,6 @@ Missing devices
* eSPI slave interface
 
  * Ethernet controllers (GMAC and EMC)
- * USB host (USBH)
  * USB device (USBD)
  * SMBus controller (SMBF)
  * Peripheral SPI controller (PSPI)
diff --git a/hw/usb/hcd-ehci.h b/hw/usb/hcd-ehci.h
index fd122dd4cd..a173707d9b 100644
--- a/hw/usb/hcd-ehci.h
+++ b/hw/usb/hcd-ehci.h
@@ -344,6 +344,7 @@ struct EHCIPCIState {
 #define TYPE_PLATFORM_EHCI "platform-ehci-usb"
 #define TYPE_EXYNOS4210_EHCI "exynos4210-ehci-usb"
 #define TYPE_AW_H3_EHCI "aw-h3-ehci-usb"
+#define TYPE_NPCM7XX_EHCI "npcm7xx-ehci-usb"
 #define TYPE_TEGRA2_EHCI "tegra2-ehci-usb"
 #define TYPE_PPC4xx_EHCI "ppc4xx-ehci-usb"
 #define TYPE_FUSBH200_EHCI "fusbh200-ehci-usb"
diff --git a/include/hw/arm/npcm7xx.h b/include/hw/arm/npcm7xx.h
index 761f9b987e..aeee1beaaa 100644
--- a/include/hw/arm/npcm7xx.h
+++ b/include/hw/arm/npcm7xx.h
@@ -25,6 +25,8 @@
 #include "hw/nvram/npcm7xx_otp.h"
 #include "hw/timer/npcm7xx_timer.h"
 #include "hw/ssi/npcm7xx_fiu.h"
+#include "hw/usb/hcd-ehci.h"
+#include "hw/usb/hcd-ohci.h"
 #include "target/arm/cpu.h"
 
 #define NPCM7XX_MAX_NUM_CPUS(2)
@@ -77,6 +79,8 @@ typedef struct NPCM7xxState {
 NPCM7xxOTPState fuse_array;
 NPCM7xxMCState  mc;
 NPCM7xxRNGState rng;
+EHCISysBusState ehci;
+OHCISysBusState ohci;
 NPCM7xxFIUState fiu[2];
 } NPCM7xxState;
 
diff --git a/hw/arm/npcm7xx.c b/hw/arm/npcm7xx.c
index cb4db41c54..c1d122576b 100644
--- a/hw/arm/npcm7xx.c
+++ b/hw/arm/npcm7xx.c
@@ -46,6 +46,10 @@
 #define NPCM7XX_MC_BA   (0xf0824000)
 #define NPCM7XX_RNG_BA  (0xf000b000)
 
+/* USB Host modules */
+#define NPCM7XX_EHCI_BA (0xf0806000)
+#define NPCM7XX_OHCI_BA (0xf0807000)
+
 /* Internal AHB SRAM */
 #define NPCM7XX_RAM3_BA (0xc0008000)
 #define NPCM7XX_RAM3_SZ (4 * KiB)
@@ -90,6 +94,8 @@ enum NPCM7xxInterrupt {
 NPCM7XX_WDG0_IRQ= 47,   /* Timer Module 0 Watchdog */
 NPCM7XX_WDG1_IRQ,   /* Timer Module 1 Watchdog */
 NPCM7XX_WDG2_IRQ,   /* Timer Module 2 Watchdog */
+NPCM7XX_EHCI_IRQ= 61,
+NPCM7XX_OHCI_IRQ= 62,
 };
 
 /* Total number of GIC interrupts, including internal Cortex-A9 interrupts. */
@@ -263,6 +269,9 @@ static void npcm7xx_init(Object *obj)
 object_initialize_child(obj, "tim[*]", &s->tim[i], TYPE_NPCM7XX_TIMER);
 }
 
+object_initialize_child(obj, "ehci", &s->ehci, TYPE_NPCM7XX_EHCI);
+object_initialize_child(obj, "ohci", &s->ohci, TYPE_SYSBUS_OHCI);
+
 QEMU_BUILD_BUG_ON(ARRAY_SIZE(npcm7xx_fiu) != ARRAY_SIZE(s->fiu));
 for (i = 0; i < ARRAY_SIZE(s->fiu); i++) {
 object_initialize_child(obj, npcm7xx_fiu[i].name, &s->fiu[i],
@@ -380,6 +389,22 @@ static void npcm7xx_realize(DeviceState *dev, Error **errp)
 sysbus_realize(SYS_BUS_DEVICE(&s->rng), &error_abort);
 sysbus_mmio_map(SYS_BUS_DEVICE(&s->rng), 0, NPCM7XX_RNG_BA);
 
+/* USB Host */
+object_property_set_bool(OBJECT(&s->ehci), "companion-enable", true,
+ &error_abort);
+sysbus_realize(SYS_BUS_DEVICE(&s->ehci), &error_abort);
+sysbus_mmio_map(SYS_BUS_DEVICE(&s->ehci), 0, NPCM7XX_EHCI_BA);
+sysbus_connect_irq(SYS_BUS_DEVICE(&s->ehci), 0,
+   npcm7xx_irq(s, NPCM7XX_EHCI_IRQ));
+
+object_property_set_str(OBJECT(&s->ohci), "masterbu

[PATCH v6 2/6] migration: Introduce migrate_send_rp_message_req_pages()

[Peter Xu]

This is another layer wrapper for sending a page request to the source VM.  The
new migrate_send_rp_message_req_pages() will be used elsewhere in coming
patches.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
---
 migration/migration.c | 10 --
 migration/migration.h |  2 ++
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 0575ecb379..255e69c8aa 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -316,8 +316,8 @@ error:
  *   Start: Address offset within the RB
  *   Len: Length in bytes required - must be a multiple of pagesize
  */
-int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb,
-  ram_addr_t start)
+int migrate_send_rp_message_req_pages(MigrationIncomingState *mis,
+  RAMBlock *rb, ram_addr_t start)
 {
 uint8_t bufc[12 + 1 + 255]; /* start (8), len (4), rbname up to 256 */
 size_t msglen = 12; /* start + len */
@@ -353,6 +353,12 @@ int migrate_send_rp_req_pages(MigrationIncomingState *mis, 
RAMBlock *rb,
 return migrate_send_rp_message(mis, msg_type, msglen, bufc);
 }
 
+int migrate_send_rp_req_pages(MigrationIncomingState *mis,
+  RAMBlock *rb, ram_addr_t start)
+{
+return migrate_send_rp_message_req_pages(mis, rb, start);
+}
+
 static bool migration_colo_enabled;
 bool migration_incoming_colo_enabled(void)
 {
diff --git a/migration/migration.h b/migration/migration.h
index deb411aaad..e853ccf8b1 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -333,6 +333,8 @@ void migrate_send_rp_pong(MigrationIncomingState *mis,
   uint32_t value);
 int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb,
   ram_addr_t start);
+int migrate_send_rp_message_req_pages(MigrationIncomingState *mis,
+  RAMBlock *rb, ram_addr_t start);
 void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis,
  char *block_name);
 void migrate_send_rp_resume_ack(MigrationIncomingState *mis, uint32_t value);
-- 
2.26.2

[PATCH v6 0/6] migration/postcopy: Sync faulted addresses after network recovered

[Peter Xu]

v6:
- fix page mask to use ramblock psize [Dave]

v5:
- added one test patch for easier debugging for migration-test
- added one fix patch [1] for another postcopy race
- fixed a bug that could trigger when host/guest page size differs

v4:
- use "void */ulong" instead of "uint64_t" where proper in patch 3/4 [Dave]

v3:
- fix build on 32bit hosts & rebase
- remove r-bs for the last 2 patches for Dave due to the changes

v2:
- add r-bs for Dave
- add patch "migration: Properly destroy variables on incoming side" as patch 1
- destroy page_request_mutex in migration_incoming_state_destroy() too [Dave]
- use WITH_QEMU_LOCK_GUARD in two places where we can [Dave]

We've seen conditional guest hangs on destination VM after postcopy recovered.
However the hang will resolve itself after a few minutes.

The problem is: after a postcopy recovery, the prioritized postcopy queue on
the source VM is actually missing.  So all the faulted threads before the
postcopy recovery happened will keep halted until (accidentally) the page got
copied by the background precopy migration stream.

The solution is to also refresh this information after postcopy recovery.  To
achieve this, we need to maintain a list of faulted addresses on the
destination node, so that we can resend the list when necessary.  This work is
done via patch 2-5.

With that, the last thing we need to do is to send this extra information to
source VM after recovered.  Very luckily, this synchronization can be
"emulated" by sending a bunch of page requests (although these pages have been
sent previously!) to source VM just like when we've got a page fault.  Even in
the 1st version of the postcopy code we'll handle duplicated pages well.  So
this fix does not even need a new capability bit and it'll work smoothly on old
QEMUs when we migrate from them to the new QEMUs.

Please review, thanks.

Peter Xu (6):
  migration: Pass incoming state into qemu_ufd_copy_ioctl()
  migration: Introduce migrate_send_rp_message_req_pages()
  migration: Maintain postcopy faulted addresses
  migration: Sync requested pages after postcopy recovery
  migration/postcopy: Release fd before going into 'postcopy-pause'
  migration-test: Only hide error if !QTEST_LOG

 migration/migration.c| 55 ++
 migration/migration.h| 21 -
 migration/postcopy-ram.c | 25 
 migration/savevm.c   | 57 
 migration/trace-events   |  3 ++
 tests/qtest/migration-test.c |  6 +++-
 6 files changed, 154 insertions(+), 13 deletions(-)

-- 
2.26.2

[PATCH v2 5/5] hw/gpio: Add GPIO model for Nuvoton NPCM7xx

[Havard Skinnemoen via]

The NPCM7xx chips have multiple GPIO controllers that are mostly
identical except for some minor differences like the reset values of
some registers. Each controller controls up to 32 pins.

Each individual pin is modeled as a pair of unnamed GPIOs -- one for
emitting the actual pin state, and one for driving the pin externally.
Like the nRF51 GPIO controller, a gpio level may be negative, which
means the pin is not driven, or floating.

Reviewed-by: Tyrone Ting 
Signed-off-by: Havard Skinnemoen 
---
 docs/system/arm/nuvoton.rst |   2 +-
 include/hw/arm/npcm7xx.h|   2 +
 include/hw/gpio/npcm7xx_gpio.h  |  55 +
 hw/arm/npcm7xx.c|  80 ++
 hw/gpio/npcm7xx_gpio.c  | 424 
 tests/qtest/npcm7xx_gpio-test.c | 385 +
 hw/gpio/meson.build |   1 +
 hw/gpio/trace-events|   7 +
 tests/qtest/meson.build |   1 +
 9 files changed, 956 insertions(+), 1 deletion(-)
 create mode 100644 include/hw/gpio/npcm7xx_gpio.h
 create mode 100644 hw/gpio/npcm7xx_gpio.c
 create mode 100644 tests/qtest/npcm7xx_gpio-test.c

diff --git a/docs/system/arm/nuvoton.rst b/docs/system/arm/nuvoton.rst
index 99fc61c740..b00d405d52 100644
--- a/docs/system/arm/nuvoton.rst
+++ b/docs/system/arm/nuvoton.rst
@@ -40,11 +40,11 @@ Supported devices
  * Flash Interface Unit (FIU; no protection features)
  * Random Number Generator (RNG)
  * USB host (USBH)
+ * GPIO controller
 
 Missing devices
 ---
 
- * GPIO controller
  * LPC/eSPI host-to-BMC interface, including
 
* Keyboard and mouse controller interface (KBCI)
diff --git a/include/hw/arm/npcm7xx.h b/include/hw/arm/npcm7xx.h
index aeee1beaaa..5469247e38 100644
--- a/include/hw/arm/npcm7xx.h
+++ b/include/hw/arm/npcm7xx.h
@@ -18,6 +18,7 @@
 
 #include "hw/boards.h"
 #include "hw/cpu/a9mpcore.h"
+#include "hw/gpio/npcm7xx_gpio.h"
 #include "hw/mem/npcm7xx_mc.h"
 #include "hw/misc/npcm7xx_clk.h"
 #include "hw/misc/npcm7xx_gcr.h"
@@ -79,6 +80,7 @@ typedef struct NPCM7xxState {
 NPCM7xxOTPState fuse_array;
 NPCM7xxMCState  mc;
 NPCM7xxRNGState rng;
+NPCM7xxGPIOStategpio[8];
 EHCISysBusState ehci;
 OHCISysBusState ohci;
 NPCM7xxFIUState fiu[2];
diff --git a/include/hw/gpio/npcm7xx_gpio.h b/include/hw/gpio/npcm7xx_gpio.h
new file mode 100644
index 00..b1d771bd77
--- /dev/null
+++ b/include/hw/gpio/npcm7xx_gpio.h
@@ -0,0 +1,55 @@
+/*
+ * Nuvoton NPCM7xx General Purpose Input / Output (GPIO)
+ *
+ * Copyright 2020 Google LLC
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+#ifndef NPCM7XX_GPIO_H
+#define NPCM7XX_GPIO_H
+
+#include "exec/memory.h"
+#include "hw/sysbus.h"
+
+/* Number of pins managed by each controller. */
+#define NPCM7XX_GPIO_NR_PINS (32)
+
+/*
+ * Number of registers in our device state structure. Don't change this without
+ * incrementing the version_id in the vmstate.
+ */
+#define NPCM7XX_GPIO_NR_REGS (0x80 / sizeof(uint32_t))
+
+typedef struct NPCM7xxGPIOState {
+SysBusDevice parent;
+
+/* Properties to be defined by the SoC */
+uint32_t reset_pu;
+uint32_t reset_pd;
+uint32_t reset_osrc;
+uint32_t reset_odsc;
+
+MemoryRegion mmio;
+
+qemu_irq irq;
+qemu_irq output[NPCM7XX_GPIO_NR_PINS];
+
+uint32_t pin_level;
+uint32_t ext_level;
+uint32_t ext_driven;
+
+uint32_t regs[NPCM7XX_GPIO_NR_REGS];
+} NPCM7xxGPIOState;
+
+#define TYPE_NPCM7XX_GPIO "npcm7xx-gpio"
+#define NPCM7XX_GPIO(obj) \
+OBJECT_CHECK(NPCM7xxGPIOState, (obj), TYPE_NPCM7XX_GPIO)
+
+#endif /* NPCM7XX_GPIO_H */
diff --git a/hw/arm/npcm7xx.c b/hw/arm/npcm7xx.c
index c1d122576b..47e2b6fc40 100644
--- a/hw/arm/npcm7xx.c
+++ b/hw/arm/npcm7xx.c
@@ -96,6 +96,14 @@ enum NPCM7xxInterrupt {
 NPCM7XX_WDG2_IRQ,   /* Timer Module 2 Watchdog */
 NPCM7XX_EHCI_IRQ= 61,
 NPCM7XX_OHCI_IRQ= 62,
+NPCM7XX_GPIO0_IRQ   = 116,
+NPCM7XX_GPIO1_IRQ,
+NPCM7XX_GPIO2_IRQ,
+NPCM7XX_GPIO3_IRQ,
+NPCM7XX_GPIO4_IRQ,
+NPCM7XX_GPIO5_IRQ,
+NPCM7XX_GPIO6_IRQ,
+NPCM7XX_GPIO7_IRQ,
 };
 
 /* Total number of GIC interrupts, including internal Cortex-A9 interrupts. */
@@ -130,6 +138,55 @@ static const hwaddr npcm7xx_fiu3_flash_addr[] = {
 0xb800, /* CS3 */
 };
 
+static const struct {
+hwaddr regs_addr;
+uint32_t unconnected_pins;
+uint32_t reset_pu;
+uint32_t reset_pd;
+uint32_t reset_osrc;
+uint32_t reset_odsc;
+} npcm7xx_gpio[] = {
+{
+.regs_addr = 0xf001,
+.r

[PATCH v2 3/5] hw/misc: Add npcm7xx random number generator

[Havard Skinnemoen via]

The RNG module returns a byte of randomness when the Data Valid bit is
set.

This implementation ignores the prescaler setting, and loads a new value
into RNGD every time RNGCS is read while the RNG is enabled and random
data is available.

A qtest featuring some simple randomness tests is included.

Reviewed-by: Tyrone Ting 
Reviewed-by: Peter Maydell 
Signed-off-by: Havard Skinnemoen 
---
 docs/system/arm/nuvoton.rst|   2 +-
 include/hw/arm/npcm7xx.h   |   2 +
 include/hw/misc/npcm7xx_rng.h  |  34 
 hw/arm/npcm7xx.c   |   7 +-
 hw/misc/npcm7xx_rng.c  | 180 +
 tests/qtest/npcm7xx_rng-test.c | 278 +
 hw/misc/meson.build|   1 +
 hw/misc/trace-events   |   4 +
 tests/qtest/meson.build|   1 +
 9 files changed, 507 insertions(+), 2 deletions(-)
 create mode 100644 include/hw/misc/npcm7xx_rng.h
 create mode 100644 hw/misc/npcm7xx_rng.c
 create mode 100644 tests/qtest/npcm7xx_rng-test.c

diff --git a/docs/system/arm/nuvoton.rst b/docs/system/arm/nuvoton.rst
index e3e1a3a3a7..4342434df4 100644
--- a/docs/system/arm/nuvoton.rst
+++ b/docs/system/arm/nuvoton.rst
@@ -38,6 +38,7 @@ Supported devices
  * DDR4 memory controller (dummy interface indicating memory training is done)
  * OTP controllers (no protection features)
  * Flash Interface Unit (FIU; no protection features)
+ * Random Number Generator (RNG)
 
 Missing devices
 ---
@@ -59,7 +60,6 @@ Missing devices
  * Peripheral SPI controller (PSPI)
  * Analog to Digital Converter (ADC)
  * SD/MMC host
- * Random Number Generator (RNG)
  * PECI interface
  * Pulse Width Modulation (PWM)
  * Tachometer
diff --git a/include/hw/arm/npcm7xx.h b/include/hw/arm/npcm7xx.h
index 13106af215..761f9b987e 100644
--- a/include/hw/arm/npcm7xx.h
+++ b/include/hw/arm/npcm7xx.h
@@ -21,6 +21,7 @@
 #include "hw/mem/npcm7xx_mc.h"
 #include "hw/misc/npcm7xx_clk.h"
 #include "hw/misc/npcm7xx_gcr.h"
+#include "hw/misc/npcm7xx_rng.h"
 #include "hw/nvram/npcm7xx_otp.h"
 #include "hw/timer/npcm7xx_timer.h"
 #include "hw/ssi/npcm7xx_fiu.h"
@@ -75,6 +76,7 @@ typedef struct NPCM7xxState {
 NPCM7xxOTPState key_storage;
 NPCM7xxOTPState fuse_array;
 NPCM7xxMCState  mc;
+NPCM7xxRNGState rng;
 NPCM7xxFIUState fiu[2];
 } NPCM7xxState;
 
diff --git a/include/hw/misc/npcm7xx_rng.h b/include/hw/misc/npcm7xx_rng.h
new file mode 100644
index 00..5e85fd439d
--- /dev/null
+++ b/include/hw/misc/npcm7xx_rng.h
@@ -0,0 +1,34 @@
+/*
+ * Nuvoton NPCM7xx Random Number Generator.
+ *
+ * Copyright 2020 Google LLC
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+ * for more details.
+ */
+#ifndef NPCM7XX_RNG_H
+#define NPCM7XX_RNG_H
+
+#include "hw/sysbus.h"
+
+typedef struct NPCM7xxRNGState {
+SysBusDevice parent;
+
+MemoryRegion iomem;
+
+uint8_t rngcs;
+uint8_t rngd;
+uint8_t rngmode;
+} NPCM7xxRNGState;
+
+#define TYPE_NPCM7XX_RNG "npcm7xx-rng"
+#define NPCM7XX_RNG(obj) OBJECT_CHECK(NPCM7xxRNGState, (obj), TYPE_NPCM7XX_RNG)
+
+#endif /* NPCM7XX_RNG_H */
diff --git a/hw/arm/npcm7xx.c b/hw/arm/npcm7xx.c
index c341dcab8b..cb4db41c54 100644
--- a/hw/arm/npcm7xx.c
+++ b/hw/arm/npcm7xx.c
@@ -44,6 +44,7 @@
 #define NPCM7XX_GCR_BA  (0xf080)
 #define NPCM7XX_CLK_BA  (0xf0801000)
 #define NPCM7XX_MC_BA   (0xf0824000)
+#define NPCM7XX_RNG_BA  (0xf000b000)
 
 /* Internal AHB SRAM */
 #define NPCM7XX_RAM3_BA (0xc0008000)
@@ -256,6 +257,7 @@ static void npcm7xx_init(Object *obj)
 object_initialize_child(obj, "otp2", &s->fuse_array,
 TYPE_NPCM7XX_FUSE_ARRAY);
 object_initialize_child(obj, "mc", &s->mc, TYPE_NPCM7XX_MC);
+object_initialize_child(obj, "rng", &s->rng, TYPE_NPCM7XX_RNG);
 
 for (i = 0; i < ARRAY_SIZE(s->tim); i++) {
 object_initialize_child(obj, "tim[*]", &s->tim[i], TYPE_NPCM7XX_TIMER);
@@ -374,6 +376,10 @@ static void npcm7xx_realize(DeviceState *dev, Error **errp)
serial_hd(i), DEVICE_LITTLE_ENDIAN);
 }
 
+/* Random Number Generator. Cannot fail. */
+sysbus_realize(SYS_BUS_DEVICE(&s->rng), &error_abort);
+sysbus_mmio_map(SYS_BUS_DEVICE(&s->rng), 0, NPCM7XX_RNG_BA);
+
 /*
  * Flash Interface Unit (FIU). Can fail if incorrect number of chip selects
  * specified, but this is a programming error.
@@ -412,7 +418,6 @@ static void npcm7xx_realize(DeviceState *dev, Error **errp)
 create_unimplemented_device("npcm7xx.vdmx", 0xe080, 

[PATCH v6 16/16] scripts/oss-fuzz: remove the generic-fuzz target

[Alexander Bulekov]

generic-fuzz is not a standalone fuzzer - it requires some env variables
to be set. On oss-fuzz, we set these with some predefined
generic-fuzz-{...} targets, that are thin wrappers around generic-fuzz.
Remove generic-fuzz from the oss-fuzz build, so oss-fuzz does not treat
it as a standalone fuzzer.

Signed-off-by: Alexander Bulekov 
---
 scripts/oss-fuzz/build.sh | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/scripts/oss-fuzz/build.sh b/scripts/oss-fuzz/build.sh
index 0c3ca9e06f..37cd7f9e25 100755
--- a/scripts/oss-fuzz/build.sh
+++ b/scripts/oss-fuzz/build.sh
@@ -97,5 +97,11 @@ do
 cp qemu-fuzz-i386 "$DEST_DIR/qemu-fuzz-i386-target-$target"
 done
 
+# Remove the generic-fuzz target, as it requires some environment variables to
+# be configured. We have some generic-fuzz-{pc-q35, floppy, ...} targets that
+# are thin wrappers around this target that set the required environment
+# variables according to predefined configs.
+rm "$DEST_DIR/qemu-fuzz-i386-target-generic-fuzz"
+
 echo "Done. The fuzzers are located in $DEST_DIR"
 exit 0
-- 
2.28.0

[PATCH v2 1/5] Move npcm7xx_timer_reached_zero call out of npcm7xx_timer_pause

[Havard Skinnemoen via]

This allows us to reuse npcm7xx_timer_pause for the watchdog timer.

Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Havard Skinnemoen 
---
 hw/timer/npcm7xx_timer.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/hw/timer/npcm7xx_timer.c b/hw/timer/npcm7xx_timer.c
index 5703e43d40..2df9e3e496 100644
--- a/hw/timer/npcm7xx_timer.c
+++ b/hw/timer/npcm7xx_timer.c
@@ -157,9 +157,6 @@ static void npcm7xx_timer_pause(NPCM7xxTimer *t)
 timer_del(&t->qtimer);
 now = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
 t->remaining_ns = t->expires_ns - now;
-if (t->remaining_ns <= 0) {
-npcm7xx_timer_reached_zero(t);
-}
 }
 
 /*
@@ -239,6 +236,9 @@ static void npcm7xx_timer_write_tcsr(NPCM7xxTimer *t, 
uint32_t new_tcsr)
 } else {
 t->tcsr &= ~NPCM7XX_TCSR_CACT;
 npcm7xx_timer_pause(t);
+if (t->remaining_ns <= 0) {
+npcm7xx_timer_reached_zero(t);
+}
 }
 }
 }
-- 
2.29.0.rc1.297.gfa9743e501-goog

[PATCH v2 2/5] hw/timer: Adding watchdog for NPCM7XX Timer.

[Havard Skinnemoen via]

From: Hao Wu 

The watchdog is part of NPCM7XX's timer module. Its behavior is
controlled by the WTCR register in the timer.

When enabled, the watchdog issues an interrupt signal after a pre-set
amount of cycles, and issues a reset signal shortly after that.

Reviewed-by: Tyrone Ting 
Signed-off-by: Hao Wu 
Signed-off-by: Havard Skinnemoen 
---
 include/hw/misc/npcm7xx_clk.h |   2 +
 include/hw/timer/npcm7xx_timer.h  |  48 +++-
 hw/arm/npcm7xx.c  |  12 +
 hw/misc/npcm7xx_clk.c |  28 ++
 hw/timer/npcm7xx_timer.c  | 266 ++
 tests/qtest/npcm7xx_watchdog_timer-test.c | 313 ++
 MAINTAINERS   |   1 +
 tests/qtest/meson.build   |   1 +
 8 files changed, 618 insertions(+), 53 deletions(-)
 create mode 100644 tests/qtest/npcm7xx_watchdog_timer-test.c

diff --git a/include/hw/misc/npcm7xx_clk.h b/include/hw/misc/npcm7xx_clk.h
index cdcc9e8534..2338fbbdb5 100644
--- a/include/hw/misc/npcm7xx_clk.h
+++ b/include/hw/misc/npcm7xx_clk.h
@@ -31,6 +31,8 @@
  */
 #define NPCM7XX_CLK_NR_REGS (0x70 / sizeof(uint32_t))
 
+#define NPCM7XX_WATCHDOG_RESET_GPIO_IN "npcm7xx-clk-watchdog-reset-gpio-in"
+
 typedef struct NPCM7xxCLKState {
 SysBusDevice parent;
 
diff --git a/include/hw/timer/npcm7xx_timer.h b/include/hw/timer/npcm7xx_timer.h
index 878a365a79..6993fd723a 100644
--- a/include/hw/timer/npcm7xx_timer.h
+++ b/include/hw/timer/npcm7xx_timer.h
@@ -29,14 +29,31 @@
  */
 #define NPCM7XX_TIMER_NR_REGS (0x54 / sizeof(uint32_t))
 
+/* The basic watchdog timer period is 2^14 clock cycles. */
+#define NPCM7XX_WATCHDOG_BASETIME_SHIFT 14
+
+#define NPCM7XX_WATCHDOG_RESET_GPIO_OUT "npcm7xx-clk-watchdog-reset-gpio-out"
+
 typedef struct NPCM7xxTimerCtrlState NPCM7xxTimerCtrlState;
 
 /**
- * struct NPCM7xxTimer - Individual timer state.
- * @irq: GIC interrupt line to fire on expiration (if enabled).
+ * struct NPCM7xxBaseTimer - Basic functionality that both regular timer and
+ * watchdog timer use.
  * @qtimer: QEMU timer that notifies us on expiration.
  * @expires_ns: Absolute virtual expiration time.
  * @remaining_ns: Remaining time until expiration if timer is paused.
+ */
+typedef struct NPCM7xxBaseTimer {
+QEMUTimer   qtimer;
+int64_t expires_ns;
+int64_t remaining_ns;
+} NPCM7xxBaseTimer;
+
+/**
+ * struct NPCM7xxTimer - Individual timer state.
+ * @ctrl: The timer module that owns this timer.
+ * @irq: GIC interrupt line to fire on expiration (if enabled).
+ * @base_timer: The basic timer functionality for this timer.
  * @tcsr: The Timer Control and Status Register.
  * @ticr: The Timer Initial Count Register.
  */
@@ -44,21 +61,38 @@ typedef struct NPCM7xxTimer {
 NPCM7xxTimerCtrlState *ctrl;
 
 qemu_irqirq;
-QEMUTimer   qtimer;
-int64_t expires_ns;
-int64_t remaining_ns;
+NPCM7xxBaseTimer base_timer;
 
 uint32_ttcsr;
 uint32_tticr;
 } NPCM7xxTimer;
 
+/**
+ * struct NPCM7xxWatchdogTimer - The watchdog timer state.
+ * @ctrl: The timer module that owns this timer.
+ * @irq: GIC interrupt line to fire on expiration (if enabled).
+ * @reset_signal: The GPIO used to send a reset signal.
+ * @base_timer: The basic timer functionality for this timer.
+ * @wtcr: The Watchdog Timer Control Register.
+ */
+typedef struct NPCM7xxWatchdogTimer {
+NPCM7xxTimerCtrlState *ctrl;
+
+qemu_irqirq;
+qemu_irqreset_signal;
+NPCM7xxBaseTimer base_timer;
+
+uint32_twtcr;
+} NPCM7xxWatchdogTimer;
+
 /**
  * struct NPCM7xxTimerCtrlState - Timer Module device state.
  * @parent: System bus device.
  * @iomem: Memory region through which registers are accessed.
+ * @index: The index of this timer module.
  * @tisr: The Timer Interrupt Status Register.
- * @wtcr: The Watchdog Timer Control Register.
  * @timer: The five individual timers managed by this module.
+ * @watchdog_timer: The watchdog timer managed by this module.
  */
 struct NPCM7xxTimerCtrlState {
 SysBusDevice parent;
@@ -66,9 +100,9 @@ struct NPCM7xxTimerCtrlState {
 MemoryRegion iomem;
 
 uint32_ttisr;
-uint32_twtcr;
 
 NPCM7xxTimer timer[NPCM7XX_TIMERS_PER_CTRL];
+NPCM7xxWatchdogTimer watchdog_timer;
 };
 
 #define TYPE_NPCM7XX_TIMER "npcm7xx-timer"
diff --git a/hw/arm/npcm7xx.c b/hw/arm/npcm7xx.c
index 037f3a26f2..c341dcab8b 100644
--- a/hw/arm/npcm7xx.c
+++ b/hw/arm/npcm7xx.c
@@ -86,6 +86,9 @@ enum NPCM7xxInterrupt {
 NPCM7XX_TIMER12_IRQ,
 NPCM7XX_TIMER13_IRQ,
 NPCM7XX_TIMER14_IRQ,
+NPCM7XX_WDG0_IRQ= 47,   /* Timer Module 0 Watchdog */
+NPCM7XX_WDG1_IRQ,   /* Timer Module 1 Watchdog */
+NPCM7XX_WDG2_IRQ,   /* Timer Module 2 Watchdog */
 };
 
 /* Total number of GIC interrupts, including internal Cortex-A9 interrupts. */
@@ -353,6 +356,15 @@ static void npcm7xx_realize(DeviceState *d

[PATCH v6 14/16] fuzz: add generic-fuzz configs for oss-fuzz

[Alexander Bulekov]

Predefine some generic-fuzz configs. For each of these, we will create a
separate FuzzTarget that can be selected through argv0 and, therefore,
fuzzed on oss-fuzz.

Signed-off-by: Alexander Bulekov 
---
 tests/qtest/fuzz/generic_fuzz_configs.h | 121 
 1 file changed, 121 insertions(+)
 create mode 100644 tests/qtest/fuzz/generic_fuzz_configs.h

diff --git a/tests/qtest/fuzz/generic_fuzz_configs.h 
b/tests/qtest/fuzz/generic_fuzz_configs.h
new file mode 100644
index 00..c4d925f9e6
--- /dev/null
+++ b/tests/qtest/fuzz/generic_fuzz_configs.h
@@ -0,0 +1,121 @@
+/*
+ * Generic Virtual-Device Fuzzing Target Configs
+ *
+ * Copyright Red Hat Inc., 2020
+ *
+ * Authors:
+ *  Alexander Bulekov   
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef GENERIC_FUZZ_CONFIGS_H
+#define GENERIC_FUZZ_CONFIGS_H
+
+#include "qemu/osdep.h"
+
+typedef struct generic_fuzz_config {
+const char *name, *args, *objects;
+} generic_fuzz_config;
+
+const generic_fuzz_config predefined_configs[] = {
+{
+.name = "virtio-net-pci-slirp",
+.args = "-M q35 -nodefaults "
+"-device virtio-net,netdev=net0 -netdev user,id=net0",
+.objects = "virtio*",
+},{
+.name = "virtio-blk",
+.args = "-machine q35 -device virtio-blk,drive=disk0 "
+"-drive file=null-co://,id=disk0,if=none,format=raw",
+.objects = "virtio*",
+},{
+.name = "virtio-scsi",
+.args = "-machine q35 -device virtio-scsi,num_queues=8 "
+"-device scsi-hd,drive=disk0 "
+"-drive file=null-co://,id=disk0,if=none,format=raw",
+.objects = "scsi* virtio*",
+},{
+.name = "virtio-gpu",
+.args = "-machine q35 -nodefaults -device virtio-gpu",
+.objects = "virtio*",
+},{
+.name = "virtio-vga",
+.args = "-machine q35 -nodefaults -device virtio-vga",
+.objects = "virtio*",
+},{
+.name = "virtio-rng",
+.args = "-machine q35 -nodefaults -device virtio-rng",
+.objects = "virtio*",
+},{
+.name = "virtio-balloon",
+.args = "-machine q35 -nodefaults -device virtio-balloon",
+.objects = "virtio*",
+},{
+.name = "virtio-serial",
+.args = "-machine q35 -nodefaults -device virtio-serial",
+.objects = "virtio*",
+},{
+.name = "virtio-mouse",
+.args = "-machine q35 -nodefaults -device virtio-mouse",
+.objects = "virtio*",
+},{
+.name = "e1000",
+.args = "-M q35 -nodefaults "
+"-device e1000,netdev=net0 -netdev user,id=net0",
+.objects = "e1000",
+},{
+.name = "e1000e",
+.args = "-M q35 -nodefaults "
+"-device e1000e,netdev=net0 -netdev user,id=net0",
+.objects = "e1000e",
+},{
+.name = "cirrus-vga",
+.args = "-machine q35 -nodefaults -device cirrus-vga",
+.objects = "cirrus*",
+},{
+.name = "bochs-display",
+.args = "-machine q35 -nodefaults -device bochs-display",
+.objects = "bochs*",
+},{
+.name = "intel-hda",
+.args = "-machine q35 -nodefaults -device intel-hda,id=hda0 "
+"-device hda-output,bus=hda0.0 -device hda-micro,bus=hda0.0 "
+"-device hda-duplex,bus=hda0.0",
+.objects = "intel-hda",
+},{
+.name = "ide-hd",
+.args = "-machine q35 -nodefaults "
+"-drive file=null-co://,if=none,format=raw,id=disk0 "
+"-device ide-hd,drive=disk0",
+.objects = "ahci*",
+},{
+.name = "floppy",
+.args = "-machine pc -nodefaults -device floppy,id=floppy0 "
+"-drive id=disk0,file=null-co://,file.read-zeroes=on,if=none "
+"-device floppy,drive=disk0,drive-type=288",
+.objects = "fd* floppy*",
+},{
+.name = "xhci",
+.args = "-machine q35 -nodefaults "
+"-drive file=null-co://,if=none,format=raw,id=disk0 "
+"-device qemu-xhci,id=xhci -device usb-tablet,bus=xhci.0 "
+"-device usb-bot -device usb-storage,drive=disk0 "
+"-chardev null,id=cd0 -chardev null,id=cd1 "
+"-device usb-braille,chardev=cd0 -device usb-ccid -device usb-ccid "
+"-device usb-kbd -device usb-mouse -device usb-serial,chardev=cd1 "
+"-device usb-tablet -device usb-wacom-tablet -device usb-audio",
+.objects = "*usb* *uhci* *xhci*",
+},{
+.name = "pc-i440fx",
+.args = "-machine pc",
+.objects = "*",
+},{
+.name = "pc-q35",
+.args = "-machine q35",
+.objects = "*",
+}
+};
+
+#endif
-- 
2.28.0

[PATCH v6 6/6] migration-test: Only hide error if !QTEST_LOG

[Peter Xu]

The errors are very useful when debugging qtest failures, especially when
QTEST_LOG=1 is set.  Let's allow override MigrateStart.hide_stderr when
QTEST_LOG=1 is specified, because that means the user wants to be verbose.

Not very nice to introduce the first QTEST_LOG env access in migration-test.c,
however it should be handy.  Without this patch, I was hacking error_report()
when debugging such errors.  Let's make things easier.

Signed-off-by: Peter Xu 
---
 tests/qtest/migration-test.c | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index f410ec5996..f2142fbd3c 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -464,6 +464,10 @@ static void migrate_postcopy_start(QTestState *from, 
QTestState *to)
 }
 
 typedef struct {
+/*
+ * QTEST_LOG=1 may override this.  When QTEST_LOG=1, we always dump errors
+ * unconditionally, because it means the user would like to be verbose.
+ */
 bool hide_stderr;
 bool use_shmem;
 /* only launch the target process */
@@ -557,7 +561,7 @@ static int test_migrate_start(QTestState **from, QTestState 
**to,
 
 g_free(bootpath);
 
-if (args->hide_stderr) {
+if (!getenv("QTEST_LOG") && args->hide_stderr) {
 ignore_stderr = "2>/dev/null";
 } else {
 ignore_stderr = "";
-- 
2.26.2

[PATCH v2 0/5] Additional NPCM7xx features, devices and tests

[Havard Skinnemoen via]

This is an update to the initial NPCM7xx patch series adding

  - Watchdog timer support. This makes the reboot command work.
  - Random Number Generator device.
  - USB Host Controllers.
  - GPIO Controllers.

The watchdog was implemented by my new teammate Hao Wu. Expect to see more
patches from him in the near future.

This series has also been pushed to the npcm7xx-5.2-update branch of my github
repository at

  https://github.com/hskinnemoen/qemu

Changes since v1:

  - Dropped the timer test since it's already applied (thanks Peter).
  - Watchdog reset signaling is now using GPIOs instead of a custom API
(thanks Peter for suggesting, and Hao for implementing it).
  - A few comment updates for the watchdog timer.
  - VMState added to GPIO device.
  - Missing VMstate terminator added to RNG device.
  - Include order in RNG test fixed.

Again, thanks a lot for reviewing!

Havard

Hao Wu (1):
  hw/timer: Adding watchdog for NPCM7XX Timer.

Havard Skinnemoen (4):
  Move npcm7xx_timer_reached_zero call out of npcm7xx_timer_pause
  hw/misc: Add npcm7xx random number generator
  hw/arm/npcm7xx: Add EHCI and OHCI controllers
  hw/gpio: Add GPIO model for Nuvoton NPCM7xx

 docs/system/arm/nuvoton.rst   |   6 +-
 hw/usb/hcd-ehci.h |   1 +
 include/hw/arm/npcm7xx.h  |   8 +
 include/hw/gpio/npcm7xx_gpio.h|  55 +++
 include/hw/misc/npcm7xx_clk.h |   2 +
 include/hw/misc/npcm7xx_rng.h |  34 ++
 include/hw/timer/npcm7xx_timer.h  |  48 ++-
 hw/arm/npcm7xx.c  | 126 ++-
 hw/gpio/npcm7xx_gpio.c| 424 ++
 hw/misc/npcm7xx_clk.c |  28 ++
 hw/misc/npcm7xx_rng.c | 180 +
 hw/timer/npcm7xx_timer.c  | 270 +++---
 hw/usb/hcd-ehci-sysbus.c  |  19 +
 tests/qtest/npcm7xx_gpio-test.c   | 385 
 tests/qtest/npcm7xx_rng-test.c| 278 ++
 tests/qtest/npcm7xx_watchdog_timer-test.c | 313 
 MAINTAINERS   |   1 +
 hw/gpio/meson.build   |   1 +
 hw/gpio/trace-events  |   7 +
 hw/misc/meson.build   |   1 +
 hw/misc/trace-events  |   4 +
 tests/qtest/meson.build   |   3 +
 22 files changed, 2133 insertions(+), 61 deletions(-)
 create mode 100644 include/hw/gpio/npcm7xx_gpio.h
 create mode 100644 include/hw/misc/npcm7xx_rng.h
 create mode 100644 hw/gpio/npcm7xx_gpio.c
 create mode 100644 hw/misc/npcm7xx_rng.c
 create mode 100644 tests/qtest/npcm7xx_gpio-test.c
 create mode 100644 tests/qtest/npcm7xx_rng-test.c
 create mode 100644 tests/qtest/npcm7xx_watchdog_timer-test.c

-- 
2.29.0.rc1.297.gfa9743e501-goog

[PATCH v6 12/16] fuzz: Add instructions for using generic-fuzz

[Alexander Bulekov]

Reviewed-by: Darren Kenny 
Signed-off-by: Alexander Bulekov 
---
 docs/devel/fuzzing.txt | 39 +++
 1 file changed, 39 insertions(+)

diff --git a/docs/devel/fuzzing.txt b/docs/devel/fuzzing.txt
index 96d71c94d7..03585c1a9b 100644
--- a/docs/devel/fuzzing.txt
+++ b/docs/devel/fuzzing.txt
@@ -125,6 +125,45 @@ provided by libfuzzer. Libfuzzer passes a byte array and 
length. Commonly the
 fuzzer loops over the byte-array interpreting it as a list of qtest commands,
 addresses, or values.
 
+== The Generic Fuzzer ==
+Writing a fuzz target can be a lot of effort (especially if a device driver has
+not be built-out within libqos). Many devices can be fuzzed to some degree,
+without any device-specific code, using the generic-fuzz target.
+
+The generic-fuzz target is capable of fuzzing devices over their PIO, MMIO,
+and DMA input-spaces. To apply the generic-fuzz to a device, we need to define
+two env-variables, at minimum:
+
+QEMU_FUZZ_ARGS= is the set of QEMU arguments used to configure a machine, with
+the device attached. For example, if we want to fuzz the virtio-net device
+attached to a pc-i440fx machine, we can specify:
+QEMU_FUZZ_ARGS="-M pc -nodefaults -netdev user,id=user0 \
+-device virtio-net,netdev=user0"
+
+QEMU_FUZZ_OBJECTS= is a set of space-delimited strings used to identify the
+MemoryRegions that will be fuzzed. These strings are compared against
+MemoryRegion names and MemoryRegion owner names, to decide whether each
+MemoryRegion should be fuzzed. These strings support globbing. For the
+virtio-net example, we could use QEMU_FUZZ_OBJECTS=
+ * 'virtio-net'
+ * 'virtio*'
+ * 'virtio* pcspk' (Fuzz the virtio devices and the PC speaker...)
+ * '*' (Fuzz the whole machine)
+
+The "info mtree" and "info qom-tree" monitor commands can be especially useful
+for identifying the MemoryRegion and Object names used for matching.
+
+As a generic rule-of-thumb, the more MemoryRegions/Devices we match, the 
greater
+the input-space, and the smaller the probability of finding crashing inputs for
+individual devices. As such, it is usually a good idea to limit the fuzzer to
+only a few MemoryRegions.
+
+To ensure that these env variables have been configured correctly, we can use:
+
+./qemu-fuzz-i386 --fuzz-target=generic-fuzz -runs=0
+
+The output should contain a complete list of matched MemoryRegions.
+
 = Implementation Details =
 
 == The Fuzzer's Lifecycle ==
-- 
2.28.0

[PATCH v6 4/6] migration: Sync requested pages after postcopy recovery

[Peter Xu]

We synchronize the requested pages right after a postcopy recovery happens.
This helps to synchronize the prioritized pages on source so that the faulted
threads can be served faster.

Reported-by: Xiaohui Li 
Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
---
 migration/savevm.c | 57 ++
 migration/trace-events |  1 +
 2 files changed, 58 insertions(+)

diff --git a/migration/savevm.c b/migration/savevm.c
index d2e141f7b1..33acbba1a4 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -2011,6 +2011,49 @@ static int 
loadvm_postcopy_handle_run(MigrationIncomingState *mis)
 return LOADVM_QUIT;
 }
 
+/* We must be with page_request_mutex held */
+static gboolean postcopy_sync_page_req(gpointer key, gpointer value,
+   gpointer data)
+{
+MigrationIncomingState *mis = data;
+void *host_addr = (void *) key;
+ram_addr_t rb_offset;
+RAMBlock *rb;
+int ret;
+
+rb = qemu_ram_block_from_host(host_addr, true, &rb_offset);
+if (!rb) {
+/*
+ * This should _never_ happen.  However be nice for a migrating VM to
+ * not crash/assert.  Post an error (note: intended to not use *_once
+ * because we do want to see all the illegal addresses; and this can
+ * never be triggered by the guest so we're safe) and move on next.
+ */
+error_report("%s: illegal host addr %p", __func__, host_addr);
+/* Try the next entry */
+return FALSE;
+}
+
+ret = migrate_send_rp_message_req_pages(mis, rb, rb_offset);
+if (ret) {
+/* Please refer to above comment. */
+error_report("%s: send rp message failed for addr %p",
+ __func__, host_addr);
+return FALSE;
+}
+
+trace_postcopy_page_req_sync(host_addr);
+
+return FALSE;
+}
+
+static void migrate_send_rp_req_pages_pending(MigrationIncomingState *mis)
+{
+WITH_QEMU_LOCK_GUARD(&mis->page_request_mutex) {
+g_tree_foreach(mis->page_requested, postcopy_sync_page_req, mis);
+}
+}
+
 static int loadvm_postcopy_handle_resume(MigrationIncomingState *mis)
 {
 if (mis->state != MIGRATION_STATUS_POSTCOPY_RECOVER) {
@@ -2033,6 +2076,20 @@ static int 
loadvm_postcopy_handle_resume(MigrationIncomingState *mis)
 /* Tell source that "we are ready" */
 migrate_send_rp_resume_ack(mis, MIGRATION_RESUME_ACK_VALUE);
 
+/*
+ * After a postcopy recovery, the source should have lost the postcopy
+ * queue, or potentially the requested pages could have been lost during
+ * the network down phase.  Let's re-sync with the source VM by re-sending
+ * all the pending pages that we eagerly need, so these threads won't get
+ * blocked too long due to the recovery.
+ *
+ * Without this procedure, the faulted destination VM threads (waiting for
+ * page requests right before the postcopy is interrupted) can keep hanging
+ * until the pages are sent by the source during the background copying of
+ * pages, or another thread faulted on the same address accidentally.
+ */
+migrate_send_rp_req_pages_pending(mis);
+
 return 0;
 }
 
diff --git a/migration/trace-events b/migration/trace-events
index e4d5eb94ca..0fbfd2da60 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -49,6 +49,7 @@ vmstate_save(const char *idstr, const char *vmsd_name) "%s, 
%s"
 vmstate_load(const char *idstr, const char *vmsd_name) "%s, %s"
 postcopy_pause_incoming(void) ""
 postcopy_pause_incoming_continued(void) ""
+postcopy_page_req_sync(void *host_addr) "sync page req %p"
 
 # vmstate.c
 vmstate_load_field_error(const char *field, int ret) "field \"%s\" load 
failed, ret = %d"
-- 
2.26.2

[PATCH v6 5/6] migration/postcopy: Release fd before going into 'postcopy-pause'

[Peter Xu]

Logically below race could trigger with the old code:

  test programmigration thread
  
   wait_until('postcopy-pause')
  postcopy_pause()
set_state('postcopy-pause')
   do_postcopy_recover()
 arm s->to_dst_file with new fd
release s->to_dst_file [1]

Here [1] could have released the just-installed recoverying channel.  Then the
migration could hang without really resuming.

Instead, it should be very safe to release the fd before setting the state into
'postcopy-pause', because there's no reason for any other thread to touch it
during 'postcopy-active'.

Dave reported a very rare postcopy recovery hang that the migration-test
program waited for the migration to complete in migrate_postcopy_complete().
We do suspect it's the same thing that we're gonna fix here.  Hard to tell.
However since we've noticed this, fix this irrelevant of the hang report.

Cc: Dr. David Alan Gilbert 
Cc: Juan Quintela 
Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
---
 migration/migration.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index e3a958b299..20e04855d9 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -3178,9 +3178,6 @@ static MigThrError postcopy_pause(MigrationState *s)
 while (true) {
 QEMUFile *file;
 
-migrate_set_state(&s->state, s->state,
-  MIGRATION_STATUS_POSTCOPY_PAUSED);
-
 /* Current channel is possibly broken. Release it. */
 assert(s->to_dst_file);
 qemu_mutex_lock(&s->qemu_file_lock);
@@ -3191,6 +3188,9 @@ static MigThrError postcopy_pause(MigrationState *s)
 qemu_file_shutdown(file);
 qemu_fclose(file);
 
+migrate_set_state(&s->state, s->state,
+  MIGRATION_STATUS_POSTCOPY_PAUSED);
+
 error_report("Detected IO failure for postcopy. "
  "Migration paused.");
 
-- 
2.26.2

[PATCH v6 3/6] migration: Maintain postcopy faulted addresses

[Peter Xu]

Maintain a list of faulted addresses on the destination host for which we're
waiting on.  This is implemented using a GTree rather than a real list to make
sure even there're plenty of vCPUs/threads that are faulting, the lookup will
still be fast with O(log(N)) (because we'll do that after placing each page).
It should bring a slight overhead, but ideally that shouldn't be a big problem
simply because in most cases the requested page list will be short.

Actually we did similar things for postcopy blocktime measurements.  This patch
didn't use that simply because:

  (1) blocktime measurement is towards vcpu threads only, but here we need to
  record all faulted addresses, including main thread and external
  thread (like, DPDK via vhost-user).

  (2) blocktime measurement will require UFFD_FEATURE_THREAD_ID, but here we
  don't want to add that extra dependency on the kernel version since not
  necessary.  E.g., we don't need to know which thread faulted on which
  page, we also don't care about multiple threads faulting on the same
  page.  But we only care about what addresses are faulted so waiting for a
  page copying from src.

  (3) blocktime measurement is not enabled by default.  However we need this by
  default especially for postcopy recover.

Another thing to mention is that this patch introduced a new mutex to serialize
the receivedmap and the page_requested tree, however that serialization does
not cover other procedures like UFFDIO_COPY.

Signed-off-by: Peter Xu 
---
 migration/migration.c| 41 +++-
 migration/migration.h| 19 ++-
 migration/postcopy-ram.c | 17 ++---
 migration/trace-events   |  2 ++
 4 files changed, 74 insertions(+), 5 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 255e69c8aa..e3a958b299 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -143,6 +143,13 @@ static int migration_maybe_pause(MigrationState *s,
  int new_state);
 static void migrate_fd_cancel(MigrationState *s);
 
+static gint page_request_addr_cmp(gconstpointer ap, gconstpointer bp)
+{
+uintptr_t a = (uintptr_t) ap, b = (uintptr_t) bp;
+
+return (a > b) - (a < b);
+}
+
 void migration_object_init(void)
 {
 MachineState *ms = MACHINE(qdev_get_machine());
@@ -165,6 +172,8 @@ void migration_object_init(void)
 qemu_event_init(¤t_incoming->main_thread_load_event, false);
 qemu_sem_init(¤t_incoming->postcopy_pause_sem_dst, 0);
 qemu_sem_init(¤t_incoming->postcopy_pause_sem_fault, 0);
+qemu_mutex_init(¤t_incoming->page_request_mutex);
+current_incoming->page_requested = g_tree_new(page_request_addr_cmp);
 
 if (!migration_object_check(current_migration, &err)) {
 error_report_err(err);
@@ -240,6 +249,11 @@ void migration_incoming_state_destroy(void)
 
 qemu_event_reset(&mis->main_thread_load_event);
 
+if (mis->page_requested) {
+g_tree_destroy(mis->page_requested);
+mis->page_requested = NULL;
+}
+
 if (mis->socket_address_list) {
 qapi_free_SocketAddressList(mis->socket_address_list);
 mis->socket_address_list = NULL;
@@ -354,8 +368,33 @@ int 
migrate_send_rp_message_req_pages(MigrationIncomingState *mis,
 }
 
 int migrate_send_rp_req_pages(MigrationIncomingState *mis,
-  RAMBlock *rb, ram_addr_t start)
+  RAMBlock *rb, ram_addr_t start, uint64_t haddr)
 {
+void *aligned = (void *)(uintptr_t)(haddr & (-qemu_ram_pagesize(rb)));
+bool received;
+
+WITH_QEMU_LOCK_GUARD(&mis->page_request_mutex) {
+received = ramblock_recv_bitmap_test_byte_offset(rb, start);
+if (!received && !g_tree_lookup(mis->page_requested, aligned)) {
+/*
+ * The page has not been received, and it's not yet in the page
+ * request list.  Queue it.  Set the value of element to 1, so that
+ * things like g_tree_lookup() will return TRUE (1) when found.
+ */
+g_tree_insert(mis->page_requested, aligned, (gpointer)1);
+mis->page_requested_count++;
+trace_postcopy_page_req_add(aligned, mis->page_requested_count);
+}
+}
+
+/*
+ * If the page is there, skip sending the message.  We don't even need the
+ * lock because as long as the page arrived, it'll be there forever.
+ */
+if (received) {
+return 0;
+}
+
 return migrate_send_rp_message_req_pages(mis, rb, start);
 }
 
diff --git a/migration/migration.h b/migration/migration.h
index e853ccf8b1..8d2d1ce839 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -104,6 +104,23 @@ struct MigrationIncomingState {
 
 /* List of listening socket addresses  */
 SocketAddressList *socket_address_list;
+
+/* A tree of pages that we requested to the source VM */
+GTree *page_requested;
+/* F

[PATCH v6 08/16] fuzz: add a DISABLE_PCI op to generic-fuzzer

[Alexander Bulekov]

This new operation is used in the next commit, which concatenates two
fuzzer-generated inputs. With this operation, we can prevent the second
input from clobbering the PCI configuration performed by the first.

Signed-off-by: Alexander Bulekov 
Reviewed-by: Darren Kenny 
---
 tests/qtest/fuzz/generic_fuzz.c | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/tests/qtest/fuzz/generic_fuzz.c b/tests/qtest/fuzz/generic_fuzz.c
index e356873ae0..fb8bf4a112 100644
--- a/tests/qtest/fuzz/generic_fuzz.c
+++ b/tests/qtest/fuzz/generic_fuzz.c
@@ -39,6 +39,7 @@ enum cmds {
 OP_WRITE,
 OP_PCI_READ,
 OP_PCI_WRITE,
+OP_DISABLE_PCI,
 OP_ADD_DMA_PATTERN,
 OP_CLEAR_DMA_PATTERNS,
 OP_CLOCK_STEP,
@@ -116,6 +117,7 @@ static GArray *dma_regions;
 
 static GArray *dma_patterns;
 static int dma_pattern_index;
+static bool pci_disabled;
 
 /*
  * Allocate a block of memory and populate it with a pattern.
@@ -479,7 +481,7 @@ static void op_pci_read(QTestState *s, const unsigned char 
* data, size_t len)
 uint8_t base;
 uint8_t offset;
 } a;
-if (len < sizeof(a) || fuzzable_pci_devices->len == 0) {
+if (len < sizeof(a) || fuzzable_pci_devices->len == 0 || pci_disabled) {
 return;
 }
 memcpy(&a, data, sizeof(a));
@@ -509,7 +511,7 @@ static void op_pci_write(QTestState *s, const unsigned char 
* data, size_t len)
 uint8_t offset;
 uint32_t value;
 } a;
-if (len < sizeof(a) || fuzzable_pci_devices->len == 0) {
+if (len < sizeof(a) || fuzzable_pci_devices->len == 0 || pci_disabled) {
 return;
 }
 memcpy(&a, data, sizeof(a));
@@ -564,6 +566,11 @@ static void op_clock_step(QTestState *s, const unsigned 
char *data, size_t len)
 qtest_clock_step_next(s);
 }
 
+static void op_disable_pci(QTestState *s, const unsigned char *data, size_t 
len)
+{
+pci_disabled = true;
+}
+
 static void handle_timeout(int sig)
 {
 if (qtest_log_enabled) {
@@ -619,6 +626,7 @@ static void generic_fuzz(QTestState *s, const unsigned char 
*Data, size_t Size)
 [OP_WRITE]  = op_write,
 [OP_PCI_READ]   = op_pci_read,
 [OP_PCI_WRITE]  = op_pci_write,
+[OP_DISABLE_PCI]= op_disable_pci,
 [OP_ADD_DMA_PATTERN]= op_add_dma_pattern,
 [OP_CLEAR_DMA_PATTERNS] = op_clear_dma_patterns,
 [OP_CLOCK_STEP] = op_clock_step,
@@ -651,6 +659,7 @@ static void generic_fuzz(QTestState *s, const unsigned char 
*Data, size_t Size)
 }
 
 op_clear_dma_patterns(s, NULL, 0);
+pci_disabled = false;
 
 while (cmd && Size) {
 /* Get the length until the next command or end of input */
-- 
2.28.0

[PATCH v6 1/6] migration: Pass incoming state into qemu_ufd_copy_ioctl()

[Peter Xu]

It'll be used in follow up patches to access more fields out of it.  Meanwhile
fetch the userfaultfd inside the function.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
---
 migration/postcopy-ram.c | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
index 0a2f88a87d..722034dc01 100644
--- a/migration/postcopy-ram.c
+++ b/migration/postcopy-ram.c
@@ -1128,10 +1128,12 @@ int postcopy_ram_incoming_setup(MigrationIncomingState 
*mis)
 return 0;
 }
 
-static int qemu_ufd_copy_ioctl(int userfault_fd, void *host_addr,
+static int qemu_ufd_copy_ioctl(MigrationIncomingState *mis, void *host_addr,
void *from_addr, uint64_t pagesize, RAMBlock 
*rb)
 {
+int userfault_fd = mis->userfault_fd;
 int ret;
+
 if (from_addr) {
 struct uffdio_copy copy_struct;
 copy_struct.dst = (uint64_t)(uintptr_t)host_addr;
@@ -1185,7 +1187,7 @@ int postcopy_place_page(MigrationIncomingState *mis, void 
*host, void *from,
  * which would be slightly cheaper, but we'd have to be careful
  * of the order of updating our page state.
  */
-if (qemu_ufd_copy_ioctl(mis->userfault_fd, host, from, pagesize, rb)) {
+if (qemu_ufd_copy_ioctl(mis, host, from, pagesize, rb)) {
 int e = errno;
 error_report("%s: %s copy host: %p from: %p (size: %zd)",
  __func__, strerror(e), host, from, pagesize);
@@ -1212,7 +1214,7 @@ int postcopy_place_page_zero(MigrationIncomingState *mis, 
void *host,
  * but it's not available for everything (e.g. hugetlbpages)
  */
 if (qemu_ram_is_uf_zeroable(rb)) {
-if (qemu_ufd_copy_ioctl(mis->userfault_fd, host, NULL, pagesize, rb)) {
+if (qemu_ufd_copy_ioctl(mis, host, NULL, pagesize, rb)) {
 int e = errno;
 error_report("%s: %s zero host: %p",
  __func__, strerror(e), host);
-- 
2.26.2

[PATCH v6 06/16] fuzz: Add fuzzer callbacks to DMA-read functions

[Alexander Bulekov]

We should be careful to not call any functions besides fuzz_dma_read_cb.
Without --enable-fuzzing, fuzz_dma_read_cb is an empty inlined function.

Signed-off-by: Alexander Bulekov 
Reviewed-by: Darren Kenny 
---
 include/exec/memory.h | 1 +
 include/exec/memory_ldst_cached.h.inc | 3 +++
 memory_ldst.c.inc | 4 
 softmmu/memory.c  | 1 +
 softmmu/physmem.c | 2 ++
 5 files changed, 11 insertions(+)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index 4aaf578ce6..aff6ef7605 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -2462,6 +2462,7 @@ address_space_read_cached(MemoryRegionCache *cache, 
hwaddr addr,
   void *buf, hwaddr len)
 {
 assert(addr < cache->len && len <= cache->len - addr);
+fuzz_dma_read_cb(cache->xlat + addr, len, cache->mrs.mr, false);
 if (likely(cache->ptr)) {
 memcpy(buf, cache->ptr + addr, len);
 return MEMTX_OK;
diff --git a/include/exec/memory_ldst_cached.h.inc 
b/include/exec/memory_ldst_cached.h.inc
index fd4bbb40e7..aff574039f 100644
--- a/include/exec/memory_ldst_cached.h.inc
+++ b/include/exec/memory_ldst_cached.h.inc
@@ -28,6 +28,7 @@ static inline uint32_t 
ADDRESS_SPACE_LD_CACHED(l)(MemoryRegionCache *cache,
 hwaddr addr, MemTxAttrs attrs, MemTxResult *result)
 {
 assert(addr < cache->len && 4 <= cache->len - addr);
+fuzz_dma_read_cb(cache->xlat + addr, 4, cache->mrs.mr, false);
 if (likely(cache->ptr)) {
 return LD_P(l)(cache->ptr + addr);
 } else {
@@ -39,6 +40,7 @@ static inline uint64_t 
ADDRESS_SPACE_LD_CACHED(q)(MemoryRegionCache *cache,
 hwaddr addr, MemTxAttrs attrs, MemTxResult *result)
 {
 assert(addr < cache->len && 8 <= cache->len - addr);
+fuzz_dma_read_cb(cache->xlat + addr, 8, cache->mrs.mr, false);
 if (likely(cache->ptr)) {
 return LD_P(q)(cache->ptr + addr);
 } else {
@@ -50,6 +52,7 @@ static inline uint32_t 
ADDRESS_SPACE_LD_CACHED(uw)(MemoryRegionCache *cache,
 hwaddr addr, MemTxAttrs attrs, MemTxResult *result)
 {
 assert(addr < cache->len && 2 <= cache->len - addr);
+fuzz_dma_read_cb(cache->xlat + addr, 2, cache->mrs.mr, false);
 if (likely(cache->ptr)) {
 return LD_P(uw)(cache->ptr + addr);
 } else {
diff --git a/memory_ldst.c.inc b/memory_ldst.c.inc
index c54aee4a95..8d45d2eeff 100644
--- a/memory_ldst.c.inc
+++ b/memory_ldst.c.inc
@@ -42,6 +42,7 @@ static inline uint32_t glue(address_space_ldl_internal, 
SUFFIX)(ARG1_DECL,
 MO_32 | devend_memop(endian), attrs);
 } else {
 /* RAM case */
+fuzz_dma_read_cb(addr, 4, mr, false);
 ptr = qemu_map_ram_ptr(mr->ram_block, addr1);
 switch (endian) {
 case DEVICE_LITTLE_ENDIAN:
@@ -110,6 +111,7 @@ static inline uint64_t glue(address_space_ldq_internal, 
SUFFIX)(ARG1_DECL,
 MO_64 | devend_memop(endian), attrs);
 } else {
 /* RAM case */
+fuzz_dma_read_cb(addr, 8, mr, false);
 ptr = qemu_map_ram_ptr(mr->ram_block, addr1);
 switch (endian) {
 case DEVICE_LITTLE_ENDIAN:
@@ -175,6 +177,7 @@ uint32_t glue(address_space_ldub, SUFFIX)(ARG1_DECL,
 r = memory_region_dispatch_read(mr, addr1, &val, MO_8, attrs);
 } else {
 /* RAM case */
+fuzz_dma_read_cb(addr, 1, mr, false);
 ptr = qemu_map_ram_ptr(mr->ram_block, addr1);
 val = ldub_p(ptr);
 r = MEMTX_OK;
@@ -212,6 +215,7 @@ static inline uint32_t glue(address_space_lduw_internal, 
SUFFIX)(ARG1_DECL,
 MO_16 | devend_memop(endian), attrs);
 } else {
 /* RAM case */
+fuzz_dma_read_cb(addr, 2, mr, false);
 ptr = qemu_map_ram_ptr(mr->ram_block, addr1);
 switch (endian) {
 case DEVICE_LITTLE_ENDIAN:
diff --git a/softmmu/memory.c b/softmmu/memory.c
index d3cdb46459..d7fdca2603 100644
--- a/softmmu/memory.c
+++ b/softmmu/memory.c
@@ -1429,6 +1429,7 @@ MemTxResult memory_region_dispatch_read(MemoryRegion *mr,
 unsigned size = memop_size(op);
 MemTxResult r;
 
+fuzz_dma_read_cb(addr, size, mr, false);
 if (!memory_region_access_valid(mr, addr, size, false, attrs)) {
 *pval = unassigned_mem_read(mr, addr, size);
 return MEMTX_DECODE_ERROR;
diff --git a/softmmu/physmem.c b/softmmu/physmem.c
index e319fb2a1e..a9adedb9f8 100644
--- a/softmmu/physmem.c
+++ b/softmmu/physmem.c
@@ -2832,6 +2832,7 @@ MemTxResult flatview_read_continue(FlatView *fv, hwaddr 
addr,
 stn_he_p(buf, l, val);
 } else {
 /* RAM case */
+fuzz_dma_read_cb(addr, len, mr, false);
 ram_ptr = qemu_ram_ptr_length(mr->ram_block, addr1, &l, false);
 memcpy(buf, ram_ptr, l);
 }
@@ -3192,6 +3193,7 @@ void *address_space_map(AddressSpace *as,
 memory_region_ref(mr);
 *plen = f

[PATCH v6 15/16] fuzz: register predefined generic-fuzz configs

[Alexander Bulekov]

We call get_generic_fuzz_configs, which fills an array with
predefined {name, args, objects} triples. For each of these, we add a
new FuzzTarget, that uses a small wrapper to set
QEMU_FUZZ_{ARGS,OBJECTS} to the corresponding predefined values.

Signed-off-by: Alexander Bulekov 
---
 tests/qtest/fuzz/generic_fuzz.c | 32 
 1 file changed, 32 insertions(+)

diff --git a/tests/qtest/fuzz/generic_fuzz.c b/tests/qtest/fuzz/generic_fuzz.c
index f739937827..bff98fe3c8 100644
--- a/tests/qtest/fuzz/generic_fuzz.c
+++ b/tests/qtest/fuzz/generic_fuzz.c
@@ -26,6 +26,7 @@
 #include "hw/qdev-core.h"
 #include "hw/pci/pci.h"
 #include "hw/boards.h"
+#include "generic_fuzz_configs.h"
 
 /*
  * SEPARATOR is used to separate "operations" in the fuzz input
@@ -901,6 +902,17 @@ static GString *generic_fuzz_cmdline(FuzzTarget *t)
 return cmd_line;
 }
 
+static GString *generic_fuzz_predefined_config_cmdline(FuzzTarget *t)
+{
+const generic_fuzz_config *config;
+g_assert(t->opaque);
+
+config = t->opaque;
+setenv("QEMU_FUZZ_ARGS", config->args, 1);
+setenv("QEMU_FUZZ_OBJECTS", config->objects, 1);
+return generic_fuzz_cmdline(t);
+}
+
 static void register_generic_fuzz_targets(void)
 {
 fuzz_add_target(&(FuzzTarget){
@@ -911,6 +923,26 @@ static void register_generic_fuzz_targets(void)
 .fuzz = generic_fuzz,
 .crossover = generic_fuzz_crossover
 });
+
+GString *name;
+const generic_fuzz_config *config;
+
+for (int i = 0;
+ i < sizeof(predefined_configs) / sizeof(generic_fuzz_config);
+ i++) {
+config = predefined_configs + i;
+name = g_string_new("generic-fuzz");
+g_string_append_printf(name, "-%s", config->name);
+fuzz_add_target(&(FuzzTarget){
+.name = name->str,
+.description = "Predefined generic-fuzz config.",
+.get_init_cmdline = generic_fuzz_predefined_config_cmdline,
+.pre_fuzz = generic_pre_fuzz,
+.fuzz = generic_fuzz,
+.crossover = generic_fuzz_crossover,
+.opaque = (void *)config
+});
+}
 }
 
 fuzz_target_init(register_generic_fuzz_targets);
-- 
2.28.0

[PATCH v6 10/16] scripts/oss-fuzz: Add script to reorder a generic-fuzzer trace

[Alexander Bulekov]

The generic-fuzzer uses hooks to fulfill DMA requests just-in-time.
This means that if we try to use QTEST_LOG=1 to build a reproducer, the
DMA writes will be logged _after_ the in/out/read/write that triggered
the DMA read. To work work around this, the generic-fuzzer annotates
these just-in time DMA fulfilments with a tag that we can use to
discern them. This script simply iterates over a raw qtest
trace (including log messages, errors, timestamps etc), filters it and
re-orders it so that DMA fulfillments are placed directly _before_ the
qtest command that will cause the DMA access.

Signed-off-by: Alexander Bulekov 
Reviewed-by: Darren Kenny 
---
 .../oss-fuzz/reorder_fuzzer_qtest_trace.py| 103 ++
 1 file changed, 103 insertions(+)
 create mode 100755 scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py

diff --git a/scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py 
b/scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py
new file mode 100755
index 00..890e1def85
--- /dev/null
+++ b/scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py
@@ -0,0 +1,103 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+"""
+Use this to convert qtest log info from a generic fuzzer input into a qtest
+trace that you can feed into a standard qemu-system process. Example usage:
+
+QEMU_FUZZ_ARGS="-machine q35,accel=qtest" QEMU_FUZZ_OBJECTS="*" \
+./i386-softmmu/qemu-fuzz-i386 --fuzz-target=generic-pci-fuzz
+# .. Finds some crash
+QTEST_LOG=1 FUZZ_SERIALIZE_QTEST=1 \
+QEMU_FUZZ_ARGS="-machine q35,accel=qtest" QEMU_FUZZ_OBJECTS="*" \
+./i386-softmmu/qemu-fuzz-i386 --fuzz-target=generic-pci-fuzz
+/path/to/crash 2> qtest_log_output
+scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py qtest_log_output > qtest_trace
+./i386-softmmu/qemu-fuzz-i386 -machine q35,accel=qtest \
+-qtest stdin < qtest_trace
+
+### Details ###
+
+Some fuzzer make use of hooks that allow us to populate some memory range, just
+before a DMA read from that range. This means that the fuzzer can produce
+activity that looks like:
+[start] read from mmio addr
+[end]   read from mmio addr
+[start] write to pio addr
+[start] fill a DMA buffer just in time
+[end]   fill a DMA buffer just in time
+[start] fill a DMA buffer just in time
+[end]   fill a DMA buffer just in time
+[end]   write to pio addr
+[start] read from mmio addr
+[end]   read from mmio addr
+
+We annotate these "nested" DMA writes, so with QTEST_LOG=1 the QTest trace
+might look something like:
+[R +0.028431] readw 0x1
+[R +0.028434] outl 0xc000 0xbeef  # Triggers a DMA read from 0xbeef and 0xbf00
+[DMA][R +0.034639] write 0xbeef 0x2 0x
+[DMA][R +0.034639] write 0xbf00 0x2 0x
+[R +0.028431] readw 0xfc000
+
+This script would reorder the above trace so it becomes:
+readw 0x1
+write 0xbeef 0x2 0x
+write 0xbf00 0x2 0x
+outl 0xc000 0xbeef
+readw 0xfc000
+
+I.e. by the time, 0xc000 tries to read from DMA, those DMA buffers have already
+been set up, removing the need for the DMA hooks. We can simply provide this
+reordered trace via -qtest stdio to reproduce the input
+
+Note: this won't work for traces where the device tries to read from the same
+DMA region twice in between MMIO/PIO commands. E.g:
+[R +0.028434] outl 0xc000 0xbeef
+[DMA][R +0.034639] write 0xbeef 0x2 0x
+[DMA][R +0.034639] write 0xbeef 0x2 0x
+
+The fuzzer will annotate suspected double-fetches with [DOUBLE-FETCH]. This
+script looks for these tags and warns the users that the resulting trace might
+not reproduce the bug.
+"""
+
+import sys
+
+__author__ = "Alexander Bulekov "
+__copyright__  = "Copyright (C) 2020, Red Hat, Inc."
+__license__= "GPL version 2 or (at your option) any later version"
+
+__maintainer__ = "Alexander Bulekov"
+__email__  = "alx...@bu.edu"
+
+
+def usage():
+sys.exit("Usage: {} /path/to/qtest_log_output".format((sys.argv[0])))
+
+
+def main(filename):
+with open(filename, "r") as f:
+trace = f.readlines()
+
+# Leave only lines that look like logged qtest commands
+trace[:] = [x.strip() for x in trace if "[R +" in x
+or "[S +" in x and "CLOSED" not in x]
+
+for i in range(len(trace)):
+if i+1 < len(trace):
+if "[DMA]" in trace[i+1]:
+if "[DOUBLE-FETCH]" in trace[i+1]:
+sys.stderr.write("Warning: Likely double fetch on line"
+ "{}.\n There will likely be problems "
+ "reproducing behavior with the "
+ "resulting qtest trace\n\n".format(i+1))
+trace[i], trace[i+1] = trace[i+1], trace[i]
+for line in trace:
+print(line.split("]")[-1].strip())
+
+
+if __name__ == '__main__':
+if len(sys.argv) == 1:
+usage()
+main(sys.argv[1])
-- 
2.28.0

[PATCH v6 09/16] fuzz: add a crossover function to generic-fuzzer

[Alexander Bulekov]

Reviewed-by: Darren Kenny 
Signed-off-by: Alexander Bulekov 
---
 tests/qtest/fuzz/generic_fuzz.c | 86 +
 1 file changed, 86 insertions(+)

diff --git a/tests/qtest/fuzz/generic_fuzz.c b/tests/qtest/fuzz/generic_fuzz.c
index fb8bf4a112..f739937827 100644
--- a/tests/qtest/fuzz/generic_fuzz.c
+++ b/tests/qtest/fuzz/generic_fuzz.c
@@ -804,6 +804,91 @@ static void generic_pre_fuzz(QTestState *s)
 counter_shm_init();
 }
 
+/*
+ * When libfuzzer gives us two inputs to combine, return a new input with the
+ * following structure:
+ *
+ * Input 1 (data1)
+ * SEPARATOR
+ * Clear out the DMA Patterns
+ * SEPARATOR
+ * Disable the pci_read/write instructions
+ * SEPARATOR
+ * Input 2 (data2)
+ *
+ * The idea is to collate the core behaviors of the two inputs.
+ * For example:
+ * Input 1: maps a device's BARs, sets up three DMA patterns, and triggers
+ *  device functionality A
+ * Input 2: maps a device's BARs, sets up one DMA pattern, and triggers device
+ *  functionality B
+ *
+ * This function attempts to produce an input that:
+ * Ouptut: maps a device's BARs, set up three DMA patterns, triggers
+ *  functionality A device, replaces the DMA patterns with a single
+ *  patten, and triggers device functionality B.
+ */
+static size_t generic_fuzz_crossover(const uint8_t *data1, size_t size1, const
+ uint8_t *data2, size_t size2, uint8_t 
*out,
+ size_t max_out_size, unsigned int seed)
+{
+size_t copy_len = 0, size = 0;
+
+/* Check that we have enough space for data1 and at least part of data2 */
+if (max_out_size <= size1 + strlen(SEPARATOR) * 3 + 2) {
+return 0;
+}
+
+/* Copy_Len in the first input */
+copy_len = size1;
+memcpy(out + size, data1, copy_len);
+size += copy_len;
+max_out_size -= copy_len;
+
+/* Append a separator */
+copy_len = strlen(SEPARATOR);
+memcpy(out + size, SEPARATOR, copy_len);
+size += copy_len;
+max_out_size -= copy_len;
+
+/* Clear out the DMA Patterns */
+copy_len = 1;
+if (copy_len) {
+out[size] = OP_CLEAR_DMA_PATTERNS;
+}
+size += copy_len;
+max_out_size -= copy_len;
+
+/* Append a separator */
+copy_len = strlen(SEPARATOR);
+memcpy(out + size, SEPARATOR, copy_len);
+size += copy_len;
+max_out_size -= copy_len;
+
+/* Disable PCI ops. Assume data1 took care of setting up PCI */
+copy_len = 1;
+if (copy_len) {
+out[size] = OP_DISABLE_PCI;
+}
+size += copy_len;
+max_out_size -= copy_len;
+
+/* Append a separator */
+copy_len = strlen(SEPARATOR);
+memcpy(out + size, SEPARATOR, copy_len);
+size += copy_len;
+max_out_size -= copy_len;
+
+/* Copy_Len over the second input */
+copy_len = MIN(size2, max_out_size);
+memcpy(out + size, data2, copy_len);
+size += copy_len;
+max_out_size -= copy_len;
+
+return  size;
+}
+
+
 static GString *generic_fuzz_cmdline(FuzzTarget *t)
 {
 GString *cmd_line = g_string_new(TARGET_NAME);
@@ -824,6 +909,7 @@ static void register_generic_fuzz_targets(void)
 .get_init_cmdline = generic_fuzz_cmdline,
 .pre_fuzz = generic_pre_fuzz,
 .fuzz = generic_fuzz,
+.crossover = generic_fuzz_crossover
 });
 }
 
-- 
2.28.0

[PATCH v6 02/16] fuzz: Add generic virtual-device fuzzer

[Alexander Bulekov]

This is a generic fuzzer designed to fuzz a virtual device's
MemoryRegions, as long as they exist within the Memory or Port IO (if it
exists) AddressSpaces. The fuzzer's input is interpreted into a sequence
of qtest commands (outb, readw, etc). The interpreted commands are
separated by a magic seaparator, which should be easy for the fuzzer to
guess. Without ASan, the separator can be specified as a "dictionary
value" using the -dict argument (see libFuzzer documentation).

Signed-off-by: Alexander Bulekov 
---
 tests/qtest/fuzz/generic_fuzz.c | 512 
 tests/qtest/fuzz/meson.build|   1 +
 2 files changed, 513 insertions(+)
 create mode 100644 tests/qtest/fuzz/generic_fuzz.c

diff --git a/tests/qtest/fuzz/generic_fuzz.c b/tests/qtest/fuzz/generic_fuzz.c
new file mode 100644
index 00..f69e9583ce
--- /dev/null
+++ b/tests/qtest/fuzz/generic_fuzz.c
@@ -0,0 +1,512 @@
+/*
+ * Generic Virtual-Device Fuzzing Target
+ *
+ * Copyright Red Hat Inc., 2020
+ *
+ * Authors:
+ *  Alexander Bulekov   
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+
+#include 
+
+#include "hw/core/cpu.h"
+#include "tests/qtest/libqos/libqtest.h"
+#include "fuzz.h"
+#include "fork_fuzz.h"
+#include "exec/address-spaces.h"
+#include "string.h"
+#include "exec/memory.h"
+#include "exec/ramblock.h"
+#include "exec/address-spaces.h"
+#include "hw/qdev-core.h"
+
+/*
+ * SEPARATOR is used to separate "operations" in the fuzz input
+ */
+#define SEPARATOR "FUZZ"
+
+enum cmds {
+OP_IN,
+OP_OUT,
+OP_READ,
+OP_WRITE,
+OP_CLOCK_STEP,
+};
+
+#define DEFAULT_TIMEOUT_US 10
+#define USEC_IN_SEC 10
+
+typedef struct {
+ram_addr_t addr;
+ram_addr_t size; /* The number of bytes until the end of the I/O region */
+} address_range;
+
+static useconds_t timeout = DEFAULT_TIMEOUT_US;
+
+static bool qtest_log_enabled;
+
+/*
+ * List of memory regions that are children of QOM objects specified by the
+ * user for fuzzing.
+ */
+static GHashTable *fuzzable_memoryregions;
+
+struct get_io_cb_info {
+int index;
+int found;
+address_range result;
+};
+
+static int get_io_address_cb(Int128 start, Int128 size,
+  const MemoryRegion *mr, void *opaque) {
+struct get_io_cb_info *info = opaque;
+if (g_hash_table_lookup(fuzzable_memoryregions, mr)) {
+if (info->index == 0) {
+info->result.addr = (ram_addr_t)start;
+info->result.size = (ram_addr_t)size;
+info->found = 1;
+return 1;
+}
+info->index--;
+}
+return 0;
+}
+
+/*
+ * Here we want to convert a fuzzer-provided [io-region-index, offset] to
+ * a physical address. To do this, we iterate over all of the matched
+ * MemoryRegions. Check whether each region exists within the particular io
+ * space. Return the absolute address of the offset within the index'th region
+ * that is a subregion of the io_space and the distance until the end of the
+ * memory region.
+ */
+static bool get_io_address(address_range *result, AddressSpace *as,
+uint8_t index,
+uint32_t offset) {
+FlatView *view;
+view = as->current_map;
+g_assert(view);
+struct get_io_cb_info cb_info = {};
+
+cb_info.index = index;
+
+/*
+ * Loop around the FlatView until we match "index" number of
+ * fuzzable_memoryregions, or until we know that there are no matching
+ * memory_regions.
+ */
+do {
+flatview_for_each_range(view, get_io_address_cb , &cb_info);
+} while (cb_info.index != index && !cb_info.found);
+
+*result = cb_info.result;
+return cb_info.found;
+}
+static bool get_pio_address(address_range *result,
+uint8_t index, uint16_t offset)
+{
+/*
+ * PIO BARs can be set past the maximum port address (0x). Thus, result
+ * can contain an addr that extends past the PIO space. When we pass this
+ * address to qtest_in/qtest_out, it is cast to a uint16_t, so we might end
+ * up fuzzing a completely different MemoryRegion/Device. Therefore, check
+ * that the address here is within the PIO space limits.
+ */
+bool found = get_io_address(result, &address_space_io, index, offset);
+return result->addr <= 0x ? found : false;
+}
+static bool get_mmio_address(address_range *result,
+ uint8_t index, uint32_t offset)
+{
+return get_io_address(result, &address_space_memory, index, offset);
+}
+
+static void op_in(QTestState *s, const unsigned char * data, size_t len)
+{
+enum Sizes {Byte, Word, Long, end_sizes};
+struct {
+uint8_t size;
+uint8_t base;
+uint16_t offset;
+} a;
+address_range abs;
+
+if (len < sizeof(a)) {
+return;
+}
+memcpy(&a, data, sizeof(a));
+i

[PATCH v6 13/16] fuzz: add an "opaque" to the FuzzTarget struct

[Alexander Bulekov]

It can be useful to register FuzzTargets that have nearly-identical
initialization handlers (e.g. for using the same fuzzing code, with
different configuration options). Add an opaque pointer to the
FuzzTarget struct, so that FuzzTargets can hold some data, useful for
storing target-specific configuration options, that can be read by the
get_init_cmdline function.

Signed-off-by: Alexander Bulekov 
---
 tests/qtest/fuzz/fuzz.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/qtest/fuzz/fuzz.h b/tests/qtest/fuzz/fuzz.h
index ed9ce17154..08e9560a79 100644
--- a/tests/qtest/fuzz/fuzz.h
+++ b/tests/qtest/fuzz/fuzz.h
@@ -100,6 +100,7 @@ typedef struct FuzzTarget {
uint8_t *out, size_t max_out_size,
unsigned int seed);
 
+void *opaque;
 } FuzzTarget;
 
 void flush_events(QTestState *);
-- 
2.28.0

[PATCH v6 05/16] fuzz: Declare DMA Read callback function

[Alexander Bulekov]

This patch declares the fuzz_dma_read_cb function and uses the
preprocessor and linker(weak symbols) to handle these cases:

When we build softmmu/all with --enable-fuzzing, there should be no
strong symbol defined for fuzz_dma_read_cb, and we link against a weak
stub function.

When we build softmmu/fuzz with --enable-fuzzing, we link against the
strong symbol in generic_fuzz.c

When we build softmmu/all without --enable-fuzzing, fuzz_dma_read_cb is
an empty, inlined function. As long as we don't call any other functions
when building the arguments, there should be no overhead.

Signed-off-by: Alexander Bulekov 
Reviewed-by: Darren Kenny 
---
 include/exec/memory.h |  8 
 softmmu/memory.c  | 13 +
 2 files changed, 21 insertions(+)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index 93d27bff26..4aaf578ce6 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -47,6 +47,14 @@ void fuzz_dma_read_cb(size_t addr,
   size_t len,
   MemoryRegion *mr,
   bool is_write);
+#else
+static inline void fuzz_dma_read_cb(size_t addr,
+size_t len,
+MemoryRegion *mr,
+bool is_write)
+{
+/* Do Nothing */
+}
 #endif
 
 extern bool global_dirty_log;
diff --git a/softmmu/memory.c b/softmmu/memory.c
index c46b0c6d65..d3cdb46459 100644
--- a/softmmu/memory.c
+++ b/softmmu/memory.c
@@ -3242,6 +3242,19 @@ void memory_region_init_rom_device(MemoryRegion *mr,
 vmstate_register_ram(mr, owner_dev);
 }
 
+/*
+ * Support softmmu builds with CONFIG_FUZZ using a weak symbol and a stub for
+ * the fuzz_dma_read_cb callback
+ */
+#ifdef CONFIG_FUZZ
+void __attribute__((weak)) fuzz_dma_read_cb(size_t addr,
+  size_t len,
+  MemoryRegion *mr,
+  bool is_write)
+{
+}
+#endif
+
 static const TypeInfo memory_region_info = {
 .parent = TYPE_OBJECT,
 .name   = TYPE_MEMORY_REGION,
-- 
2.28.0

[PATCH v6 00/16] Add a Generic Virtual Device Fuzzer

[Alexander Bulekov]

v6:
- Some More "General" -> "Generic"
- Fix broken build between commits (build-tested after each commit
  and through gitlab CI)
- Fix some predefined generic-fuzz configs that failed to run
v5:
- Replace GArray-based predefined fuzzer configs with a static
  struct array
- "General" -> "Generic"
- Fix bugs with wrong timeout denominator and skipping DMA memwrites
  without QTEST_LOG
v4:
- Replace yaml + c template-based oss-fuzz configs, with C code to
  register a FuzzTarget for each config (as suggested by Paolo)
- Replicate the functionality of address_space_write_rom to ensure
  matching behavior when QTEST_LOG is enabled
- Improve code documentation/comments
- Small formatting changes
v3:
- Use flatviews to help select regions for fuzzing 
- Meson-related changes
- Add some documentation
- Improve minimalization script to trim write{bwlq} commands
v2:
- Remove QOS dependency.
- Add a custom crossover function
- Fix broken minimization scripts
- Fixes to the IO region and DMA handling code

This is a general virtual-device fuzzer, designed to fuzz devices over Port IO,
MMIO, and DMA.

To get started with this:
 1. Build the fuzzers (see docs/devel/fuzzing.txt)
Note: Build with --enable-sanitizers, or create a "dictionary file":
echo kw1=\"FUZZ\" > dict
and pass it as an argument to libFuzzer with -dict=./dict
This magic value is a command separator that lets the fuzzer perform
multiple IO actions with a single input.

 2. Pick the qemu arguments you wish to fuzz:
export QEMU_FUZZ_ARGS="-M q35 -device virtio-balloon"

 3. Tell the fuzzer which QOM objects or MemoryRegion names to fuzz. I find the
 "info qom-tree", "info qtree" and "info mtree" commands useful for identifying
 these. Supports globbing. Here I will try to simultaneously fuzz(for no good
 reason) virtio-balloon and e1000e, which is included by default in the q35:
export QEMU_FUZZ_OBJECTS='virtio* e1000*'
You can also try to fuzz the whole machine:
export QEMU_FUZZ_OBJECTS='*'

 4. Run the fuzzer for 0 inputs. The fuzzer should output a list of
 MemoryRegions/PCI Devices it will try to fuzz. Confirm that these match your
 expectations.
./i386-softmmu/qemu-fuzz-i386 --fuzz-target=general-fuzz -runs=0

 5. Run the fuzzer:
./i386-softmmu/qemu-fuzz-i386 --fuzz-target=general-fuzz 


Basically, at the core, this fuzzer is an interpreter that splits the input
into a series of commands, such as mmio_write, pio_write, etc. We structure
these commands to hit only MemoryRegions that are associated with the devices
specified in QEMU_FUZZ_OBJECTS. Additionally, these patches add "hooks" to
functions that are typically used by virtual-devices to read from RAM (DMA).
These hooks attempt to populate these DMA regions with fuzzed data, just in
time.

Some of the issues I have found or reproduced with this fuzzer:
https://bugs.launchpad.net/bugs/1525123
https://bugs.launchpad.net/bugs/1681439
https://bugs.launchpad.net/bugs/1777315
https://bugs.launchpad.net/bugs/1878034
https://bugs.launchpad.net/bugs/1878043
https://bugs.launchpad.net/bugs/1878054
https://bugs.launchpad.net/bugs/1878057
https://bugs.launchpad.net/bugs/1878067
https://bugs.launchpad.net/bugs/1878134
https://bugs.launchpad.net/bugs/1878136
https://bugs.launchpad.net/bugs/1878253
https://bugs.launchpad.net/bugs/1878255
https://bugs.launchpad.net/bugs/1878259
https://bugs.launchpad.net/bugs/1878263
https://bugs.launchpad.net/bugs/1878323
https://bugs.launchpad.net/bugs/1878641
https://bugs.launchpad.net/bugs/1878642
https://bugs.launchpad.net/bugs/1878645
https://bugs.launchpad.net/bugs/1878651
https://bugs.launchpad.net/bugs/1879223
https://bugs.launchpad.net/bugs/1879227
https://bugs.launchpad.net/bugs/1879531
https://bugs.launchpad.net/bugs/1880355
https://bugs.launchpad.net/bugs/1880539
https://bugs.launchpad.net/bugs/1884693
https://bugs.launchpad.net/bugs/1886362
https://bugs.launchpad.net/bugs/1887303
https://bugs.launchpad.net/bugs/1887309
https://bugs.launchpad.net/bugs/697510
v5:
- Replace GArray-based predefined fuzzer configs with a static
  struct array
- "General" -> "Generic"
- Fix bugs with wrong timeout denominator and skipping DMA memwrites
  without QTEST_LOG

Alexander Bulekov (16):
  memory: Add FlatView foreach function
  fuzz: Add generic virtual-device fuzzer
  fuzz: Add PCI features to the generic fuzzer
  fuzz: Add DMA support to the generic-fuzzer
  fuzz: Declare DMA Read callback function
  fuzz: Add fuzzer callbacks to DMA-read functions
  fuzz: Add support for custom crossover functions
  fuzz: add a DISABLE_PCI op to generic-fuzzer
  fuzz: add a crossover function to generic-fuzzer
  scripts/oss-fuzz: Add script to reorder a generic-fuzzer trace
  scripts/oss-fuzz: Add crash trace minimization script
  fuzz: Add instructions for using generic-fuzz
  fuzz: add an "opaque" to the 

[PATCH v6 07/16] fuzz: Add support for custom crossover functions

[Alexander Bulekov]

libfuzzer supports a "custom crossover function". Libfuzzer often tries
to blend two inputs to create a new interesting input. Sometimes, we
have a better idea about how to blend inputs together. This change
allows fuzzers to specify a custom function for blending two inputs
together.

Signed-off-by: Alexander Bulekov 
Reviewed-by: Darren Kenny 
---
 tests/qtest/fuzz/fuzz.c | 13 +
 tests/qtest/fuzz/fuzz.h | 27 +++
 2 files changed, 40 insertions(+)

diff --git a/tests/qtest/fuzz/fuzz.c b/tests/qtest/fuzz/fuzz.c
index eb0070437f..7be7226bc0 100644
--- a/tests/qtest/fuzz/fuzz.c
+++ b/tests/qtest/fuzz/fuzz.c
@@ -118,6 +118,19 @@ static FuzzTarget *fuzz_get_target(char* name)
 }
 
 
+/* Sometimes called by libfuzzer to mutate two inputs into one */
+size_t LLVMFuzzerCustomCrossOver(const uint8_t *data1, size_t size1,
+ const uint8_t *data2, size_t size2,
+ uint8_t *out, size_t max_out_size,
+ unsigned int seed)
+{
+if (fuzz_target->crossover) {
+return fuzz_target->crossover(data1, size1, data2, size2, out,
+  max_out_size, seed);
+}
+return 0;
+}
+
 /* Executed for each fuzzing-input */
 int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size)
 {
diff --git a/tests/qtest/fuzz/fuzz.h b/tests/qtest/fuzz/fuzz.h
index 8eb765edc8..ed9ce17154 100644
--- a/tests/qtest/fuzz/fuzz.h
+++ b/tests/qtest/fuzz/fuzz.h
@@ -77,6 +77,29 @@ typedef struct FuzzTarget {
  */
 void(*fuzz)(QTestState *, const unsigned char *, size_t);
 
+/*
+ * The fuzzer can specify a "Custom Crossover" function for combining two
+ * inputs from the corpus. This function is sometimes called by libfuzzer
+ * when mutating inputs.
+ *
+ * data1: location of first input
+ * size1: length of first input
+ * data1: location of second input
+ * size1: length of second input
+ * out: where to place the resulting, mutated input
+ * max_out_size: the maximum length of the input that can be placed in out
+ * seed: the seed that should be used to make mutations deterministic, when
+ *   needed
+ *
+ * See libfuzzer's LLVMFuzzerCustomCrossOver API for more info.
+ *
+ * Can be NULL
+ */
+size_t(*crossover)(const uint8_t *data1, size_t size1,
+   const uint8_t *data2, size_t size2,
+   uint8_t *out, size_t max_out_size,
+   unsigned int seed);
+
 } FuzzTarget;
 
 void flush_events(QTestState *);
@@ -91,6 +114,10 @@ void fuzz_qtest_set_serialize(bool option);
  */
 void fuzz_add_target(const FuzzTarget *target);
 
+size_t LLVMFuzzerCustomCrossOver(const uint8_t *data1, size_t size1,
+ const uint8_t *data2, size_t size2,
+ uint8_t *out, size_t max_out_size,
+ unsigned int seed);
 int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size);
 int LLVMFuzzerInitialize(int *argc, char ***argv, char ***envp);
 
-- 
2.28.0

[PATCH v6 11/16] scripts/oss-fuzz: Add crash trace minimization script

[Alexander Bulekov]

Once we find a crash, we can convert it into a QTest trace. Usually this
trace will contain many operations that are unneeded to reproduce the
crash. This script tries to minimize the crashing trace, by removing
operations and trimming QTest bufwrite(write addr len data...) commands.

Signed-off-by: Alexander Bulekov 
Reviewed-by: Darren Kenny 
---
 scripts/oss-fuzz/minimize_qtest_trace.py | 157 +++
 1 file changed, 157 insertions(+)
 create mode 100755 scripts/oss-fuzz/minimize_qtest_trace.py

diff --git a/scripts/oss-fuzz/minimize_qtest_trace.py 
b/scripts/oss-fuzz/minimize_qtest_trace.py
new file mode 100755
index 00..5e405a0d5f
--- /dev/null
+++ b/scripts/oss-fuzz/minimize_qtest_trace.py
@@ -0,0 +1,157 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+"""
+This takes a crashing qtest trace and tries to remove superflous operations
+"""
+
+import sys
+import os
+import subprocess
+import time
+import struct
+
+QEMU_ARGS = None
+QEMU_PATH = None
+TIMEOUT = 5
+CRASH_TOKEN = None
+
+write_suffix_lookup = {"b": (1, "B"),
+   "w": (2, "H"),
+   "l": (4, "L"),
+   "q": (8, "Q")}
+
+def usage():
+sys.exit("""\
+Usage: QEMU_PATH="/path/to/qemu" QEMU_ARGS="args" {} input_trace output_trace
+By default, will try to use the second-to-last line in the output to identify
+whether the crash occred. Optionally, manually set a string that idenitifes the
+crash by setting CRASH_TOKEN=
+""".format((sys.argv[0])))
+
+def check_if_trace_crashes(trace, path):
+global CRASH_TOKEN
+with open(path, "w") as tracefile:
+tracefile.write("".join(trace))
+
+rc = subprocess.Popen("timeout -s 9 {timeout}s {qemu_path} {qemu_args} 
2>&1\
+< {trace_path}".format(timeout=TIMEOUT,
+   qemu_path=QEMU_PATH,
+   qemu_args=QEMU_ARGS,
+   trace_path=path),
+  shell=True,
+  stdin=subprocess.PIPE,
+  stdout=subprocess.PIPE)
+stdo = rc.communicate()[0]
+output = stdo.decode('unicode_escape')
+if rc.returncode == 137:# Timed Out
+return False
+if len(output.splitlines()) < 2:
+return False
+
+if CRASH_TOKEN is None:
+CRASH_TOKEN = output.splitlines()[-2]
+
+return CRASH_TOKEN in output
+
+
+def minimize_trace(inpath, outpath):
+global TIMEOUT
+with open(inpath) as f:
+trace = f.readlines()
+start = time.time()
+if not check_if_trace_crashes(trace, outpath):
+sys.exit("The input qtest trace didn't cause a crash...")
+end = time.time()
+print("Crashed in {} seconds".format(end-start))
+TIMEOUT = (end-start)*5
+print("Setting the timeout for {} seconds".format(TIMEOUT))
+print("Identifying Crashes by this string: {}".format(CRASH_TOKEN))
+
+i = 0
+newtrace = trace[:]
+# For each line
+while i < len(newtrace):
+# 1.) Try to remove it completely and reproduce the crash. If it works,
+# we're done.
+prior = newtrace[i]
+print("Trying to remove {}".format(newtrace[i]))
+# Try to remove the line completely
+newtrace[i] = ""
+if check_if_trace_crashes(newtrace, outpath):
+i += 1
+continue
+newtrace[i] = prior
+
+# 2.) Try to replace write{bwlq} commands with a write addr, len
+# command. Since this can require swapping endianness, try both LE and
+# BE options. We do this, so we can "trim" the writes in (3)
+if (newtrace[i].startswith("write") and not
+newtrace[i].startswith("write ")):
+suffix = newtrace[i].split()[0][-1]
+assert(suffix in write_suffix_lookup)
+addr = int(newtrace[i].split()[1], 16)
+value = int(newtrace[i].split()[2], 16)
+for endianness in ['<', '>']:
+data = struct.pack("{end}{size}".format(end=endianness,
+   size=write_suffix_lookup[suffix][1]),
+   value)
+newtrace[i] = "write {addr} {size} 0x{data}\n".format(
+addr=hex(addr),
+size=hex(write_suffix_lookup[suffix][0]),
+data=data.hex())
+if(check_if_trace_crashes(newtrace, outpath)):
+break
+else:
+newtrace[i] = prior
+
+# 3.) If it is a qtest write command: write addr len data, try to split
+# it into two separate write commands. If splitting the write down the
+# middle does not work, try to move the pivot "left" and retry, until
+# there is no space left. The idea is to prune unneccessary bytes from
+# long writes, while accommodating arbitrary MemoryRegion access sizes
+# and alignments.
+if newtrace[i].startswith("write "):
+add

[PATCH v6 01/16] memory: Add FlatView foreach function

[Alexander Bulekov]

Acked-by: Paolo Bonzini 
Signed-off-by: Alexander Bulekov 
---
 include/exec/memory.h | 5 +
 softmmu/memory.c  | 9 +
 2 files changed, 14 insertions(+)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index 622207bde1..042918dd16 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -719,6 +719,11 @@ static inline FlatView 
*address_space_to_flatview(AddressSpace *as)
 return qatomic_rcu_read(&as->current_map);
 }
 
+typedef int (*flatview_cb)(Int128 start,
+   Int128 len,
+   const MemoryRegion*, void*);
+
+void flatview_for_each_range(FlatView *fv, flatview_cb cb , void *opaque);
 
 /**
  * struct MemoryRegionSection: describes a fragment of a #MemoryRegion
diff --git a/softmmu/memory.c b/softmmu/memory.c
index 403ff3abc9..c46b0c6d65 100644
--- a/softmmu/memory.c
+++ b/softmmu/memory.c
@@ -656,6 +656,15 @@ static void render_memory_region(FlatView *view,
 }
 }
 
+void flatview_for_each_range(FlatView *fv, flatview_cb cb , void *opaque)
+{
+FlatRange *fr;
+FOR_EACH_FLAT_RANGE(fr, fv) {
+if (cb(fr->addr.start, fr->addr.size, fr->mr, opaque))
+break;
+}
+}
+
 static MemoryRegion *memory_region_get_flatview_root(MemoryRegion *mr)
 {
 while (mr->enabled) {
-- 
2.28.0

[PATCH 12/22] vl: move bios_name out of softmmu/vl.c

[Paolo Bonzini]

bios_name is a legacy variable used by machine code.  Hide it
from softmmu/vl.c.

Signed-off-by: Paolo Bonzini 
---
 hw/core/machine.c | 5 +
 softmmu/vl.c  | 2 --
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index 3c674bb05e..e4dac350d4 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -212,6 +212,8 @@ GlobalProperty hw_compat_2_1[] = {
 };
 const size_t hw_compat_2_1_len = G_N_ELEMENTS(hw_compat_2_1);
 
+const char *bios_name = NULL;
+
 static char *machine_get_kernel(Object *obj, Error **errp)
 {
 MachineState *ms = MACHINE(obj);
@@ -396,6 +398,9 @@ static void machine_set_firmware(Object *obj, const char 
*value, Error **errp)
 
 g_free(ms->firmware);
 ms->firmware = g_strdup(value);
+
+/* HACK */
+bios_name = ms->firmware;
 }
 
 static void machine_set_suppress_vmdesc(Object *obj, bool value, Error **errp)
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 52e7d317d7..e32e209a82 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -123,7 +123,6 @@ static int data_dir_idx;
 static const char *mem_path;
 static const char *boot_order;
 static const char *boot_once;
-const char *bios_name = NULL;
 enum vga_retrace_method vga_retrace_method = VGA_RETRACE_DUMB;
 int display_opengl;
 const char* keyboard_layout = NULL;
@@ -4314,7 +4313,6 @@ void qemu_init(int argc, char **argv, char **envp)
 kernel_filename = qemu_opt_get(machine_opts, "kernel");
 initrd_filename = qemu_opt_get(machine_opts, "initrd");
 kernel_cmdline = qemu_opt_get(machine_opts, "append");
-bios_name = qemu_opt_get(machine_opts, "firmware");
 
 opts = qemu_opts_find(qemu_find_opts("boot-opts"), NULL);
 if (opts) {
-- 
2.26.2

[PATCH 21/22] vl: separate qemu_create_early_backends

[Paolo Bonzini]

"Early" backends are created before the machine and can be used as
machine options.

Signed-off-by: Paolo Bonzini 
---
 softmmu/vl.c | 123 +++
 1 file changed, 65 insertions(+), 58 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index e9391929f6..866df5bb7c 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -117,6 +117,14 @@
 
 #define MAX_VIRTIO_CONSOLES 1
 
+typedef struct BlockdevOptionsQueueEntry {
+BlockdevOptions *bdo;
+Location loc;
+QSIMPLEQ_ENTRY(BlockdevOptionsQueueEntry) entry;
+} BlockdevOptionsQueueEntry;
+
+typedef QSIMPLEQ_HEAD(, BlockdevOptionsQueueEntry) BlockdevOptionsQueue;
+
 static const char *cpu_option;
 static const char *data_dir[16];
 static int data_dir_idx;
@@ -126,7 +134,9 @@ static const char *boot_once;
 static const char *incoming;
 static const char *loadvm;
 static int display_remote;
+static int snapshot;
 static QemuPluginList plugin_list = QTAILQ_HEAD_INITIALIZER(plugin_list);
+static BlockdevOptionsQueue bdo_queue = QSIMPLEQ_HEAD_INITIALIZER(bdo_queue);
 static bool nographic = false;
 enum vga_retrace_method vga_retrace_method = VGA_RETRACE_DUMB;
 int mem_prealloc; /* force preallocation of physical target memory */
@@ -1043,14 +1053,6 @@ static void default_drive(int enable, int snapshot, 
BlockInterfaceType type,
 
 }
 
-typedef struct BlockdevOptionsQueueEntry {
-BlockdevOptions *bdo;
-Location loc;
-QSIMPLEQ_ENTRY(BlockdevOptionsQueueEntry) entry;
-} BlockdevOptionsQueueEntry;
-
-typedef QSIMPLEQ_HEAD(, BlockdevOptionsQueueEntry) BlockdevOptionsQueue;
-
 static void configure_blockdev(BlockdevOptionsQueue *bdo_queue,
MachineClass *machine_class, int snapshot)
 {
@@ -2640,7 +2642,7 @@ static int machine_set_property(void *opaque,
  * cannot be created here, as it depends on the chardev
  * already existing.
  */
-static bool object_create_initial(const char *type, QemuOpts *opts)
+static bool object_create_early(const char *type, QemuOpts *opts)
 {
 if (user_creatable_print_help(type, opts)) {
 exit(0);
@@ -2692,6 +2694,58 @@ static bool object_create_initial(const char *type, 
QemuOpts *opts)
 return true;
 }
 
+static void qemu_create_early_backends(void)
+{
+MachineClass *machine_class = MACHINE_GET_CLASS(current_machine);
+
+if ((alt_grab || ctrl_grab) && dpy.type != DISPLAY_TYPE_SDL) {
+error_report("-alt-grab and -ctrl-grab are only valid "
+ "for SDL, ignoring option");
+}
+if (dpy.has_window_close &&
+(dpy.type != DISPLAY_TYPE_GTK && dpy.type != DISPLAY_TYPE_SDL)) {
+error_report("-no-quit is only valid for GTK and SDL, "
+ "ignoring option");
+}
+
+qemu_display_early_init(&dpy);
+qemu_console_early_init();
+
+if (dpy.has_gl && dpy.gl != DISPLAYGL_MODE_OFF && display_opengl == 0) {
+#if defined(CONFIG_OPENGL)
+error_report("OpenGL is not supported by the display");
+#else
+error_report("OpenGL support is disabled");
+#endif
+exit(1);
+}
+
+qemu_opts_foreach(qemu_find_opts("object"),
+  user_creatable_add_opts_foreach,
+  object_create_early, &error_fatal);
+
+/* spice needs the timers to be initialized by this point */
+/* spice must initialize before audio as it changes the default auiodev */
+/* spice must initialize before chardevs (for spicevmc and spiceport) */
+qemu_spice_init();
+
+qemu_opts_foreach(qemu_find_opts("chardev"),
+  chardev_init_func, NULL, &error_fatal);
+
+#ifdef CONFIG_VIRTFS
+qemu_opts_foreach(qemu_find_opts("fsdev"),
+  fsdev_init_func, NULL, &error_fatal);
+#endif
+
+/*
+ * Note: we need to create audio and block backends before
+ * machine_set_property(), so machine properties can refer to
+ * them.
+ */
+configure_blockdev(&bdo_queue, machine_class, snapshot);
+audio_init_audiodevs();
+}
+
 
 /*
  * The remainder of object creation happens after the
@@ -2699,7 +2753,7 @@ static bool object_create_initial(const char *type, 
QemuOpts *opts)
  */
 static bool object_create_delayed(const char *type, QemuOpts *opts)
 {
-return !object_create_initial(type, opts);
+return !object_create_early(type, opts);
 }
 
 
@@ -3308,7 +3362,6 @@ void qemu_finish_machine_init(void)
 
 void qemu_init(int argc, char **argv, char **envp)
 {
-int snapshot = 0;
 QemuOpts *opts, *machine_opts;
 QemuOpts *icount_opts = NULL, *accel_opts = NULL;
 QemuOptsList *olist;
@@ -3321,7 +3374,6 @@ void qemu_init(int argc, char **argv, char **envp)
 FILE *vmstate_dump_file = NULL;
 Error *err = NULL;
 bool have_custom_ram_size;
-BlockdevOptionsQueue bdo_queue = QSIMPLEQ_HEAD_INITIALIZER(bdo_queue);
 
 qemu_add_opts(&qemu_drive_opts);
 qemu_add_drive_opts(&qemu_legacy_drive_opts);
@@ -4291,52 +4343,7 @@ void qemu_init(int argc, char **argv, cha

[PATCH 13/22] vl: extract various command line validation snippets to a new function

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini 
---
 softmmu/vl.c | 78 ++--
 1 file changed, 39 insertions(+), 39 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index e32e209a82..f8b1ffb46e 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -123,6 +123,7 @@ static int data_dir_idx;
 static const char *mem_path;
 static const char *boot_order;
 static const char *boot_once;
+static const char *incoming;
 enum vga_retrace_method vga_retrace_method = VGA_RETRACE_DUMB;
 int display_opengl;
 const char* keyboard_layout = NULL;
@@ -2866,6 +2867,39 @@ static char *find_datadir(void)
 return get_relocated_path(CONFIG_QEMU_DATADIR);
 }
 
+static void qemu_validate_options(void)
+{
+QemuOpts *machine_opts = qemu_get_machine_opts();
+const char *kernel_filename = qemu_opt_get(machine_opts, "kernel");
+const char *initrd_filename = qemu_opt_get(machine_opts, "initrd");
+const char *kernel_cmdline = qemu_opt_get(machine_opts, "append");
+
+if (kernel_filename == NULL) {
+ if (kernel_cmdline != NULL) {
+  error_report("-append only allowed with -kernel option");
+  exit(1);
+  }
+
+  if (initrd_filename != NULL) {
+  error_report("-initrd only allowed with -kernel option");
+  exit(1);
+  }
+}
+
+if (incoming && !preconfig_exit_requested) {
+error_report("'preconfig' and 'incoming' options are "
+ "mutually exclusive");
+exit(EXIT_FAILURE);
+}
+
+#ifdef CONFIG_CURSES
+if (is_daemonized() && dpy.type == DISPLAY_TYPE_CURSES) {
+error_report("curses display cannot be used with -daemonize");
+exit(1);
+}
+#endif
+}
+
 static void qemu_process_early_options(void)
 {
 char **dirs;
@@ -3122,9 +3156,6 @@ void qemu_init(int argc, char **argv, char **envp)
 {
 int i;
 int snapshot = 0;
-int linux_boot;
-const char *initrd_filename;
-const char *kernel_filename, *kernel_cmdline;
 QemuOpts *opts, *machine_opts;
 QemuOpts *icount_opts = NULL, *accel_opts = NULL;
 QemuOptsList *olist;
@@ -3133,7 +3164,6 @@ void qemu_init(int argc, char **argv, char **envp)
 const char *loadvm = NULL;
 MachineClass *machine_class;
 const char *vga_model = NULL;
-const char *incoming = NULL;
 bool userconfig = true;
 bool nographic = false;
 int display_remote = 0;
@@ -4050,6 +4080,8 @@ void qemu_init(int argc, char **argv, char **envp)
  */
 loc_set_none();
 
+qemu_validate_options();
+
 /* These options affect everything else and should be processed
  * before daemonizing.
  */
@@ -4064,12 +4096,6 @@ void qemu_init(int argc, char **argv, char **envp)
 user_register_global_props();
 replay_configure(icount_opts);
 
-if (incoming && !preconfig_exit_requested) {
-error_report("'preconfig' and 'incoming' options are "
- "mutually exclusive");
-exit(EXIT_FAILURE);
-}
-
 configure_rtc(qemu_find_opts_singleton("rtc"));
 
 machine_class = select_machine();
@@ -4173,12 +4199,6 @@ void qemu_init(int argc, char **argv, char **envp)
 error_report("-nographic cannot be used with -daemonize");
 exit(1);
 }
-#ifdef CONFIG_CURSES
-if (dpy.type == DISPLAY_TYPE_CURSES) {
-error_report("curses display cannot be used with -daemonize");
-exit(1);
-}
-#endif
 }
 
 if (nographic) {
@@ -4309,11 +4329,6 @@ void qemu_init(int argc, char **argv, char **envp)
 qtest_server_init(qtest_chrdev, qtest_log, &error_fatal);
 }
 
-machine_opts = qemu_get_machine_opts();
-kernel_filename = qemu_opt_get(machine_opts, "kernel");
-initrd_filename = qemu_opt_get(machine_opts, "initrd");
-kernel_cmdline = qemu_opt_get(machine_opts, "append");
-
 opts = qemu_opts_find(qemu_find_opts("boot-opts"), NULL);
 if (opts) {
 boot_order = qemu_opt_get(opts, "order");
@@ -4334,24 +4349,9 @@ void qemu_init(int argc, char **argv, char **envp)
 boot_order = machine_class->default_boot_order;
 }
 
-if (!kernel_cmdline) {
-kernel_cmdline = "";
-current_machine->kernel_cmdline = (char *)kernel_cmdline;
-}
-
-linux_boot = (kernel_filename != NULL);
-
-if (!linux_boot && *kernel_cmdline != '\0') {
-error_report("-append only allowed with -kernel option");
-exit(1);
-}
-
-if (!linux_boot && initrd_filename != NULL) {
-error_report("-initrd only allowed with -kernel option");
-exit(1);
-}
-
-if (semihosting_enabled() && !semihosting_get_argc() && kernel_filename) {
+if (semihosting_enabled() && !semihosting_get_argc()) {
+const char *kernel_filename = qemu_opt_get(machine_opts, "kernel");
+const char *kernel_cmdline = qemu_opt_get(machine_opts, "append");
 /* fall back to the -kernel/-append */
 semihosting_arg_f

[PATCH v6 03/16] fuzz: Add PCI features to the generic fuzzer

[Alexander Bulekov]

This patch compares TYPE_PCI_DEVICE objects against the user-provided
matching pattern. If there is a match, we use some hacks and leverage
QOS to map each possible BAR for that device. Now fuzzed inputs might be
converted to pci_read/write commands which target specific. This means
that we can fuzz a particular device's PCI configuration space,

Signed-off-by: Alexander Bulekov 
Reviewed-by: Darren Kenny 
---
 tests/qtest/fuzz/generic_fuzz.c | 81 +
 1 file changed, 81 insertions(+)

diff --git a/tests/qtest/fuzz/generic_fuzz.c b/tests/qtest/fuzz/generic_fuzz.c
index f69e9583ce..4ab28c2ec7 100644
--- a/tests/qtest/fuzz/generic_fuzz.c
+++ b/tests/qtest/fuzz/generic_fuzz.c
@@ -24,6 +24,7 @@
 #include "exec/ramblock.h"
 #include "exec/address-spaces.h"
 #include "hw/qdev-core.h"
+#include "hw/pci/pci.h"
 
 /*
  * SEPARATOR is used to separate "operations" in the fuzz input
@@ -35,12 +36,17 @@ enum cmds {
 OP_OUT,
 OP_READ,
 OP_WRITE,
+OP_PCI_READ,
+OP_PCI_WRITE,
 OP_CLOCK_STEP,
 };
 
 #define DEFAULT_TIMEOUT_US 10
 #define USEC_IN_SEC 10
 
+#define PCI_HOST_BRIDGE_CFG 0xcf8
+#define PCI_HOST_BRIDGE_DATA 0xcfc
+
 typedef struct {
 ram_addr_t addr;
 ram_addr_t size; /* The number of bytes until the end of the I/O region */
@@ -55,6 +61,7 @@ static bool qtest_log_enabled;
  * user for fuzzing.
  */
 static GHashTable *fuzzable_memoryregions;
+static GPtrArray *fuzzable_pci_devices;
 
 struct get_io_cb_info {
 int index;
@@ -280,6 +287,65 @@ static void op_write(QTestState *s, const unsigned char * 
data, size_t len)
 break;
 }
 }
+static void op_pci_read(QTestState *s, const unsigned char * data, size_t len)
+{
+enum Sizes {Byte, Word, Long, end_sizes};
+struct {
+uint8_t size;
+uint8_t base;
+uint8_t offset;
+} a;
+if (len < sizeof(a) || fuzzable_pci_devices->len == 0) {
+return;
+}
+memcpy(&a, data, sizeof(a));
+PCIDevice *dev = g_ptr_array_index(fuzzable_pci_devices,
+  a.base % fuzzable_pci_devices->len);
+int devfn = dev->devfn;
+qtest_outl(s, PCI_HOST_BRIDGE_CFG, (1U << 31) | (devfn << 8) | a.offset);
+switch (a.size %= end_sizes) {
+case Byte:
+qtest_inb(s, PCI_HOST_BRIDGE_DATA);
+break;
+case Word:
+qtest_inw(s, PCI_HOST_BRIDGE_DATA);
+break;
+case Long:
+qtest_inl(s, PCI_HOST_BRIDGE_DATA);
+break;
+}
+}
+
+static void op_pci_write(QTestState *s, const unsigned char * data, size_t len)
+{
+enum Sizes {Byte, Word, Long, end_sizes};
+struct {
+uint8_t size;
+uint8_t base;
+uint8_t offset;
+uint32_t value;
+} a;
+if (len < sizeof(a) || fuzzable_pci_devices->len == 0) {
+return;
+}
+memcpy(&a, data, sizeof(a));
+PCIDevice *dev = g_ptr_array_index(fuzzable_pci_devices,
+  a.base % fuzzable_pci_devices->len);
+int devfn = dev->devfn;
+qtest_outl(s, PCI_HOST_BRIDGE_CFG, (1U << 31) | (devfn << 8) | a.offset);
+switch (a.size %= end_sizes) {
+case Byte:
+qtest_outb(s, PCI_HOST_BRIDGE_DATA, a.value & 0xFF);
+break;
+case Word:
+qtest_outw(s, PCI_HOST_BRIDGE_DATA, a.value & 0x);
+break;
+case Long:
+qtest_outl(s, PCI_HOST_BRIDGE_DATA, a.value & 0x);
+break;
+}
+}
+
 static void op_clock_step(QTestState *s, const unsigned char *data, size_t len)
 {
 qtest_clock_step_next(s);
@@ -338,6 +404,8 @@ static void generic_fuzz(QTestState *s, const unsigned char 
*Data, size_t Size)
 [OP_OUT]= op_out,
 [OP_READ]   = op_read,
 [OP_WRITE]  = op_write,
+[OP_PCI_READ]   = op_pci_read,
+[OP_PCI_WRITE]  = op_pci_write,
 [OP_CLOCK_STEP] = op_clock_step,
 };
 const unsigned char *cmd = Data;
@@ -428,6 +496,18 @@ static int locate_fuzz_objects(Object *child, void *opaque)
 /* Find and save ptrs to any child MemoryRegions */
 object_child_foreach_recursive(child, locate_fuzz_memory_regions, 
NULL);
 
+/*
+ * We matched an object. If its a PCI device, store a pointer to it so
+ * we can map BARs and fuzz its config space.
+ */
+if (object_dynamic_cast(OBJECT(child), TYPE_PCI_DEVICE)) {
+/*
+ * Don't want duplicate pointers to the same PCIDevice, so remove
+ * copies of the pointer, before adding it.
+ */
+g_ptr_array_remove_fast(fuzzable_pci_devices, PCI_DEVICE(child));
+g_ptr_array_add(fuzzable_pci_devices, PCI_DEVICE(child));
+}
 } else if (object_dynamic_cast(OBJECT(child), TYPE_MEMORY_REGION)) {
 if (g_pattern_match_simple(pattern,
 object_get_canonical_path_component(child))) {
@@ -460,6 +540,7 @@ static void gen

[PATCH 10/22] vl: extract qemu_init_subsystems

[Paolo Bonzini]

Group a bunch of subsystem initializations that can be done right
after command line parsing.  Remove initializations that can be done
simply as global variable initializers.

Signed-off-by: Paolo Bonzini 
---
 softmmu/vl.c | 94 
 1 file changed, 43 insertions(+), 51 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index a9f69a7d11..e731d754ec 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -129,7 +129,7 @@ bool enable_mlock = false;
 bool enable_cpu_pm = false;
 int nb_nics;
 NICInfo nd_table[MAX_NICS];
-int autostart;
+int autostart = 1;
 static enum {
 RTC_BASE_UTC,
 RTC_BASE_LOCALTIME,
@@ -1229,7 +1229,8 @@ struct VMChangeStateEntry {
 int priority;
 };
 
-static QTAILQ_HEAD(, VMChangeStateEntry) vm_change_state_head;
+static QTAILQ_HEAD(, VMChangeStateEntry) vm_change_state_head =
+QTAILQ_HEAD_INITIALIZER(vm_change_state_head);
 
 /**
  * qemu_add_vm_change_state_handler_prio:
@@ -2962,11 +2963,45 @@ static void qemu_maybe_daemonize(const char *pid_file)
 qemu_add_exit_notifier(&qemu_unlink_pidfile_notifier);
 }
 
+static void qemu_init_subsystems(void)
+{
+Error *err;
+
+os_set_line_buffering();
+
+module_call_init(MODULE_INIT_TRACE);
+
+qemu_init_cpu_list();
+qemu_init_cpu_loop();
+qemu_mutex_lock_iothread();
+
+atexit(qemu_run_exit_notifiers);
+
+module_call_init(MODULE_INIT_QOM);
+module_call_init(MODULE_INIT_MIGRATION);
+
+runstate_init();
+precopy_infrastructure_init();
+postcopy_infrastructure_init();
+monitor_init_globals();
+
+if (qcrypto_init(&err) < 0) {
+error_reportf_err(err, "cannot initialize crypto: ");
+exit(1);
+}
+
+os_setup_early_signal_handling();
+
+bdrv_init_with_whitelist();
+page_size_init();
+socket_init();
+}
 
 void qemu_init(int argc, char **argv, char **envp)
 {
 int i;
-int snapshot, linux_boot;
+int snapshot = 0;
+int linux_boot;
 const char *initrd_filename;
 const char *kernel_filename, *kernel_cmdline;
 const char *boot_order = NULL;
@@ -2987,7 +3022,6 @@ void qemu_init(int argc, char **argv, char **envp)
 ram_addr_t maxram_size;
 uint64_t ram_slots = 0;
 FILE *vmstate_dump_file = NULL;
-Error *main_loop_err = NULL;
 Error *err = NULL;
 const char *mem_path = NULL;
 bool have_custom_ram_size;
@@ -2995,22 +3029,6 @@ void qemu_init(int argc, char **argv, char **envp)
 QemuPluginList plugin_list = QTAILQ_HEAD_INITIALIZER(plugin_list);
 int mem_prealloc = 0; /* force preallocation of physical target memory */
 
-os_set_line_buffering();
-
-error_init(argv[0]);
-module_call_init(MODULE_INIT_TRACE);
-
-qemu_init_cpu_list();
-qemu_init_cpu_loop();
-
-qemu_mutex_lock_iothread();
-
-atexit(qemu_run_exit_notifiers);
-qemu_init_exec_dir(argv[0]);
-
-module_call_init(MODULE_INIT_QOM);
-module_call_init(MODULE_INIT_MIGRATION);
-
 qemu_add_opts(&qemu_drive_opts);
 qemu_add_drive_opts(&qemu_legacy_drive_opts);
 qemu_add_drive_opts(&qemu_common_drive_opts);
@@ -3045,27 +3063,10 @@ void qemu_init(int argc, char **argv, char **envp)
 qemu_add_opts(&qemu_fw_cfg_opts);
 module_call_init(MODULE_INIT_OPTS);
 
-runstate_init();
-precopy_infrastructure_init();
-postcopy_infrastructure_init();
-monitor_init_globals();
-
-if (qcrypto_init(&err) < 0) {
-error_reportf_err(err, "cannot initialize crypto: ");
-exit(1);
-}
-
-QTAILQ_INIT(&vm_change_state_head);
-os_setup_early_signal_handling();
-
-cpu_option = NULL;
-snapshot = 0;
-
-nb_nics = 0;
-
-bdrv_init_with_whitelist();
+error_init(argv[0]);
+qemu_init_exec_dir(argv[0]);
 
-autostart = 1;
+qemu_init_subsystems();
 
 /* first pass of option parsing */
 optind = 1;
@@ -3940,13 +3941,10 @@ void qemu_init(int argc, char **argv, char **envp)
 qemu_process_help_options();
 qemu_maybe_daemonize(pid_file);
 
-if (qemu_init_main_loop(&main_loop_err)) {
-error_report_err(main_loop_err);
-exit(1);
-}
+qemu_init_main_loop(&error_fatal);
+cpu_timers_init();
 
 user_register_global_props();
-
 replay_configure(icount_opts);
 
 if (incoming && !preconfig_exit_requested) {
@@ -4125,9 +4123,6 @@ void qemu_init(int argc, char **argv, char **envp)
 exit(1);
 }
 
-page_size_init();
-socket_init();
-
 qemu_opts_foreach(qemu_find_opts("object"),
   user_creatable_add_opts_foreach,
   object_create_initial, &error_fatal);
@@ -4245,9 +4240,6 @@ void qemu_init(int argc, char **argv, char **envp)
 semihosting_arg_fallback(kernel_filename, kernel_cmdline);
 }
 
-/* initialize cpu timers and VCPU throttle modules */
-cpu_timers_init();
-
 if (default_net) {
 QemuOptsList *net = qemu_find_opts("net");
 qemu_opts_set(net, NULL, "type", "nic"

[PATCH 19/22] vl: extract default devices to separate functions

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini 
---
 softmmu/vl.c | 216 +++
 1 file changed, 114 insertions(+), 102 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index e58572dbd8..0a6f47e7d6 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -125,7 +125,9 @@ static const char *boot_order;
 static const char *boot_once;
 static const char *incoming;
 static const char *loadvm;
+static int display_remote;
 static QemuPluginList plugin_list = QTAILQ_HEAD_INITIALIZER(plugin_list);
+static bool nographic = false;
 enum vga_retrace_method vga_retrace_method = VGA_RETRACE_DUMB;
 int mem_prealloc; /* force preallocation of physical target memory */
 int display_opengl;
@@ -147,6 +149,7 @@ static int rtc_host_datetime_offset = -1; /* valid & used 
only with
  RTC_BASE_DATETIME */
 QEMUClockType rtc_clock;
 int vga_interface_type = VGA_NONE;
+static const char *vga_model = NULL;
 static DisplayOptions dpy;
 static int num_serial_hds;
 static Chardev **serial_hds;
@@ -2224,6 +2227,115 @@ static int foreach_device_config(int type, int 
(*func)(const char *cmdline))
 return 0;
 }
 
+static void qemu_disable_default_devices(void)
+{
+MachineClass *machine_class = MACHINE_GET_CLASS(current_machine);
+
+qemu_opts_foreach(qemu_find_opts("device"),
+  default_driver_check, NULL, NULL);
+qemu_opts_foreach(qemu_find_opts("global"),
+  default_driver_check, NULL, NULL);
+
+if (!vga_model && !default_vga) {
+vga_interface_type = VGA_DEVICE;
+}
+if (!has_defaults || machine_class->no_serial) {
+default_serial = 0;
+}
+if (!has_defaults || machine_class->no_parallel) {
+default_parallel = 0;
+}
+if (!has_defaults || machine_class->no_floppy) {
+default_floppy = 0;
+}
+if (!has_defaults || machine_class->no_cdrom) {
+default_cdrom = 0;
+}
+if (!has_defaults || machine_class->no_sdcard) {
+default_sdcard = 0;
+}
+if (!has_defaults) {
+default_monitor = 0;
+default_net = 0;
+default_vga = 0;
+}
+}
+
+static void qemu_create_default_devices(void)
+{
+MachineClass *machine_class = MACHINE_GET_CLASS(current_machine);
+
+if (is_daemonized()) {
+/* According to documentation and historically, -nographic redirects
+ * serial port, parallel port and monitor to stdio, which does not work
+ * with -daemonize.  We can redirect these to null instead, but since
+ * -nographic is legacy, let's just error out.
+ * We disallow -nographic only if all other ports are not redirected
+ * explicitly, to not break existing legacy setups which uses
+ * -nographic _and_ redirects all ports explicitly - this is valid
+ * usage, -nographic is just a no-op in this case.
+ */
+if (nographic
+&& (default_parallel || default_serial || default_monitor)) {
+error_report("-nographic cannot be used with -daemonize");
+exit(1);
+}
+}
+
+if (nographic) {
+if (default_parallel)
+add_device_config(DEV_PARALLEL, "null");
+if (default_serial && default_monitor) {
+add_device_config(DEV_SERIAL, "mon:stdio");
+} else {
+if (default_serial)
+add_device_config(DEV_SERIAL, "stdio");
+if (default_monitor)
+monitor_parse("stdio", "readline", false);
+}
+} else {
+if (default_serial)
+add_device_config(DEV_SERIAL, "vc:80Cx24C");
+if (default_parallel)
+add_device_config(DEV_PARALLEL, "vc:80Cx24C");
+if (default_monitor)
+monitor_parse("vc:80Cx24C", "readline", false);
+}
+
+if (default_net) {
+QemuOptsList *net = qemu_find_opts("net");
+qemu_opts_set(net, NULL, "type", "nic", &error_abort);
+#ifdef CONFIG_SLIRP
+qemu_opts_set(net, NULL, "type", "user", &error_abort);
+#endif
+}
+
+#if defined(CONFIG_VNC)
+if (!QTAILQ_EMPTY(&(qemu_find_opts("vnc")->head))) {
+display_remote++;
+}
+#endif
+if (dpy.type == DISPLAY_TYPE_DEFAULT && !display_remote) {
+if (!qemu_display_find_default(&dpy)) {
+dpy.type = DISPLAY_TYPE_NONE;
+#if defined(CONFIG_VNC)
+vnc_parse("localhost:0,to=99,id=default", &error_abort);
+#endif
+}
+}
+if (dpy.type == DISPLAY_TYPE_DEFAULT) {
+dpy.type = DISPLAY_TYPE_NONE;
+}
+
+/* If no default VGA is requested, the default is "none".  */
+if (default_vga) {
+vga_model = get_default_vga_model(machine_class);
+}
+if (vga_model) {
+select_vgahw(machine_class, vga_model);
+}
+}
+
 static int serial_parse(const char *devname)
 {
 int index = num_serial_hds;
@@ -3198,10 +3310,7 @@ void qemu_init(int argc, char **argv, char **envp)
 int optind;

[PATCH 16/22] vl: create "-net nic -net user" default earlier

[Paolo Bonzini]

Create it together with other default backends, even though the processing is
done later.

Signed-off-by: Paolo Bonzini 
---
 softmmu/vl.c | 16 
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index 122bf1821b..8577667b8f 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -4235,6 +4235,14 @@ void qemu_init(int argc, char **argv, char **envp)
 monitor_parse("vc:80Cx24C", "readline", false);
 }
 
+if (default_net) {
+QemuOptsList *net = qemu_find_opts("net");
+qemu_opts_set(net, NULL, "type", "nic", &error_abort);
+#ifdef CONFIG_SLIRP
+qemu_opts_set(net, NULL, "type", "user", &error_abort);
+#endif
+}
+
 #if defined(CONFIG_VNC)
 if (!QTAILQ_EMPTY(&(qemu_find_opts("vnc")->head))) {
 display_remote++;
@@ -4370,14 +4378,6 @@ void qemu_init(int argc, char **argv, char **envp)
 semihosting_arg_fallback(kernel_filename, kernel_cmdline);
 }
 
-if (default_net) {
-QemuOptsList *net = qemu_find_opts("net");
-qemu_opts_set(net, NULL, "type", "nic", &error_abort);
-#ifdef CONFIG_SLIRP
-qemu_opts_set(net, NULL, "type", "user", &error_abort);
-#endif
-}
-
 if (net_init_clients(&err) < 0) {
 error_report_err(err);
 exit(1);
-- 
2.26.2

[PATCH 11/22] vl: move prelaunch part of qemu_init to a new function

[Paolo Bonzini]

The final part of qemu_init, starting with the completion of
board init, is already relatively clean.  Split it out of
qemu_init so that qemu_init keeps only the messy parts.

Signed-off-by: Paolo Bonzini 
---
 include/sysemu/sysemu.h |   1 +
 softmmu/vl.c| 238 +---
 2 files changed, 124 insertions(+), 115 deletions(-)

diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 817ff4cf75..9f47b2a354 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -112,6 +112,7 @@ QemuOpts *qemu_get_machine_opts(void);
 bool defaults_enabled(void);
 
 void qemu_init(int argc, char **argv, char **envp);
+void qemu_finish_machine_init(void);
 void qemu_main_loop(void);
 void qemu_cleanup(void);
 
diff --git a/softmmu/vl.c b/softmmu/vl.c
index e731d754ec..52e7d317d7 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -120,6 +120,9 @@
 static const char *cpu_option;
 static const char *data_dir[16];
 static int data_dir_idx;
+static const char *mem_path;
+static const char *boot_order;
+static const char *boot_once;
 const char *bios_name = NULL;
 enum vga_retrace_method vga_retrace_method = VGA_RETRACE_DUMB;
 int display_opengl;
@@ -2997,6 +3000,125 @@ static void qemu_init_subsystems(void)
 socket_init();
 }
 
+/* Called after leaving preconfig state.  */
+void qemu_finish_machine_init(void)
+{
+MachineClass *machine_class = MACHINE_GET_CLASS(current_machine);
+DisplayState *ds;
+
+if (machine_class->default_ram_id && current_machine->ram_size &&
+numa_uses_legacy_mem() && !current_machine->ram_memdev_id) {
+create_default_memdev(current_machine, mem_path);
+}
+
+/* from here on runstate is RUN_STATE_PRELAUNCH */
+machine_run_board_init(current_machine);
+
+/*
+ * TODO To drop support for deprecated bogus if=..., move
+ * drive_check_orphaned() here, replacing this call.  Also drop
+ * its deprecation warning, along with DriveInfo member
+ * @claimed_by_board.
+ */
+drive_mark_claimed_by_board();
+
+realtime_init();
+
+soundhw_init();
+
+if (hax_enabled()) {
+hax_sync_vcpus();
+}
+
+qemu_opts_foreach(qemu_find_opts("fw_cfg"),
+  parse_fw_cfg, fw_cfg_find(), &error_fatal);
+
+/* init USB devices */
+if (machine_usb(current_machine)) {
+if (foreach_device_config(DEV_USB, usb_parse) < 0)
+exit(1);
+}
+
+/* init generic devices */
+rom_set_order_override(FW_CFG_ORDER_OVERRIDE_DEVICE);
+qemu_opts_foreach(qemu_find_opts("device"),
+  device_init_func, NULL, &error_fatal);
+
+cpu_synchronize_all_post_init();
+
+rom_reset_order_override();
+
+/* Did we create any drives that we failed to create a device for? */
+drive_check_orphaned();
+
+/* Don't warn about the default network setup that you get if
+ * no command line -net or -netdev options are specified. There
+ * are two cases that we would otherwise complain about:
+ * (1) board doesn't support a NIC but the implicit "-net nic"
+ * requested one
+ * (2) CONFIG_SLIRP not set, in which case the implicit "-net nic"
+ * sets up a nic that isn't connected to anything.
+ */
+if (!default_net && (!qtest_enabled() || has_defaults)) {
+net_check_clients();
+}
+
+if (boot_once) {
+qemu_boot_set(boot_once, &error_fatal);
+qemu_register_reset(restore_boot_order, g_strdup(boot_order));
+}
+
+/* init local displays */
+ds = init_displaystate();
+qemu_display_init(ds, &dpy);
+
+/* must be after terminal init, SDL library changes signal handlers */
+os_setup_signal_handling();
+
+/* init remote displays */
+#ifdef CONFIG_VNC
+qemu_opts_foreach(qemu_find_opts("vnc"),
+  vnc_init_func, NULL, &error_fatal);
+#endif
+
+if (using_spice) {
+qemu_spice_display_init();
+}
+
+if (foreach_device_config(DEV_GDB, gdbserver_start) < 0) {
+exit(1);
+}
+
+qdev_machine_creation_done();
+
+/* TODO: once all bus devices are qdevified, this should be done
+ * when bus is created by qdev.c */
+/*
+ * TODO: If we had a main 'reset container' that the whole system
+ * lived in, we could reset that using the multi-phase reset
+ * APIs. For the moment, we just reset the sysbus, which will cause
+ * all devices hanging off it (and all their child buses, recursively)
+ * to be reset. Note that this will *not* reset any Device objects
+ * which are not attached to some part of the qbus tree!
+ */
+qemu_register_reset(resettable_cold_reset_fn, sysbus_get_default());
+qemu_run_machine_init_done_notifiers();
+
+if (rom_check_and_register_reset() != 0) {
+error_report("rom check and register reset failed");
+exit(1);
+}
+
+replay_start();
+
+/* This checkpoint is required by replay to separate prior clock
+   reading 

[PATCH v6 04/16] fuzz: Add DMA support to the generic-fuzzer

[Alexander Bulekov]

When a virtual-device tries to access some buffer in memory over DMA, we
add call-backs into the fuzzer(next commit). The fuzzer checks verifies
that the DMA request maps to a physical RAM address and fills the memory
with fuzzer-provided data. The patterns that we use to fill this memory
are specified using add_dma_pattern and clear_dma_patterns operations.

Signed-off-by: Alexander Bulekov 
Reviewed-by: Darren Kenny 
---
 include/exec/memory.h   |   7 +
 tests/qtest/fuzz/generic_fuzz.c | 228 
 2 files changed, 235 insertions(+)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index 042918dd16..93d27bff26 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -42,6 +42,13 @@ typedef struct IOMMUMemoryRegionClass IOMMUMemoryRegionClass;
 DECLARE_OBJ_CHECKERS(IOMMUMemoryRegion, IOMMUMemoryRegionClass,
  IOMMU_MEMORY_REGION, TYPE_IOMMU_MEMORY_REGION)
 
+#ifdef CONFIG_FUZZ
+void fuzz_dma_read_cb(size_t addr,
+  size_t len,
+  MemoryRegion *mr,
+  bool is_write);
+#endif
+
 extern bool global_dirty_log;
 
 typedef struct MemoryRegionOps MemoryRegionOps;
diff --git a/tests/qtest/fuzz/generic_fuzz.c b/tests/qtest/fuzz/generic_fuzz.c
index 4ab28c2ec7..e356873ae0 100644
--- a/tests/qtest/fuzz/generic_fuzz.c
+++ b/tests/qtest/fuzz/generic_fuzz.c
@@ -25,6 +25,7 @@
 #include "exec/address-spaces.h"
 #include "hw/qdev-core.h"
 #include "hw/pci/pci.h"
+#include "hw/boards.h"
 
 /*
  * SEPARATOR is used to separate "operations" in the fuzz input
@@ -38,12 +39,16 @@ enum cmds {
 OP_WRITE,
 OP_PCI_READ,
 OP_PCI_WRITE,
+OP_ADD_DMA_PATTERN,
+OP_CLEAR_DMA_PATTERNS,
 OP_CLOCK_STEP,
 };
 
 #define DEFAULT_TIMEOUT_US 10
 #define USEC_IN_SEC 10
 
+#define MAX_DMA_FILL_SIZE 0x1
+
 #define PCI_HOST_BRIDGE_CFG 0xcf8
 #define PCI_HOST_BRIDGE_DATA 0xcfc
 
@@ -56,6 +61,24 @@ static useconds_t timeout = DEFAULT_TIMEOUT_US;
 
 static bool qtest_log_enabled;
 
+/*
+ * A pattern used to populate a DMA region or perform a memwrite. This is
+ * useful for e.g. populating tables of unique addresses.
+ * Example {.index = 1; .stride = 2; .len = 3; .data = "\x00\x01\x02"}
+ * Renders as: 00 01 02   00 03 02   00 05 02   00 07 02 ...
+ */
+typedef struct {
+uint8_t index;  /* Index of a byte to increment by stride */
+uint8_t stride; /* Increment each index'th byte by this amount */
+size_t len;
+const uint8_t *data;
+} pattern;
+
+/* Avoid filling the same DMA region between MMIO/PIO commands ? */
+static bool avoid_double_fetches;
+
+static QTestState *qts_global; /* Need a global for the DMA callback */
+
 /*
  * List of memory regions that are children of QOM objects specified by the
  * user for fuzzing.
@@ -84,6 +107,167 @@ static int get_io_address_cb(Int128 start, Int128 size,
 return 0;
 }
 
+/*
+ * List of dma regions populated since the last fuzzing command. Used to ensure
+ * that we only write to each DMA address once, to avoid race conditions when
+ * building reproducers.
+ */
+static GArray *dma_regions;
+
+static GArray *dma_patterns;
+static int dma_pattern_index;
+
+/*
+ * Allocate a block of memory and populate it with a pattern.
+ */
+static void *pattern_alloc(pattern p, size_t len)
+{
+int i;
+uint8_t *buf = g_malloc(len);
+uint8_t sum = 0;
+
+for (i = 0; i < len; ++i) {
+buf[i] = p.data[i % p.len];
+if ((i % p.len) == p.index) {
+buf[i] += sum;
+sum += p.stride;
+}
+}
+return buf;
+}
+
+static int memory_access_size(MemoryRegion *mr, unsigned l, hwaddr addr)
+{
+unsigned access_size_max = mr->ops->valid.max_access_size;
+
+/* Regions are assumed to support 1-4 byte accesses unless
+   otherwise specified.  */
+if (access_size_max == 0) {
+access_size_max = 4;
+}
+
+/* Bound the maximum access by the alignment of the address.  */
+if (!mr->ops->impl.unaligned) {
+unsigned align_size_max = addr & -addr;
+if (align_size_max != 0 && align_size_max < access_size_max) {
+access_size_max = align_size_max;
+}
+}
+
+/* Don't attempt accesses larger than the maximum.  */
+if (l > access_size_max) {
+l = access_size_max;
+}
+l = pow2floor(l);
+
+return l;
+}
+
+/*
+ * Call-back for functions that perform DMA reads from guest memory. Confirm
+ * that the region has not already been populated since the last loop in
+ * generic_fuzz(), avoiding potential race-conditions, which we don't have
+ * a good way for reproducing right now.
+ */
+void fuzz_dma_read_cb(size_t addr, size_t len, MemoryRegion *mr, bool is_write)
+{
+/* Are we in the generic-fuzzer or are we using another fuzz-target? */
+if (!qts_global) {
+return;
+}
+
+/*
+ * Return immediately if:
+ * - We have no DMA patterns defined
+ * - The length of the DMA r

[PATCH 18/22] vl: move semihosting command line fallback to qemu_finish_machine_init

[Paolo Bonzini]

Move more sane parts of the huge qemu_init function out of it.

Signed-off-by: Paolo Bonzini 
---
 softmmu/vl.c | 12 +---
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index 75e57133ad..e58572dbd8 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -3067,6 +3067,11 @@ void qemu_finish_machine_init(void)
 
 /* from here on runstate is RUN_STATE_PRELAUNCH */
 
+if (semihosting_enabled() && !semihosting_get_argc() && 
current_machine->kernel_filename) {
+/* fall back to the -kernel/-append */
+semihosting_arg_fallback(current_machine->kernel_filename, 
current_machine->kernel_cmdline);
+}
+
 if (machine_class->default_ram_id && current_machine->ram_size &&
 numa_uses_legacy_mem() && !current_machine->ram_memdev_id) {
 create_default_memdev(current_machine, mem_path);
@@ -4372,13 +4377,6 @@ void qemu_init(int argc, char **argv, char **envp)
 boot_order = machine_class->default_boot_order;
 }
 
-if (semihosting_enabled() && !semihosting_get_argc()) {
-const char *kernel_filename = qemu_opt_get(machine_opts, "kernel");
-const char *kernel_cmdline = qemu_opt_get(machine_opts, "append");
-/* fall back to the -kernel/-append */
-semihosting_arg_fallback(kernel_filename, kernel_cmdline);
-}
-
 if (net_init_clients(&err) < 0) {
 error_report_err(err);
 exit(1);
-- 
2.26.2

[PATCH 08/22] vl: split various early command line options to a separate function

[Paolo Bonzini]

Various options affect the global state of QEMU including the rest of
qemu_init, and they need to be called very early.  Group them together
in a function that is called at the beginning.

Signed-off-by: Paolo Bonzini 
---
 softmmu/vl.c | 202 ---
 1 file changed, 113 insertions(+), 89 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index 05e661abb8..2e1714d7a4 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -117,6 +117,7 @@
 
 #define MAX_VIRTIO_CONSOLES 1
 
+static const char *cpu_option;
 static const char *data_dir[16];
 static int data_dir_idx;
 const char *bios_name = NULL;
@@ -143,6 +144,9 @@ int vga_interface_type = VGA_NONE;
 static DisplayOptions dpy;
 static int num_serial_hds;
 static Chardev **serial_hds;
+static const char *log_mask = NULL;
+static const char *log_file = NULL;
+static bool list_data_dirs = false;
 Chardev *parallel_hds[MAX_PARALLEL_PORTS];
 int win2k_install_hack = 0;
 int singlestep = 0;
@@ -2859,6 +2863,106 @@ static char *find_datadir(void)
 return get_relocated_path(CONFIG_QEMU_DATADIR);
 }
 
+static void qemu_process_early_options(void)
+{
+char **dirs;
+int i;
+
+#ifdef CONFIG_SECCOMP
+QemuOptsList *olist = qemu_find_opts_err("sandbox", NULL);
+if (olist) {
+qemu_opts_foreach(olist, parse_sandbox, NULL, &error_fatal);
+}
+#endif
+
+qemu_opts_foreach(qemu_find_opts("name"),
+  parse_name, NULL, &error_fatal);
+
+#ifndef _WIN32
+qemu_opts_foreach(qemu_find_opts("add-fd"),
+  parse_add_fd, NULL, &error_fatal);
+
+qemu_opts_foreach(qemu_find_opts("add-fd"),
+  cleanup_add_fd, NULL, &error_fatal);
+#endif
+
+if (!trace_init_backends()) {
+exit(1);
+}
+trace_init_file();
+
+/* Open the logfile at this point and set the log mask if necessary.
+ */
+qemu_set_log_filename(log_file, &error_fatal);
+if (log_mask) {
+int mask;
+mask = qemu_str_to_log_mask(log_mask);
+if (!mask) {
+qemu_print_log_usage(stdout);
+exit(1);
+}
+qemu_set_log(mask);
+} else {
+qemu_set_log(0);
+}
+
+/* add configured firmware directories */
+dirs = g_strsplit(CONFIG_QEMU_FIRMWAREPATH, G_SEARCHPATH_SEPARATOR_S, 0);
+for (i = 0; dirs[i] != NULL; i++) {
+qemu_add_data_dir(get_relocated_path(dirs[i]));
+}
+g_strfreev(dirs);
+
+/* try to find datadir relative to the executable path */
+qemu_add_data_dir(find_datadir());
+}
+
+static void qemu_process_help_options(void)
+{
+int i;
+
+/*
+ * Check for -cpu help and -device help before we call select_machine(),
+ * which will return an error if the architecture has no default machine
+ * type and the user did not specify one, so that the user doesn't need
+ * to say '-cpu help -machine something'.
+ */
+if (cpu_option && is_help_option(cpu_option)) {
+list_cpus(cpu_option);
+exit(0);
+}
+
+if (qemu_opts_foreach(qemu_find_opts("device"),
+  device_help_func, NULL, NULL)) {
+exit(0);
+}
+
+/* -L help lists the data directories and exits. */
+if (list_data_dirs) {
+for (i = 0; i < data_dir_idx; i++) {
+printf("%s\n", data_dir[i]);
+}
+exit(0);
+}
+}
+
+static void qemu_maybe_daemonize(const char *pid_file)
+{
+Error *err;
+
+os_daemonize();
+rcu_disable_atfork();
+
+if (pid_file && !qemu_write_pidfile(pid_file, &err)) {
+error_reportf_err(err, "cannot create PID file: ");
+exit(1);
+}
+
+qemu_unlink_pidfile_notifier.notify = qemu_unlink_pidfile;
+qemu_add_exit_notifier(&qemu_unlink_pidfile_notifier);
+}
+
+
 void qemu_init(int argc, char **argv, char **envp)
 {
 int i;
@@ -2875,21 +2979,16 @@ void qemu_init(int argc, char **argv, char **envp)
 const char *optarg;
 const char *loadvm = NULL;
 MachineClass *machine_class;
-const char *cpu_option;
 const char *vga_model = NULL;
 const char *incoming = NULL;
 bool userconfig = true;
 bool nographic = false;
 int display_remote = 0;
-const char *log_mask = NULL;
-const char *log_file = NULL;
 ram_addr_t maxram_size;
 uint64_t ram_slots = 0;
 FILE *vmstate_dump_file = NULL;
 Error *main_loop_err = NULL;
 Error *err = NULL;
-bool list_data_dirs = false;
-char **dirs;
 const char *mem_path = NULL;
 bool have_custom_ram_size;
 BlockdevOptionsQueue bdo_queue = QSIMPLEQ_HEAD_INITIALIZER(bdo_queue);
@@ -3833,20 +3932,17 @@ void qemu_init(int argc, char **argv, char **envp)
  */
 loc_set_none();
 
-/*
- * Check for -cpu help and -device help before we call select_machine(),
- * which will return an error if the architecture has no default machine
- * type and the user did not specify one, so that the user doesn't need
- * to say 

[PATCH 22/22] vl: separate qemu_create_late_backends

[Paolo Bonzini]

"Late" backends are created after the machine.

Signed-off-by: Paolo Bonzini 
---
 softmmu/vl.c | 64 ++--
 1 file changed, 32 insertions(+), 32 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index 866df5bb7c..f36ec16cad 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -2751,11 +2751,41 @@ static void qemu_create_early_backends(void)
  * The remainder of object creation happens after the
  * creation of chardev, fsdev, net clients and device data types.
  */
-static bool object_create_delayed(const char *type, QemuOpts *opts)
+static bool object_create_late(const char *type, QemuOpts *opts)
 {
 return !object_create_early(type, opts);
 }
 
+static void qemu_create_late_backends(void)
+{
+if (qtest_chrdev) {
+qtest_server_init(qtest_chrdev, qtest_log, &error_fatal);
+}
+
+net_init_clients(&error_fatal);
+
+qemu_opts_foreach(qemu_find_opts("object"),
+  user_creatable_add_opts_foreach,
+  object_create_late, &error_fatal);
+
+if (tpm_init() < 0) {
+exit(1);
+}
+
+qemu_opts_foreach(qemu_find_opts("mon"),
+  mon_init_func, NULL, &error_fatal);
+
+/* now chardevs have been created we may have semihosting to connect */
+qemu_semihosting_connect_chardevs();
+qemu_semihosting_console_init();
+
+if (foreach_device_config(DEV_SERIAL, serial_parse) < 0)
+exit(1);
+if (foreach_device_config(DEV_PARALLEL, parallel_parse) < 0)
+exit(1);
+if (foreach_device_config(DEV_DEBUGCON, debugcon_parse) < 0)
+exit(1);
+}
 
 static bool set_memory_options(uint64_t *ram_slots, ram_addr_t *maxram_size,
MachineClass *mc)
@@ -3372,7 +3402,6 @@ void qemu_init(int argc, char **argv, char **envp)
 ram_addr_t maxram_size;
 uint64_t ram_slots = 0;
 FILE *vmstate_dump_file = NULL;
-Error *err = NULL;
 bool have_custom_ram_size;
 
 qemu_add_opts(&qemu_drive_opts);
@@ -4385,10 +4414,6 @@ void qemu_init(int argc, char **argv, char **envp)
  */
 migration_object_init();
 
-if (qtest_chrdev) {
-qtest_server_init(qtest_chrdev, qtest_log, &error_fatal);
-}
-
 opts = qemu_opts_find(qemu_find_opts("boot-opts"), NULL);
 if (opts) {
 boot_order = qemu_opt_get(opts, "order");
@@ -4409,32 +4434,7 @@ void qemu_init(int argc, char **argv, char **envp)
 boot_order = machine_class->default_boot_order;
 }
 
-if (net_init_clients(&err) < 0) {
-error_report_err(err);
-exit(1);
-}
-
-qemu_opts_foreach(qemu_find_opts("object"),
-  user_creatable_add_opts_foreach,
-  object_create_delayed, &error_fatal);
-
-if (tpm_init() < 0) {
-exit(1);
-}
-
-qemu_opts_foreach(qemu_find_opts("mon"),
-  mon_init_func, NULL, &error_fatal);
-
-/* now chardevs have been created we may have semihosting to connect */
-qemu_semihosting_connect_chardevs();
-qemu_semihosting_console_init();
-
-if (foreach_device_config(DEV_SERIAL, serial_parse) < 0)
-exit(1);
-if (foreach_device_config(DEV_PARALLEL, parallel_parse) < 0)
-exit(1);
-if (foreach_device_config(DEV_DEBUGCON, debugcon_parse) < 0)
-exit(1);
+qemu_create_late_backends();
 
 current_machine->boot_order = boot_order;
 
-- 
2.26.2

[PATCH 15/22] vl: extract various command line desugaring snippets to a new function

[Paolo Bonzini]

Signed-off-by: Paolo Bonzini 
---
 softmmu/vl.c | 39 +--
 1 file changed, 21 insertions(+), 18 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index 3607cd4357..122bf1821b 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -126,6 +126,7 @@ static const char *boot_once;
 static const char *incoming;
 static const char *loadvm;
 enum vga_retrace_method vga_retrace_method = VGA_RETRACE_DUMB;
+int mem_prealloc; /* force preallocation of physical target memory */
 int display_opengl;
 const char* keyboard_layout = NULL;
 ram_addr_t ram_size;
@@ -159,7 +160,7 @@ int fd_bootchk = 1;
 static int no_reboot;
 int no_shutdown = 0;
 int graphic_rotate = 0;
-const char *watchdog;
+static const char *watchdog;
 QEMUOptionRom option_rom[MAX_OPTION_ROMS];
 int nb_option_roms;
 int old_param = 0;
@@ -2906,6 +2907,24 @@ static void qemu_validate_options(void)
 #endif
 }
 
+static void qemu_process_sugar_options(void)
+{
+if (mem_prealloc) {
+char *val;
+
+val = g_strdup_printf("%ld", 
qemu_opt_get_number(qemu_find_opts_singleton("smp-opts"), "cpus", 1));
+object_register_sugar_prop("memory-backend", "prealloc-threads", val);
+g_free(val);
+object_register_sugar_prop("memory-backend", "prealloc", "on");
+}
+
+if (watchdog) {
+int i = select_watchdog(watchdog);
+if (i > 0)
+exit (i == 1 ? 1 : 0);
+}
+}
+
 static void qemu_process_early_options(void)
 {
 char **dirs;
@@ -3160,7 +3179,6 @@ void qemu_finish_machine_init(void)
 
 void qemu_init(int argc, char **argv, char **envp)
 {
-int i;
 int snapshot = 0;
 QemuOpts *opts, *machine_opts;
 QemuOpts *icount_opts = NULL, *accel_opts = NULL;
@@ -3179,7 +3197,6 @@ void qemu_init(int argc, char **argv, char **envp)
 bool have_custom_ram_size;
 BlockdevOptionsQueue bdo_queue = QSIMPLEQ_HEAD_INITIALIZER(bdo_queue);
 QemuPluginList plugin_list = QTAILQ_HEAD_INITIALIZER(plugin_list);
-int mem_prealloc = 0; /* force preallocation of physical target memory */
 
 qemu_add_opts(&qemu_drive_opts);
 qemu_add_drive_opts(&qemu_legacy_drive_opts);
@@ -4086,6 +4103,7 @@ void qemu_init(int argc, char **argv, char **envp)
 loc_set_none();
 
 qemu_validate_options();
+qemu_process_sugar_options();
 
 /* These options affect everything else and should be processed
  * before daemonizing.
@@ -4137,15 +4155,6 @@ void qemu_init(int argc, char **argv, char **envp)
 machine_smp_parse(current_machine,
 qemu_opts_find(qemu_find_opts("smp-opts"), NULL), &error_fatal);
 
-if (mem_prealloc) {
-char *val;
-
-val = g_strdup_printf("%d", current_machine->smp.cpus);
-object_register_sugar_prop("memory-backend", "prealloc-threads", val);
-g_free(val);
-object_register_sugar_prop("memory-backend", "prealloc", "on");
-}
-
 /*
  * Get the default machine options from the machine if it is not already
  * specified either by the configuration file or by the command line.
@@ -4404,12 +4413,6 @@ void qemu_init(int argc, char **argv, char **envp)
 select_vgahw(machine_class, vga_model);
 }
 
-if (watchdog) {
-i = select_watchdog(watchdog);
-if (i > 0)
-exit (i == 1 ? 1 : 0);
-}
-
 /* This checkpoint is required by replay to separate prior clock
reading from the other reads, because timer polling functions query
clock values from the log. */
-- 
2.26.2

[PATCH 09/22] vl: move various initialization routines out of qemu_init

[Paolo Bonzini]

Some very simple initialization routines can be nested in existing
subsystem-level functions, do that to simplify qemu_init.

Signed-off-by: Paolo Bonzini 
---
 hw/core/machine.c  | 3 +++
 include/hw/qdev-core.h | 8 
 migration/migration.c  | 4 
 softmmu/qdev-monitor.c | 6 --
 softmmu/vl.c   | 5 -
 5 files changed, 7 insertions(+), 19 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index 70b1e5e8e8..3c674bb05e 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -873,6 +873,9 @@ static void machine_initfn(Object *obj)
 MachineState *ms = MACHINE(obj);
 MachineClass *mc = MACHINE_GET_CLASS(obj);
 
+container_get(obj, "/peripheral");
+container_get(obj, "/peripheral-anon");
+
 ms->dump_guest_core = true;
 ms->mem_merge = true;
 ms->enable_graphics = true;
diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
index 868973319e..56ce7f2d7f 100644
--- a/include/hw/qdev-core.h
+++ b/include/hw/qdev-core.h
@@ -769,14 +769,6 @@ BusState *sysbus_get_default(void);
 char *qdev_get_fw_dev_path(DeviceState *dev);
 char *qdev_get_own_fw_dev_path_from_handler(BusState *bus, DeviceState *dev);
 
-/**
- * @qdev_machine_init
- *
- * Initialize platform devices before machine init.  This is a hack until full
- * support for composition is added.
- */
-void qdev_machine_init(void);
-
 /**
  * device_legacy_reset:
  *
diff --git a/migration/migration.c b/migration/migration.c
index deb6005b8d..f48b03cac2 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -169,6 +169,10 @@ void migration_object_init(void)
 error_report_err(err);
 exit(1);
 }
+
+blk_mig_init();
+ram_mig_init();
+dirty_bitmap_mig_init();
 }
 
 void migration_shutdown(void)
diff --git a/softmmu/qdev-monitor.c b/softmmu/qdev-monitor.c
index bcfb90a08f..bcfcbac181 100644
--- a/softmmu/qdev-monitor.c
+++ b/softmmu/qdev-monitor.c
@@ -943,12 +943,6 @@ BlockBackend *blk_by_qdev_id(const char *id, Error **errp)
 return blk;
 }
 
-void qdev_machine_init(void)
-{
-qdev_get_peripheral_anon();
-qdev_get_peripheral();
-}
-
 QemuOptsList qemu_device_opts = {
 .name = "device",
 .implied_opt_name = "driver",
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 2e1714d7a4..a9f69a7d11 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -4269,10 +4269,6 @@ void qemu_init(int argc, char **argv, char **envp)
 exit(1);
 }
 
-blk_mig_init();
-ram_mig_init();
-dirty_bitmap_mig_init();
-
 qemu_opts_foreach(qemu_find_opts("mon"),
   mon_init_func, NULL, &error_fatal);
 
@@ -4305,7 +4301,6 @@ void qemu_init(int argc, char **argv, char **envp)
reading from the other reads, because timer polling functions query
clock values from the log. */
 replay_checkpoint(CHECKPOINT_INIT);
-qdev_machine_init();
 
 current_machine->boot_order = boot_order;
 
-- 
2.26.2

[PATCH 14/22] vl: preconfig and loadvm are mutually exclusive

[Paolo Bonzini]

Just like -incoming.

Signed-off-by: Paolo Bonzini 
---
 softmmu/vl.c | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index f8b1ffb46e..3607cd4357 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -124,6 +124,7 @@ static const char *mem_path;
 static const char *boot_order;
 static const char *boot_once;
 static const char *incoming;
+static const char *loadvm;
 enum vga_retrace_method vga_retrace_method = VGA_RETRACE_DUMB;
 int display_opengl;
 const char* keyboard_layout = NULL;
@@ -2886,6 +2887,11 @@ static void qemu_validate_options(void)
   }
 }
 
+if (loadvm && !preconfig_exit_requested) {
+error_report("'preconfig' and 'loadvm' options are "
+ "mutually exclusive");
+exit(EXIT_FAILURE);
+}
 if (incoming && !preconfig_exit_requested) {
 error_report("'preconfig' and 'incoming' options are "
  "mutually exclusive");
@@ -3161,7 +3167,6 @@ void qemu_init(int argc, char **argv, char **envp)
 QemuOptsList *olist;
 int optind;
 const char *optarg;
-const char *loadvm = NULL;
 MachineClass *machine_class;
 const char *vga_model = NULL;
 bool userconfig = true;
-- 
2.26.2

[PATCH 07/22] trace: remove argument from trace_init_file

[Paolo Bonzini]

It is not needed, all the callers are just saving what was
retrieved from -trace and trace_init_file can retrieve it
on its own.

Signed-off-by: Paolo Bonzini 
---
 bsd-user/main.c  |  6 ++
 linux-user/main.c|  6 ++
 qemu-img.c   |  6 ++
 qemu-io.c|  6 ++
 qemu-nbd.c   |  6 ++
 scsi/qemu-pr-helper.c|  6 ++
 softmmu/vl.c |  6 ++
 storage-daemon/qemu-storage-daemon.c |  9 +++--
 trace/control.c  | 10 --
 trace/control.h  | 12 +++-
 10 files changed, 24 insertions(+), 49 deletions(-)

diff --git a/bsd-user/main.c b/bsd-user/main.c
index ac40d79bfa..0a918e8f74 100644
--- a/bsd-user/main.c
+++ b/bsd-user/main.c
@@ -745,7 +745,6 @@ int main(int argc, char **argv)
 const char *gdbstub = NULL;
 char **target_environ, **wrk;
 envlist_t *envlist = NULL;
-char *trace_file = NULL;
 bsd_type = target_openbsd;
 
 if (argc <= 1)
@@ -851,8 +850,7 @@ int main(int argc, char **argv)
 } else if (!strcmp(r, "strace")) {
 do_strace = 1;
 } else if (!strcmp(r, "trace")) {
-g_free(trace_file);
-trace_file = trace_opt_parse(optarg);
+trace_opt_parse(optarg);
 } else {
 usage();
 }
@@ -880,7 +878,7 @@ int main(int argc, char **argv)
 if (!trace_init_backends()) {
 exit(1);
 }
-trace_init_file(trace_file);
+trace_init_file();
 
 /* Zero out regs */
 memset(regs, 0, sizeof(struct target_pt_regs));
diff --git a/linux-user/main.c b/linux-user/main.c
index 75c9785157..24d1eb73ad 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -386,11 +386,9 @@ static void handle_arg_version(const char *arg)
 exit(EXIT_SUCCESS);
 }
 
-static char *trace_file;
 static void handle_arg_trace(const char *arg)
 {
-g_free(trace_file);
-trace_file = trace_opt_parse(arg);
+trace_opt_parse(arg);
 }
 
 #if defined(TARGET_XTENSA)
@@ -672,7 +670,7 @@ int main(int argc, char **argv, char **envp)
 if (!trace_init_backends()) {
 exit(1);
 }
-trace_init_file(trace_file);
+trace_init_file();
 if (qemu_plugin_load_list(&plugins)) {
 exit(1);
 }
diff --git a/qemu-img.c b/qemu-img.c
index 2103507936..ccfdf1f48d 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -5434,7 +5434,6 @@ int main(int argc, char **argv)
 const img_cmd_t *cmd;
 const char *cmdname;
 Error *local_error = NULL;
-char *trace_file = NULL;
 int c;
 static const struct option long_options[] = {
 {"help", no_argument, 0, 'h'},
@@ -5484,8 +5483,7 @@ int main(int argc, char **argv)
 printf(QEMU_IMG_VERSION);
 return 0;
 case 'T':
-g_free(trace_file);
-trace_file = trace_opt_parse(optarg);
+trace_opt_parse(optarg);
 break;
 }
 }
@@ -5503,7 +5501,7 @@ int main(int argc, char **argv)
 if (!trace_init_backends()) {
 exit(1);
 }
-trace_init_file(trace_file);
+trace_init_file();
 qemu_set_log(LOG_TRACE);
 
 /* find the command */
diff --git a/qemu-io.c b/qemu-io.c
index 7cc832b3d6..ac88d8bd40 100644
--- a/qemu-io.c
+++ b/qemu-io.c
@@ -536,7 +536,6 @@ int main(int argc, char **argv)
 Error *local_error = NULL;
 QDict *opts = NULL;
 const char *format = NULL;
-char *trace_file = NULL;
 bool force_share = false;
 
 #ifdef CONFIG_POSIX
@@ -601,8 +600,7 @@ int main(int argc, char **argv)
 }
 break;
 case 'T':
-g_free(trace_file);
-trace_file = trace_opt_parse(optarg);
+trace_opt_parse(optarg);
 break;
 case 'V':
 printf("%s version " QEMU_FULL_VERSION "\n"
@@ -653,7 +651,7 @@ int main(int argc, char **argv)
 if (!trace_init_backends()) {
 exit(1);
 }
-trace_init_file(trace_file);
+trace_init_file();
 qemu_set_log(LOG_TRACE);
 
 /* initialize commands */
diff --git a/qemu-nbd.c b/qemu-nbd.c
index bc644a0670..37adeb0596 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -573,7 +573,6 @@ int main(int argc, char **argv)
 const char *tlscredsid = NULL;
 bool imageOpts = false;
 bool writethrough = true;
-char *trace_file = NULL;
 bool fork_process = false;
 bool list = false;
 int old_stderr = -1;
@@ -767,8 +766,7 @@ int main(int argc, char **argv)
 imageOpts = true;
 break;
 case 'T':
-g_free(trace_file);
-trace_file = trace_opt_parse(optarg);
+trace_opt_parse(optarg);
 break;
 case QEMU_NBD_OPT_TLSAUTHZ:
 tlsauthz = optarg;
@@ -815,7 +813,7 @@ int main(int argc, char **argv)
 if (!trace_init_backends()) {
 exit(1);
 }
-trace_init_file(trac

[PATCH 17/22] vl: load plugins as late as possible

[Paolo Bonzini]

There is no need to load plugins in the middle of default device processing,
move -plugin handling just before preconfig is entered.

Signed-off-by: Paolo Bonzini 
---
 softmmu/vl.c | 15 ---
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index 8577667b8f..75e57133ad 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -125,6 +125,7 @@ static const char *boot_order;
 static const char *boot_once;
 static const char *incoming;
 static const char *loadvm;
+static QemuPluginList plugin_list = QTAILQ_HEAD_INITIALIZER(plugin_list);
 enum vga_retrace_method vga_retrace_method = VGA_RETRACE_DUMB;
 int mem_prealloc; /* force preallocation of physical target memory */
 int display_opengl;
@@ -3064,12 +3065,18 @@ void qemu_finish_machine_init(void)
 MachineClass *machine_class = MACHINE_GET_CLASS(current_machine);
 DisplayState *ds;
 
+/* from here on runstate is RUN_STATE_PRELAUNCH */
+
 if (machine_class->default_ram_id && current_machine->ram_size &&
 numa_uses_legacy_mem() && !current_machine->ram_memdev_id) {
 create_default_memdev(current_machine, mem_path);
 }
 
-/* from here on runstate is RUN_STATE_PRELAUNCH */
+/* process plugin before CPUs are created, but once -smp has been parsed */
+if (qemu_plugin_load_list(&plugin_list)) {
+exit(1);
+}
+
 machine_run_board_init(current_machine);
 
 /*
@@ -3196,7 +3203,6 @@ void qemu_init(int argc, char **argv, char **envp)
 Error *err = NULL;
 bool have_custom_ram_size;
 BlockdevOptionsQueue bdo_queue = QSIMPLEQ_HEAD_INITIALIZER(bdo_queue);
-QemuPluginList plugin_list = QTAILQ_HEAD_INITIALIZER(plugin_list);
 
 qemu_add_opts(&qemu_drive_opts);
 qemu_add_drive_opts(&qemu_legacy_drive_opts);
@@ -4164,11 +4170,6 @@ void qemu_init(int argc, char **argv, char **envp)
machine_class->default_machine_opts, 0);
 }
 
-/* process plugin before CPUs are created, but once -smp has been parsed */
-if (qemu_plugin_load_list(&plugin_list)) {
-exit(1);
-}
-
 qemu_opts_foreach(qemu_find_opts("device"),
   default_driver_check, NULL, NULL);
 qemu_opts_foreach(qemu_find_opts("global"),
-- 
2.26.2

[PATCH 20/22] vl: move CHECKPOINT_INIT after preconfig

[Paolo Bonzini]

Move CHECKPOINT_INIT right before the machine initialization is
completed.  Everything before is essentially an extension of
command line parsing.

Signed-off-by: Paolo Bonzini 
---
 softmmu/vl.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index 0a6f47e7d6..e9391929f6 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -3194,6 +3194,11 @@ void qemu_finish_machine_init(void)
 exit(1);
 }
 
+/* This checkpoint is required by replay to separate prior clock
+   reading from the other reads, because timer polling functions query
+   clock values from the log. */
+replay_checkpoint(CHECKPOINT_INIT);
+
 machine_run_board_init(current_machine);
 
 /*
@@ -4424,11 +4429,6 @@ void qemu_init(int argc, char **argv, char **envp)
 if (foreach_device_config(DEV_DEBUGCON, debugcon_parse) < 0)
 exit(1);
 
-/* This checkpoint is required by replay to separate prior clock
-   reading from the other reads, because timer polling functions query
-   clock values from the log. */
-replay_checkpoint(CHECKPOINT_INIT);
-
 current_machine->boot_order = boot_order;
 
 /* parse features once if machine provides default cpu_type */
-- 
2.26.2

[PATCH 05/22] vl: extract validation of -smp to machine.c

[Paolo Bonzini]

Once smp_parse is done, the validation operates on the MachineState.
There is no reason for that code to be in vl.c.

Signed-off-by: Paolo Bonzini 
---
 hw/core/machine.c   | 23 +++
 include/hw/boards.h |  1 +
 softmmu/vl.c| 20 ++--
 3 files changed, 26 insertions(+), 18 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index 7efeac03b3..70b1e5e8e8 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -1073,6 +1073,29 @@ MemoryRegion *machine_consume_memdev(MachineState 
*machine,
 return ret;
 }
 
+bool machine_smp_parse(MachineState *ms, QemuOpts *opts, Error **errp)
+{
+MachineClass *mc = MACHINE_GET_CLASS(ms);
+
+mc->smp_parse(ms, opts);
+
+/* sanity-check smp_cpus and max_cpus against mc */
+if (ms->smp.cpus < mc->min_cpus) {
+error_setg(errp, "Invalid SMP CPUs %d. The min CPUs "
+   "supported by machine '%s' is %d",
+   ms->smp.cpus,
+   mc->name, mc->min_cpus);
+return false;
+} else if (ms->smp.max_cpus > mc->max_cpus) {
+error_setg(errp, "Invalid SMP CPUs %d. The max CPUs "
+   "supported by machine '%s' is %d",
+   current_machine->smp.max_cpus,
+   mc->name, mc->max_cpus);
+return false;
+}
+return true;
+}
+
 void machine_run_board_init(MachineState *machine)
 {
 MachineClass *machine_class = MACHINE_GET_CLASS(machine);
diff --git a/include/hw/boards.h b/include/hw/boards.h
index a49e3a6b44..4537cfb5c6 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -26,6 +26,7 @@ OBJECT_DECLARE_TYPE(MachineState, MachineClass, MACHINE)
 extern MachineState *current_machine;
 
 void machine_run_board_init(MachineState *machine);
+bool machine_smp_parse(MachineState *ms, QemuOpts *opts, Error **errp);
 bool machine_usb(MachineState *machine);
 int machine_phandle_start(MachineState *machine);
 bool machine_dump_guest_core(MachineState *machine);
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 05422a15ee..8a7d11ef05 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -3966,24 +3966,8 @@ void qemu_init(int argc, char **argv, char **envp)
 exit(0);
 }
 
-machine_class->smp_parse(current_machine,
-qemu_opts_find(qemu_find_opts("smp-opts"), NULL));
-
-/* sanity-check smp_cpus and max_cpus against machine_class */
-if (current_machine->smp.cpus < machine_class->min_cpus) {
-error_report("Invalid SMP CPUs %d. The min CPUs "
- "supported by machine '%s' is %d",
- current_machine->smp.cpus,
- machine_class->name, machine_class->min_cpus);
-exit(1);
-}
-if (current_machine->smp.max_cpus > machine_class->max_cpus) {
-error_report("Invalid SMP CPUs %d. The max CPUs "
- "supported by machine '%s' is %d",
- current_machine->smp.max_cpus,
- machine_class->name, machine_class->max_cpus);
-exit(1);
-}
+machine_smp_parse(current_machine,
+qemu_opts_find(qemu_find_opts("smp-opts"), NULL), &error_fatal);
 
 if (mem_prealloc) {
 char *val;
-- 
2.26.2

[PATCH 04/22] machine: move SMP initialization from vl.c

[Paolo Bonzini]

Initialize the object's values from the class when the object is
created, no need to have vl.c do it for us.

Signed-off-by: Paolo Bonzini 
---
 hw/core/machine.c | 7 +++
 softmmu/vl.c  | 7 ---
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index d3a8450b1f..7efeac03b3 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -908,6 +908,13 @@ static void machine_initfn(Object *obj)
 /* Register notifier when init is done for sysbus sanity checks */
 ms->sysbus_notifier.notify = machine_init_notify;
 qemu_add_machine_init_done_notifier(&ms->sysbus_notifier);
+
+/* default to mc->default_cpus */
+ms->smp.cpus = mc->default_cpus;
+ms->smp.max_cpus = mc->default_cpus;
+ms->smp.cores = 1;
+ms->smp.threads = 1;
+ms->smp.sockets = 1;
 }
 
 static void machine_finalize(Object *obj)
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 75bc686397..05422a15ee 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -3966,13 +3966,6 @@ void qemu_init(int argc, char **argv, char **envp)
 exit(0);
 }
 
-/* default to machine_class->default_cpus */
-current_machine->smp.cpus = machine_class->default_cpus;
-current_machine->smp.max_cpus = machine_class->default_cpus;
-current_machine->smp.cores = 1;
-current_machine->smp.threads = 1;
-current_machine->smp.sockets = 1;
-
 machine_class->smp_parse(current_machine,
 qemu_opts_find(qemu_find_opts("smp-opts"), NULL));
 
-- 
2.26.2

[PATCH 02/22] machine: remove deprecated -machine enforce-config-section option

[Paolo Bonzini]

Deprecated since 3.1 and complicates the initialization sequence,
remove it.

Signed-off-by: Paolo Bonzini 
---
 docs/system/deprecated.rst | 12 ++--
 hw/core/machine.c  | 24 +---
 include/hw/boards.h|  1 -
 migration/migration.c  | 10 --
 qemu-options.hx|  8 
 5 files changed, 7 insertions(+), 48 deletions(-)

diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst
index 895433c356..0ebce37a19 100644
--- a/docs/system/deprecated.rst
+++ b/docs/system/deprecated.rst
@@ -21,12 +21,6 @@ deprecated.
 System emulator command line arguments
 --
 
-``-machine enforce-config-section=on|off`` (since 3.1)
-''
-
-The ``enforce-config-section`` parameter is replaced by the
-``-global migration.send-configuration={on|off}`` option.
-
 ``-usbdevice`` (since 2.10.0)
 '
 
@@ -689,6 +683,12 @@ Support for invalid topologies is removed, the user must 
ensure
 topologies described with -smp include all possible cpus, i.e.
 *sockets* * *cores* * *threads* = *maxcpus*.
 
+``-machine enforce-config-section=on|off`` (removed 5.2)
+
+
+The ``enforce-config-section`` property was replaced by the
+``-global migration.send-configuration={on|off}`` option.
+
 Block devices
 -
 
diff --git a/hw/core/machine.c b/hw/core/machine.c
index d740a7e963..80a918895a 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -26,6 +26,7 @@
 #include "sysemu/qtest.h"
 #include "hw/pci/pci.h"
 #include "hw/mem/nvdimm.h"
+#include "migration/misc.h"
 #include "migration/vmstate.h"
 
 GlobalProperty hw_compat_5_1[] = {
@@ -411,24 +412,6 @@ static bool machine_get_suppress_vmdesc(Object *obj, Error 
**errp)
 return ms->suppress_vmdesc;
 }
 
-static void machine_set_enforce_config_section(Object *obj, bool value,
- Error **errp)
-{
-MachineState *ms = MACHINE(obj);
-
-warn_report("enforce-config-section is deprecated, please use "
-"-global migration.send-configuration=on|off instead");
-
-ms->enforce_config_section = value;
-}
-
-static bool machine_get_enforce_config_section(Object *obj, Error **errp)
-{
-MachineState *ms = MACHINE(obj);
-
-return ms->enforce_config_section;
-}
-
 static char *machine_get_memory_encryption(Object *obj, Error **errp)
 {
 MachineState *ms = MACHINE(obj);
@@ -857,11 +840,6 @@ static void machine_class_init(ObjectClass *oc, void *data)
 object_class_property_set_description(oc, "suppress-vmdesc",
 "Set on to disable self-describing migration");
 
-object_class_property_add_bool(oc, "enforce-config-section",
-machine_get_enforce_config_section, 
machine_set_enforce_config_section);
-object_class_property_set_description(oc, "enforce-config-section",
-"Set on to enforce configuration section migration");
-
 object_class_property_add_str(oc, "memory-encryption",
 machine_get_memory_encryption, machine_set_memory_encryption);
 object_class_property_set_description(oc, "memory-encryption",
diff --git a/include/hw/boards.h b/include/hw/boards.h
index bf53e8a16e..a49e3a6b44 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -268,7 +268,6 @@ struct MachineState {
 char *firmware;
 bool iommu;
 bool suppress_vmdesc;
-bool enforce_config_section;
 bool enable_graphics;
 char *memory_encryption;
 char *ram_memdev_id;
diff --git a/migration/migration.c b/migration/migration.c
index 0575ecb379..deb6005b8d 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -145,7 +145,6 @@ static void migrate_fd_cancel(MigrationState *s);
 
 void migration_object_init(void)
 {
-MachineState *ms = MACHINE(qdev_get_machine());
 Error *err = NULL;
 
 /* This can only be called once. */
@@ -170,15 +169,6 @@ void migration_object_init(void)
 error_report_err(err);
 exit(1);
 }
-
-/*
- * We cannot really do this in migration_instance_init() since at
- * that time global properties are not yet applied, then this
- * value will be definitely replaced by something else.
- */
-if (ms->enforce_config_section) {
-current_migration->send_configuration = true;
-}
 }
 
 void migration_shutdown(void)
diff --git a/qemu-options.hx b/qemu-options.hx
index 9e1ace04f7..2c83390504 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -34,7 +34,6 @@ DEF("machine", HAS_ARG, QEMU_OPTION_machine, \
 "dea-key-wrap=on|off controls support for DEA key wrapping 
(default=on)\n"
 "suppress-vmdesc=on|off disables self-describing migration 
(default=off)\n"
 "nvdimm=on|off controls NVDIMM support (default=off)\n"
-"enforce-config-section=on|off enforce configuration 
section mig

[PATCH 01/22] semihosting: fix order of initialization functions

[Paolo Bonzini]

qemu_semihosting_console_init uses semihosting.chardev which is set
by qemu_semihosting_connect_chardevs.  Thus qemu_semihosting_connect_chardevs
has to be called first.

Signed-off-by: Paolo Bonzini 
---
 softmmu/vl.c | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index 6f5b000f07..42314e6ff9 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -4288,7 +4288,8 @@ void qemu_init(int argc, char **argv, char **envp)
 qemu_opts_foreach(qemu_find_opts("mon"),
   mon_init_func, NULL, &error_fatal);
 
-/* connect semihosting console input if requested */
+/* now chardevs have been created we may have semihosting to connect */
+qemu_semihosting_connect_chardevs();
 qemu_semihosting_console_init();
 
 if (foreach_device_config(DEV_SERIAL, serial_parse) < 0)
@@ -4298,9 +4299,6 @@ void qemu_init(int argc, char **argv, char **envp)
 if (foreach_device_config(DEV_DEBUGCON, debugcon_parse) < 0)
 exit(1);
 
-/* now chardevs have been created we may have semihosting to connect */
-qemu_semihosting_connect_chardevs();
-
 /* If no default VGA is requested, the default is "none".  */
 if (default_vga) {
 vga_model = get_default_vga_model(machine_class);
-- 
2.26.2