RE: [PATCH v16 00/10] VIRTIO-IOMMU device

[Tian, Kevin]

> From: Jean-Philippe Brucker 
> Sent: Thursday, March 5, 2020 3:34 PM
> 
> On Thu, Mar 05, 2020 at 02:56:20AM +, Tian, Kevin wrote:
> > > From: Jean-Philippe Brucker 
> > > Sent: Thursday, March 5, 2020 12:47 AM
> > >
> > [...]
> > > > >
> > > > > * We can't use DVM in nested mode unless the VMID is shared with
> the
> > > > > CPU. For that we'll need the host SMMU driver to hook into the KVM
> > > VMID
> > > > > allocator, just like we do for the ASID allocator. I haven't yet
> > > > > investigated how to do that. It's possible to do vSVA without DVM
> > > > > though, by sending all TLB invalidations through the SMMU command
> > > queue.
> > > > > "
> > >
> > > Hm we're already mandating DVM for host SVA, so I'd say mandate it for
> > > vSVA as well. We'd avoid a ton of context switches, especially for the zip
> > > accelerator which doesn't require ATC invalidations. The host needs to
> pin
> > > the VMID allocated by KVM and write it in the endpoint's STE.
> > >
> >
> > Curious... what is DVM and how is it related to SVA? Is it SMMU specific?
> 
> Yes it stands for "Distributed Virtual Memory", an Arm interconnect
> protocol. When sharing a process address space, TLB invalidations from the
> CPU are broadcasted to the SMMU, so we don't have to send commands
> through
> the SMMU queue to invalidate IOTLBs. However ATCs from PCIe endpoints
> do
> not participate in DVM and still have to be invalidated by hand.
> 

ah, got it. Thanks for explanation!

Re: [PATCH v2] hw/char/pl011: Enable TxFIFO and async transmission

[Gavin Shan]

On 2/24/20 2:13 PM, Gavin Shan wrote:

The depth of TxFIFO can be 1 or 16 depending on LCR[4]. The TxFIFO is
disabled when its depth is 1. It's nice to have TxFIFO enabled if
possible because more characters can be piled and transmitted at once,
which would have less overhead. Besides, we can be blocked because of
qemu_chr_fe_write_all(), which isn't nice.

This enables TxFIFO if possible. On ther other hand, the asynchronous
transmission is enabled if needed, as we did in hw/char/cadence_uart.c

Signed-off-by: Gavin Shan 
---
v2: Put write_{count,fifo} into migration subsection
 Don't start async IO handle if it has been started, to avoid race
 Update with PL011_FLAG_{TXFF,TXFE} on changing write_count
---
  hw/char/pl011.c | 105 +---
  include/hw/char/pl011.h |   3 ++
  2 files changed, 102 insertions(+), 6 deletions(-)



ping? :)


diff --git a/hw/char/pl011.c b/hw/char/pl011.c
index 13e784f9d9..de5c4254fe 100644
--- a/hw/char/pl011.c
+++ b/hw/char/pl011.c
@@ -169,6 +169,73 @@ static void pl011_set_read_trigger(PL011State *s)
  s->read_trigger = 1;
  }
  
+static gboolean pl011_xmit(GIOChannel *chan, GIOCondition cond, void *opaque)

+{
+PL011State *s = (PL011State *)opaque;
+int ret;
+
+/* Drain FIFO if there is no backend */
+if (!qemu_chr_fe_backend_connected(&s->chr)) {
+s->write_count = 0;
+s->flags &= ~PL011_FLAG_TXFF;
+s->flags |= PL011_FLAG_TXFE;
+return FALSE;
+}
+
+/* Nothing to do */
+if (!s->write_count) {
+return FALSE;
+}
+
+ret = qemu_chr_fe_write(&s->chr, s->write_fifo, s->write_count);
+if (ret > 0) {
+s->write_count -= ret;
+memmove(s->write_fifo, s->write_fifo + ret, s->write_count);
+s->flags &= ~PL011_FLAG_TXFF;
+if (!s->write_count) {
+s->flags |= PL011_FLAG_TXFE;
+}
+}
+
+if (s->write_count) {
+s->watch_tag = qemu_chr_fe_add_watch(&s->chr, G_IO_OUT | G_IO_HUP,
+ pl011_xmit, s);
+if (!s->watch_tag) {
+s->write_count = 0;
+s->flags &= ~PL011_FLAG_TXFF;
+s->flags |= PL011_FLAG_TXFE;
+return FALSE;
+}
+}
+
+s->int_level |= PL011_INT_TX;
+pl011_update(s);
+return FALSE;
+}
+
+static void pl011_write_fifo(void *opaque, const unsigned char *buf, int size)
+{
+PL011State *s = (PL011State *)opaque;
+int depth = (s->lcr & 0x10) ? 16 : 1;
+
+if (size >= (depth - s->write_count)) {
+size = depth - s->write_count;
+}
+
+if (size > 0) {
+memcpy(s->write_fifo + s->write_count, buf, size);
+s->write_count += size;
+if (s->write_count >= depth) {
+s->flags |= PL011_FLAG_TXFF;
+}
+s->flags &= ~PL011_FLAG_TXFE;
+}
+
+if (!s->watch_tag) {
+pl011_xmit(NULL, G_IO_OUT, s);
+}
+}
+
  static void pl011_write(void *opaque, hwaddr offset,
  uint64_t value, unsigned size)
  {
@@ -179,13 +246,8 @@ static void pl011_write(void *opaque, hwaddr offset,
  
  switch (offset >> 2) {

  case 0: /* UARTDR */
-/* ??? Check if transmitter is enabled.  */
  ch = value;
-/* XXX this blocks entire thread. Rewrite to use
- * qemu_chr_fe_write and background I/O callbacks */
-qemu_chr_fe_write_all(&s->chr, &ch, 1);
-s->int_level |= PL011_INT_TX;
-pl011_update(s);
+pl011_write_fifo(opaque, &ch, 1);
  break;
  case 1: /* UARTRSR/UARTECR */
  s->rsr = 0;
@@ -207,7 +269,16 @@ static void pl011_write(void *opaque, hwaddr offset,
  if ((s->lcr ^ value) & 0x10) {
  s->read_count = 0;
  s->read_pos = 0;
+
+if (s->watch_tag) {
+g_source_remove(s->watch_tag);
+s->watch_tag = 0;
+}
+s->write_count = 0;
+s->flags &= ~PL011_FLAG_TXFF;
+s->flags |= PL011_FLAG_TXFE;
  }
+
  s->lcr = value;
  pl011_set_read_trigger(s);
  break;
@@ -292,6 +363,24 @@ static const MemoryRegionOps pl011_ops = {
  .endianness = DEVICE_NATIVE_ENDIAN,
  };
  
+static bool pl011_write_fifo_needed(void *opaque)

+{
+PL011State *s = (PL011State *)opaque;
+return s->write_count > 0;
+}
+
+static const VMStateDescription vmstate_pl011_write_fifo = {
+.name = "pl011/write_fifo",
+.version_id = 1,
+.minimum_version_id = 1,
+.needed = pl011_write_fifo_needed,
+.fields = (VMStateField[]) {
+VMSTATE_INT32(write_count, PL011State),
+VMSTATE_UINT8_ARRAY(write_fifo, PL011State, 16),
+VMSTATE_END_OF_LIST()
+}
+};
+
  static const VMStateDescription vmstate_pl011 = {
  .name = "pl011",
  .version_id = 2,
@@ -314,6 +403,10 @@ static const VMStateDescription vmstate_pl011 = {
  VMSTATE_INT32(read_count, PL011Stat

Re: [PATCH v16 00/10] VIRTIO-IOMMU device

[Jean-Philippe Brucker]

On Thu, Mar 05, 2020 at 02:56:20AM +, Tian, Kevin wrote:
> > From: Jean-Philippe Brucker 
> > Sent: Thursday, March 5, 2020 12:47 AM
> >
> [...]
> > > >
> > > > * We can't use DVM in nested mode unless the VMID is shared with the
> > > > CPU. For that we'll need the host SMMU driver to hook into the KVM
> > VMID
> > > > allocator, just like we do for the ASID allocator. I haven't yet
> > > > investigated how to do that. It's possible to do vSVA without DVM
> > > > though, by sending all TLB invalidations through the SMMU command
> > queue.
> > > > "
> > 
> > Hm we're already mandating DVM for host SVA, so I'd say mandate it for
> > vSVA as well. We'd avoid a ton of context switches, especially for the zip
> > accelerator which doesn't require ATC invalidations. The host needs to pin
> > the VMID allocated by KVM and write it in the endpoint's STE.
> > 
> 
> Curious... what is DVM and how is it related to SVA? Is it SMMU specific?

Yes it stands for "Distributed Virtual Memory", an Arm interconnect
protocol. When sharing a process address space, TLB invalidations from the
CPU are broadcasted to the SMMU, so we don't have to send commands through
the SMMU queue to invalidate IOTLBs. However ATCs from PCIe endpoints do
not participate in DVM and still have to be invalidated by hand.

Thanks,
Jean

Re: [PATCH v4 2/3] mac_via: fix incorrect creation of mos6522 device in mac_via

[Pan Nengyuan]

On 3/5/2020 2:54 PM, Pan Nengyuan wrote:
> This patch fix a bug in mac_via where it failed to actually realize devices 
> it was using.
> And move the init codes which inits the mos6522 objects and properties on 
> them from realize()
> into init(). However, we keep qdev_set_parent_bus in realize(), otherwise it 
> will cause
> device-introspect-test test fail. Then do the realize mos6522 device in the 
> mac_vir_realize.
> 
> Signed-off-by: Pan Nengyuan 
> ---
> Cc: Laurent Vivier 
> Cc: Mark Cave-Ayland 
> ---
> v4->v3:
> - split v3 into two patches, this patch fix incorrect creation of mos6522, 
> move inits and props
>   from realize into init. The v3 is:
>   https://patchwork.kernel.org/patch/11407635/
> ---
>  hw/misc/mac_via.c | 43 ++-
>  1 file changed, 30 insertions(+), 13 deletions(-)
> 
> diff --git a/hw/misc/mac_via.c b/hw/misc/mac_via.c
> index b7d0012794..4c5ad08805 100644
> --- a/hw/misc/mac_via.c
> +++ b/hw/misc/mac_via.c
> @@ -868,24 +868,24 @@ static void mac_via_reset(DeviceState *dev)
>  static void mac_via_realize(DeviceState *dev, Error **errp)
>  {
>  MacVIAState *m = MAC_VIA(dev);
> -MOS6522State *ms;
>  struct tm tm;
>  int ret;
> +Error *err = NULL;
>  
> -/* Init VIAs 1 and 2 */
> -sysbus_init_child_obj(OBJECT(dev), "via1", &m->mos6522_via1,
> -  sizeof(m->mos6522_via1), TYPE_MOS6522_Q800_VIA1);
> +qdev_set_parent_bus(DEVICE(&m->mos6522_via1), sysbus_get_default());
> +qdev_set_parent_bus(DEVICE(&m->mos6522_via2), sysbus_get_default());
>  
> -sysbus_init_child_obj(OBJECT(dev), "via2", &m->mos6522_via2,
> -  sizeof(m->mos6522_via2), TYPE_MOS6522_Q800_VIA2);
> +object_property_set_bool(OBJECT(&m->mos6522_via1), true, "realized", 
> &err);
> +if (err != NULL) {
> +error_propagate(errp, err);
> +return;
> +}
>  
> -/* Pass through mos6522 output IRQs */
> -ms = MOS6522(&m->mos6522_via1);
> -object_property_add_alias(OBJECT(dev), "irq[0]", OBJECT(ms),
> -  SYSBUS_DEVICE_GPIO_IRQ "[0]", &error_abort);
> -ms = MOS6522(&m->mos6522_via2);
> -object_property_add_alias(OBJECT(dev), "irq[1]", OBJECT(ms),
> -  SYSBUS_DEVICE_GPIO_IRQ "[0]", &error_abort);
> +object_property_set_bool(OBJECT(&m->mos6522_via2), true, "realized", 
> &err);
> +if (err != NULL) {
> +error_propagate(errp, err);
> +return;
> +}
>  
>  /* Pass through mos6522 input IRQs */
>  qdev_pass_gpios(DEVICE(&m->mos6522_via1), dev, "via1-irq");
> @@ -932,6 +932,7 @@ static void mac_via_init(Object *obj)
>  {
>  SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
>  MacVIAState *m = MAC_VIA(obj);
> +MOS6522State *ms;
>  
>  /* MMIO */
>  memory_region_init(&m->mmio, obj, "mac-via", 2 * VIA_SIZE);
> @@ -948,6 +949,22 @@ static void mac_via_init(Object *obj)
>  /* ADB */
>  qbus_create_inplace((BusState *)&m->adb_bus, sizeof(m->adb_bus),
>  TYPE_ADB_BUS, DEVICE(obj), "adb.0");
> +
> +/* Init VIAs 1 and 2 */
> +object_initialize_child(OBJECT(m), "via1", &m->mos6522_via1, 

Sorry, one more space at the end of the above line, and fail to run checkpatch.

Re: [PATCH v4 2/3] mac_via: fix incorrect creation of mos6522 device in mac_via

[Pan Nengyuan]

On 3/5/2020 2:54 PM, Pan Nengyuan wrote:
> This patch fix a bug in mac_via where it failed to actually realize devices 
> it was using.
> And move the init codes which inits the mos6522 objects and properties on 
> them from realize()
> into init(). However, we keep qdev_set_parent_bus in realize(), otherwise it 
> will cause
> device-introspect-test test fail. Then do the realize mos6522 device in the 
> mac_vir_realize.
> 
> Signed-off-by: Pan Nengyuan 
> ---
> Cc: Laurent Vivier 
> Cc: Mark Cave-Ayland 
> ---
> v4->v3:
> - split v3 into two patches, this patch fix incorrect creation of mos6522, 
> move inits and props
>   from realize into init. The v3 is:
>   https://patchwork.kernel.org/patch/11407635/
> ---
>  hw/misc/mac_via.c | 43 ++-
>  1 file changed, 30 insertions(+), 13 deletions(-)
> 
> diff --git a/hw/misc/mac_via.c b/hw/misc/mac_via.c
> index b7d0012794..4c5ad08805 100644
> --- a/hw/misc/mac_via.c
> +++ b/hw/misc/mac_via.c
> @@ -868,24 +868,24 @@ static void mac_via_reset(DeviceState *dev)
>  static void mac_via_realize(DeviceState *dev, Error **errp)
>  {
>  MacVIAState *m = MAC_VIA(dev);
> -MOS6522State *ms;
>  struct tm tm;
>  int ret;
> +Error *err = NULL;
>  
> -/* Init VIAs 1 and 2 */
> -sysbus_init_child_obj(OBJECT(dev), "via1", &m->mos6522_via1,
> -  sizeof(m->mos6522_via1), TYPE_MOS6522_Q800_VIA1);
> +qdev_set_parent_bus(DEVICE(&m->mos6522_via1), sysbus_get_default());
> +qdev_set_parent_bus(DEVICE(&m->mos6522_via2), sysbus_get_default());
>  
> -sysbus_init_child_obj(OBJECT(dev), "via2", &m->mos6522_via2,
> -  sizeof(m->mos6522_via2), TYPE_MOS6522_Q800_VIA2);
> +object_property_set_bool(OBJECT(&m->mos6522_via1), true, "realized", 
> &err);
> +if (err != NULL) {
> +error_propagate(errp, err);
> +return;
> +}
>  
> -/* Pass through mos6522 output IRQs */
> -ms = MOS6522(&m->mos6522_via1);
> -object_property_add_alias(OBJECT(dev), "irq[0]", OBJECT(ms),
> -  SYSBUS_DEVICE_GPIO_IRQ "[0]", &error_abort);
> -ms = MOS6522(&m->mos6522_via2);
> -object_property_add_alias(OBJECT(dev), "irq[1]", OBJECT(ms),
> -  SYSBUS_DEVICE_GPIO_IRQ "[0]", &error_abort);
> +object_property_set_bool(OBJECT(&m->mos6522_via2), true, "realized", 
> &err);
> +if (err != NULL) {
> +error_propagate(errp, err);
> +return;
> +}
>  
>  /* Pass through mos6522 input IRQs */
>  qdev_pass_gpios(DEVICE(&m->mos6522_via1), dev, "via1-irq");
> @@ -932,6 +932,7 @@ static void mac_via_init(Object *obj)
>  {
>  SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
>  MacVIAState *m = MAC_VIA(obj);
> +MOS6522State *ms;
>  
>  /* MMIO */
>  memory_region_init(&m->mmio, obj, "mac-via", 2 * VIA_SIZE);
> @@ -948,6 +949,22 @@ static void mac_via_init(Object *obj)
>  /* ADB */
>  qbus_create_inplace((BusState *)&m->adb_bus, sizeof(m->adb_bus),
>  TYPE_ADB_BUS, DEVICE(obj), "adb.0");
> +
> +/* Init VIAs 1 and 2 */
> +object_initialize_child(OBJECT(m), "via1", &m->mos6522_via1,Sorry, one 
> more space at the end of the above line, and fail to run checkpatch.

> +sizeof(m->mos6522_via1), TYPE_MOS6522_Q800_VIA1,
> +&error_abort, NULL);
> +object_initialize_child(OBJECT(m), "via2", &m->mos6522_via2,
> +sizeof(m->mos6522_via2), TYPE_MOS6522_Q800_VIA2,
> +&error_abort, NULL);
> +
> +/* Pass through mos6522 output IRQs */
> +ms = MOS6522(&m->mos6522_via1);
> +object_property_add_alias(OBJECT(m), "irq[0]", OBJECT(ms),
> +  SYSBUS_DEVICE_GPIO_IRQ "[0]", &error_abort);
> +ms = MOS6522(&m->mos6522_via2);
> +object_property_add_alias(OBJECT(m), "irq[1]", OBJECT(ms),
> +  SYSBUS_DEVICE_GPIO_IRQ "[0]", &error_abort);
>  }
>  
>  static void postload_update_cb(void *opaque, int running, RunState state)
> 

Re: [RFC] QEMU Gating CI

[Cleber Rosa]

On Mon, Mar 02, 2020 at 03:27:42PM +, Peter Maydell wrote:
> 
> Hi -- any progress on this front? (Maybe I missed an email; if
> so, sorry about that...)
> 
> thanks
> -- PMM
> 

Hi Peter,

Yes, I've made some progress on some of the points raised on the last
email exchanges:

 1) Jobs on non-Linux OS. I've built/setup gitlab-runner for FreeBSD,
and tested a job:
- https://gitlab.com/cleber.gnu/qemuci/-/jobs/440379169

There are some limitations on a library that gitlab-runner uses to
manage services (and that has no implementation for FreeBSD "services").
But, there are workarounds that work allright.

 2) Wrote a script that checks/waits on the pipeline:
- 
https://gitlab.com/cleber.gnu/qemuci/-/commit/d90c5cf917c43f06c0724dc025205d618521c4cc

 3) Wrote machine setup documentation/scripts.

I'm tidying it all up to send a PR in the next day or two.

Thanks for your patience!
- Cleber.


signature.asc
Description: PGP signature

Re: [PATCH 5/5] hw/scsi/spapr_vscsi: Do not mix SRP IU size with DMA buffer size

[Philippe Mathieu-Daudé]

On 3/5/20 1:43 AM, David Gibson wrote:

On Wed, Mar 04, 2020 at 04:33:11PM +0100, Philippe Mathieu-Daudé wrote:

The 'union srp_iu' is meant as a pointer to any SRP Information
Unit type, it is not related to the size of a VIO DMA buffer.

Use a plain buffer for the VIO DMA read/write calls.
We can remove the reserved buffer from the 'union srp_iu'.

This issue was noticed when replacing the zero-length arrays
from hw/scsi/srp.h with flexible array member,
'clang -fsanitize=undefined' reported:

   hw/scsi/spapr_vscsi.c:69:29: error: field 'iu' with variable sized type 
'union viosrp_iu' not at the end of a struct or class is a GNU extension 
[-Werror,-Wgnu-variable-sized-type-not-at-end]
union viosrp_iu iu;
^

Signed-off-by: Philippe Mathieu-Daudé 
---
  hw/scsi/viosrp.h  |  2 +-
  hw/scsi/spapr_vscsi.c | 10 +-
  2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/hw/scsi/viosrp.h b/hw/scsi/viosrp.h
index 25676c2383..aba3203028 100644
--- a/hw/scsi/viosrp.h
+++ b/hw/scsi/viosrp.h
@@ -49,8 +49,8 @@ union srp_iu {
  struct srp_tsk_mgmt tsk_mgmt;
  struct srp_cmd cmd;
  struct srp_rsp rsp;
-uint8_t reserved[SRP_MAX_IU_LEN];
  };
+_Static_assert(sizeof(union srp_iu) <= SRP_MAX_IU_LEN, "srp_iu size 
incorrect");


Hrm.  Given that srp_iu will be a variably sized type, is this
assertion actually testing anything meaningful?


I wanted to assert that if another SRP IU is added, the DMA buffer will 
be big enough to hold it. I'll simply remove the check.



Otherwise, LGTM.


Thanks for reviewing the series,

Phil.



  
  enum viosrp_crq_formats {

  VIOSRP_SRP_FORMAT = 0x01,
diff --git a/hw/scsi/spapr_vscsi.c b/hw/scsi/spapr_vscsi.c
index f1a0bbdc31..f9be68e44e 100644
--- a/hw/scsi/spapr_vscsi.c
+++ b/hw/scsi/spapr_vscsi.c
@@ -66,7 +66,7 @@ typedef union vscsi_crq {
  
  typedef struct vscsi_req {

  vscsi_crq   crq;
-union viosrp_iu iu;
+uint8_t viosrp_iu_buf[SRP_MAX_IU_LEN];
  
  /* SCSI request tracking */

  SCSIRequest *sreq;
@@ -99,7 +99,7 @@ typedef struct {
  
  static union viosrp_iu *req_iu(vscsi_req *req)

  {
-return (union viosrp_iu *)req->iu.srp.reserved;
+return (union viosrp_iu *)req->viosrp_iu_buf;
  }
  
  
@@ -184,7 +184,7 @@ static int vscsi_send_iu(VSCSIState *s, vscsi_req *req,
  
  /* First copy the SRP */

  rc = spapr_vio_dma_write(&s->vdev, req->crq.s.IU_data_ptr,
- &req->iu, length);
+ &req->viosrp_iu_buf, length);
  if (rc) {
  fprintf(stderr, "vscsi_send_iu: DMA write failure !\n");
  }
@@ -603,7 +603,7 @@ static const VMStateDescription vmstate_spapr_vscsi_req = {
  .minimum_version_id = 1,
  .fields = (VMStateField[]) {
  VMSTATE_BUFFER(crq.raw, vscsi_req),
-VMSTATE_BUFFER(iu.srp.reserved, vscsi_req),
+VMSTATE_BUFFER(viosrp_iu_buf, vscsi_req),
  VMSTATE_UINT32(qtag, vscsi_req),
  VMSTATE_BOOL(active, vscsi_req),
  VMSTATE_UINT32(data_len, vscsi_req),
@@ -1104,7 +1104,7 @@ static void vscsi_got_payload(VSCSIState *s, vscsi_crq 
*crq)
  }
  
  /* XXX Handle failure differently ? */

-if (spapr_vio_dma_read(&s->vdev, crq->s.IU_data_ptr, &req->iu,
+if (spapr_vio_dma_read(&s->vdev, crq->s.IU_data_ptr, &req->viosrp_iu_buf,
 crq->s.IU_length)) {
  fprintf(stderr, "vscsi_got_payload: DMA read failure !\n");
  vscsi_put_req(req);

Re: [PATCH v4 0/3] delay timer_new from init to realize to fix memleaks.

[no-reply]

Patchew URL: 
https://patchew.org/QEMU/20200305065422.12707-1-pannengy...@huawei.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [PATCH v4 0/3] delay timer_new from init to realize to fix memleaks.
Message-id: 20200305065422.12707-1-pannengy...@huawei.com
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

From https://github.com/patchew-project/qemu
 * [new tag] patchew/20200305065422.12707-1-pannengy...@huawei.com -> 
patchew/20200305065422.12707-1-pannengy...@huawei.com
Switched to a new branch 'test'
e87aa99 hw/misc/mos6522: move timer_new from init() into realize() to avoid 
memleaks
0c6588b mac_via: fix incorrect creation of mos6522 device in mac_via
1afdb4e s390x: fix memleaks in cpu_finalize

=== OUTPUT BEGIN ===
1/3 Checking commit 1afdb4e181f1 (s390x: fix memleaks in cpu_finalize)
2/3 Checking commit 0c6588bb0991 (mac_via: fix incorrect creation of mos6522 
device in mac_via)
ERROR: trailing whitespace
#71: FILE: hw/misc/mac_via.c:954:
+object_initialize_child(OBJECT(m), "via1", &m->mos6522_via1, $

total: 1 errors, 0 warnings, 66 lines checked

Patch 2/3 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

3/3 Checking commit e87aa99376d6 (hw/misc/mos6522: move timer_new from init() 
into realize() to avoid memleaks)
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20200305065422.12707-1-pannengy...@huawei.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

[PATCH v4 2/3] mac_via: fix incorrect creation of mos6522 device in mac_via

[Pan Nengyuan]

This patch fix a bug in mac_via where it failed to actually realize devices it 
was using.
And move the init codes which inits the mos6522 objects and properties on them 
from realize()
into init(). However, we keep qdev_set_parent_bus in realize(), otherwise it 
will cause
device-introspect-test test fail. Then do the realize mos6522 device in the 
mac_vir_realize.

Signed-off-by: Pan Nengyuan 
---
Cc: Laurent Vivier 
Cc: Mark Cave-Ayland 
---
v4->v3:
- split v3 into two patches, this patch fix incorrect creation of mos6522, move 
inits and props
  from realize into init. The v3 is:
  https://patchwork.kernel.org/patch/11407635/
---
 hw/misc/mac_via.c | 43 ++-
 1 file changed, 30 insertions(+), 13 deletions(-)

diff --git a/hw/misc/mac_via.c b/hw/misc/mac_via.c
index b7d0012794..4c5ad08805 100644
--- a/hw/misc/mac_via.c
+++ b/hw/misc/mac_via.c
@@ -868,24 +868,24 @@ static void mac_via_reset(DeviceState *dev)
 static void mac_via_realize(DeviceState *dev, Error **errp)
 {
 MacVIAState *m = MAC_VIA(dev);
-MOS6522State *ms;
 struct tm tm;
 int ret;
+Error *err = NULL;
 
-/* Init VIAs 1 and 2 */
-sysbus_init_child_obj(OBJECT(dev), "via1", &m->mos6522_via1,
-  sizeof(m->mos6522_via1), TYPE_MOS6522_Q800_VIA1);
+qdev_set_parent_bus(DEVICE(&m->mos6522_via1), sysbus_get_default());
+qdev_set_parent_bus(DEVICE(&m->mos6522_via2), sysbus_get_default());
 
-sysbus_init_child_obj(OBJECT(dev), "via2", &m->mos6522_via2,
-  sizeof(m->mos6522_via2), TYPE_MOS6522_Q800_VIA2);
+object_property_set_bool(OBJECT(&m->mos6522_via1), true, "realized", &err);
+if (err != NULL) {
+error_propagate(errp, err);
+return;
+}
 
-/* Pass through mos6522 output IRQs */
-ms = MOS6522(&m->mos6522_via1);
-object_property_add_alias(OBJECT(dev), "irq[0]", OBJECT(ms),
-  SYSBUS_DEVICE_GPIO_IRQ "[0]", &error_abort);
-ms = MOS6522(&m->mos6522_via2);
-object_property_add_alias(OBJECT(dev), "irq[1]", OBJECT(ms),
-  SYSBUS_DEVICE_GPIO_IRQ "[0]", &error_abort);
+object_property_set_bool(OBJECT(&m->mos6522_via2), true, "realized", &err);
+if (err != NULL) {
+error_propagate(errp, err);
+return;
+}
 
 /* Pass through mos6522 input IRQs */
 qdev_pass_gpios(DEVICE(&m->mos6522_via1), dev, "via1-irq");
@@ -932,6 +932,7 @@ static void mac_via_init(Object *obj)
 {
 SysBusDevice *sbd = SYS_BUS_DEVICE(obj);
 MacVIAState *m = MAC_VIA(obj);
+MOS6522State *ms;
 
 /* MMIO */
 memory_region_init(&m->mmio, obj, "mac-via", 2 * VIA_SIZE);
@@ -948,6 +949,22 @@ static void mac_via_init(Object *obj)
 /* ADB */
 qbus_create_inplace((BusState *)&m->adb_bus, sizeof(m->adb_bus),
 TYPE_ADB_BUS, DEVICE(obj), "adb.0");
+
+/* Init VIAs 1 and 2 */
+object_initialize_child(OBJECT(m), "via1", &m->mos6522_via1, 
+sizeof(m->mos6522_via1), TYPE_MOS6522_Q800_VIA1,
+&error_abort, NULL);
+object_initialize_child(OBJECT(m), "via2", &m->mos6522_via2,
+sizeof(m->mos6522_via2), TYPE_MOS6522_Q800_VIA2,
+&error_abort, NULL);
+
+/* Pass through mos6522 output IRQs */
+ms = MOS6522(&m->mos6522_via1);
+object_property_add_alias(OBJECT(m), "irq[0]", OBJECT(ms),
+  SYSBUS_DEVICE_GPIO_IRQ "[0]", &error_abort);
+ms = MOS6522(&m->mos6522_via2);
+object_property_add_alias(OBJECT(m), "irq[1]", OBJECT(ms),
+  SYSBUS_DEVICE_GPIO_IRQ "[0]", &error_abort);
 }
 
 static void postload_update_cb(void *opaque, int running, RunState state)
-- 
2.18.2

[PATCH v4 3/3] hw/misc/mos6522: move timer_new from init() into realize() to avoid memleaks

[Pan Nengyuan]

There are some memleaks when we call 'device_list_properties'. This patch move 
timer_new from init into realize to fix it.

Reported-by: Euler Robot 
Signed-off-by: Pan Nengyuan 
---
Cc: Laurent Vivier 
Cc: Mark Cave-Ayland 
Cc: David Gibson 
Cc: qemu-...@nongnu.org
---
v2->v1:
- no changes in this patch.
v3->v2:
- remove null check in reset, and add calls to mos6522_realize() in 
mac_via_realize to make this move to be valid.
v4->v3:
- split patch into two, this patch fix the memleaks.
---
 hw/misc/mos6522.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/hw/misc/mos6522.c b/hw/misc/mos6522.c
index 19e154b870..c1cd154a84 100644
--- a/hw/misc/mos6522.c
+++ b/hw/misc/mos6522.c
@@ -485,6 +485,11 @@ static void mos6522_init(Object *obj)
 for (i = 0; i < ARRAY_SIZE(s->timers); i++) {
 s->timers[i].index = i;
 }
+}
+
+static void mos6522_realize(DeviceState *dev, Error **errp)
+{
+MOS6522State *s = MOS6522(dev);
 
 s->timers[0].timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, mos6522_timer1, s);
 s->timers[1].timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, mos6522_timer2, s);
@@ -502,6 +507,7 @@ static void mos6522_class_init(ObjectClass *oc, void *data)
 
 dc->reset = mos6522_reset;
 dc->vmsd = &vmstate_mos6522;
+dc->realize = mos6522_realize;
 device_class_set_props(dc, mos6522_properties);
 mdc->parent_reset = dc->reset;
 mdc->set_sr_int = mos6522_set_sr_int;
-- 
2.18.2

[PATCH v4 1/3] s390x: fix memleaks in cpu_finalize

[Pan Nengyuan]

This patch fix memleaks when we call tests/qtest/cpu-plug-test on s390x. The 
leak stack is as follow:

Direct leak of 48 byte(s) in 1 object(s) allocated from:
#0 0x7fb43c7cd970 in __interceptor_calloc (/lib64/libasan.so.5+0xef970)
#1 0x7fb43be2149d in g_malloc0 (/lib64/libglib-2.0.so.0+0x5249d)
#2 0x558ba96da716 in timer_new_full 
/mnt/sdb/qemu-new/qemu/include/qemu/timer.h:530
#3 0x558ba96da716 in timer_new 
/mnt/sdb/qemu-new/qemu/include/qemu/timer.h:551
#4 0x558ba96da716 in timer_new_ns 
/mnt/sdb/qemu-new/qemu/include/qemu/timer.h:569
#5 0x558ba96da716 in s390_cpu_initfn 
/mnt/sdb/qemu-new/qemu/target/s390x/cpu.c:285
#6 0x558ba9c969ab in object_init_with_type 
/mnt/sdb/qemu-new/qemu/qom/object.c:372
#7 0x558ba9c9eb5f in object_initialize_with_type 
/mnt/sdb/qemu-new/qemu/qom/object.c:516
#8 0x558ba9c9f053 in object_new_with_type 
/mnt/sdb/qemu-new/qemu/qom/object.c:684
#9 0x558ba967ede6 in s390x_new_cpu 
/mnt/sdb/qemu-new/qemu/hw/s390x/s390-virtio-ccw.c:64
#10 0x558ba99764b3 in hmp_cpu_add 
/mnt/sdb/qemu-new/qemu/hw/core/machine-hmp-cmds.c:57
#11 0x558ba9b1c27f in handle_hmp_command 
/mnt/sdb/qemu-new/qemu/monitor/hmp.c:1082
#12 0x558ba96c1b02 in qmp_human_monitor_command 
/mnt/sdb/qemu-new/qemu/monitor/misc.c:142

Reported-by: Euler Robot 
Signed-off-by: Pan Nengyuan 
---
Cc: Richard Henderson 
Cc: David Hildenbrand 
Cc: Cornelia Huck 
Cc: qemu-s3...@nongnu.org
---
v2->v1:
- Similarly to other cleanups, move timer_new into realize(Suggested by 
Philippe Mathieu-Daudé)
v3->v2:
- Aslo do the timer_free in unrealize, it seems balanced.
v4->v3:
- Aslo do timer_free on the error path in realize() and fix some coding style.
- Use device_class_set_parent_unrealize to declare unrealize.
---
 target/s390x/cpu-qom.h |  1 +
 target/s390x/cpu.c | 41 +
 2 files changed, 38 insertions(+), 4 deletions(-)

diff --git a/target/s390x/cpu-qom.h b/target/s390x/cpu-qom.h
index dbe5346ec9..af9ffed0d8 100644
--- a/target/s390x/cpu-qom.h
+++ b/target/s390x/cpu-qom.h
@@ -61,6 +61,7 @@ typedef struct S390CPUClass {
 const char *desc;
 
 DeviceRealize parent_realize;
+DeviceUnrealize parent_unrealize;
 void (*parent_reset)(CPUState *cpu);
 void (*load_normal)(CPUState *cpu);
 void (*reset)(CPUState *cpu, cpu_reset_type type);
diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index cf84d307c6..80b987ff1b 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -182,6 +182,12 @@ static void s390_cpu_realizefn(DeviceState *dev, Error 
**errp)
 #if !defined(CONFIG_USER_ONLY)
 MachineState *ms = MACHINE(qdev_get_machine());
 unsigned int max_cpus = ms->smp.max_cpus;
+
+cpu->env.tod_timer =
+timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_tod_timer, cpu);
+cpu->env.cpu_timer =
+timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_cpu_timer, cpu);
+
 if (cpu->env.core_id >= max_cpus) {
 error_setg(&err, "Unable to add CPU with core-id: %" PRIu32
", maximum core-id: %d", cpu->env.core_id,
@@ -224,9 +230,38 @@ static void s390_cpu_realizefn(DeviceState *dev, Error 
**errp)
 
 scc->parent_realize(dev, &err);
 out:
+if (cpu->env.tod_timer) {
+timer_del(cpu->env.tod_timer);
+}
+if (cpu->env.cpu_timer) {
+timer_del(cpu->env.cpu_timer);
+}
+timer_free(cpu->env.tod_timer);
+timer_free(cpu->env.cpu_timer);
 error_propagate(errp, err);
 }
 
+static void s390_cpu_unrealizefn(DeviceState *dev, Error **errp)
+{
+S390CPUClass *scc = S390_CPU_GET_CLASS(dev);
+Error *err = NULL;
+
+#if !defined(CONFIG_USER_ONLY)
+S390CPU *cpu = S390_CPU(dev);
+
+timer_del(cpu->env.tod_timer);
+timer_del(cpu->env.cpu_timer);
+timer_free(cpu->env.tod_timer);
+timer_free(cpu->env.cpu_timer);
+#endif
+
+scc->parent_unrealize(dev, &err);
+if (err != NULL) {
+error_propagate(errp, err);
+return;
+}
+}
+
 static GuestPanicInformation *s390_cpu_get_crash_info(CPUState *cs)
 {
 GuestPanicInformation *panic_info;
@@ -279,10 +314,6 @@ static void s390_cpu_initfn(Object *obj)
 s390_cpu_get_crash_info_qom, NULL, NULL, NULL, NULL);
 s390_cpu_model_register_props(obj);
 #if !defined(CONFIG_USER_ONLY)
-cpu->env.tod_timer =
-timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_tod_timer, cpu);
-cpu->env.cpu_timer =
-timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_cpu_timer, cpu);
 s390_cpu_set_state(S390_CPU_STATE_STOPPED, cpu);
 #endif
 }
@@ -453,6 +484,8 @@ static void s390_cpu_class_init(ObjectClass *oc, void *data)
 
 device_class_set_parent_realize(dc, s390_cpu_realizefn,
 &scc->parent_realize);
+device_class_set_parent_unrealize(dc, s390_cpu_unrealizefn,
+  &scc->parent_unrealize);
 device_class_set_props(dc, s390x_cpu_properties);
 dc->user_creatable = true;
 
-- 
2.18.2

[PATCH v4 0/3] delay timer_new from init to realize to fix memleaks.

[Pan Nengyuan]

This series delay timer_new from init into realize to avoid memleaks when we 
call 'device_list_properties'.
And do timer_free only in s390x_cpu_finalize because it's hotplugable. However, 
mos6522_realize is never called 
at all due to the incorrect creation of it. So we aslo fix the incorrect 
creation in mac_via first, then move the
timer_new to mos6522_realize().

v1:
   - Delay timer_new() from init() to realize() to fix memleaks.
v2:
   - Similarly to other cleanups, move timer_new into realize in 
target/s390x/cpu.c (Suggested by Philippe Mathieu-Daudé).
   - Send these two patches as a series instead of send each as a single patch 
but with wrong subject in v1.
v3:
   - It's not valid in mos6522 if we move timer_new from init to realize, 
because it's never called at all.
 Thus, we remove null check in reset, and add calls to mos6522_realize() in 
mac_via_realize to make this move to be valid.
   - split patch by device to make it more clear.
v4:
   - Aslo do timer_free on the error path in realize() and fix some coding 
style. Then use device_class_set_parent_unrealize to declare unrealize. 
   - split the mos6522 patch into two, one to fix incorrect creation of 
mos6522, the other to fix memleak.

Pan Nengyuan (3):
  s390x: fix memleaks in cpu_finalize
  mac_via: fix incorrect creation of mos6522 device in mac_via
  hw/misc/mos6522: move timer_new from init() into realize() to avoid
memleaks

 hw/misc/mac_via.c  | 43 +-
 hw/misc/mos6522.c  |  6 ++
 target/s390x/cpu-qom.h |  1 +
 target/s390x/cpu.c | 41 
 4 files changed, 74 insertions(+), 17 deletions(-)

-- 
2.18.2

Re: [kvm-unit-tests PATCH v3 08/14] arm/arm64: ITS: its_enable_defaults

[Zenghui Yu]

Hi Eric,

On 2020/3/4 22:26, Auger Eric wrote:

Hi Zenghui,
On 2/7/20 4:20 AM, Zenghui Yu wrote:

Hi Eric,

On 2020/1/28 18:34, Eric Auger wrote:

its_enable_defaults() is the top init function that allocates the
command queue and all the requested tables (device, collection,
lpi config and pending tables), enable LPIs at distributor level
and ITS level.

gicv3_enable_defaults must be called before.

Signed-off-by: Eric Auger 

---

v2 -> v3:
- introduce its_setup_baser in this patch
- squash "arm/arm64: ITS: Init the command queue" in this patch.
---
   lib/arm/asm/gic-v3-its.h |  8 
   lib/arm/gic-v3-its.c | 89 
   2 files changed, 97 insertions(+)

diff --git a/lib/arm/asm/gic-v3-its.h b/lib/arm/asm/gic-v3-its.h
index 815c515..fe73c04 100644
--- a/lib/arm/asm/gic-v3-its.h
+++ b/lib/arm/asm/gic-v3-its.h
@@ -36,6 +36,8 @@ struct its_data {
   void *base;
   struct its_typer typer;
   struct its_baser baser[GITS_BASER_NR_REGS];
+    struct its_cmd_block *cmd_base;
+    struct its_cmd_block *cmd_write;
   };
     extern struct its_data its_data;
@@ -88,10 +90,16 @@ extern struct its_data its_data;
   #define GITS_BASER_TYPE_DEVICE    1
   #define GITS_BASER_TYPE_COLLECTION    4
   +
+struct its_cmd_block {
+    u64 raw_cmd[4];
+};
+
   extern void its_parse_typer(void);
   extern void its_init(void);
   extern int its_parse_baser(int i, struct its_baser *baser);
   extern struct its_baser *its_lookup_baser(int type);
+extern void its_enable_defaults(void);
     #else /* __arm__ */
   diff --git a/lib/arm/gic-v3-its.c b/lib/arm/gic-v3-its.c
index 2c0ce13..d1e7e52 100644
--- a/lib/arm/gic-v3-its.c
+++ b/lib/arm/gic-v3-its.c
@@ -86,3 +86,92 @@ void its_init(void)
   its_parse_baser(i, &its_data.baser[i]);
   }
   +static void its_setup_baser(int i, struct its_baser *baser)
+{
+    unsigned long n = (baser->nr_pages * baser->psz) >> PAGE_SHIFT;
+    unsigned long order = is_power_of_2(n) ? fls(n) : fls(n) + 1;
+    u64 val;
+
+    baser->table_addr = (u64)virt_to_phys(alloc_pages(order));
+
+    val = ((u64)baser->table_addr    |
+    ((u64)baser->type    << GITS_BASER_TYPE_SHIFT)    |
+    ((u64)(baser->esz - 1)    << GITS_BASER_ENTRY_SIZE_SHIFT)    |
+    ((baser->nr_pages - 1)    << GITS_BASER_PAGES_SHIFT)    |
+    (u64)baser->indirect    << 62    |


I haven't seen the 'nr_pages' and 'indirect' are programmed anywhere
except in its_parse_baser(). It looks like they're treated as RO (but
they shouldn't) and I now don't think it makes sense to parse them in
its_parse_baser(), in patch#5.


First of all please forgive me for the delay.


Never mind.



I agree with you on nr_pages. However indirect also indicates the BASER
capability to support or not 2 level tables. So I think it makes sense
to read it on init.


Yes, you're right. As the spec says, the Indirect field "is RAZ/WI for
GIC implementations that only support flat tables".


Thanks,
Zenghui

Re: [PATCH v2 4/4] qapi: Brush off some (py)lint

[Markus Armbruster]

John Snow  writes:

> On 3/4/20 10:59 AM, Markus Armbruster wrote:
>> Signed-off-by: Markus Armbruster 
>> ---
>>  scripts/qapi/commands.py   |  2 +-
>>  scripts/qapi/expr.py   |  3 +--
>>  scripts/qapi/gen.py|  9 ++---
>>  scripts/qapi/introspect.py |  2 --
>>  scripts/qapi/parser.py |  6 ++
>>  scripts/qapi/schema.py | 11 +--
>>  6 files changed, 15 insertions(+), 18 deletions(-)
>> 
>
> Looks okay. (I don't care as much about no-else-return being there or
> not, and this module is your baby.)

I admit I personally prefer to elide the else there.  But my main
argument is consistent style.  Picking the one pylint wants makes
consistency easier.

> Reviewed-by: John Snow 

Thanks!

Re: [PATCH v6 9/9] iotests: add pylintrc file

[Markus Armbruster]

John Snow  writes:

> On 3/4/20 2:22 AM, Markus Armbruster wrote:
>> John Snow  writes:
>> 
>>> Repeatable results. run `pylint iotests.py` and you should get a pass.
>> 
>> Start your sentences with a capital letter, please.
>> 
>
> The German complains about the capitalization, but not the sentence
> fragment.

Heh!

>>>
>>> Signed-off-by: John Snow 
>>> ---
>>>  tests/qemu-iotests/pylintrc | 20 
>>>  1 file changed, 20 insertions(+)
>>>  create mode 100644 tests/qemu-iotests/pylintrc
>>>
>>> diff --git a/tests/qemu-iotests/pylintrc b/tests/qemu-iotests/pylintrc
>>> new file mode 100644
>>> index 00..feed506f75
>>> --- /dev/null
>>> +++ b/tests/qemu-iotests/pylintrc
>>> @@ -0,0 +1,20 @@
>>> +[MESSAGES CONTROL]
>>> +
>>> +# Disable the message, report, category or checker with the given id(s). 
>>> You
>>> +# can either give multiple identifiers separated by comma (,) or put this
>>> +# option multiple times (only on the command line, not in the configuration
>>> +# file where it should appear only once). You can also use "--disable=all" 
>>> to
>>> +# disable everything first and then reenable specific checks. For example, 
>>> if
>>> +# you want to run only the similarities checker, you can use "--disable=all
>>> +# --enable=similarities". If you want to run only the classes checker, but 
>>> have
>>> +# no Warning level messages displayed, use "--disable=all --enable=classes
>>> +# --disable=W".
>>> +disable=invalid-name,
>>> +missing-docstring,
>>> +line-too-long,
>>> +too-many-lines,
>>> +too-few-public-methods,
>>> +too-many-arguments,
>>> +too-many-locals,
>>> +too-many-branches,
>>> +too-many-public-methods,
>>> \ No newline at end of file
>> 
>> Add the newline, please.
>> 
>> German pejorative for the too-many- and too-few- warnings: "Müsli".
>> Implies it's for muesli-knitters / granola-crunchers indulging their
>> orthorexia.
>> 
>
> They are useful at times as they can suggest when you are going a bit
> overboard on "organically grown" design. For cleaning an existing
> codebase, it's more of a hindrance to the immediate goal of establishing
> a baseline.

Yes, gentle nudges to reconsider your code organization can be useful.
But when we run pylint with the goal of getting no output, even warnings
are much more than gentle nudges.

> (*cough* I try to adhere to them in my own freshly written code, and
> disable per-line when I've decided to veto the suggestion. Not
> appropriate for a codebase like ours. As Max reminds me, it's just tests
> -- don't make them too clever or pretty.)
>
> Regardless. It's not appropriate here and now.
>
>> missing-docstring is not advisable for libraries.  Feels okay here.
>> 
>
> Ideally we do start using them, but it's out of scope here. Since I did
> some cleanup, I wanted to establish the baseline of what I adhered to.
>
> *not* suggest that it's the destination state.
>
> Adding proper docstrings should be done during mypy conversion once the
> types are determined, understood, and enforced. Not before then.
>
>> line-too-long might be worth cleaning up.  How many of them do we have
>> now?
>> 
>
> Five in iotests.py using the default length of 100. 15 if I limit to 80.
>
> If we agree that 100 is okay, I can tackle this in an amendment patch.
> If 80 is okay, I'm going to put it off as one coat of paint too many.
>
> (I will try to clean up the 100+ lines for my next version. I am
> hesitant to make a deeper cut because I have the feeling it's the type
> of series that will incur a lot of nitpicks on indent style.)

One step at a time.

Re: [PATCH 4/4] qapi: Brush off some (py)lint

[Markus Armbruster]

John Snow  writes:

> On 3/4/20 3:01 AM, Markus Armbruster wrote:
>> John Snow  writes:
>> 
>>> On 2/27/20 9:45 AM, Markus Armbruster wrote:
 Signed-off-by: Markus Armbruster 
>>>
>>> I wrote some pylint cleanup for iotests recently, too. Are you targeting
>>> a subset of pylint errors to clean here?
>>>
>>> (Do any files pass 100%?)
>> 
>> Surely you're joking, Mr. Snow!
>> 
>> I'm chipping away at pylint's gripes.  I ran it with the following
>> messages disabled:
>> 
>> bad-whitespace,
>> fixme,
>> invalid-name,
>> missing-docstring,
>> too-few-public-methods,
>> too-many-arguments,
>> too-many-branches,
>> too-many-instance-attributes,
>> too-many-lines,
>> too-many-locals,
>> too-many-statements,
>> unused-argument,
>> unused-wildcard-import,
>> 
>> These are not all obviously useless.  They're just not what I want to
>> focus on right now.
>> 
>
> Yes, understood - so my approach is disable what I don't intend to fix,
> commit the pylintrc to prevent backslide, and move on.
>
> I think we have a difference in what a pylintrc means to us (the goal
> vs. the current status.)
>
> I didn't mean "100% without caveats", just "100% in some subset of checks".
>
> (I assume the answer is still no.)

To turn the answer into a yes, I'd have to disable the messages below,
and some of them I'd rather keep.

Tacking

# pylint: disable=...

to existing troublemakers may or may not be worth the ugliness (it needs
to go on the same line, which almost invariably makes it awkwardly long).

>> Remaining:
>> 
>> 1 x C0330: Wrong continued indentation (remove 19 spaces).
>> 
>> Accident, will fix in v2.
>> 
>> 8 x R0201: Method could be a function (no-self-use)
>> 
>> Yes, but the override in a sub-class does use self.
>> 
>> 2 x W0212: Access to a protected member _body of a client class 
>> (protected-access)
>> 
>> Needs cleanup, but not now.
>> 
>> 6 x W0401: Wildcard import qapi.common (wildcard-import)
>> 
>> Not sure I care.  I'd prefer not to have more wildcard imports,
>> though.
>> 
>> 2 x W0603: Using the global statement (global-statement)
>> 
>> Cleanup is non-trivial.  Not now.
>> 
>> I also ran pycodestyle-3:
>> 
>> 1 x E127 continuation line over-indented for visual indent
>> 
>> Same as pylint's C0330, will fix in v2.
>> 
>> 3 x E261 at least two spaces before inline comment
>> 
>> I blame Emacs.  Left for another day.
>> 
>> 8 x E501 line too long
>> 
>> Left for another day.
>> 
>> 1 x E713 test for membership should be 'not in'
>> 
>> I missed that one, will fix in v2.
>> 
>>> Consider checking in a pylintrc file that lets others run the same
>>> subset of pylint tests as you are doing so that we can prevent future
>>> regressions.
>> 
>> Working towards it, slowly.
>> 
>>> Take a peek at [PATCH v6 0/9] iotests: use python logging​
>>>
>>> Thanks for this series. I had a very similar series sitting waiting to
>>> go out, but this goes further in a few places.
>> 
>> Thanks!
>> 

Re: vmx=on with -accel hax

[Markus Armbruster]

Cc'ing people listed by "scripts/get_maintainer.pl -f target/i386/hax*".

Rui Prior  writes:

> Dear qemu developers,
>
> I found a bug/limitation of qemu on windows (qemu-w64-setup-20200201.exe
> from https://qemu.weilnetz.de/w64/ ) that makes qemu terminate
> immediately with "VCPU shutdown request" (twice) if I try to use the
> "vmx=on" CPU option while simultaneously using "-accel hax".  Without
> "vmx=on", it works fine (but it prevents me from using nested
> virtualization).
>
> I am using HAXM v7.5.6.
>
> Should you need any further information, please let me know.
>
> Best regards,

[PATCH v3 1/2] spapr: Disable legacy virtio devices for pseries-5.0 and later

[David Gibson]

PAPR specifies a kind of odd, paravirtualized PCI bus, which looks to
the guess mostly like classic PCI, even if some of the individual
devices on the bus are PCI Express.  One consequence of that is that
virtio-pci devices still default to being in transitional mode, though
legacy mode is now disabled by default on current q35 x86 machine
types.

Legacy mode virtio devices aren't really necessary any more, and are
causing some problems for future changes.  Therefore, for the
pseries-5.0 machine type (and onwards), switch to modern-only
virtio-pci devices by default.

This does mean we no longer support guest kernels prior to 4.0, unless
they have modern virtio support backported (which some distro kernels
like that in RHEL7 do).

Signed-off-by: David Gibson 
---
 hw/ppc/spapr.c | 17 +
 1 file changed, 17 insertions(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 2eb0d8f70d..3cfc98ac61 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -65,6 +65,7 @@
 
 #include "hw/pci/pci.h"
 #include "hw/scsi/scsi.h"
+#include "hw/virtio/virtio-pci.h"
 #include "hw/virtio/virtio-scsi.h"
 #include "hw/virtio/vhost-scsi-common.h"
 
@@ -4566,8 +4567,20 @@ static const TypeInfo spapr_machine_info = {
 
 static void spapr_machine_latest_class_options(MachineClass *mc)
 {
+/*
+ * Most defaults for the latest behaviour are inherited from the
+ * base class, but we need to override the (non ppc specific)
+ * default behaviour for virtio.  We can't do that from the base
+ * class since it doesn't have a compat_props.
+ */
+static GlobalProperty compat[] = {
+{ TYPE_VIRTIO_PCI, "disable-legacy", "on", },
+};
+
 mc->alias = "pseries";
 mc->is_default = true;
+
+compat_props_add(mc->compat_props, compat, G_N_ELEMENTS(compat));
 }
 
 #define DEFINE_SPAPR_MACHINE(suffix, verstr, latest) \
@@ -4607,6 +4620,9 @@ DEFINE_SPAPR_MACHINE(5_0, "5.0", true);
 static void spapr_machine_4_2_class_options(MachineClass *mc)
 {
 SpaprMachineClass *smc = SPAPR_MACHINE_CLASS(mc);
+static GlobalProperty compat[] = {
+{ TYPE_VIRTIO_PCI, "disable-legacy", "auto" },
+};
 
 spapr_machine_5_0_class_options(mc);
 compat_props_add(mc->compat_props, hw_compat_4_2, hw_compat_4_2_len);
@@ -4614,6 +4630,7 @@ static void spapr_machine_4_2_class_options(MachineClass 
*mc)
 smc->default_caps.caps[SPAPR_CAP_FWNMI_MCE] = SPAPR_CAP_OFF;
 smc->rma_limit = 16 * GiB;
 mc->nvdimm_supported = false;
+compat_props_add(mc->compat_props, compat, G_N_ELEMENTS(compat));
 }
 
 DEFINE_SPAPR_MACHINE(4_2, "4.2", false);
-- 
2.24.1

[PATCH v3 0/2] spapr: Use vIOMMU translation for virtio by default

[David Gibson]

Upcoming Secure VM support for pSeries machines introduces some
complications for virtio, since the transfer buffers need to be
explicitly shared so that the hypervisor can access them.

While it's not strictly speaking dependent on it, the fact that virtio
devices bypass normal platform IOMMU translation complicates the issue
on the guest side.  Since there are some significan downsides to
bypassing the vIOMMU anyway, let's just disable that.

There's already a flag to do this in virtio, just turn it on by
default for forthcoming pseries machine types.

Any opinions on whether dropping support for the older guest kernels
is acceptable at this point?

Changes since v2:
 * Rebase and improve some comments
Changes since v1:
 * Added information on which guest kernel versions will no longer
   work with these changes
 * Use Michael Tsirkin's suggested better way of handling the machine
   type change

David Gibson (2):
  spapr: Disable legacy virtio devices for pseries-5.0 and later
  spapr: Enable virtio iommu_platform=on by default

 hw/ppc/spapr.c | 19 +++
 1 file changed, 19 insertions(+)

-- 
2.24.1

[PATCH v3 2/2] spapr: Enable virtio iommu_platform=on by default

[David Gibson]

Traditionally, virtio devices don't do DMA by the usual path on the
guest platform.  In particular they usually bypass any virtual IOMMU
the guest has, using hypervisor magic to access untranslated guest
physical addresses.

There's now the optional iommu_platform flag which can tell virtio
devices to use the platform's normal DMA path, including any IOMMUs.
That flag was motiviated for the case of hardware virtio
implementations, but there are other reasons to want it.

Specifically, the fact that the virtio device doesn't use vIOMMU
translation means that virtio devices are unsafe to pass to nested
guests, or to use with VFIO userspace drivers inside the guest.  This
is particularly noticeable on the pseries platform which *always* has
a guest-visible vIOMMU.

Not using the normal DMA path also causes difficulties for the guest
side driver when using the upcoming POWER Secure VMs (a.k.a. PEF).
While it's theoretically possible to handle this on the guest side,
it's really fiddly.  Given the other problems with the non-translated
virtio device, let's just enable vIOMMU translation for virtio devices
by default in the pseries-5.0 (and later) machine types.

This does mean the new machine type will no longer support guest
kernels older than 4.8, unless they have support for the virtio
IOMMU_PLATFORM flag backported (which some distro kernels like RHEL7
do).

Signed-off-by: David Gibson 
---
 hw/ppc/spapr.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 3cfc98ac61..5ef099536e 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -4575,6 +4575,7 @@ static void 
spapr_machine_latest_class_options(MachineClass *mc)
  */
 static GlobalProperty compat[] = {
 { TYPE_VIRTIO_PCI, "disable-legacy", "on", },
+{ TYPE_VIRTIO_DEVICE, "iommu_platform", "on", },
 };
 
 mc->alias = "pseries";
@@ -4622,6 +4623,7 @@ static void spapr_machine_4_2_class_options(MachineClass 
*mc)
 SpaprMachineClass *smc = SPAPR_MACHINE_CLASS(mc);
 static GlobalProperty compat[] = {
 { TYPE_VIRTIO_PCI, "disable-legacy", "auto" },
+{ TYPE_VIRTIO_DEVICE, "iommu_platform", "off", },
 };
 
 spapr_machine_5_0_class_options(mc);
-- 
2.24.1

Re: [PATCH] virtio-serial-bus: do cleanup on the error path in realize() to avoid memleaks

[Pan Nengyuan]

On 3/4/2020 5:23 PM, Markus Armbruster wrote:
> Pan Nengyuan  writes:
> 
>> port->bh forgot to delete on the error path, this patch add it to fix 
>> memleaks. It's easy to reproduce as follow(add a same nr port):
>> {'execute': 'device_add', 'arguments': {'id': 'virtio_serial_pci0', 
>> 'driver': 'virtio-serial-pci', 'bus': 'pci.0', 'addr': '0x5'}, 'id': 
>> 'yVkZcGgV'}
>> {'execute': 'device_add', 'arguments': {'id': 'port1', 'driver': 
>> 'virtserialport', 'name': 'port1', 'chardev': 'channel1', 'bus': 
>> 'virtio_serial_pci0.0', 'nr': 1}, 'id': '3dXdUgJA'}
>> {'execute': 'device_add', 'arguments': {'id': 'port2', 'driver': 
>> 'virtserialport', 'name': 'port2', 'chardev': 'channel2', 'bus': 
>> 'virtio_serial_pci0.0', 'nr': 1}, 'id': 'qLzcCkob'}
>> {'execute': 'device_add', 'arguments': {'id': 'port2', 'driver': 
>> 'virtserialport', 'name': 'port2', 'chardev': 'channel2', 'bus': 
>> 'virtio_serial_pci0.0', 'nr': 2}, 'id': 'qLzcCkob'}
>>
>> The leak stack:
>> Direct leak of 40 byte(s) in 1 object(s) allocated from:
>> #0 0x7f04a8008ae8 in __interceptor_malloc (/lib64/libasan.so.5+0xefae8)
>> #1 0x7f04a73cf1d5 in g_malloc (/lib64/libglib-2.0.so.0+0x531d5)
>> #2 0x56273eaee484 in aio_bh_new /mnt/sdb/backup/qemu/util/async.c:125
>> #3 0x56273eafe9a8 in qemu_bh_new 
>> /mnt/sdb/backup/qemu/util/main-loop.c:532
>> #4 0x56273d52e62e in virtser_port_device_realize 
>> /mnt/sdb/backup/qemu/hw/char/virtio-serial-bus.c:946
>> #5 0x56273dcc5040 in device_set_realized 
>> /mnt/sdb/backup/qemu/hw/core/qdev.c:891
>> #6 0x56273e5ebbce in property_set_bool 
>> /mnt/sdb/backup/qemu/qom/object.c:2238
>> #7 0x56273e5e5a9c in object_property_set 
>> /mnt/sdb/backup/qemu/qom/object.c:1324
>> #8 0x56273e5ef5f8 in object_property_set_qobject 
>> /mnt/sdb/backup/qemu/qom/qom-qobject.c:26
>> #9 0x56273e5e5e6a in object_property_set_bool 
>> /mnt/sdb/backup/qemu/qom/object.c:1390
>> #10 0x56273daa40de in qdev_device_add 
>> /mnt/sdb/backup/qemu/qdev-monitor.c:680
>> #11 0x56273daa53e9 in qmp_device_add 
>> /mnt/sdb/backup/qemu/qdev-monitor.c:805
>>
>> Reported-by: Euler Robot 
>> Signed-off-by: Pan Nengyuan 
> 
> Fixes: 199646d81522509ac2dba6d28c31e8c7d807bc93
> 
>> ---
>>  hw/char/virtio-serial-bus.c | 14 +-
>>  1 file changed, 9 insertions(+), 5 deletions(-)
>>
>> diff --git a/hw/char/virtio-serial-bus.c b/hw/char/virtio-serial-bus.c
>> index 941ed5aca9..563b845f71 100644
>> --- a/hw/char/virtio-serial-bus.c
>> +++ b/hw/char/virtio-serial-bus.c
>> @@ -957,13 +957,13 @@ static void virtser_port_device_realize(DeviceState 
>> *dev, Error **errp)
>>  if (find_port_by_id(port->vser, port->id)) {
>>  error_setg(errp, "virtio-serial-bus: A port already exists at id 
>> %u",
>> port->id);
>> -return;
>> +goto fail;
>>  }
>>  
>>  if (port->name != NULL && find_port_by_name(port->name)) {
>>  error_setg(errp, "virtio-serial-bus: A port already exists by name 
>> %s",
>> port->name);
>> -return;
>> +goto fail;
>>  }
>>  
>>  if (port->id == VIRTIO_CONSOLE_BAD_ID) {
>> @@ -974,7 +974,7 @@ static void virtser_port_device_realize(DeviceState 
>> *dev, Error **errp)
>>  if (port->id == VIRTIO_CONSOLE_BAD_ID) {
>>  error_setg(errp, "virtio-serial-bus: Maximum port limit for 
>> "
>>   "this device reached");
>> -return;
>> +goto fail;
>>  }
>>  }
>>  }
>> @@ -983,16 +983,20 @@ static void virtser_port_device_realize(DeviceState 
>> *dev, Error **errp)
>>  if (port->id >= max_nr_ports) {
>>  error_setg(errp, "virtio-serial-bus: Out-of-range port id 
>> specified, "
>>   "max. allowed: %u", max_nr_ports - 1);
>> -return;
>> +goto fail;
>>  }
>>  
>>  vsc->realize(dev, &err);
>>  if (err != NULL) {
>>  error_propagate(errp, err);
>> -return;
>> +goto fail;
>>  }
>>  
>>  port->elem = NULL;
>> +return;
>> +
>> +fail:
>> +qemu_bh_delete(port->bh);
>>  }
>>  
>>  static void virtser_port_device_plug(HotplugHandler *hotplug_dev,
> 
> Looks correct to me.
> 
> However, I wonder whether we could simply create port->bh later.  It's
> for use by virtio_serial_throttle_port(), called on incoming migration
> via virtio_serial_load_device(), virtio_load(), virtio_device_get().  It
> runs flush_queued_data(), which does nothing unless
> virtio_queue_ready(port->ovq).
> 
> Note that virtio_queue_ready() dereferences its argument.  It's safe
> only after virtser_port_device_plug() set port->ovq.  I'd expect that to
> be possible only while the device is realized.  If that's correct, we
> could simply create port->bh last in virtser_port_device_realize().

Yes, it looks correct. If there is no other points, I will change it.

Thanks.

> 
> .
> 

RE: [PATCH v16 00/10] VIRTIO-IOMMU device

[Tian, Kevin]

> From: Jean-Philippe Brucker 
> Sent: Thursday, March 5, 2020 12:47 AM
>
[...]
> > >
> > > * We can't use DVM in nested mode unless the VMID is shared with the
> > > CPU. For that we'll need the host SMMU driver to hook into the KVM
> VMID
> > > allocator, just like we do for the ASID allocator. I haven't yet
> > > investigated how to do that. It's possible to do vSVA without DVM
> > > though, by sending all TLB invalidations through the SMMU command
> queue.
> > > "
> 
> Hm we're already mandating DVM for host SVA, so I'd say mandate it for
> vSVA as well. We'd avoid a ton of context switches, especially for the zip
> accelerator which doesn't require ATC invalidations. The host needs to pin
> the VMID allocated by KVM and write it in the endpoint's STE.
> 

Curious... what is DVM and how is it related to SVA? Is it SMMU specific?

Re: [PATCH 5/5] hw/scsi/spapr_vscsi: Do not mix SRP IU size with DMA buffer size

[David Gibson]

On Wed, Mar 04, 2020 at 04:33:11PM +0100, Philippe Mathieu-Daudé wrote:
> The 'union srp_iu' is meant as a pointer to any SRP Information
> Unit type, it is not related to the size of a VIO DMA buffer.
> 
> Use a plain buffer for the VIO DMA read/write calls.
> We can remove the reserved buffer from the 'union srp_iu'.
> 
> This issue was noticed when replacing the zero-length arrays
> from hw/scsi/srp.h with flexible array member,
> 'clang -fsanitize=undefined' reported:
> 
>   hw/scsi/spapr_vscsi.c:69:29: error: field 'iu' with variable sized type 
> 'union viosrp_iu' not at the end of a struct or class is a GNU extension 
> [-Werror,-Wgnu-variable-sized-type-not-at-end]
>union viosrp_iu iu;
>^
> 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  hw/scsi/viosrp.h  |  2 +-
>  hw/scsi/spapr_vscsi.c | 10 +-
>  2 files changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/hw/scsi/viosrp.h b/hw/scsi/viosrp.h
> index 25676c2383..aba3203028 100644
> --- a/hw/scsi/viosrp.h
> +++ b/hw/scsi/viosrp.h
> @@ -49,8 +49,8 @@ union srp_iu {
>  struct srp_tsk_mgmt tsk_mgmt;
>  struct srp_cmd cmd;
>  struct srp_rsp rsp;
> -uint8_t reserved[SRP_MAX_IU_LEN];
>  };
> +_Static_assert(sizeof(union srp_iu) <= SRP_MAX_IU_LEN, "srp_iu size 
> incorrect");

Hrm.  Given that srp_iu will be a variably sized type, is this
assertion actually testing anything meaningful?


Otherwise, LGTM.

>  
>  enum viosrp_crq_formats {
>  VIOSRP_SRP_FORMAT = 0x01,
> diff --git a/hw/scsi/spapr_vscsi.c b/hw/scsi/spapr_vscsi.c
> index f1a0bbdc31..f9be68e44e 100644
> --- a/hw/scsi/spapr_vscsi.c
> +++ b/hw/scsi/spapr_vscsi.c
> @@ -66,7 +66,7 @@ typedef union vscsi_crq {
>  
>  typedef struct vscsi_req {
>  vscsi_crq   crq;
> -union viosrp_iu iu;
> +uint8_t viosrp_iu_buf[SRP_MAX_IU_LEN];
>  
>  /* SCSI request tracking */
>  SCSIRequest *sreq;
> @@ -99,7 +99,7 @@ typedef struct {
>  
>  static union viosrp_iu *req_iu(vscsi_req *req)
>  {
> -return (union viosrp_iu *)req->iu.srp.reserved;
> +return (union viosrp_iu *)req->viosrp_iu_buf;
>  }
>  
>  
> @@ -184,7 +184,7 @@ static int vscsi_send_iu(VSCSIState *s, vscsi_req *req,
>  
>  /* First copy the SRP */
>  rc = spapr_vio_dma_write(&s->vdev, req->crq.s.IU_data_ptr,
> - &req->iu, length);
> + &req->viosrp_iu_buf, length);
>  if (rc) {
>  fprintf(stderr, "vscsi_send_iu: DMA write failure !\n");
>  }
> @@ -603,7 +603,7 @@ static const VMStateDescription vmstate_spapr_vscsi_req = 
> {
>  .minimum_version_id = 1,
>  .fields = (VMStateField[]) {
>  VMSTATE_BUFFER(crq.raw, vscsi_req),
> -VMSTATE_BUFFER(iu.srp.reserved, vscsi_req),
> +VMSTATE_BUFFER(viosrp_iu_buf, vscsi_req),
>  VMSTATE_UINT32(qtag, vscsi_req),
>  VMSTATE_BOOL(active, vscsi_req),
>  VMSTATE_UINT32(data_len, vscsi_req),
> @@ -1104,7 +1104,7 @@ static void vscsi_got_payload(VSCSIState *s, vscsi_crq 
> *crq)
>  }
>  
>  /* XXX Handle failure differently ? */
> -if (spapr_vio_dma_read(&s->vdev, crq->s.IU_data_ptr, &req->iu,
> +if (spapr_vio_dma_read(&s->vdev, crq->s.IU_data_ptr, &req->viosrp_iu_buf,
> crq->s.IU_length)) {
>  fprintf(stderr, "vscsi_got_payload: DMA read failure !\n");
>  vscsi_put_req(req);

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [PATCH 4/5] hw/scsi/spapr_vscsi: Introduce req_ui() helper

[David Gibson]

On Wed, Mar 04, 2020 at 04:33:10PM +0100, Philippe Mathieu-Daudé wrote:
> Introduce the req_ui() helper which returns a pointer to
> the viosrp_iu union held in the vscsi_req structure.
> This simplifies the next patch.
> 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  hw/scsi/spapr_vscsi.c | 40 
>  1 file changed, 24 insertions(+), 16 deletions(-)
> 
> diff --git a/hw/scsi/spapr_vscsi.c b/hw/scsi/spapr_vscsi.c
> index 3cb5a38181..f1a0bbdc31 100644
> --- a/hw/scsi/spapr_vscsi.c
> +++ b/hw/scsi/spapr_vscsi.c
> @@ -97,6 +97,12 @@ typedef struct {
>  vscsi_req reqs[VSCSI_REQ_LIMIT];
>  } VSCSIState;
>  
> +static union viosrp_iu *req_iu(vscsi_req *req)
> +{
> +return (union viosrp_iu *)req->iu.srp.reserved;

I guess it doesn't really matter since you remove it in the next
patch, but this seems a really weird way of expressing
return &req->iu;

> +}
> +
> +
>  static struct vscsi_req *vscsi_get_req(VSCSIState *s)
>  {
>  vscsi_req *req;
> @@ -121,7 +127,7 @@ static struct vscsi_req *vscsi_find_req(VSCSIState *s, 
> uint64_t srp_tag)
>  
>  for (i = 0; i < VSCSI_REQ_LIMIT; i++) {
>  req = &s->reqs[i];
> -if (req->iu.srp.cmd.tag == srp_tag) {
> +if (req_iu(req)->srp.cmd.tag == srp_tag) {
>  return req;
>  }
>  }
> @@ -188,7 +194,7 @@ static int vscsi_send_iu(VSCSIState *s, vscsi_req *req,
>  req->crq.s.reserved = 0x00;
>  req->crq.s.timeout = cpu_to_be16(0x);
>  req->crq.s.IU_length = cpu_to_be16(length);
> -req->crq.s.IU_data_ptr = req->iu.srp.rsp.tag; /* right byte order */
> +req->crq.s.IU_data_ptr = req_iu(req)->srp.rsp.tag; /* right byte order */
>  
>  if (rc == 0) {
>  req->crq.s.status = VIOSRP_OK;
> @@ -224,7 +230,7 @@ static void vscsi_makeup_sense(VSCSIState *s, vscsi_req 
> *req,
>  static int vscsi_send_rsp(VSCSIState *s, vscsi_req *req,
>uint8_t status, int32_t res_in, int32_t res_out)
>  {
> -union viosrp_iu *iu = &req->iu;
> +union viosrp_iu *iu = req_iu(req);
>  uint64_t tag = iu->srp.rsp.tag;
>  int total_len = sizeof(iu->srp.rsp);
>  uint8_t sol_not = iu->srp.cmd.sol_not;
> @@ -285,7 +291,7 @@ static int vscsi_fetch_desc(VSCSIState *s, struct 
> vscsi_req *req,
>  unsigned n, unsigned buf_offset,
>  struct srp_direct_buf *ret)
>  {
> -struct srp_cmd *cmd = &req->iu.srp.cmd;
> +struct srp_cmd *cmd = &req_iu(req)->srp.cmd;
>  
>  switch (req->dma_fmt) {
>  case SRP_NO_DATA_DESC: {
> @@ -473,7 +479,7 @@ static int data_out_desc_size(struct srp_cmd *cmd)
>  
>  static int vscsi_preprocess_desc(vscsi_req *req)
>  {
> -struct srp_cmd *cmd = &req->iu.srp.cmd;
> +struct srp_cmd *cmd = &req_iu(req)->srp.cmd;
>  
>  req->cdb_offset = cmd->add_cdb_len & ~3;
>  
> @@ -655,7 +661,7 @@ static void *vscsi_load_request(QEMUFile *f, SCSIRequest 
> *sreq)
>  
>  static void vscsi_process_login(VSCSIState *s, vscsi_req *req)
>  {
> -union viosrp_iu *iu = &req->iu;
> +union viosrp_iu *iu = req_iu(req);
>  struct srp_login_rsp *rsp = &iu->srp.login_rsp;
>  uint64_t tag = iu->srp.rsp.tag;
>  
> @@ -681,7 +687,7 @@ static void vscsi_process_login(VSCSIState *s, vscsi_req 
> *req)
>  
>  static void vscsi_inquiry_no_target(VSCSIState *s, vscsi_req *req)
>  {
> -uint8_t *cdb = req->iu.srp.cmd.cdb;
> +uint8_t *cdb = req_iu(req)->srp.cmd.cdb;
>  uint8_t resp_data[36];
>  int rc, len, alen;
>  
> @@ -770,7 +776,7 @@ static void vscsi_report_luns(VSCSIState *s, vscsi_req 
> *req)
>  
>  static int vscsi_queue_cmd(VSCSIState *s, vscsi_req *req)
>  {
> -union srp_iu *srp = &req->iu.srp;
> +union srp_iu *srp = &req_iu(req)->srp;
>  SCSIDevice *sdev;
>  int n, lun;
>  
> @@ -821,7 +827,7 @@ static int vscsi_queue_cmd(VSCSIState *s, vscsi_req *req)
>  
>  static int vscsi_process_tsk_mgmt(VSCSIState *s, vscsi_req *req)
>  {
> -union viosrp_iu *iu = &req->iu;
> +union viosrp_iu *iu = req_iu(req);
>  vscsi_req *tmpreq;
>  int i, lun = 0, resp = SRP_TSK_MGMT_COMPLETE;
>  SCSIDevice *d;
> @@ -831,7 +837,8 @@ static int vscsi_process_tsk_mgmt(VSCSIState *s, 
> vscsi_req *req)
>  fprintf(stderr, "vscsi_process_tsk_mgmt %02x\n",
>  iu->srp.tsk_mgmt.tsk_mgmt_func);
>  
> -d = vscsi_device_find(&s->bus, be64_to_cpu(req->iu.srp.tsk_mgmt.lun), 
> &lun);
> +d = vscsi_device_find(&s->bus,
> +  be64_to_cpu(req_iu(req)->srp.tsk_mgmt.lun), &lun);
>  if (!d) {
>  resp = SRP_TSK_MGMT_FIELDS_INVALID;
>  } else {
> @@ -842,7 +849,7 @@ static int vscsi_process_tsk_mgmt(VSCSIState *s, 
> vscsi_req *req)
>  break;
>  }
>  
> -tmpreq = vscsi_find_req(s, req->iu.srp.tsk_mgmt.task_tag);
> +tmpreq = vscsi_find_req(s, req_iu(req)->srp.tsk_mgmt.task_tag);
>  if (tmpreq && tmpreq->sreq)

Re: [PATCH 1/5] hw/scsi/viosrp: Add missing 'hw/scsi/srp.h' include

[David Gibson]

On Wed, Mar 04, 2020 at 04:33:07PM +0100, Philippe Mathieu-Daudé wrote:
> This header use the srp_* structures declared in "hw/scsi/srp.h".
> 
> Signed-off-by: Philippe Mathieu-Daudé 

Applied to ppc-for-5.0.

> ---
>  hw/scsi/viosrp.h | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/hw/scsi/viosrp.h b/hw/scsi/viosrp.h
> index d8e365db1e..25676c2383 100644
> --- a/hw/scsi/viosrp.h
> +++ b/hw/scsi/viosrp.h
> @@ -34,6 +34,8 @@
>  #ifndef PPC_VIOSRP_H
>  #define PPC_VIOSRP_H
>  
> +#include "hw/scsi/srp.h"
> +
>  #define SRP_VERSION "16.a"
>  #define SRP_MAX_IU_LEN256
>  #define SRP_MAX_LOC_LEN 32

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [PATCH 3/5] hw/scsi/spapr_vscsi: Simplify a bit

[David Gibson]

On Wed, Mar 04, 2020 at 04:33:09PM +0100, Philippe Mathieu-Daudé wrote:
> We already have a ui pointer, use it (to simplify the next commit).
> 
> Signed-off-by: Philippe Mathieu-Daudé 

Applied to ppc-for-5.0.

> ---
>  hw/scsi/spapr_vscsi.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/scsi/spapr_vscsi.c b/hw/scsi/spapr_vscsi.c
> index 7e397ed797..3cb5a38181 100644
> --- a/hw/scsi/spapr_vscsi.c
> +++ b/hw/scsi/spapr_vscsi.c
> @@ -261,9 +261,9 @@ static int vscsi_send_rsp(VSCSIState *s, vscsi_req *req,
>  if (status) {
>  iu->srp.rsp.sol_not = (sol_not & 0x04) >> 2;
>  if (req->senselen) {
> -req->iu.srp.rsp.flags |= SRP_RSP_FLAG_SNSVALID;
> -req->iu.srp.rsp.sense_data_len = cpu_to_be32(req->senselen);
> -memcpy(req->iu.srp.rsp.data, req->sense, req->senselen);
> +iu->srp.rsp.flags |= SRP_RSP_FLAG_SNSVALID;
> +iu->srp.rsp.sense_data_len = cpu_to_be32(req->senselen);
> +memcpy(iu->srp.rsp.data, req->sense, req->senselen);
>  total_len += req->senselen;
>  }
>  } else {

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [PATCH 4/5] hw/scsi/spapr_vscsi: Introduce req_ui() helper

[David Gibson]

On Thu, Mar 05, 2020 at 11:41:37AM +1100, David Gibson wrote:
> On Wed, Mar 04, 2020 at 04:33:10PM +0100, Philippe Mathieu-Daudé wrote:
> > Introduce the req_ui() helper which returns a pointer to
> > the viosrp_iu union held in the vscsi_req structure.
> > This simplifies the next patch.
> > 
> > Signed-off-by: Philippe Mathieu-Daudé 
> > ---
> >  hw/scsi/spapr_vscsi.c | 40 
> >  1 file changed, 24 insertions(+), 16 deletions(-)
> > 
> > diff --git a/hw/scsi/spapr_vscsi.c b/hw/scsi/spapr_vscsi.c
> > index 3cb5a38181..f1a0bbdc31 100644
> > --- a/hw/scsi/spapr_vscsi.c
> > +++ b/hw/scsi/spapr_vscsi.c
> > @@ -97,6 +97,12 @@ typedef struct {
> >  vscsi_req reqs[VSCSI_REQ_LIMIT];
> >  } VSCSIState;
> >  
> > +static union viosrp_iu *req_iu(vscsi_req *req)
> > +{
> > +return (union viosrp_iu *)req->iu.srp.reserved;
> 
> I guess it doesn't really matter since you remove it in the next
> patch, but this seems a really weird way of expressing
>   return &req->iu;

Oh, also s/req_ui/req_iu/g in the commit message.

> 
> > +}
> > +
> > +
> >  static struct vscsi_req *vscsi_get_req(VSCSIState *s)
> >  {
> >  vscsi_req *req;
> > @@ -121,7 +127,7 @@ static struct vscsi_req *vscsi_find_req(VSCSIState *s, 
> > uint64_t srp_tag)
> >  
> >  for (i = 0; i < VSCSI_REQ_LIMIT; i++) {
> >  req = &s->reqs[i];
> > -if (req->iu.srp.cmd.tag == srp_tag) {
> > +if (req_iu(req)->srp.cmd.tag == srp_tag) {
> >  return req;
> >  }
> >  }
> > @@ -188,7 +194,7 @@ static int vscsi_send_iu(VSCSIState *s, vscsi_req *req,
> >  req->crq.s.reserved = 0x00;
> >  req->crq.s.timeout = cpu_to_be16(0x);
> >  req->crq.s.IU_length = cpu_to_be16(length);
> > -req->crq.s.IU_data_ptr = req->iu.srp.rsp.tag; /* right byte order */
> > +req->crq.s.IU_data_ptr = req_iu(req)->srp.rsp.tag; /* right byte order 
> > */
> >  
> >  if (rc == 0) {
> >  req->crq.s.status = VIOSRP_OK;
> > @@ -224,7 +230,7 @@ static void vscsi_makeup_sense(VSCSIState *s, vscsi_req 
> > *req,
> >  static int vscsi_send_rsp(VSCSIState *s, vscsi_req *req,
> >uint8_t status, int32_t res_in, int32_t res_out)
> >  {
> > -union viosrp_iu *iu = &req->iu;
> > +union viosrp_iu *iu = req_iu(req);
> >  uint64_t tag = iu->srp.rsp.tag;
> >  int total_len = sizeof(iu->srp.rsp);
> >  uint8_t sol_not = iu->srp.cmd.sol_not;
> > @@ -285,7 +291,7 @@ static int vscsi_fetch_desc(VSCSIState *s, struct 
> > vscsi_req *req,
> >  unsigned n, unsigned buf_offset,
> >  struct srp_direct_buf *ret)
> >  {
> > -struct srp_cmd *cmd = &req->iu.srp.cmd;
> > +struct srp_cmd *cmd = &req_iu(req)->srp.cmd;
> >  
> >  switch (req->dma_fmt) {
> >  case SRP_NO_DATA_DESC: {
> > @@ -473,7 +479,7 @@ static int data_out_desc_size(struct srp_cmd *cmd)
> >  
> >  static int vscsi_preprocess_desc(vscsi_req *req)
> >  {
> > -struct srp_cmd *cmd = &req->iu.srp.cmd;
> > +struct srp_cmd *cmd = &req_iu(req)->srp.cmd;
> >  
> >  req->cdb_offset = cmd->add_cdb_len & ~3;
> >  
> > @@ -655,7 +661,7 @@ static void *vscsi_load_request(QEMUFile *f, 
> > SCSIRequest *sreq)
> >  
> >  static void vscsi_process_login(VSCSIState *s, vscsi_req *req)
> >  {
> > -union viosrp_iu *iu = &req->iu;
> > +union viosrp_iu *iu = req_iu(req);
> >  struct srp_login_rsp *rsp = &iu->srp.login_rsp;
> >  uint64_t tag = iu->srp.rsp.tag;
> >  
> > @@ -681,7 +687,7 @@ static void vscsi_process_login(VSCSIState *s, 
> > vscsi_req *req)
> >  
> >  static void vscsi_inquiry_no_target(VSCSIState *s, vscsi_req *req)
> >  {
> > -uint8_t *cdb = req->iu.srp.cmd.cdb;
> > +uint8_t *cdb = req_iu(req)->srp.cmd.cdb;
> >  uint8_t resp_data[36];
> >  int rc, len, alen;
> >  
> > @@ -770,7 +776,7 @@ static void vscsi_report_luns(VSCSIState *s, vscsi_req 
> > *req)
> >  
> >  static int vscsi_queue_cmd(VSCSIState *s, vscsi_req *req)
> >  {
> > -union srp_iu *srp = &req->iu.srp;
> > +union srp_iu *srp = &req_iu(req)->srp;
> >  SCSIDevice *sdev;
> >  int n, lun;
> >  
> > @@ -821,7 +827,7 @@ static int vscsi_queue_cmd(VSCSIState *s, vscsi_req 
> > *req)
> >  
> >  static int vscsi_process_tsk_mgmt(VSCSIState *s, vscsi_req *req)
> >  {
> > -union viosrp_iu *iu = &req->iu;
> > +union viosrp_iu *iu = req_iu(req);
> >  vscsi_req *tmpreq;
> >  int i, lun = 0, resp = SRP_TSK_MGMT_COMPLETE;
> >  SCSIDevice *d;
> > @@ -831,7 +837,8 @@ static int vscsi_process_tsk_mgmt(VSCSIState *s, 
> > vscsi_req *req)
> >  fprintf(stderr, "vscsi_process_tsk_mgmt %02x\n",
> >  iu->srp.tsk_mgmt.tsk_mgmt_func);
> >  
> > -d = vscsi_device_find(&s->bus, be64_to_cpu(req->iu.srp.tsk_mgmt.lun), 
> > &lun);
> > +d = vscsi_device_find(&s->bus,
> > +  be64_to_cpu(req_iu(req)->srp.tsk_mgmt.lun), 
> > &lun);
> >  if (!d) {
> >   

Re: [PATCH 2/5] hw/scsi/spapr_vscsi: Use SRP_MAX_IU_LEN instead of sizeof flexible array

[David Gibson]

On Wed, Mar 04, 2020 at 04:33:08PM +0100, Philippe Mathieu-Daudé wrote:
> Replace sizeof() flexible arrays union srp_iu/viosrp_iu by the
> SRP_MAX_IU_LEN definition, which is what this code actually meant
> to use.
> 
> Signed-off-by: Philippe Mathieu-Daudé 

Applied to ppc-for-5.0

> ---
>  hw/scsi/spapr_vscsi.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/scsi/spapr_vscsi.c b/hw/scsi/spapr_vscsi.c
> index 7d584e7732..7e397ed797 100644
> --- a/hw/scsi/spapr_vscsi.c
> +++ b/hw/scsi/spapr_vscsi.c
> @@ -671,8 +671,8 @@ static void vscsi_process_login(VSCSIState *s, vscsi_req 
> *req)
>   */
>  rsp->req_lim_delta = cpu_to_be32(VSCSI_REQ_LIMIT-2);
>  rsp->tag = tag;
> -rsp->max_it_iu_len = cpu_to_be32(sizeof(union srp_iu));
> -rsp->max_ti_iu_len = cpu_to_be32(sizeof(union srp_iu));
> +rsp->max_it_iu_len = cpu_to_be32(SRP_MAX_IU_LEN);
> +rsp->max_ti_iu_len = cpu_to_be32(SRP_MAX_IU_LEN);
>  /* direct and indirect */
>  rsp->buf_fmt = cpu_to_be16(SRP_BUF_FORMAT_DIRECT | 
> SRP_BUF_FORMAT_INDIRECT);
>  
> @@ -1088,7 +1088,7 @@ static void vscsi_got_payload(VSCSIState *s, vscsi_crq 
> *crq)
>   * in our 256 bytes IUs. If not we'll have to increase the size
>   * of the structure.
>   */
> -if (crq->s.IU_length > sizeof(union viosrp_iu)) {
> +if (crq->s.IU_length > SRP_MAX_IU_LEN) {
>  fprintf(stderr, "VSCSI: SRP IU too long (%d bytes) !\n",
>  crq->s.IU_length);
>  vscsi_put_req(req);

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

[PATCH 2/3] hw/net/e1000: Move macreg[] arrays to .rodata to save 1MiB of .data

[Philippe Mathieu-Daudé]

Each array consumes 256KiB of .data. As we do not reassign entries,
we can move it to the .rodata section, and save a total of 1MiB of
.data (size reported on x86_64 host).

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/net/e1000.c   | 4 ++--
 hw/net/e1000e_core.c | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/hw/net/e1000.c b/hw/net/e1000.c
index 972d9b5083..9233248c9a 100644
--- a/hw/net/e1000.c
+++ b/hw/net/e1000.c
@@ -1151,7 +1151,7 @@ set_ims(E1000State *s, int index, uint32_t val)
 
 #define getreg(x)[x] = mac_readreg
 typedef uint32_t (*readops)(E1000State *, int);
-static readops macreg_readops[] = {
+static const readops macreg_readops[] = {
 getreg(PBA),  getreg(RCTL), getreg(TDH),  getreg(TXDCTL),
 getreg(WUFC), getreg(TDT),  getreg(CTRL), getreg(LEDCTL),
 getreg(MANC), getreg(MDIC), getreg(SWSM), getreg(STATUS),
@@ -1207,7 +1207,7 @@ enum { NREADOPS = ARRAY_SIZE(macreg_readops) };
 
 #define putreg(x)[x] = mac_writereg
 typedef void (*writeops)(E1000State *, int, uint32_t);
-static writeops macreg_writeops[] = {
+static const writeops macreg_writeops[] = {
 putreg(PBA),  putreg(EERD), putreg(SWSM), putreg(WUFC),
 putreg(TDBAL),putreg(TDBAH),putreg(TXDCTL),   putreg(RDBAH),
 putreg(RDBAL),putreg(LEDCTL),   putreg(VET),  putreg(FCRUC),
diff --git a/hw/net/e1000e_core.c b/hw/net/e1000e_core.c
index 38bdb90114..df957e0c1a 100644
--- a/hw/net/e1000e_core.c
+++ b/hw/net/e1000e_core.c
@@ -2856,7 +2856,7 @@ e1000e_set_gcr(E1000ECore *core, int index, uint32_t val)
 
 #define e1000e_getreg(x)[x] = e1000e_mac_readreg
 typedef uint32_t (*readops)(E1000ECore *, int);
-static readops e1000e_macreg_readops[] = {
+static const readops e1000e_macreg_readops[] = {
 e1000e_getreg(PBA),
 e1000e_getreg(WUFC),
 e1000e_getreg(MANC),
@@ -3063,7 +3063,7 @@ enum { E1000E_NREADOPS = 
ARRAY_SIZE(e1000e_macreg_readops) };
 
 #define e1000e_putreg(x)[x] = e1000e_mac_writereg
 typedef void (*writeops)(E1000ECore *, int, uint32_t);
-static writeops e1000e_macreg_writeops[] = {
+static const writeops e1000e_macreg_writeops[] = {
 e1000e_putreg(PBA),
 e1000e_putreg(SWSM),
 e1000e_putreg(WUFC),
-- 
2.21.1

[PATCH 3/3] virtfs-proxy-helper: Make the helper_opts[] array const

[Philippe Mathieu-Daudé]

Reduce a bit the memory footprint by making the helper_opts[]
array const.

Signed-off-by: Philippe Mathieu-Daudé 
---
 fsdev/virtfs-proxy-helper.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fsdev/virtfs-proxy-helper.c b/fsdev/virtfs-proxy-helper.c
index aa1ab2590d..de061a8a0e 100644
--- a/fsdev/virtfs-proxy-helper.c
+++ b/fsdev/virtfs-proxy-helper.c
@@ -43,7 +43,7 @@
 #define BTRFS_SUPER_MAGIC 0x9123683E
 #endif
 
-static struct option helper_opts[] = {
+static const struct option helper_opts[] = {
 {"fd", required_argument, NULL, 'f'},
 {"path", required_argument, NULL, 'p'},
 {"nodaemon", no_argument, NULL, 'n'},
-- 
2.21.1

[PATCH 0/3] hw/net,virtfs-proxy-helper: Reduce .data footprint

[Philippe Mathieu-Daudé]

More memory footprint reduction, similar to:
https://lists.gnu.org/archive/html/qemu-devel/2020-03/msg00984.html

The elf-dissector tool [1] [2] helped to notice the big array.

[1] https://phabricator.kde.org/source/elf-dissector/
[2] https://www.volkerkrause.eu/2019/06/22/elf-dissector-aarch64-support.html

Philippe Mathieu-Daudé (3):
  hw/net/e1000: Add readops/writeops typedefs
  hw/net/e1000: Move macreg[] arrays to .rodata to save 1MiB of .data
  virtfs-proxy-helper: Make the helper_opts[] array const

 fsdev/virtfs-proxy-helper.c | 2 +-
 hw/net/e1000.c  | 6 --
 hw/net/e1000e_core.c| 6 --
 3 files changed, 9 insertions(+), 5 deletions(-)

-- 
2.21.1

[PATCH 1/3] hw/net/e1000: Add readops/writeops typedefs

[Philippe Mathieu-Daudé]

Express the macreg[] arrays using typedefs.
No logical changes introduced here.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/net/e1000.c   | 6 --
 hw/net/e1000e_core.c | 6 --
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/hw/net/e1000.c b/hw/net/e1000.c
index 0b833d5a15..972d9b5083 100644
--- a/hw/net/e1000.c
+++ b/hw/net/e1000.c
@@ -1150,7 +1150,8 @@ set_ims(E1000State *s, int index, uint32_t val)
 }
 
 #define getreg(x)[x] = mac_readreg
-static uint32_t (*macreg_readops[])(E1000State *, int) = {
+typedef uint32_t (*readops)(E1000State *, int);
+static readops macreg_readops[] = {
 getreg(PBA),  getreg(RCTL), getreg(TDH),  getreg(TXDCTL),
 getreg(WUFC), getreg(TDT),  getreg(CTRL), getreg(LEDCTL),
 getreg(MANC), getreg(MDIC), getreg(SWSM), getreg(STATUS),
@@ -1205,7 +1206,8 @@ static uint32_t (*macreg_readops[])(E1000State *, int) = {
 enum { NREADOPS = ARRAY_SIZE(macreg_readops) };
 
 #define putreg(x)[x] = mac_writereg
-static void (*macreg_writeops[])(E1000State *, int, uint32_t) = {
+typedef void (*writeops)(E1000State *, int, uint32_t);
+static writeops macreg_writeops[] = {
 putreg(PBA),  putreg(EERD), putreg(SWSM), putreg(WUFC),
 putreg(TDBAL),putreg(TDBAH),putreg(TXDCTL),   putreg(RDBAH),
 putreg(RDBAL),putreg(LEDCTL),   putreg(VET),  putreg(FCRUC),
diff --git a/hw/net/e1000e_core.c b/hw/net/e1000e_core.c
index 94ea34dca5..38bdb90114 100644
--- a/hw/net/e1000e_core.c
+++ b/hw/net/e1000e_core.c
@@ -2855,7 +2855,8 @@ e1000e_set_gcr(E1000ECore *core, int index, uint32_t val)
 }
 
 #define e1000e_getreg(x)[x] = e1000e_mac_readreg
-static uint32_t (*e1000e_macreg_readops[])(E1000ECore *, int) = {
+typedef uint32_t (*readops)(E1000ECore *, int);
+static readops e1000e_macreg_readops[] = {
 e1000e_getreg(PBA),
 e1000e_getreg(WUFC),
 e1000e_getreg(MANC),
@@ -3061,7 +3062,8 @@ static uint32_t (*e1000e_macreg_readops[])(E1000ECore *, 
int) = {
 enum { E1000E_NREADOPS = ARRAY_SIZE(e1000e_macreg_readops) };
 
 #define e1000e_putreg(x)[x] = e1000e_mac_writereg
-static void (*e1000e_macreg_writeops[])(E1000ECore *, int, uint32_t) = {
+typedef void (*writeops)(E1000ECore *, int, uint32_t);
+static writeops e1000e_macreg_writeops[] = {
 e1000e_putreg(PBA),
 e1000e_putreg(SWSM),
 e1000e_putreg(WUFC),
-- 
2.21.1

[PATCH 2/2] Makefile: Let the 'help' target list the tools targets

[Philippe Mathieu-Daudé]

List the name of the tool targets when calling 'make help':

  $ make help
  [...]
  Tools targets:
qemu-ga- Build qemu-ga tool
qemu-keymap- Build qemu-keymap tool
elf2dmp- Build elf2dmp tool
ivshmem-client - Build ivshmem-client tool
ivshmem-server - Build ivshmem-server tool
qemu-nbd   - Build qemu-nbd tool
qemu-img   - Build qemu-img tool
qemu-io- Build qemu-io tool
qemu-edid  - Build qemu-edid tool
fsdev/virtfs-proxy-helper  - Build virtfs-proxy-helper tool
scsi/qemu-pr-helper- Build qemu-pr-helper tool

Signed-off-by: Philippe Mathieu-Daudé 
---
 Makefile | 5 +
 1 file changed, 5 insertions(+)

diff --git a/Makefile b/Makefile
index a9ede4f428..4a1d3e2a79 100644
--- a/Makefile
+++ b/Makefile
@@ -1240,6 +1240,11 @@ endif
$(foreach t, $(TARGET_DIRS), \
$(call print-help-run,$(t)/all,Build for $(t));) \
echo '')
+   @$(if $(TOOLS), \
+   echo 'Tools targets:'; \
+   $(foreach t, $(TOOLS), \
+   $(call print-help-run,$(t),Build $(shell basename $(t)) tool);) 
\
+   echo '')
@echo  'Cleaning targets:'
$(call print-help,clean,Remove most generated files but keep the config)
 ifdef CONFIG_GCOV
-- 
2.21.1

[PATCH 1/2] Makefile: Align 'help' target output

[Philippe Mathieu-Daudé]

The 'help' target is displayed unaligned. Add a print-help
function and use it. Now if someone want to change the
indentation, there is a single place to modify.

Signed-off-by: Philippe Mathieu-Daudé 
---
 Makefile | 44 +++-
 1 file changed, 23 insertions(+), 21 deletions(-)

diff --git a/Makefile b/Makefile
index aa9cc0b584..a9ede4f428 100644
--- a/Makefile
+++ b/Makefile
@@ -1220,50 +1220,52 @@ endif
 include $(SRC_PATH)/tests/docker/Makefile.include
 include $(SRC_PATH)/tests/vm/Makefile.include
 
+print-help-run = printf "  %-30s - %s\\n" "$1" "$2"
+print-help = $(quiet-@)$(call print-help-run,$1,$2)
+
 .PHONY: help
 help:
@echo  'Generic targets:'
-   @echo  '  all - Build all'
+   $(call print-help,all,Build all)
 ifdef CONFIG_MODULES
-   @echo  '  modules - Build all modules'
+   $(call print-help,modules,Build all modules)
 endif
-   @echo  '  dir/file.o  - Build specified target only'
-   @echo  '  install - Install QEMU, documentation and tools'
-   @echo  '  ctags/TAGS  - Generate tags file for editors'
-   @echo  '  cscope  - Generate cscope index'
+   $(call print-help,dir/file.o,Build specified target only)
+   $(call print-help,install,Install QEMU, documentation and tools)
+   $(call print-help,ctags/TAGS,Generate tags file for editors)
+   $(call print-help,cscope,Generate cscope index)
@echo  ''
@$(if $(TARGET_DIRS), \
echo 'Architecture specific targets:'; \
$(foreach t, $(TARGET_DIRS), \
-   printf "  %-30s - Build for %s\\n" $(t)/all $(t);) \
+   $(call print-help-run,$(t)/all,Build for $(t));) \
echo '')
@echo  'Cleaning targets:'
-   @echo  '  clean   - Remove most generated files but keep the 
config'
+   $(call print-help,clean,Remove most generated files but keep the config)
 ifdef CONFIG_GCOV
-   @echo  '  clean-coverage  - Remove coverage files'
+   $(call print-help,clean-coverage,Remove coverage files)
 endif
-   @echo  '  distclean   - Remove all generated files'
-   @echo  '  dist- Build a distributable tarball'
+   $(call print-help,distclean,Remove all generated files)
+   $(call print-help,dist,Build a distributable tarball)
@echo  ''
@echo  'Test targets:'
-   @echo  '  check   - Run all tests (check-help for details)'
-   @echo  '  docker  - Help about targets running tests inside 
containers'
-   @echo  '  vm-help - Help about targets running tests inside VM'
+   $(call print-help,check,Run all tests (check-help for details))
+   $(call print-help,docker,Help about targets running tests inside 
containers)
+   $(call print-help,vm-help,Help about targets running tests inside VM)
@echo  ''
@echo  'Documentation targets:'
-   @echo  '  html info pdf txt'
-   @echo  '  - Build documentation in specified format'
+   $(call print-help,html info pdf txt,Build documentation in specified 
format)
 ifdef CONFIG_GCOV
-   @echo  '  coverage-report - Create code coverage report'
+   $(call print-help,coverage-report,Create code coverage report)
 endif
@echo  ''
 ifdef CONFIG_WIN32
@echo  'Windows targets:'
-   @echo  '  installer   - Build NSIS-based installer for QEMU'
+   $(call print-help,installer,Build NSIS-based installer for QEMU)
 ifdef QEMU_GA_MSI_ENABLED
-   @echo  '  msi - Build MSI-based installer for qemu-ga'
+   $(call print-help,msi,Build MSI-based installer for qemu-ga)
 endif
@echo  ''
 endif
-   @echo  '  $(MAKE) [targets]  (quiet build, default)'
-   @echo  '  $(MAKE) V=1 [targets]  (verbose build)'
+   $(call print-help,$(MAKE) [targets],(quiet build, default))
+   $(call print-help,$(MAKE) V=1 [targets],(verbose build))
-- 
2.21.1

[PATCH 0/2] Makefile: Re-indent 'help' target and list tools

[Philippe Mathieu-Daudé]

This series improves the 'make help' output:
- re-indent current output
- list tools targets:

$ make help
[...]

Architecture specific targets:
  x86_64-softmmu/all - Build for x86_64-softmmu

Tools targets:
  qemu-ga- Build qemu-ga tool
  qemu-keymap- Build qemu-keymap tool
  elf2dmp- Build elf2dmp tool
  ivshmem-client - Build ivshmem-client tool
  ivshmem-server - Build ivshmem-server tool
  qemu-nbd   - Build qemu-nbd tool
  qemu-img   - Build qemu-img tool
  qemu-io- Build qemu-io tool
  qemu-edid  - Build qemu-edid tool
  fsdev/virtfs-proxy-helper  - Build virtfs-proxy-helper tool
  scsi/qemu-pr-helper- Build qemu-pr-helper tool

Cleaning targets:
  clean  - Remove most generated files but keep t=
he config
  distclean  - Remove all generated files
  dist   - Build a distributable tarball

Philippe Mathieu-Daud=C3=A9 (2):
  Makefile: Align 'help' target output
  Makefile: Let the 'help' target list the tools targets

 Makefile | 49 -
 1 file changed, 28 insertions(+), 21 deletions(-)

--=20
2.21.1

[PATCH] linux-user: do prlimit selectively

[Tobias Koch]

Analogous to what commit 5dfa88f7 did for setrlimit, this commit
selectively ignores limits for memory-related resources in prlimit64
calls. This is to prevent too restrictive limits from causing QEMU
itself to malfunction.

Signed-off-by: Tobias Koch 
---
 linux-user/syscall.c | 16 ++--
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 8d27d10807..8554c77a38 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -11872,13 +11872,17 @@ static abi_long do_syscall1(void *cpu_env, int
num, abi_long arg1,
 struct host_rlimit64 rnew, rold, *rnewp = 0;
 int resource = target_to_host_resource(arg2);
 if (arg3) {
-    if (!lock_user_struct(VERIFY_READ, target_rnew, arg3, 1)) {
-    return -TARGET_EFAULT;
+    if (resource != RLIMIT_AS &&
+    resource != RLIMIT_DATA &&
+    resource != RLIMIT_STACK) {
+    if (!lock_user_struct(VERIFY_READ, target_rnew, arg3, 1)) {
+    return -TARGET_EFAULT;
+    }
+    rnew.rlim_cur = tswap64(target_rnew->rlim_cur);
+    rnew.rlim_max = tswap64(target_rnew->rlim_max);
+    unlock_user_struct(target_rnew, arg3, 0);
+    rnewp = &rnew;
 }
-    rnew.rlim_cur = tswap64(target_rnew->rlim_cur);
-    rnew.rlim_max = tswap64(target_rnew->rlim_max);
-    unlock_user_struct(target_rnew, arg3, 0);
-    rnewp = &rnew;
 }
 
 ret = get_errno(sys_prlimit64(arg1, resource, rnewp, arg4 ?
&rold : 0));
-- 
2.20.1

Re: [PATCH] hw/ide: Remove status register read side effect

[jasper.lowell]

> Yes, that's correct. However I'm quite confident from booting other
> non-Solaris OSs
> under qemu-system-sparc64 that PCI native mode is being used,
> particularly as it is
> possible to see the related PCI sabre IRQ routing configuration
> changes.

Considering that Solaris 10 is accessing CFR and ARTTIM23 I don't think
there is any doubt that it is using the chip in PCI native mode. I
don't think Solaris 10 even has support for legacy mode.

Thanks,
Jasper Lowell.

On Wed, 2020-03-04 at 21:07 +, Mark Cave-Ayland wrote:
> On 04/03/2020 03:11, jasper.low...@bt.com wrote:
> 
> > > cmd646_update_irq() only seems to raise PCI interrupt, should it
> > > also
> > > have 
> > > an option to use INT 14 and 15 in legacy mode similar to what my
> > > patch 
> > > does for via-ide?
> > 
> > Looking through /qemu/hw/ide/cmd646.c it doesn't look like QEMU has
> > support for legacy mode. At the very least, it looks like we
> > default to
> > PCI native mode:
> > 
> > static void pci_cmd646_ide_realize(PCIDevice *dev, Error **errp)
> > ...
> > pci_conf[PCI_CLASS_PROG] = 0x8f;
> > ...
> > 
> > To add support for legacy mode it would require changing
> > cmd646_update_irq() and maybe cmd646_set_irq() so that interrupts
> > are
> > conditionally raised on IRQ14 and/or IRQ15 when the ports are in
> > legacy
> > mode.
> 
> Yes, that's correct. However I'm quite confident from booting other
> non-Solaris OSs
> under qemu-system-sparc64 that PCI native mode is being used,
> particularly as it is
> possible to see the related PCI sabre IRQ routing configuration
> changes.
> 
> 
> ATB,
> 
> Mark.

Re: [PATCH v3 12/20] hw/ide: Let the DMAIntFunc prototype use a boolean 'is_write' argument

[John Snow]

On 2/20/20 8:05 AM, Philippe Mathieu-Daudé wrote:
> The 'is_write' argument is either 0 or 1.
> Convert it to a boolean type.
> 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  include/hw/ide/internal.h | 2 +-
>  hw/dma/rc4030.c   | 6 +++---
>  hw/ide/ahci.c | 2 +-
>  hw/ide/core.c | 2 +-
>  hw/ide/macio.c| 2 +-
>  hw/ide/pci.c  | 2 +-
>  6 files changed, 8 insertions(+), 8 deletions(-)

Acked-by: John Snow 

Re: [PATCH v3 11/20] hw/ide/internal: Remove unused DMARestartFunc typedef

[John Snow]

On 2/20/20 8:05 AM, Philippe Mathieu-Daudé wrote:
> The IDE DMA restart callback has been removed in commit fe09c7c9f0.
> 
> Fixes: fe09c7c9f0
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  include/hw/ide/internal.h | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/include/hw/ide/internal.h b/include/hw/ide/internal.h
> index 52ec197da0..ce766ac485 100644
> --- a/include/hw/ide/internal.h
> +++ b/include/hw/ide/internal.h
> @@ -326,7 +326,6 @@ typedef int DMAIntFunc(IDEDMA *, int);
>  typedef int32_t DMAInt32Func(IDEDMA *, int32_t len);
>  typedef void DMAu32Func(IDEDMA *, uint32_t);
>  typedef void DMAStopFunc(IDEDMA *, bool);
> -typedef void DMARestartFunc(void *, int, RunState);
>  
>  struct unreported_events {
>  bool eject_request;
> 

Acked-by: John Snow 

[PATCH v2 3/3] iotests: modify test 040 to use JobRunner

[John Snow]

Instead of having somewhat reproduced it for itself.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/040 | 51 +-
 1 file changed, 25 insertions(+), 26 deletions(-)

diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040
index 90b59081ff..e2ef3bb812 100755
--- a/tests/qemu-iotests/040
+++ b/tests/qemu-iotests/040
@@ -483,34 +483,33 @@ class TestErrorHandling(iotests.QMPTestCase):
   file=('top-dbg' if top_debug else 'top-file'),
   backing='mid-fmt')
 
+
+class TestJobRunner(iotests.JobRunner):
+expected_events = ('BLOCK_JOB_COMPLETED',
+   'BLOCK_JOB_ERROR',
+   'BLOCK_JOB_READY')
+
+def __init__(self, *args, test, **kwargs):
+super().__init__(*args, **kwargs)
+self.log = []
+self.test = test
+
+def on_pause(self, event):
+super().on_pause(event)
+result = self._vm.qmp('block-job-resume', device=self._id)
+self.test.assert_qmp(result, 'return', {})
+
+def on_block_job_event(self, event):
+if event['event'] not in self.expected_events:
+self.test.fail("Unexpected event: %s" % event)
+super().on_block_job_event(event)
+self.log.append(event)
+
 def run_job(self, expected_events, error_pauses_job=False):
-match_device = {'data': {'device': 'job0'}}
-events = {
-'BLOCK_JOB_COMPLETED': match_device,
-'BLOCK_JOB_CANCELLED': match_device,
-'BLOCK_JOB_ERROR': match_device,
-'BLOCK_JOB_READY': match_device,
-}
-
-completed = False
-log = []
-while not completed:
-ev = self.vm.events_wait(events, timeout=5.0)
-if ev['event'] == 'BLOCK_JOB_COMPLETED':
-completed = True
-elif ev['event'] == 'BLOCK_JOB_ERROR':
-if error_pauses_job:
-result = self.vm.qmp('block-job-resume', device='job0')
-self.assert_qmp(result, 'return', {})
-elif ev['event'] == 'BLOCK_JOB_READY':
-result = self.vm.qmp('block-job-complete', device='job0')
-self.assert_qmp(result, 'return', {})
-else:
-self.fail("Unexpected event: %s" % ev)
-log.append(iotests.filter_qmp_event(ev))
-
+job = self.TestJobRunner(self.vm, 'job0', test=self)
+job.run()
 self.maxDiff = None
-self.assertEqual(expected_events, log)
+self.assertEqual(expected_events, job.log)
 
 def event_error(self, op, action):
 return {
-- 
2.21.1

[PATCH v2 1/3] qmp.py: change event_wait to use a dict

[John Snow]

It's easier to work with than a list of tuples, because we can check the
keys for membership.

Signed-off-by: John Snow 
---
 python/qemu/machine.py| 10 +-
 tests/qemu-iotests/040| 12 ++--
 tests/qemu-iotests/260|  5 +++--
 tests/qemu-iotests/iotests.py | 16 
 4 files changed, 22 insertions(+), 21 deletions(-)

diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 183d8f3d38..748de5f322 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -476,21 +476,21 @@ def event_wait(self, name, timeout=60.0, match=None):
 timeout: QEMUMonitorProtocol.pull_event timeout parameter.
 match: Optional match criteria. See event_match for details.
 """
-return self.events_wait([(name, match)], timeout)
+return self.events_wait({name: match}, timeout)
 
 def events_wait(self, events, timeout=60.0):
 """
 events_wait waits for and returns a named event from QMP with a 
timeout.
 
-events: a sequence of (name, match_criteria) tuples.
+events: a mapping containing {name: match_criteria}.
 The match criteria are optional and may be None.
 See event_match for details.
 timeout: QEMUMonitorProtocol.pull_event timeout parameter.
 """
 def _match(event):
-for name, match in events:
-if event['event'] == name and self.event_match(event, match):
-return True
+name = event['event']
+if name in events:
+return self.event_match(event, events[name])
 return False
 
 # Search cached events
diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040
index 32c82b4ec6..90b59081ff 100755
--- a/tests/qemu-iotests/040
+++ b/tests/qemu-iotests/040
@@ -485,12 +485,12 @@ class TestErrorHandling(iotests.QMPTestCase):
 
 def run_job(self, expected_events, error_pauses_job=False):
 match_device = {'data': {'device': 'job0'}}
-events = [
-('BLOCK_JOB_COMPLETED', match_device),
-('BLOCK_JOB_CANCELLED', match_device),
-('BLOCK_JOB_ERROR', match_device),
-('BLOCK_JOB_READY', match_device),
-]
+events = {
+'BLOCK_JOB_COMPLETED': match_device,
+'BLOCK_JOB_CANCELLED': match_device,
+'BLOCK_JOB_ERROR': match_device,
+'BLOCK_JOB_READY': match_device,
+}
 
 completed = False
 log = []
diff --git a/tests/qemu-iotests/260 b/tests/qemu-iotests/260
index 804a7addb9..729f031122 100755
--- a/tests/qemu-iotests/260
+++ b/tests/qemu-iotests/260
@@ -67,8 +67,9 @@ def test(persistent, restart):
 
 vm.qmp_log('block-commit', device='drive0', top=top,
filters=[iotests.filter_qmp_testfiles])
-ev = vm.events_wait((('BLOCK_JOB_READY', None),
- ('BLOCK_JOB_COMPLETED', None)))
+ev = vm.events_wait({
+'BLOCK_JOB_READY': None,
+'BLOCK_JOB_COMPLETED': None })
 log(filter_qmp_event(ev))
 if (ev['event'] == 'BLOCK_JOB_COMPLETED'):
 vm.shutdown()
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 3e606c35ef..2625001978 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -614,14 +614,14 @@ def run_job(self, job, auto_finalize=True, 
auto_dismiss=False,
 """
 match_device = {'data': {'device': job}}
 match_id = {'data': {'id': job}}
-events = [
-('BLOCK_JOB_COMPLETED', match_device),
-('BLOCK_JOB_CANCELLED', match_device),
-('BLOCK_JOB_ERROR', match_device),
-('BLOCK_JOB_READY', match_device),
-('BLOCK_JOB_PENDING', match_id),
-('JOB_STATUS_CHANGE', match_id)
-]
+events = {
+'BLOCK_JOB_COMPLETED': match_device,
+'BLOCK_JOB_CANCELLED': match_device,
+'BLOCK_JOB_ERROR': match_device,
+'BLOCK_JOB_READY': match_device,
+'BLOCK_JOB_PENDING': match_id,
+'JOB_STATUS_CHANGE': match_id,
+}
 error = None
 while True:
 ev = filter_qmp_event(self.events_wait(events, timeout=wait))
-- 
2.21.1

[PATCH v2 0/3] iotests: add JobRunner framework

[John Snow]

Requires: 20200304213818.15341-1-js...@redhat.com

(This requires the iotests pylint & logging series.)

The basic idea is to make a generic job runtime manager and allow
callers to subclass the manager. Then, instead of adding callback
arguments to the function all the time, we have à la carte customization
of the loop.

To showcase this a little bit, I removed the pre_finalization argument
and made existing callers use a custom JobRunner; and then converted
test 040 to use this style of job runner.

Is it a simplification? No. Is it cool? Maybe. Did it remove the
duplicated job-running code in 040? yes.

V2:
 - Rebased on logging series; logging conditionals are pretty now.
 - Inlined callback login in 257
 - No longer based on bitmap-populate job (no test 287)
 - Moved super() call to the beginning of test 040's callback
 - Added docstrings and type annotations

John Snow (3):
  qmp.py: change event_wait to use a dict
  iotests: add JobRunner class
  iotests: modify test 040 to use JobRunner

 python/qemu/machine.py|  10 +-
 tests/qemu-iotests/040|  51 +
 tests/qemu-iotests/255|   9 +-
 tests/qemu-iotests/257|  54 +
 tests/qemu-iotests/260|   5 +-
 tests/qemu-iotests/iotests.py | 201 +-
 tests/qemu-iotests/pylintrc   |  11 ++
 7 files changed, 233 insertions(+), 108 deletions(-)

-- 
2.21.1

[PATCH v2 2/3] iotests: add JobRunner class

[John Snow]

The idea is that instead of increasing the arguments to job_run all the
time, create a more general-purpose job runner that can be subclassed to
do interesting things with.

pylint note: the 'callbacks' option guards against unused warning
arguments in functions designated as callbacks. It does not currently
guard against "no-self-use" though; hence a once-off ignore.

mypy note: QapiEvent is only a weak alias; it's fully interchangable
with the type it's declared as. In the future, we may wish to tighten
these types. For now, this communicates the rough shape of the type and
(more importantly) the intent.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/255|   9 +-
 tests/qemu-iotests/257|  54 +
 tests/qemu-iotests/iotests.py | 201 +-
 tests/qemu-iotests/pylintrc   |  11 ++
 4 files changed, 200 insertions(+), 75 deletions(-)

diff --git a/tests/qemu-iotests/255 b/tests/qemu-iotests/255
index 8f08f741da..e66cdfd672 100755
--- a/tests/qemu-iotests/255
+++ b/tests/qemu-iotests/255
@@ -71,8 +71,13 @@ with iotests.FilePath('t.qcow2') as disk_path, \
 result = vm.qmp_log('block-commit', job_id='job0', auto_finalize=False,
 device='overlay', top_node='mid')
 
-vm.run_job('job0', auto_finalize=False, pre_finalize=start_requests,
-auto_dismiss=True)
+class TestJobRunner(iotests.JobRunner):
+def on_pending(self, event):
+start_requests()
+super().on_pending(event)
+
+runner = TestJobRunner(vm, 'job0', auto_finalize=False, auto_dismiss=True)
+runner.run()
 
 vm.shutdown()
 
diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index 004a433b8b..95341c330f 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -352,30 +352,40 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', 
failure=None):
 job = backup(drive0, 1, bsync1, msync_mode,
  bitmap="bitmap0", bitmap_mode=bsync_mode)
 
-def _callback():
-"""Issue writes while the job is open to test bitmap divergence."""
-# Note: when `failure` is 'intermediate', this isn't called.
-log('')
-bitmaps = perform_writes(drive0, 2, filter_node_name='backup-top')
-# Named bitmap (static, should be unchanged)
-ebitmap.compare(vm.get_bitmap(drive0.node, 'bitmap0',
-  bitmaps=bitmaps))
-# Anonymous bitmap (dynamic, shows new writes)
-anonymous = EmulatedBitmap()
-anonymous.dirty_group(2)
-anonymous.compare(vm.get_bitmap(drive0.node, '', recording=True,
-bitmaps=bitmaps))
 
-# Simulate the order in which this will happen:
-# group 1 gets cleared first, then group two gets written.
-if ((bsync_mode == 'on-success' and not failure) or
-(bsync_mode == 'always')):
-ebitmap.clear()
-ebitmap.dirty_group(2)
+class TestJobRunner(iotests.JobRunner):
+def on_pending(self, event):
+"""
+Issue writes while the job is open to test bitmap divergence.
+"""
+
+# Note: when `failure` is 'intermediate', this isn't called.
+log('')
+bitmaps = perform_writes(drive0, 2,
+ filter_node_name='backup-top')
+# Named bitmap (static, should be unchanged)
+ebitmap.compare(vm.get_bitmap(drive0.node, 'bitmap0',
+  bitmaps=bitmaps))
+# Anonymous bitmap (dynamic, shows new writes)
+anonymous = EmulatedBitmap()
+anonymous.dirty_group(2)
+anonymous.compare(vm.get_bitmap(drive0.node, '', 
recording=True,
+bitmaps=bitmaps))
+
+# Simulate the order in which this will happen:
+# group 1 gets cleared first, then group two gets written.
+if ((bsync_mode == 'on-success' and not failure) or
+(bsync_mode == 'always')):
+ebitmap.clear()
+ebitmap.dirty_group(2)
+
+super().on_pending(event)
+
+
+runner = TestJobRunner(vm, job, cancel=(failure == 'simulated'),
+   auto_finalize=False, auto_dismiss=True)
+runner.run()
 
-vm.run_job(job, auto_dismiss=True, auto_finalize=False,
-   pre_finalize=_callback,
-   cancel=(failure == 'simulated'))
 bitmaps = vm.query_bitmaps()
 log({'bitmaps': bitmaps}, indent=2)
 log('')
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 2625001978..90d42cdff1 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-io

Re: [PATCH v1 1/3] riscv/sifive_u: Fix up file ordering

[Alistair Francis]

On Wed, Mar 4, 2020 at 6:10 AM Bin Meng  wrote:
>
> On Wed, Mar 4, 2020 at 9:37 AM Alistair Francis
>  wrote:
> >
> > Split the file into clear machine and SoC sections.
> >
>
> Yep, I found functions in this file are a little bit confusing as well ..
>
> > Signed-off-by: Alistair Francis 
> > ---
> >  hw/riscv/sifive_u.c | 107 ++--
> >  1 file changed, 54 insertions(+), 53 deletions(-)
> >
> > diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
> > index 156a003642..9a0145b5b4 100644
> > --- a/hw/riscv/sifive_u.c
> > +++ b/hw/riscv/sifive_u.c
> > @@ -399,6 +399,60 @@ static void riscv_sifive_u_init(MachineState *machine)
>
> So while we are here, could we rename this to something like:
> sifive_u_machine_init()? ie: drop the riscv_ prefix.
>
> >&address_space_memory);
> >  }
> >
> > +static bool sifive_u_get_start_in_flash(Object *obj, Error **errp)
>
> and sifive_u_machine_get_start_in_flash()? and other functions too?
>
> > +{
> > +SiFiveUState *s = RISCV_U_MACHINE(obj);
> > +
> > +return s->start_in_flash;
> > +}
> > +
> > +static void sifive_u_set_start_in_flash(Object *obj, bool value, Error 
> > **errp)
> > +{
> > +SiFiveUState *s = RISCV_U_MACHINE(obj);
> > +
> > +s->start_in_flash = value;
> > +}
> > +
> > +static void riscv_sifive_u_machine_instance_init(Object *obj)
> > +{
> > +SiFiveUState *s = RISCV_U_MACHINE(obj);
> > +
> > +s->start_in_flash = false;
> > +object_property_add_bool(obj, "start-in-flash", 
> > sifive_u_get_start_in_flash,
> > + sifive_u_set_start_in_flash, NULL);
> > +object_property_set_description(obj, "start-in-flash",
> > +"Set on to tell QEMU's ROM to jump to 
> > " \
> > +"flash. Otherwise QEMU will jump to 
> > DRAM",
> > +NULL);
> > +}
> > +
> > +
> > +static void riscv_sifive_u_machine_class_init(ObjectClass *oc, void *data)
> > +{
> > +MachineClass *mc = MACHINE_CLASS(oc);
> > +
> > +mc->desc = "RISC-V Board compatible with SiFive U SDK";
> > +mc->init = riscv_sifive_u_init;
> > +mc->max_cpus = SIFIVE_U_MANAGEMENT_CPU_COUNT + 
> > SIFIVE_U_COMPUTE_CPU_COUNT;
> > +mc->min_cpus = SIFIVE_U_MANAGEMENT_CPU_COUNT + 1;
> > +mc->default_cpus = mc->min_cpus;
> > +}
> > +
> > +static const TypeInfo riscv_sifive_u_machine_typeinfo = {
> > +.name   = MACHINE_TYPE_NAME("sifive_u"),
> > +.parent = TYPE_MACHINE,
> > +.class_init = riscv_sifive_u_machine_class_init,
> > +.instance_init = riscv_sifive_u_machine_instance_init,
> > +.instance_size = sizeof(SiFiveUState),
> > +};
> > +
> > +static void riscv_sifive_u_machine_init_register_types(void)
> > +{
> > +type_register_static(&riscv_sifive_u_machine_typeinfo);
> > +}
> > +
> > +type_init(riscv_sifive_u_machine_init_register_types)
> > +
> >  static void riscv_sifive_u_soc_init(Object *obj)
>
> Similarly this can be renamed to: sifive_u_soc_init(), and other soc 
> functions.

I missed this in v2, fixed in the next version.

Alistair

>
> >  {
> >  MachineState *ms = MACHINE(qdev_get_machine());
> > @@ -439,33 +493,6 @@ static void riscv_sifive_u_soc_init(Object *obj)
> >TYPE_CADENCE_GEM);
> >  }
> >
> > -static bool sifive_u_get_start_in_flash(Object *obj, Error **errp)
> > -{
> > -SiFiveUState *s = RISCV_U_MACHINE(obj);
> > -
> > -return s->start_in_flash;
> > -}
> > -
> > -static void sifive_u_set_start_in_flash(Object *obj, bool value, Error 
> > **errp)
> > -{
> > -SiFiveUState *s = RISCV_U_MACHINE(obj);
> > -
> > -s->start_in_flash = value;
> > -}
> > -
> > -static void riscv_sifive_u_machine_instance_init(Object *obj)
> > -{
> > -SiFiveUState *s = RISCV_U_MACHINE(obj);
> > -
> > -s->start_in_flash = false;
> > -object_property_add_bool(obj, "start-in-flash", 
> > sifive_u_get_start_in_flash,
> > - sifive_u_set_start_in_flash, NULL);
> > -object_property_set_description(obj, "start-in-flash",
> > -"Set on to tell QEMU's ROM to jump to 
> > " \
> > -"flash. Otherwise QEMU will jump to 
> > DRAM",
> > -NULL);
> > -}
> > -
> >  static void riscv_sifive_u_soc_realize(DeviceState *dev, Error **errp)
> >  {
> >  MachineState *ms = MACHINE(qdev_get_machine());
> > @@ -603,29 +630,3 @@ static void riscv_sifive_u_soc_register_types(void)
> >  }
> >
> >  type_init(riscv_sifive_u_soc_register_types)
> > -
> > -static void riscv_sifive_u_machine_class_init(ObjectClass *oc, void *data)
> > -{
> > -MachineClass *mc = MACHINE_CLASS(oc);
> > -
> > -mc->desc = "RISC-V Board compatible with SiFive U SDK";
> > -mc->init = riscv_sifive_u_init;
> > -mc->max_cpus = SIFIVE_U_MANAGEMENT_CPU_COUNT + 
> > SIFIVE_U_COMPUTE_CPU_COUNT;
> > -mc->

Re: [PATCH v2 0/3] hw/riscv: Add a serial property to the sifive_u machine

[no-reply]

Patchew URL: 
https://patchew.org/QEMU/cover.1583362888.git.alistair.fran...@wdc.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing 
commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

qemu-system-x86_64: -accel kvm: failed to initialize kvm: No such file or 
directory
qemu-system-x86_64: falling back to tcg
**
ERROR:/tmp/qemu-test/src/tests/qtest/migration-helpers.c:119:check_migration_status:
 assertion failed (current_status != "completed"): ("completed" != "completed")
ERROR - Bail out! 
ERROR:/tmp/qemu-test/src/tests/qtest/migration-helpers.c:119:check_migration_status:
 assertion failed (current_status != "completed"): ("completed" != "completed")
make: *** [check-qtest-x86_64] Error 1
make: *** Waiting for unfinished jobs
  TESTiotest-qcow2: 061
  TESTiotest-qcow2: 062
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', 
'--label', 'com.qemu.instance.uuid=7818f7bb68234e3daee3c182689ac0db', '-u', 
'1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', 
'-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 
'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', 
'/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', 
'/var/tmp/patchew-tester-tmp-fgt66d82/src/docker-src.2020-03-04-18.40.35.9848:/var/tmp/qemu:z,ro',
 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit 
status 2.
filter=--filter=label=com.qemu.instance.uuid=7818f7bb68234e3daee3c182689ac0db
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-fgt66d82/src'
make: *** [docker-run-test-quick@centos7] Error 2

real12m6.842s
user0m5.918s


The full log is available at
http://patchew.org/logs/cover.1583362888.git.alistair.fran...@wdc.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [PULL 1/1] qxl: introduce hardware revision 5

[Laszlo Ersek]

Hi Gerd,

On 02/13/20 10:06, Gerd Hoffmann wrote:
> The only difference to hardware revision 4 is that the device doesn't
> switch to VGA mode in case someone happens to touch a VGA register,
> which should make things more robust in configurations with multiple
> vga devices.
>
> Swtiching back to VGA mode happens on reset, either full machine
> reset or qxl device reset (QXL_IO_RESET ioport command).
>
> Signed-off-by: Gerd Hoffmann 
> Reviewed-by: Maxim Levitsky 
> Message-id: 20200206074358.4274-1-kra...@redhat.com
> ---
>  hw/display/qxl.h  | 2 +-
>  hw/core/machine.c | 2 ++
>  hw/display/qxl.c  | 7 ++-
>  3 files changed, 9 insertions(+), 2 deletions(-)
>
> diff --git a/hw/display/qxl.h b/hw/display/qxl.h
> index 80eb0d267269..707631a1f573 100644
> --- a/hw/display/qxl.h
> +++ b/hw/display/qxl.h
> @@ -144,7 +144,7 @@ typedef struct PCIQXLDevice {
>  }   \
>  } while (0)
>
> -#define QXL_DEFAULT_REVISION QXL_REVISION_STABLE_V12
> +#define QXL_DEFAULT_REVISION (QXL_REVISION_STABLE_V12 + 1)
>
>  /* qxl.c */
>  void *qxl_phys2virt(PCIQXLDevice *qxl, QXLPHYSICAL phys, int group_id);
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index d8e30e4895d8..84812a1d1cc1 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -34,6 +34,8 @@ GlobalProperty hw_compat_4_2[] = {
>  { "vhost-blk-device", "seg_max_adjust", "off"},
>  { "usb-host", "suppress-remote-wake", "off" },
>  { "usb-redir", "suppress-remote-wake", "off" },
> +{ "qxl", "revision", "4" },
> +{ "qxl-vga", "revision", "4" },
>  };
>  const size_t hw_compat_4_2_len = G_N_ELEMENTS(hw_compat_4_2);
>
> diff --git a/hw/display/qxl.c b/hw/display/qxl.c
> index c33b1915a52c..64884da70857 100644
> --- a/hw/display/qxl.c
> +++ b/hw/display/qxl.c
> @@ -1309,7 +1309,8 @@ static void qxl_vga_ioport_write(void *opaque, uint32_t 
> addr, uint32_t val)
>  PCIQXLDevice *qxl = container_of(vga, PCIQXLDevice, vga);
>
>  trace_qxl_io_write_vga(qxl->id, qxl_mode_to_string(qxl->mode), addr, 
> val);
> -if (qxl->mode != QXL_MODE_VGA) {
> +if (qxl->mode != QXL_MODE_VGA &&
> +qxl->revision <= QXL_REVISION_STABLE_V12) {
>  qxl_destroy_primary(qxl, QXL_SYNC);
>  qxl_soft_reset(qxl);
>  }
> @@ -2121,6 +2122,10 @@ static void qxl_realize_common(PCIQXLDevice *qxl, 
> Error **errp)
>  pci_device_rev = QXL_REVISION_STABLE_V12;
>  io_size = pow2ceil(QXL_IO_RANGE_SIZE);
>  break;
> +case 5: /* qxl-5 */
> +pci_device_rev = QXL_REVISION_STABLE_V12 + 1;
> +io_size = pow2ceil(QXL_IO_RANGE_SIZE);
> +break;
>  default:
>  error_setg(errp, "Invalid revision %d for qxl device (max %d)",
> qxl->revision, QXL_DEFAULT_REVISION);
>

this patch -- commit ed71c09ffd6f -- disables ACPI S3 in the Windows 10
guest for me, using OVMF and QXL.

The "Sleep" menu entry disappears from the power button icon/menu at the
login screen, and "psshutdown -d -t 3" (from the pstools package) also
stops working (it reports that the computer does not support S3).

At the parent commit (e18e5501d8ac), S3 suspend/resume works.

Here's the bisection log:

> git bisect start
> # good: [928173659d6e5dc368284f73f90ea1d129e1f57d] Merge remote-tracking 
> branch 'remotes/pmaydell/tags/pull-target-arm-20200130' into staging
> git bisect good 928173659d6e5dc368284f73f90ea1d129e1f57d
> # good: [93c86fff53a267f657e79ec07dcd04b63882e330] Merge remote-tracking 
> branch 'remotes/pmaydell/tags/pull-target-arm-20200207' into staging
> git bisect good 93c86fff53a267f657e79ec07dcd04b63882e330
> # bad: [db736e0437aa6fd7c1b7e4599c17f9619ab6b837] Merge remote-tracking 
> branch 'remotes/bonzini/tags/for-upstream' into staging
> git bisect bad db736e0437aa6fd7c1b7e4599c17f9619ab6b837
> # bad: [8ee06e4ccb0f447caf9dc884b98986c155915e5c] ppc/mac_oldworld: use 
> memdev for RAM
> git bisect bad 8ee06e4ccb0f447caf9dc884b98986c155915e5c
> # good: [dc7a88d0810ad272bdcd2e0869359af78fdd9114] target/arm: Implement 
> ARMv8.1-VMID16 extension
> git bisect good dc7a88d0810ad272bdcd2e0869359af78fdd9114
> # bad: [652c5bbd7e7d3cb3d27a2e0590118dc79fb6f4d8] Merge remote-tracking 
> branch 'remotes/vivier2/tags/linux-user-for-5.0-pull-request' into staging
> git bisect bad 652c5bbd7e7d3cb3d27a2e0590118dc79fb6f4d8
> # bad: [e050e426782ec4ae6bf84e5ec75ca502187f69cb] qapi: Use explicit bulleted 
> lists
> git bisect bad e050e426782ec4ae6bf84e5ec75ca502187f69cb
> # good: [5d6542bea780ad443c4f7f1496e64706101f525c] Merge remote-tracking 
> branch 'remotes/rth/tags/pull-tcg-20200212' into staging
> git bisect good 5d6542bea780ad443c4f7f1496e64706101f525c
> # bad: [72ec8bf362b24ebbd45452c298a3b14fb617eebb] qga/qapi-schema.json: Fix 
> missing '-' in GuestDiskBusType doc comment
> git bisect bad 72ec8bf362b24ebbd45452c298a3b14fb617eebb
> # bad: [2eb054c237d0b5427f62499f3c31e90cf87821d7] configure: Allow user to 
> specify sphin

Re: [PATCH v2] riscv: sifive_u: Add a "serial" property for board serial number

[Alistair Francis]

On Wed, Mar 4, 2020 at 3:00 PM Palmer Dabbelt  wrote:
>
> On Sun, 16 Feb 2020 05:55:17 PST (-0800), bmeng...@gmail.com wrote:
> > At present the board serial number is hard-coded to 1, and passed
> > to OTP model during initialization. Firmware (FSBL, U-Boot) uses
> > the serial number to generate a unique MAC address for the on-chip
> > ethernet controller. When multiple QEMU 'sifive_u' instances are
> > created and connected to the same subnet, they all have the same
> > MAC address hence it creates a unusable network.
> >
> > A new "serial" property is introduced to specify the board serial
> > number. When not given, the default serial number 1 is used.
> >
> > Signed-off-by: Bin Meng 
> >
> > ---
> >
> > Changes in v2:
> > - Move setting OTP serial number property from riscv_sifive_u_soc_init()
> >   to riscv_sifive_u_soc_realize(), to fix the 'check-qtest-riscv' error.
> >   I am not really sure why doing so could fix the 'make check' error.
> >   The v1 patch worked fine and nothing seems wrong.
> >
> >  hw/riscv/sifive_u.c | 21 -
> >  include/hw/riscv/sifive_u.h |  1 +
> >  2 files changed, 21 insertions(+), 1 deletion(-)
> >
> > diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
> > index 0e12b3c..ca561d3 100644
> > --- a/hw/riscv/sifive_u.c
> > +++ b/hw/riscv/sifive_u.c
> > @@ -34,6 +34,7 @@
> >  #include "qemu/log.h"
> >  #include "qemu/error-report.h"
> >  #include "qapi/error.h"
> > +#include "qapi/visitor.h"
> >  #include "hw/boards.h"
> >  #include "hw/loader.h"
> >  #include "hw/sysbus.h"
> > @@ -434,7 +435,6 @@ static void riscv_sifive_u_soc_init(Object *obj)
> >TYPE_SIFIVE_U_PRCI);
> >  sysbus_init_child_obj(obj, "otp", &s->otp, sizeof(s->otp),
> >TYPE_SIFIVE_U_OTP);
> > -qdev_prop_set_uint32(DEVICE(&s->otp), "serial", OTP_SERIAL);
> >  sysbus_init_child_obj(obj, "gem", &s->gem, sizeof(s->gem),
> >TYPE_CADENCE_GEM);
> >  }
> > @@ -453,6 +453,18 @@ static void sifive_u_set_start_in_flash(Object *obj, 
> > bool value, Error **errp)
> >  s->start_in_flash = value;
> >  }
> >
> > +static void sifive_u_get_serial(Object *obj, Visitor *v, const char *name,
> > +void *opaque, Error **errp)
> > +{
> > +visit_type_uint32(v, name, (uint32_t *)opaque, errp);
> > +}
> > +
> > +static void sifive_u_set_serial(Object *obj, Visitor *v, const char *name,
> > +void *opaque, Error **errp)
> > +{
> > +visit_type_uint32(v, name, (uint32_t *)opaque, errp);
> > +}
> > +
> >  static void riscv_sifive_u_machine_instance_init(Object *obj)
> >  {
> >  SiFiveUState *s = RISCV_U_MACHINE(obj);
> > @@ -464,11 +476,17 @@ static void 
> > riscv_sifive_u_machine_instance_init(Object *obj)
> >  "Set on to tell QEMU's ROM to jump to 
> > " \
> >  "flash. Otherwise QEMU will jump to 
> > DRAM",
> >  NULL);
> > +
> > +s->serial = OTP_SERIAL;
> > +object_property_add(obj, "serial", "uint32", sifive_u_get_serial,
> > +sifive_u_set_serial, NULL, &s->serial, NULL);
> > +object_property_set_description(obj, "serial", "Board serial number", 
> > NULL);
> >  }
> >
> >  static void riscv_sifive_u_soc_realize(DeviceState *dev, Error **errp)
> >  {
> >  MachineState *ms = MACHINE(qdev_get_machine());
> > +SiFiveUState *us = RISCV_U_MACHINE(ms);
> >  SiFiveUSoCState *s = RISCV_U_SOC(dev);
> >  const struct MemmapEntry *memmap = sifive_u_memmap;
> >  MemoryRegion *system_memory = get_system_memory();
> > @@ -554,6 +572,7 @@ static void riscv_sifive_u_soc_realize(DeviceState 
> > *dev, Error **errp)
> >  object_property_set_bool(OBJECT(&s->prci), true, "realized", &err);
> >  sysbus_mmio_map(SYS_BUS_DEVICE(&s->prci), 0, 
> > memmap[SIFIVE_U_PRCI].base);
> >
> > +qdev_prop_set_uint32(DEVICE(&s->otp), "serial", us->serial);
> >  object_property_set_bool(OBJECT(&s->otp), true, "realized", &err);
> >  sysbus_mmio_map(SYS_BUS_DEVICE(&s->otp), 0, memmap[SIFIVE_U_OTP].base);
> >
> > diff --git a/include/hw/riscv/sifive_u.h b/include/hw/riscv/sifive_u.h
> > index 82667b5..7cf742e 100644
> > --- a/include/hw/riscv/sifive_u.h
> > +++ b/include/hw/riscv/sifive_u.h
> > @@ -59,6 +59,7 @@ typedef struct SiFiveUState {
> >  int fdt_size;
> >
> >  bool start_in_flash;
> > +uint32_t serial;
> >  } SiFiveUState;
> >
> >  enum {
>
> Reviewed-by: Palmer Dabbelt 
>
> Thanks.  This is in the queue for the soft freeze.

This patch isn't correct. The SoC gets the sifive_u machine state
which is a bad idea. If the SoC is ever user on a different machine
this will crash.

I have sent a patch series that adds a machine and SoC property instead.

Alistair

>

Re: [PATCH v7 06/10] iotests: limit line length to 79 chars

[John Snow]

On 3/4/20 4:58 PM, Philippe Mathieu-Daudé wrote:
> Do we want to indent Python like C and align argument below opening
> parenthesis? Except when using sys.stderr.write() you seem to do it.

This isn't an argument to write, it's an argument to the format string,
so I will say "no."

For *where* I've placed the line break, this is the correct indentation.
emacs's python mode will settle on this indent, too.

https://python.org/dev/peps/pep-0008/#indentation

(If anyone quotes Guido's belittling comment in this email, I will
become cross.)


But there are other places to put the line break. This is also
technically valid:

sys.stderr.write('qemu-img received signal %i: %s\n'
 % (-exitcode, ' '.join(qemu_img_args + list(args

And so is this:

sys.stderr.write('qemu-img received signal %i: %s\n' %
 (-exitcode, ' '.join(qemu_img_args + list(args

(And so would be any other number of rewrites to use format codes,
f-strings, or temporary variables to otherwise reduce the length of the
line.)

I will reluctantly admit that wrapping to 79 columns is useful in some
contexts. The beauty of line continuations is something I have little
desire to litigate, though.

If there's some consensus on the true and beautiful way to do it, I will
oblige -- but the thought of spinning more iterations until we find a
color swatch we like is an unpleasant one.

--js

Re: [PATCH v1 2/3] riscv/sifive_u: Add a serial property to the sifive_u SoC

[Alistair Francis]

On Wed, Mar 4, 2020 at 6:47 AM Bin Meng  wrote:
>
> Hi Alistair,
>
> On Wed, Mar 4, 2020 at 9:37 AM Alistair Francis
>  wrote:
> >
> > At present the board serial number is hard-coded to 1, and passed
> > to OTP model during initialization. Firmware (FSBL, U-Boot) uses
> > the serial number to generate a unique MAC address for the on-chip
> > ethernet controller. When multiple QEMU 'sifive_u' instances are
> > created and connected to the same subnet, they all have the same
> > MAC address hence it creates a unusable network.
> >
> > A new "serial" property is introduced to the sifive_u SoC to specify
> > the board serial number. When not given, the default serial number
> > 1 is used.
> >
> > Suggested-by: Bin Meng 
> > Signed-off-by: Alistair Francis 
> > ---
> >  hw/riscv/sifive_u.c | 8 +++-
> >  include/hw/riscv/sifive_u.h | 2 ++
> >  2 files changed, 9 insertions(+), 1 deletion(-)
> >
> > diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
> > index 9a0145b5b4..e52f9d0bd4 100644
> > --- a/hw/riscv/sifive_u.c
> > +++ b/hw/riscv/sifive_u.c
> > @@ -488,7 +488,7 @@ static void riscv_sifive_u_soc_init(Object *obj)
> >TYPE_SIFIVE_U_PRCI);
> >  sysbus_init_child_obj(obj, "otp", &s->otp, sizeof(s->otp),
> >TYPE_SIFIVE_U_OTP);
> > -qdev_prop_set_uint32(DEVICE(&s->otp), "serial", OTP_SERIAL);
> > +qdev_prop_set_uint32(DEVICE(&s->otp), "serial", s->serial);
> >  sysbus_init_child_obj(obj, "gem", &s->gem, sizeof(s->gem),
> >TYPE_CADENCE_GEM);
> >  }
> > @@ -607,10 +607,16 @@ static void riscv_sifive_u_soc_realize(DeviceState 
> > *dev, Error **errp)
> >  memmap[SIFIVE_U_GEM_MGMT].base, memmap[SIFIVE_U_GEM_MGMT].size);
> >  }
> >
> > +static Property riscv_sifive_u_soc_props[] = {
> > +DEFINE_PROP_UINT32("serial", SiFiveUSoCState, serial, OTP_SERIAL),
> > +DEFINE_PROP_END_OF_LIST()
>
> I am not sure how adding another level of property in the SoC could
> solve the 'make check' error.

The problem is that you were adding a machine property and then you
had the SoC reach up to the machine object to get the serial value.
This isn't correct and is why the tests fail.

This patch series instead adds a property to the machine and the SoC,
where the machine sets the SoC property.

>
> > +};
> > +
> >  static void riscv_sifive_u_soc_class_init(ObjectClass *oc, void *data)
> >  {
> >  DeviceClass *dc = DEVICE_CLASS(oc);
> >
> > +device_class_set_props(dc, riscv_sifive_u_soc_props);
> >  dc->realize = riscv_sifive_u_soc_realize;
> >  /* Reason: Uses serial_hds in realize function, thus can't be used 
> > twice */
> >  dc->user_creatable = false;
> > diff --git a/include/hw/riscv/sifive_u.h b/include/hw/riscv/sifive_u.h
> > index 82667b5746..a2baa1de5f 100644
> > --- a/include/hw/riscv/sifive_u.h
> > +++ b/include/hw/riscv/sifive_u.h
> > @@ -42,6 +42,8 @@ typedef struct SiFiveUSoCState {
> >  SiFiveUPRCIState prci;
> >  SiFiveUOTPState otp;
> >  CadenceGEMState gem;
> > +
> > +uint32_t serial;
> >  } SiFiveUSoCState;
> >
> >  #define TYPE_RISCV_U_MACHINE MACHINE_TYPE_NAME("sifive_u")
> > --
>
> But anyway this patch does not actually work as expected. See below:
>
> $ ./riscv64-softmmu/qemu-system-riscv64 -M sifive_u,serial=3
> -nographic -m 2G -bios opensbi_u-boot_sifive_u_64.bin
>
> OpenSBI v0.5 (Oct 31 2019 18:38:50)
>_  _
>   / __ \  / |  _ \_   _|
>  | |  | |_ __   ___ _ __ | (___ | |_) || |
>  | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
>  | |__| | |_) |  __/ | | |) | |_) || |_
>   \/| .__/ \___|_| |_|_/|/_|
> | |
> |_|
>
> Platform Name  : SiFive Freedom U540
> Platform HART Features : RV64ACDFIMSU
> Platform Max HARTs : 5
> Current Hart   : 1
> Firmware Base  : 0x8000
> Firmware Size  : 96 KB
> Runtime SBI Version: 0.2
>
> PMP0: 0x8000-0x8001 (A)
> PMP1: 0x-0x (A,R,W,X)
>
>
> U-Boot 2019.10 (Oct 31 2019 - 18:38:33 +0800)
>
> CPU:   rv64imafdcsu
> Model: SiFive HiFive Unleashed A00
> DRAM:  2 GiB
> MMC:
> In:serial@1001
> Out:   serial@1001
> Err:   serial@1001
> Net:
> Warning: ethernet@1009 MAC addresses don't match:
> Address in ROM is  52:54:00:12:34:56
> Address in environment is  70:b3:d5:92:f0:01
> eth0: ethernet@1009
> Hit any key to stop autoboot:  0
>
>
> See this line:
> Address in environment is  70:b3:d5:92:f0:01
>
> It should be: 70:b3:d5:92:f0:03 since I specified serial number as 3.

Fixed, The problem was setting the serial property too early.

Alistair

>
> Regards,
> Bin

[PATCH v2 3/3] riscv/sifive_u: Add a serial property to the sifive_u machine

[Alistair Francis]

From: Bin Meng 

At present the board serial number is hard-coded to 1, and passed
to OTP model during initialization. Firmware (FSBL, U-Boot) uses
the serial number to generate a unique MAC address for the on-chip
ethernet controller. When multiple QEMU 'sifive_u' instances are
created and connected to the same subnet, they all have the same
MAC address hence it creates a unusable network.

A new "serial" property is introduced to specify the board serial
number. When not given, the default serial number 1 is used.

Signed-off-by: Bin Meng 
Reviewed-by: Palmer Dabbelt 
Reviewed-by: Alistair Francis 
Message-Id: <1573916930-19068-1-git-send-email-bmeng...@gmail.com>
[ Changed by AF:
 - Use the SoC's serial property to pass the info to the SoC
 - Fixup commit title
 - Rebase on file restructuring
]
Signed-off-by: Alistair Francis 
---
 hw/riscv/sifive_u.c | 20 
 include/hw/riscv/sifive_u.h |  1 +
 2 files changed, 21 insertions(+)

diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
index aed8249d0b..4299a5addc 100644
--- a/hw/riscv/sifive_u.c
+++ b/hw/riscv/sifive_u.c
@@ -34,6 +34,7 @@
 #include "qemu/log.h"
 #include "qemu/error-report.h"
 #include "qapi/error.h"
+#include "qapi/visitor.h"
 #include "hw/boards.h"
 #include "hw/loader.h"
 #include "hw/sysbus.h"
@@ -322,6 +323,8 @@ static void riscv_sifive_u_init(MachineState *machine)
 object_initialize_child(OBJECT(machine), "soc", &s->soc,
 sizeof(s->soc), TYPE_RISCV_U_SOC,
 &error_abort, NULL);
+object_property_set_uint(OBJECT(&s->soc), s->serial, "serial",
+&error_abort);
 object_property_set_bool(OBJECT(&s->soc), true, "realized",
 &error_abort);
 
@@ -413,6 +416,18 @@ static void sifive_u_set_start_in_flash(Object *obj, bool 
value, Error **errp)
 s->start_in_flash = value;
 }
 
+static void sifive_u_get_serial(Object *obj, Visitor *v, const char *name,
+void *opaque, Error **errp)
+{
+visit_type_uint32(v, name, (uint32_t *)opaque, errp);
+}
+
+static void sifive_u_set_serial(Object *obj, Visitor *v, const char *name,
+void *opaque, Error **errp)
+{
+visit_type_uint32(v, name, (uint32_t *)opaque, errp);
+}
+
 static void riscv_sifive_u_machine_instance_init(Object *obj)
 {
 SiFiveUState *s = RISCV_U_MACHINE(obj);
@@ -424,6 +439,11 @@ static void riscv_sifive_u_machine_instance_init(Object 
*obj)
 "Set on to tell QEMU's ROM to jump to " \
 "flash. Otherwise QEMU will jump to DRAM",
 NULL);
+
+s->serial = OTP_SERIAL;
+object_property_add(obj, "serial", "uint32", sifive_u_get_serial,
+sifive_u_set_serial, NULL, &s->serial, NULL);
+object_property_set_description(obj, "serial", "Board serial number", 
NULL);
 }
 
 
diff --git a/include/hw/riscv/sifive_u.h b/include/hw/riscv/sifive_u.h
index a2baa1de5f..16c297ec5f 100644
--- a/include/hw/riscv/sifive_u.h
+++ b/include/hw/riscv/sifive_u.h
@@ -61,6 +61,7 @@ typedef struct SiFiveUState {
 int fdt_size;
 
 bool start_in_flash;
+uint32_t serial;
 } SiFiveUState;
 
 enum {
-- 
2.25.1

[PATCH v2 1/3] riscv/sifive_u: Fix up file ordering

[Alistair Francis]

Split the file into clear machine and SoC sections.

Signed-off-by: Alistair Francis 
---
 hw/riscv/sifive_u.c | 107 ++--
 1 file changed, 54 insertions(+), 53 deletions(-)

diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
index 156a003642..9a0145b5b4 100644
--- a/hw/riscv/sifive_u.c
+++ b/hw/riscv/sifive_u.c
@@ -399,6 +399,60 @@ static void riscv_sifive_u_init(MachineState *machine)
   &address_space_memory);
 }
 
+static bool sifive_u_get_start_in_flash(Object *obj, Error **errp)
+{
+SiFiveUState *s = RISCV_U_MACHINE(obj);
+
+return s->start_in_flash;
+}
+
+static void sifive_u_set_start_in_flash(Object *obj, bool value, Error **errp)
+{
+SiFiveUState *s = RISCV_U_MACHINE(obj);
+
+s->start_in_flash = value;
+}
+
+static void riscv_sifive_u_machine_instance_init(Object *obj)
+{
+SiFiveUState *s = RISCV_U_MACHINE(obj);
+
+s->start_in_flash = false;
+object_property_add_bool(obj, "start-in-flash", 
sifive_u_get_start_in_flash,
+ sifive_u_set_start_in_flash, NULL);
+object_property_set_description(obj, "start-in-flash",
+"Set on to tell QEMU's ROM to jump to " \
+"flash. Otherwise QEMU will jump to DRAM",
+NULL);
+}
+
+
+static void riscv_sifive_u_machine_class_init(ObjectClass *oc, void *data)
+{
+MachineClass *mc = MACHINE_CLASS(oc);
+
+mc->desc = "RISC-V Board compatible with SiFive U SDK";
+mc->init = riscv_sifive_u_init;
+mc->max_cpus = SIFIVE_U_MANAGEMENT_CPU_COUNT + SIFIVE_U_COMPUTE_CPU_COUNT;
+mc->min_cpus = SIFIVE_U_MANAGEMENT_CPU_COUNT + 1;
+mc->default_cpus = mc->min_cpus;
+}
+
+static const TypeInfo riscv_sifive_u_machine_typeinfo = {
+.name   = MACHINE_TYPE_NAME("sifive_u"),
+.parent = TYPE_MACHINE,
+.class_init = riscv_sifive_u_machine_class_init,
+.instance_init = riscv_sifive_u_machine_instance_init,
+.instance_size = sizeof(SiFiveUState),
+};
+
+static void riscv_sifive_u_machine_init_register_types(void)
+{
+type_register_static(&riscv_sifive_u_machine_typeinfo);
+}
+
+type_init(riscv_sifive_u_machine_init_register_types)
+
 static void riscv_sifive_u_soc_init(Object *obj)
 {
 MachineState *ms = MACHINE(qdev_get_machine());
@@ -439,33 +493,6 @@ static void riscv_sifive_u_soc_init(Object *obj)
   TYPE_CADENCE_GEM);
 }
 
-static bool sifive_u_get_start_in_flash(Object *obj, Error **errp)
-{
-SiFiveUState *s = RISCV_U_MACHINE(obj);
-
-return s->start_in_flash;
-}
-
-static void sifive_u_set_start_in_flash(Object *obj, bool value, Error **errp)
-{
-SiFiveUState *s = RISCV_U_MACHINE(obj);
-
-s->start_in_flash = value;
-}
-
-static void riscv_sifive_u_machine_instance_init(Object *obj)
-{
-SiFiveUState *s = RISCV_U_MACHINE(obj);
-
-s->start_in_flash = false;
-object_property_add_bool(obj, "start-in-flash", 
sifive_u_get_start_in_flash,
- sifive_u_set_start_in_flash, NULL);
-object_property_set_description(obj, "start-in-flash",
-"Set on to tell QEMU's ROM to jump to " \
-"flash. Otherwise QEMU will jump to DRAM",
-NULL);
-}
-
 static void riscv_sifive_u_soc_realize(DeviceState *dev, Error **errp)
 {
 MachineState *ms = MACHINE(qdev_get_machine());
@@ -603,29 +630,3 @@ static void riscv_sifive_u_soc_register_types(void)
 }
 
 type_init(riscv_sifive_u_soc_register_types)
-
-static void riscv_sifive_u_machine_class_init(ObjectClass *oc, void *data)
-{
-MachineClass *mc = MACHINE_CLASS(oc);
-
-mc->desc = "RISC-V Board compatible with SiFive U SDK";
-mc->init = riscv_sifive_u_init;
-mc->max_cpus = SIFIVE_U_MANAGEMENT_CPU_COUNT + SIFIVE_U_COMPUTE_CPU_COUNT;
-mc->min_cpus = SIFIVE_U_MANAGEMENT_CPU_COUNT + 1;
-mc->default_cpus = mc->min_cpus;
-}
-
-static const TypeInfo riscv_sifive_u_machine_typeinfo = {
-.name   = MACHINE_TYPE_NAME("sifive_u"),
-.parent = TYPE_MACHINE,
-.class_init = riscv_sifive_u_machine_class_init,
-.instance_init = riscv_sifive_u_machine_instance_init,
-.instance_size = sizeof(SiFiveUState),
-};
-
-static void riscv_sifive_u_machine_init_register_types(void)
-{
-type_register_static(&riscv_sifive_u_machine_typeinfo);
-}
-
-type_init(riscv_sifive_u_machine_init_register_types)
-- 
2.25.1

[PATCH v2 0/3] hw/riscv: Add a serial property to the sifive_u machine

[Alistair Francis]

At present the board serial number is hard-coded to 1, and passed
to OTP model during initialization. Firmware (FSBL, U-Boot) uses
the serial number to generate a unique MAC address for the on-chip
ethernet controller. When multiple QEMU 'sifive_u' instances are
created and connected to the same subnet, they all have the same
MAC address hence it creates a unusable network.

A new "serial" property is introduced to specify the board serial
number. When not given, the default serial number 1 is used.

v2:
 - Fix the serial setting so it correctly sets

Alistair Francis (2):
  riscv/sifive_u: Fix up file ordering
  riscv/sifive_u: Add a serial property to the sifive_u SoC

Bin Meng (1):
  riscv/sifive_u: Add a serial property to the sifive_u machine

 hw/riscv/sifive_u.c | 135 +---
 include/hw/riscv/sifive_u.h |   3 +
 2 files changed, 84 insertions(+), 54 deletions(-)

-- 
2.25.1

[PATCH v2 2/3] riscv/sifive_u: Add a serial property to the sifive_u SoC

[Alistair Francis]

At present the board serial number is hard-coded to 1, and passed
to OTP model during initialization. Firmware (FSBL, U-Boot) uses
the serial number to generate a unique MAC address for the on-chip
ethernet controller. When multiple QEMU 'sifive_u' instances are
created and connected to the same subnet, they all have the same
MAC address hence it creates a unusable network.

A new "serial" property is introduced to the sifive_u SoC to specify
the board serial number. When not given, the default serial number
1 is used.

Suggested-by: Bin Meng 
Signed-off-by: Alistair Francis 
---
 hw/riscv/sifive_u.c | 8 +++-
 include/hw/riscv/sifive_u.h | 2 ++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
index 9a0145b5b4..aed8249d0b 100644
--- a/hw/riscv/sifive_u.c
+++ b/hw/riscv/sifive_u.c
@@ -488,7 +488,6 @@ static void riscv_sifive_u_soc_init(Object *obj)
   TYPE_SIFIVE_U_PRCI);
 sysbus_init_child_obj(obj, "otp", &s->otp, sizeof(s->otp),
   TYPE_SIFIVE_U_OTP);
-qdev_prop_set_uint32(DEVICE(&s->otp), "serial", OTP_SERIAL);
 sysbus_init_child_obj(obj, "gem", &s->gem, sizeof(s->gem),
   TYPE_CADENCE_GEM);
 }
@@ -581,6 +580,7 @@ static void riscv_sifive_u_soc_realize(DeviceState *dev, 
Error **errp)
 object_property_set_bool(OBJECT(&s->prci), true, "realized", &err);
 sysbus_mmio_map(SYS_BUS_DEVICE(&s->prci), 0, memmap[SIFIVE_U_PRCI].base);
 
+qdev_prop_set_uint32(DEVICE(&s->otp), "serial", s->serial);
 object_property_set_bool(OBJECT(&s->otp), true, "realized", &err);
 sysbus_mmio_map(SYS_BUS_DEVICE(&s->otp), 0, memmap[SIFIVE_U_OTP].base);
 
@@ -607,10 +607,16 @@ static void riscv_sifive_u_soc_realize(DeviceState *dev, 
Error **errp)
 memmap[SIFIVE_U_GEM_MGMT].base, memmap[SIFIVE_U_GEM_MGMT].size);
 }
 
+static Property riscv_sifive_u_soc_props[] = {
+DEFINE_PROP_UINT32("serial", SiFiveUSoCState, serial, OTP_SERIAL),
+DEFINE_PROP_END_OF_LIST()
+};
+
 static void riscv_sifive_u_soc_class_init(ObjectClass *oc, void *data)
 {
 DeviceClass *dc = DEVICE_CLASS(oc);
 
+device_class_set_props(dc, riscv_sifive_u_soc_props);
 dc->realize = riscv_sifive_u_soc_realize;
 /* Reason: Uses serial_hds in realize function, thus can't be used twice */
 dc->user_creatable = false;
diff --git a/include/hw/riscv/sifive_u.h b/include/hw/riscv/sifive_u.h
index 82667b5746..a2baa1de5f 100644
--- a/include/hw/riscv/sifive_u.h
+++ b/include/hw/riscv/sifive_u.h
@@ -42,6 +42,8 @@ typedef struct SiFiveUSoCState {
 SiFiveUPRCIState prci;
 SiFiveUOTPState otp;
 CadenceGEMState gem;
+
+uint32_t serial;
 } SiFiveUSoCState;
 
 #define TYPE_RISCV_U_MACHINE MACHINE_TYPE_NAME("sifive_u")
-- 
2.25.1

Re: [PATCH v2] riscv: sifive_u: Add a "serial" property for board serial number

[Palmer Dabbelt]

On Sun, 16 Feb 2020 05:55:17 PST (-0800), bmeng...@gmail.com wrote:

At present the board serial number is hard-coded to 1, and passed
to OTP model during initialization. Firmware (FSBL, U-Boot) uses
the serial number to generate a unique MAC address for the on-chip
ethernet controller. When multiple QEMU 'sifive_u' instances are
created and connected to the same subnet, they all have the same
MAC address hence it creates a unusable network.

A new "serial" property is introduced to specify the board serial
number. When not given, the default serial number 1 is used.

Signed-off-by: Bin Meng 

---

Changes in v2:
- Move setting OTP serial number property from riscv_sifive_u_soc_init()
  to riscv_sifive_u_soc_realize(), to fix the 'check-qtest-riscv' error.
  I am not really sure why doing so could fix the 'make check' error.
  The v1 patch worked fine and nothing seems wrong.

 hw/riscv/sifive_u.c | 21 -
 include/hw/riscv/sifive_u.h |  1 +
 2 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
index 0e12b3c..ca561d3 100644
--- a/hw/riscv/sifive_u.c
+++ b/hw/riscv/sifive_u.c
@@ -34,6 +34,7 @@
 #include "qemu/log.h"
 #include "qemu/error-report.h"
 #include "qapi/error.h"
+#include "qapi/visitor.h"
 #include "hw/boards.h"
 #include "hw/loader.h"
 #include "hw/sysbus.h"
@@ -434,7 +435,6 @@ static void riscv_sifive_u_soc_init(Object *obj)
   TYPE_SIFIVE_U_PRCI);
 sysbus_init_child_obj(obj, "otp", &s->otp, sizeof(s->otp),
   TYPE_SIFIVE_U_OTP);
-qdev_prop_set_uint32(DEVICE(&s->otp), "serial", OTP_SERIAL);
 sysbus_init_child_obj(obj, "gem", &s->gem, sizeof(s->gem),
   TYPE_CADENCE_GEM);
 }
@@ -453,6 +453,18 @@ static void sifive_u_set_start_in_flash(Object *obj, bool 
value, Error **errp)
 s->start_in_flash = value;
 }

+static void sifive_u_get_serial(Object *obj, Visitor *v, const char *name,
+void *opaque, Error **errp)
+{
+visit_type_uint32(v, name, (uint32_t *)opaque, errp);
+}
+
+static void sifive_u_set_serial(Object *obj, Visitor *v, const char *name,
+void *opaque, Error **errp)
+{
+visit_type_uint32(v, name, (uint32_t *)opaque, errp);
+}
+
 static void riscv_sifive_u_machine_instance_init(Object *obj)
 {
 SiFiveUState *s = RISCV_U_MACHINE(obj);
@@ -464,11 +476,17 @@ static void riscv_sifive_u_machine_instance_init(Object 
*obj)
 "Set on to tell QEMU's ROM to jump to " \
 "flash. Otherwise QEMU will jump to DRAM",
 NULL);
+
+s->serial = OTP_SERIAL;
+object_property_add(obj, "serial", "uint32", sifive_u_get_serial,
+sifive_u_set_serial, NULL, &s->serial, NULL);
+object_property_set_description(obj, "serial", "Board serial number", 
NULL);
 }

 static void riscv_sifive_u_soc_realize(DeviceState *dev, Error **errp)
 {
 MachineState *ms = MACHINE(qdev_get_machine());
+SiFiveUState *us = RISCV_U_MACHINE(ms);
 SiFiveUSoCState *s = RISCV_U_SOC(dev);
 const struct MemmapEntry *memmap = sifive_u_memmap;
 MemoryRegion *system_memory = get_system_memory();
@@ -554,6 +572,7 @@ static void riscv_sifive_u_soc_realize(DeviceState *dev, 
Error **errp)
 object_property_set_bool(OBJECT(&s->prci), true, "realized", &err);
 sysbus_mmio_map(SYS_BUS_DEVICE(&s->prci), 0, memmap[SIFIVE_U_PRCI].base);

+qdev_prop_set_uint32(DEVICE(&s->otp), "serial", us->serial);
 object_property_set_bool(OBJECT(&s->otp), true, "realized", &err);
 sysbus_mmio_map(SYS_BUS_DEVICE(&s->otp), 0, memmap[SIFIVE_U_OTP].base);

diff --git a/include/hw/riscv/sifive_u.h b/include/hw/riscv/sifive_u.h
index 82667b5..7cf742e 100644
--- a/include/hw/riscv/sifive_u.h
+++ b/include/hw/riscv/sifive_u.h
@@ -59,6 +59,7 @@ typedef struct SiFiveUState {
 int fdt_size;

 bool start_in_flash;
+uint32_t serial;
 } SiFiveUState;

 enum {


Reviewed-by: Palmer Dabbelt 

Thanks.  This is in the queue for the soft freeze.

Re: [EXTERNAL] Re: PATCH] WHPX: TSC get and set should be dependent on VM state

[Paolo Bonzini]

On 04/03/20 23:44, Sunil Muthuswamy wrote:
>> You don't need the "git config --local commit.gpgsign true" command, but
>> you will then create a signed tag with
>>
>>  git tag -s -f qemu-for-upstream
>>  # let's say "mirror" is your github repo
>>  git push mirror +tags/for-upstream
>>
>> and send it to Peter.  I really think we should document this final step
>> ("send it to Peter") better in the wiki, because the git tools for
>> sending pull requests leave a lot to be desired.
>
> Thanks. Ok, I am setup with GPG. Where should I be sending the pull requests 
> to? Who is "Peter"? Do I have to send it to you?

Peter is Peter Maydell, but for now you can send them to me.  I'll get
round to documenting the remaining steps.

Unfortunately all the scripts I have for this use the Unix shell, but
they are just a few lines so I might try to rewrite them in Python.
This way you can use them from Windows without needing to bring up WSL
or anything like that.

Paolo

RE: [EXTERNAL] Re: PATCH] WHPX: TSC get and set should be dependent on VM state

[Sunil Muthuswamy]

> -Original Message-
> From: Paolo Bonzini  On Behalf Of Paolo Bonzini
> Sent: Tuesday, March 3, 2020 9:53 AM
> To: Sunil Muthuswamy ; Richard Henderson 
> ; Eduardo Habkost 
> Cc: qemu-devel@nongnu.org; Stefan Weil 
> Subject: Re: [EXTERNAL] Re: PATCH] WHPX: TSC get and set should be dependent 
> on VM state
> 
> On 02/03/20 20:59, Sunil Muthuswamy wrote:
> >> You'd be using it to include a signed tags in a pull requests; that is,
> >> the git tag that you ask to pull has a cryptographic signature attached
> >> to it.
> > Great. Is there a link that I can use to read up on how to get the GPG key
> > and how to include the signature or what process should I be following?
> 
> This guide seems good, though I haven't tried:
> 
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmedium.com%2F%40ryanmillerc%2Fuse-gpg-signing-keys-with-git-
> on-windows-10-github-
> 4acbced49f68&data=02%7C01%7Csunilmut%40microsoft.com%7C826eb46e9f9f44281bd008d7bf9bb70d%7C72f988bf86f141af91ab2
> d7cd011db47%7C1%7C0%7C637188548242292250&sdata=E%2BQ0ar57y65EI6%2FDXXOnEqHM682SOhu1WyzrXrILWGs%3D&res
> erved=0
> 
> You don't need the "git config --local commit.gpgsign true" command, but
> you will then create a signed tag with
> 
>   git tag -s -f qemu-for-upstream
>   # let's say "mirror" is your github repo
>   git push mirror +tags/for-upstream
> 
> and send it to Peter.  I really think we should document this final step
> ("send it to Peter") better in the wiki, because the git tools for
> sending pull requests leave a lot to be desired.
> 
> Paolo

Thanks. Ok, I am setup with GPG. Where should I be sending the pull requests 
to? Who is "Peter"? Do I have to send it to you?

Re: [PATCH v7 01/10] iotests: do a light delinting

[John Snow]

On 3/4/20 4:45 PM, Philippe Mathieu-Daudé wrote:
> I don't understand the rational for this change, but the result is
> correct anyway, so thanks for this nice cleanup!

Leftover from when 'no-else-return' was enabled.

It's mostly harmless.

Re: [PATCH v5 14/50] mutli-process: build remote command line args

[Elena Ufimtseva]

On Wed, Mar 04, 2020 at 04:33:57PM +, Daniel P. Berrangé wrote:
> On Wed, Mar 04, 2020 at 08:25:34AM -0800, Elena Ufimtseva wrote:
> > On Wed, Mar 04, 2020 at 11:00:32AM +, Daniel P. Berrangé wrote:
> > > On Mon, Mar 02, 2020 at 02:39:37PM -0800, Elena Ufimtseva wrote:
> > > > On Mon, Mar 02, 2020 at 05:47:45PM +, Daniel P. Berrangé wrote:
> > > > > On Mon, Mar 02, 2020 at 06:36:13PM +0100, Philippe Mathieu-Daudé 
> > > > > wrote:
> > > > > > typo "multi" in patch subject.
> > > > > >
> > > > Thank Philippe, will fix.
> > > >  
> > > > > > On 2/24/20 9:55 PM, Jagannathan Raman wrote:
> > > > > > > From: Elena Ufimtseva 
> > > > > > > 
> > > > > > > Signed-off-by: Elena Ufimtseva 
> > > > > > > Signed-off-by: Jagannathan Raman 
> > > > > > > Signed-off-by: John G Johnson 
> > > > > > > ---
> > > > > > >   v4 -> v5:
> > > > > > >- Added "exec" suboption to get the executable's name
> > > > > > >- Addressed feedback about variable names
> > > > > > >- Removed redundant check for spawning a process
> > > > > > > 
> > > > > > >   hw/proxy/qemu-proxy.c | 68 
> > > > > > > +--
> > > > > > >   include/hw/proxy/qemu-proxy.h |  2 +-
> > > > > > >   2 files changed, 54 insertions(+), 16 deletions(-)
> > > > > > > 
> > > > > > > diff --git a/hw/proxy/qemu-proxy.c b/hw/proxy/qemu-proxy.c
> > > > > > > index 828bbd7..d792e86 100644
> > > > > > > --- a/hw/proxy/qemu-proxy.c
> > > > > > > +++ b/hw/proxy/qemu-proxy.c
> > > > > > > @@ -19,19 +19,50 @@
> > > > > > >   static void pci_proxy_dev_realize(PCIDevice *dev, Error **errp);
> > > > > > > +static int add_argv(char *opts_str, char **argv, int argc)
> > > > > > > +{
> > > > > > > +int max_args = 64;
> > > > > > > +
> > > > > > > +if (argc < max_args - 1) {
> > > > > > > +argv[argc++] = opts_str;
> > > > > > > +argv[argc] = 0;
> > > > > > > +} else {
> > > > > > > +return 0;
> > > > > > > +}
> > > > > > > +
> > > > > > > +return argc;
> > > > > > > +}
> > > > > > > +
> > > > > > > +static int make_argv(char *opts_str, char **argv, int argc)
> > > > > > > +{
> > > > > > > +int max_args = 64;
> > > > > > > +
> > > > > > > +char *p2 = strtok(opts_str, " ");
> > > > > > > +while (p2 && argc < max_args - 1) {
> > > > > > > +argv[argc++] = p2;
> > > > > > > +p2 = strtok(0, " ");
> > > > > > > +}
> > > > > > > +argv[argc] = 0;
> > > > > > 
> > > > > > Is there a GLib function to do that?
> > > > >
> > > > 
> > > > Hi Daniel
> > > > 
> > > > > g_shell_parse_argv() perhaps
> > > > >
> > > > 
> > > > Thanks for the suggestion.
> > > > 
> > > > >   
> > > > > https://developer.gnome.org/glib/stable/glib-Shell-related-Utilities.html
> > > > > 
> > > > > 
> > > > > Though my preference would be to avoid the need to do this at all, by
> > > > > not accepting a raw shell command line string in the first place.
> > > > >
> > > > Can you please clarify? Did you mean that it would be better if Qemu 
> > > > somehow
> > > > verifies the options and then passes it to a remote process via a 
> > > > message?
> > > 
> > > I've not been able to trace the code paths back all the way, so I can't
> > > point to where I think needs fixing. I assuming that something, somewhere
> > > in this patch series should starts out with a binary name and a list of 
> > > argv
> > > as an array of char *. ie a "char **argv".  At some point this array gets
> > > mashed together into a single 'char *' string where all the argv are 
> > > separated
> > > by a space. This patch now tries to parse this and turn it back into a
> > > "char **argv" array.
> > > 
> > > So my key point is that we should try hard to avoid this intermediate
> > > shell command line string stage entirely. Always keep the argv in an array
> > > form, and never mash them together such that they then need parsing again.
> > >
> > Hi Daniel
> > 
> > Thank you for explanation.
> > At this point there is no intermediate stage as we grab the arguments
> > as a raw string from the command line option -remote:
> > 
> > -remote rid=8,exec=qemu-scsi-dev,command="-drive 
> > id=drive_image2,,file=/root/remote-process-disk.img"
> > 
> > So the command="" string is being later parsed into the array and remote 
> > process
> > gets spawned with the "char **argv".
> > 
> > Stefan expressed his concern that its not convenient to use due to
> > the double escaping commas, spaces, quotes and we do agree with that.
> > We were seeking an advice on what is the better approach.
> 
> I've not looked closely enough to understand the range of different
> options we need to be able to pass to the remote QEMU ? Is it just
> "-drive" options, or can it be absolutely any QEMU option ?
> 
> If it is just -drive, then I could imagine a -remote-drive option
> such that we end up with with a set of args
> 
>$QEMU \
>-remote rid=8,exec=qemu-scsi-dev \
>-remote-drive rid=8,id=drive_image1,file=/root/remote-proces

Re: [PATCH 2/2] via-ide: Also emulate non 100% native mode

[BALATON Zoltan]

On Wed, 4 Mar 2020, Mark Cave-Ayland wrote:

On 04/03/2020 00:22, BALATON Zoltan wrote:

So on that basis the best explanation as to what is happening is the
comment in the link you provided here:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/powerpc/platforms/chrp/pci.c?h=v4.14.172#n353

/* Pegasos2 firmware version 20040810 configures the built-in IDE controller
* in legacy mode, but sets the PCI registers to PCI native mode.
* The chip can only operate in legacy mode, so force the PCI class into legacy
* mode as well. The same fixup must be done to the class-code property in
* the IDE node /pci@8000/ide@C,1
*/


I'm not sure that it makes much sense that the firmware configures the chip one 
way
then puts info about a different way in the device tree. There could be bugs 
but this
is not likely. Especially because I see in traces that the firmware does try to
configure the device in native mode. These are the first few accesses of the 
firmware
to via-ide:


But that is exactly what is happening! The comment above clearly indicates the
firmware incorrectly sets the IDE controller in native mode which is in exact
agreement with the trace you provide below. And in fact if you look at


I may be reading the comment wrong but to me that says that "firmware 
configures IDE
in _legacy_ mode" whereas the trace shows it actually configures it in _native_ 
mode
which is complying to the CHRP doc above. But since it cannot comply to the 
"native
devices using OpenPIC" part it probably tries to apply the "ISA devices 
embedded in
PCI" part and locks IRQ to 14 and 15. Or it just wants to avoid sharing IRQs as 
much
as possible and the designers decided they will use IRQ14 and 15 for IDE.


Interesting. My interpretation of the comment was that the hardware can only 
operate
in legacy mode, even though the firmware configures the PCI registers to enable
native mode (which is why the class-code and IRQ routing are wrong).


I think you may give more significance to this comment than it really has. 
I don't know who put it there but maybe was also guessing what really 
happens and it's not really an authorative answer why it behaves like 
that. It seems to come from this commit and not from Genesi/bPlan but 
sounds more like a bugfix from some user:


https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/arch/powerpc/platforms/chrp/pci.c?h=v3.16.82&id=556ecf9be66f4d493e19bc71a7ce84366d512b71

I give more credibility to what MorphOS expects because the developers of 
that were closely cooperating with the board designers:


https://en.wikipedia.org/wiki/Bplan#Community_support
https://en.wikipedia.org/wiki/Pegasos#Operating_system_support


https://www.powerdeveloper.org/platforms/pegasos/devicetree you can see the 
nvramrc
hack that was released in order to fix the device tree to boot Linux which 
alters the
class-code and sets interrupts to 14 (which I believe is invalid, but seemingly 
good
enough here).


Isn't this the same fixup that newer Linux kernels already include? Maybe this 
was
needed before Linux properly supported Pegasos2 but later kernels will do this
anyway. This does not give us any new info we did not have before I think maybe 
just
easier to see all fixups in one place.


pci_cfg_write via-ide 12:1 @0x9 <- 0xf
pci_cfg_write via-ide 12:1 @0x40 <- 0xb
pci_cfg_write via-ide 12:1 @0x41 <- 0xf2
pci_cfg_write via-ide 12:1 @0x43 <- 0x35
pci_cfg_write via-ide 12:1 @0x44 <- 0x18
pci_cfg_write via-ide 12:1 @0x45 <- 0x1c
pci_cfg_write via-ide 12:1 @0x46 <- 0xc0
pci_cfg_write via-ide 12:1 @0x50 <- 0x17171717
pci_cfg_write via-ide 12:1 @0x54 <- 0x14
pci_cfg_read via-ide 12:1 @0x0 -> 0x5711106
pci_cfg_read via-ide 12:1 @0x0 -> 0x5711106
pci_cfg_read via-ide 12:1 @0x8 -> 0x1018f06
pci_cfg_read via-ide 12:1 @0xc -> 0x0
pci_cfg_read via-ide 12:1 @0x2c -> 0x11001af4
pci_cfg_read via-ide 12:1 @0x3c -> 0x10e
pci_cfg_read via-ide 12:1 @0x4 -> 0x2800080
pci_cfg_read via-ide 12:1 @0x3c -> 0x10e
pci_cfg_write via-ide 12:1 @0x3c <- 0x109

The very first write is to turn on native mode, so I think it's not about what 
the
firmware does but something about how hardware is wired on Pegasos II or the 
VT8231
chip itself that only allows legacy interrupts instead of 100% native mode for 
IDE.


Given that the DT is wrong, then we should assume that all OSs would have to
compensate for this in the same way as Linux, and therefore this should be 
handled
automatically.

AFAICT this then only leaves the question: why does the firmware set
PCI_INTERRUPT_LINE to 9, which is presumably why you are seeing problems running
MorphOS under QEMU.


Linux does try to handle both true native mode and half-native mode. It only 
uses
half-native mode if finds IRQ14 on Pegasos, otherwise skips Pegasos specific 
fixup
and uses true native mode setup. I don't know what MorphOS does but I think it 
justs
knows that Pegasos2 has this quirk and does not look at the device tree at all.


I think th

[PATCH 0/6] hw,ui: Reduce QEMU .rodata/.bss footprint

[Philippe Mathieu-Daudé]

This series reduce the footprint of the QEMU binary:
.bss: 106KiB (moved to .heap)
.rodata: 4.34MiB
(sizes on x86_64 building with -Os)

The elf-dissector tool [1] [2] helped to notice the big array.

[1] https://phabricator.kde.org/source/elf-dissector/
[2] https://www.volkerkrause.eu/2019/06/22/elf-dissector-aarch64-support.html

Philippe Mathieu-Daudé (6):
  hw/audio/fmopl: Fix a typo twice
  hw/audio/fmopl: Move ENV_CURVE to .heap to save 32KiB of .bss
  hw/usb/quirks: Use smaller types to reduce .rodata by 10KiB
  hw/audio/intel-hda: Use memory region alias to reduce .rodata by
4.34MB
  ui/curses: Make control_characters[] array const
  ui/curses: Move arrays to .heap to save 74KiB of .bss

 hw/usb/quirks.h  | 10 +-
 hw/audio/fmopl.c |  8 +---
 hw/audio/intel-hda.c | 24 ++--
 ui/curses.c  | 10 +++---
 4 files changed, 27 insertions(+), 25 deletions(-)

-- 
2.21.1

[PATCH 5/6] ui/curses: Make control_characters[] array const

[Philippe Mathieu-Daudé]

As we only use this array as input, make it const.

Signed-off-by: Philippe Mathieu-Daudé 
---
 ui/curses.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ui/curses.c b/ui/curses.c
index 3a1b71451c..3bafc10c1c 100644
--- a/ui/curses.c
+++ b/ui/curses.c
@@ -529,7 +529,7 @@ static void font_setup(void)
  * Control characters are normally non-printable, but VGA does have
  * well-known glyphs for them.
  */
-static uint16_t control_characters[0x20] = {
+static const uint16_t control_characters[0x20] = {
   0x0020,
   0x263a,
   0x263b,
-- 
2.21.1

[PATCH 6/6] ui/curses: Move arrays to .heap to save 74KiB of .bss

[Philippe Mathieu-Daudé]

We only need these arrays when using the curses display.
Move them from the .bss to the .heap (sizes reported on
x86_64 host: screen[] is 64KiB, vga_to_curses 7KiB).

Signed-off-by: Philippe Mathieu-Daudé 
---
 ui/curses.c | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/ui/curses.c b/ui/curses.c
index 3bafc10c1c..a59b23a9cf 100644
--- a/ui/curses.c
+++ b/ui/curses.c
@@ -54,13 +54,13 @@ enum maybe_keycode {
 };
 
 static DisplayChangeListener *dcl;
-static console_ch_t screen[160 * 100];
+static console_ch_t *screen;
 static WINDOW *screenpad = NULL;
 static int width, height, gwidth, gheight, invalidate;
 static int px, py, sminx, sminy, smaxx, smaxy;
 
 static const char *font_charset = "CP437";
-static cchar_t vga_to_curses[256];
+static cchar_t *vga_to_curses;
 
 static void curses_update(DisplayChangeListener *dcl,
   int x, int y, int w, int h)
@@ -405,6 +405,8 @@ static void curses_refresh(DisplayChangeListener *dcl)
 static void curses_atexit(void)
 {
 endwin();
+g_free(vga_to_curses);
+g_free(screen);
 }
 
 /*
@@ -783,6 +785,8 @@ static void curses_display_init(DisplayState *ds, 
DisplayOptions *opts)
 if (opts->u.curses.charset) {
 font_charset = opts->u.curses.charset;
 }
+screen = g_new0(console_ch_t, 160 * 100);
+vga_to_curses = g_new0(cchar_t, 256);
 curses_setup();
 curses_keyboard_setup();
 atexit(curses_atexit);
-- 
2.21.1

[PATCH 1/6] hw/audio/fmopl: Fix a typo twice

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/audio/fmopl.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/audio/fmopl.c b/hw/audio/fmopl.c
index 9f50a89b4a..173a7521f2 100644
--- a/hw/audio/fmopl.c
+++ b/hw/audio/fmopl.c
@@ -1066,7 +1066,7 @@ static void OPLResetChip(FM_OPL *OPL)
}
 }
 
-/* --  Create one of vietual YM3812 --   */
+/* --  Create one of virtual YM3812 --   */
 /* 'rate'  is sampling rate and 'bufsiz' is the size of the  */
 FM_OPL *OPLCreate(int clock, int rate)
 {
@@ -1115,7 +1115,7 @@ FM_OPL *OPLCreate(int clock, int rate)
return OPL;
 }
 
-/* --  Destroy one of vietual YM3812 --   */
+/* --  Destroy one of virtual YM3812 --   */
 void OPLDestroy(FM_OPL *OPL)
 {
 #ifdef OPL_OUTPUT_LOG
-- 
2.21.1

[PATCH 3/6] hw/usb/quirks: Use smaller types to reduce .rodata by 10KiB

[Philippe Mathieu-Daudé]

The USB descriptor sizes are specified as 16-bit for idVendor /
idProduct, and 8-bit for bInterfaceClass / bInterfaceSubClass /
bInterfaceProtocol. Doing so we reduce the usbredir_raw_serial_ids[]
and usbredir_ftdi_serial_ids[] arrays from 16KiB to 6KiB (size
reported on x86_64 host, building with --extra-cflags=-Os).

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/usb/quirks.h | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/hw/usb/quirks.h b/hw/usb/quirks.h
index 89480befd7..794d89a356 100644
--- a/hw/usb/quirks.h
+++ b/hw/usb/quirks.h
@@ -21,11 +21,11 @@
 #include "quirks-pl2303-ids.h"
 
 struct usb_device_id {
-int vendor_id;
-int product_id;
-int interface_class;
-int interface_subclass;
-int interface_protocol;
+int16_t vendor_id;
+int16_t product_id;
+int8_t interface_class;
+int8_t interface_subclass;
+int8_t interface_protocol;
 };
 
 #define USB_DEVICE(vendor, product) \
-- 
2.21.1

[PATCH 4/6] hw/audio/intel-hda: Use memory region alias to reduce .rodata by 4.34MB

[Philippe Mathieu-Daudé]

The intel-hda model uses an array of register indexed by the
register address. This array also contains a pair of aliased
registers at offset 0x2000. This creates a huge hole in the
array, which ends up eating 4.6MiB of .rodata (size reported
on x86_64 host, building with --extra-cflags=-Os).

By using a memory region alias, we reduce this array to 132kB.

Before:

  (qemu) info mtree
febd4000-febd7fff (prio 1, i/o): intel-hda

After:

  (qemu) info mtree
febd4000-febd7fff (prio 1, i/o): intel-hda
febd4000-febd7fff (prio 1, i/o): intel-hda-container
  febd4000-febd5fff (prio 0, i/o): intel-hda
  febd6000-febd7fff (prio 0, i/o): alias intel-hda-alias 
@intel-hda -1fff

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/audio/intel-hda.c | 24 ++--
 1 file changed, 10 insertions(+), 14 deletions(-)

diff --git a/hw/audio/intel-hda.c b/hw/audio/intel-hda.c
index 1bcc3e5cf8..e8d18b7c58 100644
--- a/hw/audio/intel-hda.c
+++ b/hw/audio/intel-hda.c
@@ -181,7 +181,9 @@ struct IntelHDAState {
 IntelHDAStream st[8];
 
 /* state */
+MemoryRegion container;
 MemoryRegion mmio;
+MemoryRegion alias;
 uint32_t rirb_count;
 int64_t wall_base_ns;
 
@@ -670,12 +672,6 @@ static const struct IntelHDAReg regtab[] = {
 .offset   = offsetof(IntelHDAState, wall_clk),
 .rhandler = intel_hda_get_wall_clk,
 },
-[ ICH6_REG_WALLCLK + 0x2000 ] = {
-.name = "WALLCLK(alias)",
-.size = 4,
-.offset   = offsetof(IntelHDAState, wall_clk),
-.rhandler = intel_hda_get_wall_clk,
-},
 
 /* dma engine */
 [ ICH6_REG_CORBLBASE ] = {
@@ -837,12 +833,6 @@ static const struct IntelHDAReg regtab[] = {
 .size = 4,\
 .offset   = offsetof(IntelHDAState, st[_i].lpib), \
 },\
-[ ST_REG(_i, ICH6_REG_SD_LPIB) + 0x2000 ] = { \
-.stream   = _i,   \
-.name = _t stringify(_i) " LPIB(alias)",  \
-.size = 4,\
-.offset   = offsetof(IntelHDAState, st[_i].lpib), \
-},\
 [ ST_REG(_i, ICH6_REG_SD_CBL) ] = {   \
 .stream   = _i,   \
 .name = _t stringify(_i) " CBL",  \
@@ -1125,9 +1115,15 @@ static void intel_hda_realize(PCIDevice *pci, Error 
**errp)
 error_free(err);
 }
 
+memory_region_init(&d->container, OBJECT(d),
+   "intel-hda-container", 0x4000);
 memory_region_init_io(&d->mmio, OBJECT(d), &intel_hda_mmio_ops, d,
-  "intel-hda", 0x4000);
-pci_register_bar(&d->pci, 0, 0, &d->mmio);
+  "intel-hda", 0x2000);
+memory_region_add_subregion(&d->container, 0x, &d->mmio);
+memory_region_init_alias(&d->alias, OBJECT(d), "intel-hda-alias",
+ &d->mmio, 0, 0x2000);
+memory_region_add_subregion(&d->container, 0x2000, &d->alias);
+pci_register_bar(&d->pci, 0, 0, &d->container);
 
 hda_codec_bus_init(DEVICE(pci), &d->codecs, sizeof(d->codecs),
intel_hda_response, intel_hda_xfer);
-- 
2.21.1

[PATCH 2/6] hw/audio/fmopl: Move ENV_CURVE to .heap to save 32KiB of .bss

[Philippe Mathieu-Daudé]

This buffer is only used by the adlib audio device. Move it to
the .heap to release 32KiB of .bss (size reported on x86_64 host).

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/audio/fmopl.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/hw/audio/fmopl.c b/hw/audio/fmopl.c
index 173a7521f2..356d4dfbca 100644
--- a/hw/audio/fmopl.c
+++ b/hw/audio/fmopl.c
@@ -186,7 +186,7 @@ static int32_t *VIB_TABLE;
 
 /* envelope output curve table */
 /* attack + decay + OFF */
-static int32_t ENV_CURVE[2*EG_ENT+1];
+static int32_t *ENV_CURVE;
 
 /* multiple table */
 #define ML 2
@@ -1090,6 +1090,7 @@ FM_OPL *OPLCreate(int clock, int rate)
OPL->clock = clock;
OPL->rate  = rate;
OPL->max_ch = max_ch;
+ENV_CURVE = g_new(int32_t, 2 * EG_ENT + 1);
/* init grobal tables */
OPL_initialize(OPL);
/* reset chip */
@@ -1127,6 +1128,7 @@ void OPLDestroy(FM_OPL *OPL)
 #endif
OPL_UnLockTable();
free(OPL);
+g_free(ENV_CURVE);
 }
 
 /* --  Option handlers --   */
-- 
2.21.1

Re: [PATCH v7 04/10] iotests: replace mutable list default args

[Philippe Mathieu-Daudé]

On 3/4/20 10:38 PM, John Snow wrote:

It's bad hygiene: if we modify this list, it will be modified across all
invocations.

(Remaining bad usages are fixed in a subsequent patch which changes the
function signature anyway.)

Signed-off-by: John Snow 
---
  tests/qemu-iotests/iotests.py | 24 
  1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index fd65b90691..54d68774e1 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -136,7 +136,7 @@ def qemu_img_log(*args):
  log(result, filters=[filter_testfiles])
  return result
  
-def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[]):

+def img_info_log(filename, filter_path=None, imgopts=False, extra_args=()):
  args = ['info']
  if imgopts:
  args.append('--image-opts')
@@ -350,7 +350,7 @@ def _filter(_key, value):
  return value
  return filter_qmp(qmsg, _filter)
  
-def log(msg, filters=[], indent=None):

+def log(msg, filters=(), indent=None):
  '''Logs either a string message or a JSON serializable message (like QMP).
  If indent is provided, JSON serializable messages are pretty-printed.'''
  for flt in filters:
@@ -566,7 +566,7 @@ def get_qmp_events_filtered(self, wait=60.0):
  result.append(filter_qmp_event(ev))
  return result
  
-def qmp_log(self, cmd, filters=[], indent=None, **kwargs):

+def qmp_log(self, cmd, filters=(), indent=None, **kwargs):
  full_cmd = OrderedDict((
  ("execute", cmd),
  ("arguments", ordered_qmp(kwargs))
@@ -967,7 +967,7 @@ def case_notrun(reason):
  open('%s/%s.casenotrun' % (output_dir, seq), 'a').write(
  '[case not run] ' + reason + '\n')
  
-def verify_image_format(supported_fmts=[], unsupported_fmts=[]):

+def verify_image_format(supported_fmts=(), unsupported_fmts=()):
  assert not (supported_fmts and unsupported_fmts)
  
  if 'generic' in supported_fmts and \

@@ -981,7 +981,7 @@ def verify_image_format(supported_fmts=[], 
unsupported_fmts=[]):
  if not_sup or (imgfmt in unsupported_fmts):
  notrun('not suitable for this image format: %s' % imgfmt)
  
-def verify_protocol(supported=[], unsupported=[]):

+def verify_protocol(supported=(), unsupported=()):
  assert not (supported and unsupported)
  
  if 'generic' in supported:

@@ -1000,11 +1000,11 @@ def verify_platform(supported=None, unsupported=None):
  if not any((sys.platform.startswith(x) for x in supported)):
  notrun('not suitable for this OS: %s' % sys.platform)
  
-def verify_cache_mode(supported_cache_modes=[]):

+def verify_cache_mode(supported_cache_modes=()):
  if supported_cache_modes and (cachemode not in supported_cache_modes):
  notrun('not suitable for this cache mode: %s' % cachemode)
  
-def verify_aio_mode(supported_aio_modes=[]):

+def verify_aio_mode(supported_aio_modes=()):
  if supported_aio_modes and (aiomode not in supported_aio_modes):
  notrun('not suitable for this aio mode: %s' % aiomode)
  
@@ -1044,7 +1044,7 @@ def supported_formats(read_only=False):
  
  return supported_formats.formats[read_only]
  
-def skip_if_unsupported(required_formats=[], read_only=False):

+def skip_if_unsupported(required_formats=(), read_only=False):
  '''Skip Test Decorator
 Runs the test if all the required formats are whitelisted'''
  def skip_test_decorator(func):
@@ -1095,11 +1095,11 @@ def execute_unittest(output, verbosity, debug):
  sys.stderr.write(out)
  
  def execute_test(test_function=None,

- supported_fmts=[],
+ supported_fmts=(),
   supported_platforms=None,
- supported_cache_modes=[], supported_aio_modes={},
- unsupported_fmts=[], supported_protocols=[],
- unsupported_protocols=[]):
+ supported_cache_modes=(), supported_aio_modes=(),
+ unsupported_fmts=(), supported_protocols=(),
+ unsupported_protocols=()):
  """Run either unittest or script-style tests."""
  
  # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to




Reviewed-by: Philippe Mathieu-Daudé 

Re: [PATCH v7 06/10] iotests: limit line length to 79 chars

[Philippe Mathieu-Daudé]

On 3/4/20 10:38 PM, John Snow wrote:

79 is the PEP8 recommendation. This recommendation works well for
reading patch diffs in TUI email clients.

Signed-off-by: John Snow 
---
  tests/qemu-iotests/iotests.py | 69 ++-
  tests/qemu-iotests/pylintrc   |  6 ++-
  2 files changed, 48 insertions(+), 27 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 54d68774e1..1be11f491f 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -76,9 +76,11 @@
  def qemu_img(*args):
  '''Run qemu-img and return the exit code'''
  devnull = open('/dev/null', 'r+')
-exitcode = subprocess.call(qemu_img_args + list(args), stdin=devnull, 
stdout=devnull)
+exitcode = subprocess.call(qemu_img_args + list(args),
+   stdin=devnull, stdout=devnull)
  if exitcode < 0:
-sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' 
'.join(qemu_img_args + list(args
+sys.stderr.write('qemu-img received signal %i: %s\n' % (
+-exitcode, ' '.join(qemu_img_args + list(args


Do we want to indent Python like C and align argument below opening 
parenthesis? Except when using sys.stderr.write() you seem to do it.



  return exitcode
  
  def ordered_qmp(qmsg, conv_keys=True):

@@ -117,7 +119,8 @@ def qemu_img_verbose(*args):
  '''Run qemu-img without suppressing its output and return the exit code'''
  exitcode = subprocess.call(qemu_img_args + list(args))
  if exitcode < 0:
-sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' 
'.join(qemu_img_args + list(args
+sys.stderr.write('qemu-img received signal %i: %s\n' % (
+-exitcode, ' '.join(qemu_img_args + list(args
  return exitcode
  
  def qemu_img_pipe(*args):

@@ -128,7 +131,8 @@ def qemu_img_pipe(*args):
  universal_newlines=True)
  exitcode = subp.wait()
  if exitcode < 0:
-sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' 
'.join(qemu_img_args + list(args
+sys.stderr.write('qemu-img received signal %i: %s\n' % (
+-exitcode, ' '.join(qemu_img_args + list(args
  return subp.communicate()[0]
  
  def qemu_img_log(*args):

@@ -158,7 +162,8 @@ def qemu_io(*args):
  universal_newlines=True)
  exitcode = subp.wait()
  if exitcode < 0:
-sys.stderr.write('qemu-io received signal %i: %s\n' % (-exitcode, ' 
'.join(args)))
+sys.stderr.write('qemu-io received signal %i: %s\n' % (
+-exitcode, ' '.join(args)))
  return subp.communicate()[0]
  
  def qemu_io_log(*args):

@@ -280,10 +285,13 @@ def filter_test_dir(msg):
  def filter_win32(msg):
  return win32_re.sub("", msg)
  
-qemu_io_re = re.compile(r"[0-9]* ops; [0-9\/:. sec]* \([0-9\/.inf]* [EPTGMKiBbytes]*\/sec and [0-9\/.inf]* ops\/sec\)")

+qemu_io_re = re.compile(r"[0-9]* ops; [0-9\/:. sec]* "
+r"\([0-9\/.inf]* [EPTGMKiBbytes]*\/sec "
+r"and [0-9\/.inf]* ops\/sec\)")
  def filter_qemu_io(msg):
  msg = filter_win32(msg)
-return qemu_io_re.sub("X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)", 
msg)
+return qemu_io_re.sub("X ops; XX:XX:XX.X "
+  "(XXX YYY/sec and XXX ops/sec)", msg)
  
  chown_re = re.compile(r"chown [0-9]+:[0-9]+")

  def filter_chown(msg):
@@ -335,7 +343,9 @@ def filter_img_info(output, filename):
  line = line.replace(filename, 'TEST_IMG') \
 .replace(imgfmt, 'IMGFMT')
  line = re.sub('iters: [0-9]+', 'iters: XXX', line)
-line = re.sub('uuid: [-a-f0-9]+', 'uuid: 
----', line)
+line = re.sub('uuid: [-a-f0-9]+',
+  'uuid: ----',
+  line)
  line = re.sub('cid: [0-9]+', 'cid: XX', line)
  lines.append(line)
  return '\n'.join(lines)
@@ -356,12 +366,9 @@ def log(msg, filters=(), indent=None):
  for flt in filters:
  msg = flt(msg)
  if isinstance(msg, (dict, list)):
-# Python < 3.4 needs to know not to add whitespace when 
pretty-printing:
-separators = (', ', ': ') if indent is None else (',', ': ')
  # Don't sort if it's already sorted
  do_sort = not isinstance(msg, OrderedDict)
-print(json.dumps(msg, sort_keys=do_sort,
- indent=indent, separators=separators))
+print(json.dumps(msg, sort_keys=do_sort, indent=indent))


Unrelated change. Maybe worth a separate patch?


  else:
  print(msg)
  
@@ -529,11 +536,13 @@ def pause_drive(self, drive, event=None):

  self.pause_drive(drive, "write_aio")
  return
  self.qmp('human-monitor-command',
- command_line='qemu-io %s "break %s bp_%s"' % (drive, event, 

Re: [PATCH v7 09/10] iotests: Mark verify functions as private

[Philippe Mathieu-Daudé]

On 3/4/20 10:38 PM, John Snow wrote:

Discourage their use.


I recommend you to repeat the subject, else it is harder to review 
looking only at patch description.




(Also, make pending patches not yet using the new entry points fail in a
very obvious way.)

Reviewed-by: Max Reitz 
Signed-off-by: John Snow 
---
  tests/qemu-iotests/iotests.py | 20 ++--
  1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 1f88d2fa2a..23678f2daa 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -982,7 +982,7 @@ def case_notrun(reason):
  open('%s/%s.casenotrun' % (output_dir, seq), 'a').write(
  '[case not run] ' + reason + '\n')
  
-def verify_image_format(supported_fmts=(), unsupported_fmts=()):

+def _verify_image_format(supported_fmts=(), unsupported_fmts=()):
  assert not (supported_fmts and unsupported_fmts)
  
  if 'generic' in supported_fmts and \

@@ -996,7 +996,7 @@ def verify_image_format(supported_fmts=(), 
unsupported_fmts=()):
  if not_sup or (imgfmt in unsupported_fmts):
  notrun('not suitable for this image format: %s' % imgfmt)
  
-def verify_protocol(supported=(), unsupported=()):

+def _verify_protocol(supported=(), unsupported=()):
  assert not (supported and unsupported)
  
  if 'generic' in supported:

@@ -1006,7 +1006,7 @@ def verify_protocol(supported=(), unsupported=()):
  if not_sup or (imgproto in unsupported):
  notrun('not suitable for this protocol: %s' % imgproto)
  
-def verify_platform(supported=(), unsupported=()):

+def _verify_platform(supported=(), unsupported=()):
  if any((sys.platform.startswith(x) for x in unsupported)):
  notrun('not suitable for this OS: %s' % sys.platform)
  
@@ -1014,11 +1014,11 @@ def verify_platform(supported=(), unsupported=()):

  if not any((sys.platform.startswith(x) for x in supported)):
  notrun('not suitable for this OS: %s' % sys.platform)
  
-def verify_cache_mode(supported_cache_modes=()):

+def _verify_cache_mode(supported_cache_modes=()):
  if supported_cache_modes and (cachemode not in supported_cache_modes):
  notrun('not suitable for this cache mode: %s' % cachemode)
  
-def verify_aio_mode(supported_aio_modes=()):

+def _verify_aio_mode(supported_aio_modes=()):
  if supported_aio_modes and (aiomode not in supported_aio_modes):
  notrun('not suitable for this aio mode: %s' % aiomode)
  
@@ -1145,11 +1145,11 @@ def execute_setup_common(supported_fmts: Collection[str] = (),

  sys.stderr.write('Please run this test via the "check" script\n')
  sys.exit(os.EX_USAGE)
  
-verify_image_format(supported_fmts, unsupported_fmts)

-verify_protocol(supported_protocols, unsupported_protocols)
-verify_platform(supported=supported_platforms)
-verify_cache_mode(supported_cache_modes)
-verify_aio_mode(supported_aio_modes)
+_verify_image_format(supported_fmts, unsupported_fmts)
+_verify_protocol(supported_protocols, unsupported_protocols)
+_verify_platform(supported=supported_platforms)
+_verify_cache_mode(supported_cache_modes)
+_verify_aio_mode(supported_aio_modes)
  
  debug = '-d' in sys.argv

  if debug:

[PATCH v7 10/10] iotests: use python logging for iotests.log()

[John Snow]

We can turn logging on/off globally instead of per-function.

Remove use_log from run_job, and use python logging to turn on
diffable output when we run through a script entry point.

iotest 245 changes output order due to buffering reasons.


An extended note on python logging:

A NullHandler is added to `qemu.iotests` to stop output from being
generated if this code is used as a library without configuring logging.
A NullHandler is only needed at the root, so a duplicate handler is not
needed for `qemu.iotests.diff_io`.

When logging is not configured, messages at the 'WARNING' levels or
above are printed with default settings. The NullHandler stops this from
occurring, which is considered good hygiene for code used as a library.

See https://docs.python.org/3/howto/logging.html#library-config

When logging is actually enabled (always at the behest of an explicit
call by a client script), a root logger is implicitly created at the
root, which allows messages to propagate upwards and be handled/emitted
from the root logger with default settings.

When we want iotest logging, we attach a handler to the
qemu.iotests.diff_io logger and disable propagation to avoid possible
double-printing.

For more information on python logging infrastructure, I highly
recommend downloading the pip package `logging_tree`, which provides
convenient visualizations of the hierarchical logging configuration
under different circumstances.

See https://pypi.org/project/logging_tree/ for more information.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/030|  4 +--
 tests/qemu-iotests/245|  1 +
 tests/qemu-iotests/245.out| 24 +-
 tests/qemu-iotests/iotests.py | 48 +--
 4 files changed, 44 insertions(+), 33 deletions(-)

diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030
index aa911d266a..104e3cee1b 100755
--- a/tests/qemu-iotests/030
+++ b/tests/qemu-iotests/030
@@ -411,8 +411,8 @@ class TestParallelOps(iotests.QMPTestCase):
 result = self.vm.qmp('block-job-set-speed', device='drive0', speed=0)
 self.assert_qmp(result, 'return', {})
 
-self.vm.run_job(job='drive0', auto_dismiss=True, use_log=False)
-self.vm.run_job(job='node4', auto_dismiss=True, use_log=False)
+self.vm.run_job(job='drive0', auto_dismiss=True)
+self.vm.run_job(job='node4', auto_dismiss=True)
 self.assert_no_active_block_jobs()
 
 # Test a block-stream and a block-commit job in parallel
diff --git a/tests/qemu-iotests/245 b/tests/qemu-iotests/245
index 489bf78bd0..edb40a4ae8 100755
--- a/tests/qemu-iotests/245
+++ b/tests/qemu-iotests/245
@@ -1002,5 +1002,6 @@ class TestBlockdevReopen(iotests.QMPTestCase):
 self.reopen(opts, {'backing': 'hd2'})
 
 if __name__ == '__main__':
+iotests.activate_logging()
 iotests.main(supported_fmts=["qcow2"],
  supported_protocols=["file"])
diff --git a/tests/qemu-iotests/245.out b/tests/qemu-iotests/245.out
index a19de5214d..15c3630e92 100644
--- a/tests/qemu-iotests/245.out
+++ b/tests/qemu-iotests/245.out
@@ -1,17 +1,17 @@
+{"execute": "job-finalize", "arguments": {"id": "commit0"}}
+{"return": {}}
+{"data": {"id": "commit0", "type": "commit"}, "event": "BLOCK_JOB_PENDING", 
"timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "commit0", "len": 3145728, "offset": 3145728, "speed": 0, 
"type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": 
{"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "stream0"}}
+{"return": {}}
+{"data": {"id": "stream0", "type": "stream"}, "event": "BLOCK_JOB_PENDING", 
"timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "stream0", "len": 3145728, "offset": 3145728, "speed": 0, 
"type": "stream"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": 
{"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-finalize", "arguments": {"id": "stream0"}}
+{"return": {}}
+{"data": {"id": "stream0", "type": "stream"}, "event": "BLOCK_JOB_PENDING", 
"timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "stream0", "len": 3145728, "offset": 3145728, "speed": 0, 
"type": "stream"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": 
{"microseconds": "USECS", "seconds": "SECS"}}
 ..
 --
 Ran 18 tests
 
 OK
-{"execute": "job-finalize", "arguments": {"id": "commit0"}}
-{"return": {}}
-{"data": {"id": "commit0", "type": "commit"}, "event": "BLOCK_JOB_PENDING", 
"timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "commit0", "len": 3145728, "offset": 3145728, "speed": 0, 
"type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": 
{"microseconds": "USECS", "seconds": "SECS"}}
-{"execute": "job-finalize", "arguments": {"id": "stream0"}}
-{"return": {}}
-{"data": {"id": "stream0", "type": "stream"}, "event": "BLOCK_JOB

Re: [PATCH v7 01/10] iotests: do a light delinting

[Philippe Mathieu-Daudé]

On 3/4/20 10:38 PM, John Snow wrote:

This doesn't fix everything in here, but it does help clean up the
pylint report considerably.

This should be 100% style changes only; the intent is to make pylint
more useful by working on establishing a baseline for iotests that we
can gate against in the future. This will be important if (when?) we
begin adding type hints to our code base.

Signed-off-by: John Snow 
---
  tests/qemu-iotests/iotests.py | 83 ++-
  1 file changed, 43 insertions(+), 40 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 8815052eb5..c3aa857140 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -16,11 +16,9 @@
  # along with this program.  If not, see .
  #
  
-import errno

  import os
  import re
  import subprocess
-import string
  import unittest
  import sys
  import struct
@@ -34,7 +32,7 @@
  sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
  from qemu import qtest
  
-assert sys.version_info >= (3,6)

+assert sys.version_info >= (3, 6)
  
  # This will not work if arguments contain spaces but is necessary if we

  # want to support the override options that ./check supports.
@@ -138,11 +136,11 @@ def qemu_img_log(*args):
  return result
  
  def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[]):

-args = [ 'info' ]
+args = ['info']
  if imgopts:
  args.append('--image-opts')
  else:
-args += [ '-f', imgfmt ]
+args += ['-f', imgfmt]
  args += extra_args
  args.append(filename)
  
@@ -221,7 +219,7 @@ def cmd(self, cmd):

  # quit command is in close(), '\n' is added automatically
  assert '\n' not in cmd
  cmd = cmd.strip()
-assert cmd != 'q' and cmd != 'quit'
+assert cmd not in ('q', 'quit')
  self._p.stdin.write(cmd + '\n')
  self._p.stdin.flush()
  return self._read_output()
@@ -243,10 +241,8 @@ def qemu_nbd_early_pipe(*args):
  sys.stderr.write('qemu-nbd received signal %i: %s\n' %
   (-exitcode,
' '.join(qemu_nbd_args + ['--fork'] + list(args
-if exitcode == 0:
-return exitcode, ''
-else:
-return exitcode, subp.communicate()[0]
+
+return exitcode, subp.communicate()[0] if exitcode else ''
  
  def qemu_nbd_popen(*args):

  '''Run qemu-nbd in daemon mode and return the parent's exit code'''
@@ -310,7 +306,7 @@ def filter_qmp(qmsg, filter_fn):
  items = qmsg.items()
  
  for k, v in items:

-if isinstance(v, list) or isinstance(v, dict):
+if isinstance(v, (dict, list)):
  qmsg[k] = filter_qmp(v, filter_fn)
  else:
  qmsg[k] = filter_fn(k, v)
@@ -321,7 +317,7 @@ def filter_testfiles(msg):
  return msg.replace(prefix, 'TEST_DIR/PID-')
  
  def filter_qmp_testfiles(qmsg):

-def _filter(key, value):
+def _filter(_key, value):
  if is_str(value):
  return filter_testfiles(value)
  return value
@@ -347,7 +343,7 @@ def filter_imgfmt(msg):
  return msg.replace(imgfmt, 'IMGFMT')
  
  def filter_qmp_imgfmt(qmsg):

-def _filter(key, value):
+def _filter(_key, value):
  if is_str(value):
  return filter_imgfmt(value)
  return value
@@ -358,7 +354,7 @@ def log(msg, filters=[], indent=None):
  If indent is provided, JSON serializable messages are pretty-printed.'''
  for flt in filters:
  msg = flt(msg)
-if isinstance(msg, dict) or isinstance(msg, list):
+if isinstance(msg, (dict, list)):
  # Python < 3.4 needs to know not to add whitespace when 
pretty-printing:
  separators = (', ', ': ') if indent is None else (',', ': ')
  # Don't sort if it's already sorted
@@ -369,14 +365,14 @@ def log(msg, filters=[], indent=None):
  print(msg)
  
  class Timeout:

-def __init__(self, seconds, errmsg = "Timeout"):
+def __init__(self, seconds, errmsg="Timeout"):
  self.seconds = seconds
  self.errmsg = errmsg
  def __enter__(self):
  signal.signal(signal.SIGALRM, self.timeout)
  signal.setitimer(signal.ITIMER_REAL, self.seconds)
  return self
-def __exit__(self, type, value, traceback):
+def __exit__(self, exc_type, value, traceback):
  signal.setitimer(signal.ITIMER_REAL, 0)
  return False
  def timeout(self, signum, frame):
@@ -385,7 +381,7 @@ def timeout(self, signum, frame):
  def file_pattern(name):
  return "{0}-{1}".format(os.getpid(), name)
  
-class FilePaths(object):

+class FilePaths:
  """
  FilePaths is an auto-generated filename that cleans itself up.
  
@@ -532,11 +528,11 @@ def pause_drive(self, drive, event=None):

  self.pause_drive(drive, "write_aio")
  return
  self.qmp('human-m

[PATCH v7 07/10] iotests: add script_initialize

[John Snow]

Like script_main, but doesn't require a single point of entry.
Replace all existing initialization sections with this drop-in replacement.

This brings debug support to all existing script-style iotests.

Reviewed-by: Max Reitz 
Signed-off-by: John Snow 
---
 tests/qemu-iotests/149|  3 +-
 tests/qemu-iotests/194|  4 +-
 tests/qemu-iotests/202|  4 +-
 tests/qemu-iotests/203|  4 +-
 tests/qemu-iotests/206|  2 +-
 tests/qemu-iotests/207|  6 ++-
 tests/qemu-iotests/208|  2 +-
 tests/qemu-iotests/209|  2 +-
 tests/qemu-iotests/210|  6 ++-
 tests/qemu-iotests/211|  6 ++-
 tests/qemu-iotests/212|  6 ++-
 tests/qemu-iotests/213|  6 ++-
 tests/qemu-iotests/216|  4 +-
 tests/qemu-iotests/218|  2 +-
 tests/qemu-iotests/219|  2 +-
 tests/qemu-iotests/222|  7 ++--
 tests/qemu-iotests/224|  4 +-
 tests/qemu-iotests/228|  6 ++-
 tests/qemu-iotests/234|  4 +-
 tests/qemu-iotests/235|  4 +-
 tests/qemu-iotests/236|  2 +-
 tests/qemu-iotests/237|  2 +-
 tests/qemu-iotests/238|  2 +
 tests/qemu-iotests/242|  2 +-
 tests/qemu-iotests/246|  2 +-
 tests/qemu-iotests/248|  2 +-
 tests/qemu-iotests/254|  2 +-
 tests/qemu-iotests/255|  2 +-
 tests/qemu-iotests/256|  2 +-
 tests/qemu-iotests/258|  7 ++--
 tests/qemu-iotests/260|  4 +-
 tests/qemu-iotests/262|  4 +-
 tests/qemu-iotests/264|  4 +-
 tests/qemu-iotests/277|  2 +
 tests/qemu-iotests/280|  8 ++--
 tests/qemu-iotests/283|  4 +-
 tests/qemu-iotests/iotests.py | 73 +++
 37 files changed, 128 insertions(+), 80 deletions(-)

diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149
index b4a21bf7b7..852768f80a 100755
--- a/tests/qemu-iotests/149
+++ b/tests/qemu-iotests/149
@@ -382,8 +382,7 @@ def test_once(config, qemu_img=False):
 
 
 # Obviously we only work with the luks image format
-iotests.verify_image_format(supported_fmts=['luks'])
-iotests.verify_platform()
+iotests.script_initialize(supported_fmts=['luks'])
 
 # We need sudo in order to run cryptsetup to create
 # dm-crypt devices. This is safe to use on any
diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194
index 9dc1bd3510..8b1f720af4 100755
--- a/tests/qemu-iotests/194
+++ b/tests/qemu-iotests/194
@@ -21,8 +21,8 @@
 
 import iotests
 
-iotests.verify_image_format(supported_fmts=['qcow2', 'qed', 'raw'])
-iotests.verify_platform(['linux'])
+iotests.script_initialize(supported_fmts=['qcow2', 'qed', 'raw'],
+  supported_platforms=['linux'])
 
 with iotests.FilePath('source.img') as source_img_path, \
  iotests.FilePath('dest.img') as dest_img_path, \
diff --git a/tests/qemu-iotests/202 b/tests/qemu-iotests/202
index 920a8683ef..e3900a44d1 100755
--- a/tests/qemu-iotests/202
+++ b/tests/qemu-iotests/202
@@ -24,8 +24,8 @@
 
 import iotests
 
-iotests.verify_image_format(supported_fmts=['qcow2'])
-iotests.verify_platform(['linux'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+  supported_platforms=['linux'])
 
 with iotests.FilePath('disk0.img') as disk0_img_path, \
  iotests.FilePath('disk1.img') as disk1_img_path, \
diff --git a/tests/qemu-iotests/203 b/tests/qemu-iotests/203
index 49eff5d405..4b4bd3307d 100755
--- a/tests/qemu-iotests/203
+++ b/tests/qemu-iotests/203
@@ -24,8 +24,8 @@
 
 import iotests
 
-iotests.verify_image_format(supported_fmts=['qcow2'])
-iotests.verify_platform(['linux'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+  supported_platforms=['linux'])
 
 with iotests.FilePath('disk0.img') as disk0_img_path, \
  iotests.FilePath('disk1.img') as disk1_img_path, \
diff --git a/tests/qemu-iotests/206 b/tests/qemu-iotests/206
index e2b50ae24d..f42432a838 100755
--- a/tests/qemu-iotests/206
+++ b/tests/qemu-iotests/206
@@ -23,7 +23,7 @@
 import iotests
 from iotests import imgfmt
 
-iotests.verify_image_format(supported_fmts=['qcow2'])
+iotests.script_initialize(supported_fmts=['qcow2'])
 
 with iotests.FilePath('t.qcow2') as disk_path, \
  iotests.FilePath('t.qcow2.base') as backing_path, \
diff --git a/tests/qemu-iotests/207 b/tests/qemu-iotests/207
index 3d9c1208ca..a6621410da 100755
--- a/tests/qemu-iotests/207
+++ b/tests/qemu-iotests/207
@@ -24,8 +24,10 @@ import iotests
 import subprocess
 import re
 
-iotests.verify_image_format(supported_fmts=['raw'])
-iotests.verify_protocol(supported=['ssh'])
+iotests.script_initialize(
+supported_fmts=['raw'],
+supported_protocols=['ssh'],
+)
 
 def filter_hash(qmsg):
 def _filter(key, value):
diff --git a/tests/qemu-iotests/208 b/tests/qemu-iotests/208
index 1c3fc8c7fd..6cb642f821 100755
--- a/tests/qemu-iotests/208
+++ b/tests/qemu-iotests/208
@@ -22,7 +22,7 @@
 
 import iotests
 
-iotests.verify_image_format(supp

[PATCH v7 08/10] iotest 258: use script_main

[John Snow]

Since this one is nicely factored to use a single entry point,
use script_main to run the tests.

Signed-off-by: John Snow 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Max Reitz 
---
 tests/qemu-iotests/258 | 11 ---
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/tests/qemu-iotests/258 b/tests/qemu-iotests/258
index a65151dda6..e305a1502f 100755
--- a/tests/qemu-iotests/258
+++ b/tests/qemu-iotests/258
@@ -23,12 +23,6 @@ import iotests
 from iotests import log, qemu_img, qemu_io_silent, \
 filter_qmp_testfiles, filter_qmp_imgfmt
 
-# Need backing file and change-backing-file support
-iotests.script_initialize(
-supported_fmts=['qcow2', 'qed'],
-supported_platforms=['linux'],
-)
-
 # Returns a node for blockdev-add
 def node(node_name, path, backing=None, fmt=None, throttle=None):
 if fmt is None:
@@ -161,4 +155,7 @@ def main():
 test_concurrent_finish(False)
 
 if __name__ == '__main__':
-main()
+# Need backing file and change-backing-file support
+iotests.script_main(main,
+supported_fmts=['qcow2', 'qed'],
+supported_platforms=['linux'])
-- 
2.21.1

[PATCH v7 09/10] iotests: Mark verify functions as private

[John Snow]

Discourage their use.

(Also, make pending patches not yet using the new entry points fail in a
very obvious way.)

Reviewed-by: Max Reitz 
Signed-off-by: John Snow 
---
 tests/qemu-iotests/iotests.py | 20 ++--
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 1f88d2fa2a..23678f2daa 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -982,7 +982,7 @@ def case_notrun(reason):
 open('%s/%s.casenotrun' % (output_dir, seq), 'a').write(
 '[case not run] ' + reason + '\n')
 
-def verify_image_format(supported_fmts=(), unsupported_fmts=()):
+def _verify_image_format(supported_fmts=(), unsupported_fmts=()):
 assert not (supported_fmts and unsupported_fmts)
 
 if 'generic' in supported_fmts and \
@@ -996,7 +996,7 @@ def verify_image_format(supported_fmts=(), 
unsupported_fmts=()):
 if not_sup or (imgfmt in unsupported_fmts):
 notrun('not suitable for this image format: %s' % imgfmt)
 
-def verify_protocol(supported=(), unsupported=()):
+def _verify_protocol(supported=(), unsupported=()):
 assert not (supported and unsupported)
 
 if 'generic' in supported:
@@ -1006,7 +1006,7 @@ def verify_protocol(supported=(), unsupported=()):
 if not_sup or (imgproto in unsupported):
 notrun('not suitable for this protocol: %s' % imgproto)
 
-def verify_platform(supported=(), unsupported=()):
+def _verify_platform(supported=(), unsupported=()):
 if any((sys.platform.startswith(x) for x in unsupported)):
 notrun('not suitable for this OS: %s' % sys.platform)
 
@@ -1014,11 +1014,11 @@ def verify_platform(supported=(), unsupported=()):
 if not any((sys.platform.startswith(x) for x in supported)):
 notrun('not suitable for this OS: %s' % sys.platform)
 
-def verify_cache_mode(supported_cache_modes=()):
+def _verify_cache_mode(supported_cache_modes=()):
 if supported_cache_modes and (cachemode not in supported_cache_modes):
 notrun('not suitable for this cache mode: %s' % cachemode)
 
-def verify_aio_mode(supported_aio_modes=()):
+def _verify_aio_mode(supported_aio_modes=()):
 if supported_aio_modes and (aiomode not in supported_aio_modes):
 notrun('not suitable for this aio mode: %s' % aiomode)
 
@@ -1145,11 +1145,11 @@ def execute_setup_common(supported_fmts: 
Collection[str] = (),
 sys.stderr.write('Please run this test via the "check" script\n')
 sys.exit(os.EX_USAGE)
 
-verify_image_format(supported_fmts, unsupported_fmts)
-verify_protocol(supported_protocols, unsupported_protocols)
-verify_platform(supported=supported_platforms)
-verify_cache_mode(supported_cache_modes)
-verify_aio_mode(supported_aio_modes)
+_verify_image_format(supported_fmts, unsupported_fmts)
+_verify_protocol(supported_protocols, unsupported_protocols)
+_verify_platform(supported=supported_platforms)
+_verify_cache_mode(supported_cache_modes)
+_verify_aio_mode(supported_aio_modes)
 
 debug = '-d' in sys.argv
 if debug:
-- 
2.21.1

[PATCH v7 06/10] iotests: limit line length to 79 chars

[John Snow]

79 is the PEP8 recommendation. This recommendation works well for
reading patch diffs in TUI email clients.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/iotests.py | 69 ++-
 tests/qemu-iotests/pylintrc   |  6 ++-
 2 files changed, 48 insertions(+), 27 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 54d68774e1..1be11f491f 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -76,9 +76,11 @@
 def qemu_img(*args):
 '''Run qemu-img and return the exit code'''
 devnull = open('/dev/null', 'r+')
-exitcode = subprocess.call(qemu_img_args + list(args), stdin=devnull, 
stdout=devnull)
+exitcode = subprocess.call(qemu_img_args + list(args),
+   stdin=devnull, stdout=devnull)
 if exitcode < 0:
-sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' 
'.join(qemu_img_args + list(args
+sys.stderr.write('qemu-img received signal %i: %s\n' % (
+-exitcode, ' '.join(qemu_img_args + list(args
 return exitcode
 
 def ordered_qmp(qmsg, conv_keys=True):
@@ -117,7 +119,8 @@ def qemu_img_verbose(*args):
 '''Run qemu-img without suppressing its output and return the exit code'''
 exitcode = subprocess.call(qemu_img_args + list(args))
 if exitcode < 0:
-sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' 
'.join(qemu_img_args + list(args
+sys.stderr.write('qemu-img received signal %i: %s\n' % (
+-exitcode, ' '.join(qemu_img_args + list(args
 return exitcode
 
 def qemu_img_pipe(*args):
@@ -128,7 +131,8 @@ def qemu_img_pipe(*args):
 universal_newlines=True)
 exitcode = subp.wait()
 if exitcode < 0:
-sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' 
'.join(qemu_img_args + list(args
+sys.stderr.write('qemu-img received signal %i: %s\n' % (
+-exitcode, ' '.join(qemu_img_args + list(args
 return subp.communicate()[0]
 
 def qemu_img_log(*args):
@@ -158,7 +162,8 @@ def qemu_io(*args):
 universal_newlines=True)
 exitcode = subp.wait()
 if exitcode < 0:
-sys.stderr.write('qemu-io received signal %i: %s\n' % (-exitcode, ' 
'.join(args)))
+sys.stderr.write('qemu-io received signal %i: %s\n' % (
+-exitcode, ' '.join(args)))
 return subp.communicate()[0]
 
 def qemu_io_log(*args):
@@ -280,10 +285,13 @@ def filter_test_dir(msg):
 def filter_win32(msg):
 return win32_re.sub("", msg)
 
-qemu_io_re = re.compile(r"[0-9]* ops; [0-9\/:. sec]* \([0-9\/.inf]* 
[EPTGMKiBbytes]*\/sec and [0-9\/.inf]* ops\/sec\)")
+qemu_io_re = re.compile(r"[0-9]* ops; [0-9\/:. sec]* "
+r"\([0-9\/.inf]* [EPTGMKiBbytes]*\/sec "
+r"and [0-9\/.inf]* ops\/sec\)")
 def filter_qemu_io(msg):
 msg = filter_win32(msg)
-return qemu_io_re.sub("X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)", 
msg)
+return qemu_io_re.sub("X ops; XX:XX:XX.X "
+  "(XXX YYY/sec and XXX ops/sec)", msg)
 
 chown_re = re.compile(r"chown [0-9]+:[0-9]+")
 def filter_chown(msg):
@@ -335,7 +343,9 @@ def filter_img_info(output, filename):
 line = line.replace(filename, 'TEST_IMG') \
.replace(imgfmt, 'IMGFMT')
 line = re.sub('iters: [0-9]+', 'iters: XXX', line)
-line = re.sub('uuid: [-a-f0-9]+', 'uuid: 
----', line)
+line = re.sub('uuid: [-a-f0-9]+',
+  'uuid: ----',
+  line)
 line = re.sub('cid: [0-9]+', 'cid: XX', line)
 lines.append(line)
 return '\n'.join(lines)
@@ -356,12 +366,9 @@ def log(msg, filters=(), indent=None):
 for flt in filters:
 msg = flt(msg)
 if isinstance(msg, (dict, list)):
-# Python < 3.4 needs to know not to add whitespace when 
pretty-printing:
-separators = (', ', ': ') if indent is None else (',', ': ')
 # Don't sort if it's already sorted
 do_sort = not isinstance(msg, OrderedDict)
-print(json.dumps(msg, sort_keys=do_sort,
- indent=indent, separators=separators))
+print(json.dumps(msg, sort_keys=do_sort, indent=indent))
 else:
 print(msg)
 
@@ -529,11 +536,13 @@ def pause_drive(self, drive, event=None):
 self.pause_drive(drive, "write_aio")
 return
 self.qmp('human-monitor-command',
- command_line='qemu-io %s "break %s bp_%s"' % (drive, event, 
drive))
+ command_line='qemu-io %s "break %s bp_%s"' % (
+ drive, event, drive))
 
 def resume_drive(self, drive):
 self.qmp('human-monitor-command',
- command_line='qemu-io %s "remove_break bp_%s"' % (drive, 
drive))
+  

[PATCH v7 04/10] iotests: replace mutable list default args

[John Snow]

It's bad hygiene: if we modify this list, it will be modified across all
invocations.

(Remaining bad usages are fixed in a subsequent patch which changes the
function signature anyway.)

Signed-off-by: John Snow 
---
 tests/qemu-iotests/iotests.py | 24 
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index fd65b90691..54d68774e1 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -136,7 +136,7 @@ def qemu_img_log(*args):
 log(result, filters=[filter_testfiles])
 return result
 
-def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[]):
+def img_info_log(filename, filter_path=None, imgopts=False, extra_args=()):
 args = ['info']
 if imgopts:
 args.append('--image-opts')
@@ -350,7 +350,7 @@ def _filter(_key, value):
 return value
 return filter_qmp(qmsg, _filter)
 
-def log(msg, filters=[], indent=None):
+def log(msg, filters=(), indent=None):
 '''Logs either a string message or a JSON serializable message (like QMP).
 If indent is provided, JSON serializable messages are pretty-printed.'''
 for flt in filters:
@@ -566,7 +566,7 @@ def get_qmp_events_filtered(self, wait=60.0):
 result.append(filter_qmp_event(ev))
 return result
 
-def qmp_log(self, cmd, filters=[], indent=None, **kwargs):
+def qmp_log(self, cmd, filters=(), indent=None, **kwargs):
 full_cmd = OrderedDict((
 ("execute", cmd),
 ("arguments", ordered_qmp(kwargs))
@@ -967,7 +967,7 @@ def case_notrun(reason):
 open('%s/%s.casenotrun' % (output_dir, seq), 'a').write(
 '[case not run] ' + reason + '\n')
 
-def verify_image_format(supported_fmts=[], unsupported_fmts=[]):
+def verify_image_format(supported_fmts=(), unsupported_fmts=()):
 assert not (supported_fmts and unsupported_fmts)
 
 if 'generic' in supported_fmts and \
@@ -981,7 +981,7 @@ def verify_image_format(supported_fmts=[], 
unsupported_fmts=[]):
 if not_sup or (imgfmt in unsupported_fmts):
 notrun('not suitable for this image format: %s' % imgfmt)
 
-def verify_protocol(supported=[], unsupported=[]):
+def verify_protocol(supported=(), unsupported=()):
 assert not (supported and unsupported)
 
 if 'generic' in supported:
@@ -1000,11 +1000,11 @@ def verify_platform(supported=None, unsupported=None):
 if not any((sys.platform.startswith(x) for x in supported)):
 notrun('not suitable for this OS: %s' % sys.platform)
 
-def verify_cache_mode(supported_cache_modes=[]):
+def verify_cache_mode(supported_cache_modes=()):
 if supported_cache_modes and (cachemode not in supported_cache_modes):
 notrun('not suitable for this cache mode: %s' % cachemode)
 
-def verify_aio_mode(supported_aio_modes=[]):
+def verify_aio_mode(supported_aio_modes=()):
 if supported_aio_modes and (aiomode not in supported_aio_modes):
 notrun('not suitable for this aio mode: %s' % aiomode)
 
@@ -1044,7 +1044,7 @@ def supported_formats(read_only=False):
 
 return supported_formats.formats[read_only]
 
-def skip_if_unsupported(required_formats=[], read_only=False):
+def skip_if_unsupported(required_formats=(), read_only=False):
 '''Skip Test Decorator
Runs the test if all the required formats are whitelisted'''
 def skip_test_decorator(func):
@@ -1095,11 +1095,11 @@ def execute_unittest(output, verbosity, debug):
 sys.stderr.write(out)
 
 def execute_test(test_function=None,
- supported_fmts=[],
+ supported_fmts=(),
  supported_platforms=None,
- supported_cache_modes=[], supported_aio_modes={},
- unsupported_fmts=[], supported_protocols=[],
- unsupported_protocols=[]):
+ supported_cache_modes=(), supported_aio_modes=(),
+ unsupported_fmts=(), supported_protocols=(),
+ unsupported_protocols=()):
 """Run either unittest or script-style tests."""
 
 # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to
-- 
2.21.1

[PATCH v7 03/10] iotests: ignore import warnings from pylint

[John Snow]

The right way to solve this is to come up with a virtual environment
infrastructure that sets all the paths correctly, and/or to create
installable python modules that can be imported normally.

That's hard, so just silence this error for now.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/iotests.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index cd0f185d30..fd65b90691 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -29,6 +29,7 @@
 import io
 from collections import OrderedDict
 
+# pylint: disable=import-error, wrong-import-position
 sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
 from qemu import qtest
 
-- 
2.21.1

[PATCH v7 01/10] iotests: do a light delinting

[John Snow]

This doesn't fix everything in here, but it does help clean up the
pylint report considerably.

This should be 100% style changes only; the intent is to make pylint
more useful by working on establishing a baseline for iotests that we
can gate against in the future. This will be important if (when?) we
begin adding type hints to our code base.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/iotests.py | 83 ++-
 1 file changed, 43 insertions(+), 40 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 8815052eb5..c3aa857140 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -16,11 +16,9 @@
 # along with this program.  If not, see .
 #
 
-import errno
 import os
 import re
 import subprocess
-import string
 import unittest
 import sys
 import struct
@@ -34,7 +32,7 @@
 sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
 from qemu import qtest
 
-assert sys.version_info >= (3,6)
+assert sys.version_info >= (3, 6)
 
 # This will not work if arguments contain spaces but is necessary if we
 # want to support the override options that ./check supports.
@@ -138,11 +136,11 @@ def qemu_img_log(*args):
 return result
 
 def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[]):
-args = [ 'info' ]
+args = ['info']
 if imgopts:
 args.append('--image-opts')
 else:
-args += [ '-f', imgfmt ]
+args += ['-f', imgfmt]
 args += extra_args
 args.append(filename)
 
@@ -221,7 +219,7 @@ def cmd(self, cmd):
 # quit command is in close(), '\n' is added automatically
 assert '\n' not in cmd
 cmd = cmd.strip()
-assert cmd != 'q' and cmd != 'quit'
+assert cmd not in ('q', 'quit')
 self._p.stdin.write(cmd + '\n')
 self._p.stdin.flush()
 return self._read_output()
@@ -243,10 +241,8 @@ def qemu_nbd_early_pipe(*args):
 sys.stderr.write('qemu-nbd received signal %i: %s\n' %
  (-exitcode,
   ' '.join(qemu_nbd_args + ['--fork'] + list(args
-if exitcode == 0:
-return exitcode, ''
-else:
-return exitcode, subp.communicate()[0]
+
+return exitcode, subp.communicate()[0] if exitcode else ''
 
 def qemu_nbd_popen(*args):
 '''Run qemu-nbd in daemon mode and return the parent's exit code'''
@@ -310,7 +306,7 @@ def filter_qmp(qmsg, filter_fn):
 items = qmsg.items()
 
 for k, v in items:
-if isinstance(v, list) or isinstance(v, dict):
+if isinstance(v, (dict, list)):
 qmsg[k] = filter_qmp(v, filter_fn)
 else:
 qmsg[k] = filter_fn(k, v)
@@ -321,7 +317,7 @@ def filter_testfiles(msg):
 return msg.replace(prefix, 'TEST_DIR/PID-')
 
 def filter_qmp_testfiles(qmsg):
-def _filter(key, value):
+def _filter(_key, value):
 if is_str(value):
 return filter_testfiles(value)
 return value
@@ -347,7 +343,7 @@ def filter_imgfmt(msg):
 return msg.replace(imgfmt, 'IMGFMT')
 
 def filter_qmp_imgfmt(qmsg):
-def _filter(key, value):
+def _filter(_key, value):
 if is_str(value):
 return filter_imgfmt(value)
 return value
@@ -358,7 +354,7 @@ def log(msg, filters=[], indent=None):
 If indent is provided, JSON serializable messages are pretty-printed.'''
 for flt in filters:
 msg = flt(msg)
-if isinstance(msg, dict) or isinstance(msg, list):
+if isinstance(msg, (dict, list)):
 # Python < 3.4 needs to know not to add whitespace when 
pretty-printing:
 separators = (', ', ': ') if indent is None else (',', ': ')
 # Don't sort if it's already sorted
@@ -369,14 +365,14 @@ def log(msg, filters=[], indent=None):
 print(msg)
 
 class Timeout:
-def __init__(self, seconds, errmsg = "Timeout"):
+def __init__(self, seconds, errmsg="Timeout"):
 self.seconds = seconds
 self.errmsg = errmsg
 def __enter__(self):
 signal.signal(signal.SIGALRM, self.timeout)
 signal.setitimer(signal.ITIMER_REAL, self.seconds)
 return self
-def __exit__(self, type, value, traceback):
+def __exit__(self, exc_type, value, traceback):
 signal.setitimer(signal.ITIMER_REAL, 0)
 return False
 def timeout(self, signum, frame):
@@ -385,7 +381,7 @@ def timeout(self, signum, frame):
 def file_pattern(name):
 return "{0}-{1}".format(os.getpid(), name)
 
-class FilePaths(object):
+class FilePaths:
 """
 FilePaths is an auto-generated filename that cleans itself up.
 
@@ -532,11 +528,11 @@ def pause_drive(self, drive, event=None):
 self.pause_drive(drive, "write_aio")
 return
 self.qmp('human-monitor-command',
-command_line='qemu-io %s "break %s bp_%s"' % (drive, 
event, drive))
+ command_l

[PATCH v7 05/10] iotests: add pylintrc file

[John Snow]

This allows others to get repeatable results with pylint. If you run
`pylint iotests.py`, you should see a 100% pass.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/pylintrc | 22 ++
 1 file changed, 22 insertions(+)
 create mode 100644 tests/qemu-iotests/pylintrc

diff --git a/tests/qemu-iotests/pylintrc b/tests/qemu-iotests/pylintrc
new file mode 100644
index 00..8720b6a0de
--- /dev/null
+++ b/tests/qemu-iotests/pylintrc
@@ -0,0 +1,22 @@
+[MESSAGES CONTROL]
+
+# Disable the message, report, category or checker with the given id(s). You
+# can either give multiple identifiers separated by comma (,) or put this
+# option multiple times (only on the command line, not in the configuration
+# file where it should appear only once). You can also use "--disable=all" to
+# disable everything first and then reenable specific checks. For example, if
+# you want to run only the similarities checker, you can use "--disable=all
+# --enable=similarities". If you want to run only the classes checker, but have
+# no Warning level messages displayed, use "--disable=all --enable=classes
+# --disable=W".
+disable=invalid-name,
+no-else-return,
+too-many-lines,
+too-few-public-methods,
+too-many-arguments,
+too-many-locals,
+too-many-branches,
+too-many-public-methods,
+# These are temporary, and should be removed:
+missing-docstring,
+line-too-long,
-- 
2.21.1

[PATCH v7 02/10] iotests: don't use 'format' for drive_add

[John Snow]

It shadows (with a different type) the built-in format.
Use something else.

Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Max Reitz 
Signed-off-by: John Snow 
---
 tests/qemu-iotests/055| 3 ++-
 tests/qemu-iotests/iotests.py | 6 +++---
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/tests/qemu-iotests/055 b/tests/qemu-iotests/055
index 82b9f5f47d..4175fff5e4 100755
--- a/tests/qemu-iotests/055
+++ b/tests/qemu-iotests/055
@@ -469,7 +469,8 @@ class TestDriveCompression(iotests.QMPTestCase):
 qemu_img('create', '-f', fmt, blockdev_target_img,
  str(TestDriveCompression.image_len), *args)
 if attach_target:
-self.vm.add_drive(blockdev_target_img, format=fmt, 
interface="none")
+self.vm.add_drive(blockdev_target_img,
+  img_format=fmt, interface="none")
 
 self.vm.launch()
 
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index c3aa857140..cd0f185d30 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -482,21 +482,21 @@ def add_drive_raw(self, opts):
 self._args.append(opts)
 return self
 
-def add_drive(self, path, opts='', interface='virtio', format=imgfmt):
+def add_drive(self, path, opts='', interface='virtio', img_format=imgfmt):
 '''Add a virtio-blk drive to the VM'''
 options = ['if=%s' % interface,
'id=drive%d' % self._num_drives]
 
 if path is not None:
 options.append('file=%s' % path)
-options.append('format=%s' % format)
+options.append('format=%s' % img_format)
 options.append('cache=%s' % cachemode)
 options.append('aio=%s' % aiomode)
 
 if opts:
 options.append(opts)
 
-if format == 'luks' and 'key-secret' not in opts:
+if img_format == 'luks' and 'key-secret' not in opts:
 # default luks support
 if luks_default_secret_object not in self._args:
 self.add_object(luks_default_secret_object)
-- 
2.21.1

[PATCH v7 00/10] iotests: use python logging

[John Snow]

This series uses python logging to enable output conditionally on
iotests.log(). We unify an initialization call (which also enables
debugging output for those tests with -d) and then make the switch
inside of iotests.

It will help alleviate the need to create logged/unlogged versions
of all the various helpers we have made.

Also, I got lost and accidentally delinted iotests while I was here.
Sorry about that.

V7:

[] : patches are identical
[] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/10:[0025] [FC] 'iotests: do a light delinting'
002/10:[] [--] 'iotests: don't use 'format' for drive_add'
003/10:[] [-C] 'iotests: ignore import warnings from pylint'
004/10:[0008] [FC] 'iotests: replace mutable list default args'
005/10:[0006] [FC] 'iotests: add pylintrc file'
006/10:[down] 'iotests: limit line length to 79 chars'
007/10:[0008] [FC] 'iotests: add script_initialize'
008/10:[] [--] 'iotest 258: use script_main'
009/10:[] [--] 'iotests: Mark verify functions as private'
010/10:[0006] [FC] 'iotests: use python logging for iotests.log()'

- All delinting patches are now entirely front-loaded.
- Redid delinting to avoid "correcting" no-else-return statements.
- Moved more mutable list corrections into patch 4, to make it standalone.
- Moved pylintrc up to patch 5. Disabled no-else-return.
- Added patch 6 to require line length checks.
  (Some python 3.4 compatibility code is removed as a consequence.)
- Patch 7 changes slightly as a result of patch 4 changes.
- Added some logging explainer into patch 10.
  (Patch changes slightly because of patch 6.)

V6:
 - It's been so long since V5, let's just look at it anew.
 - Dropped patch 1, rebased, added more delinting.
 - I'm not touching the supported_platforms thing.
   Not interested in rehashing that debate.

V5:
 - Rebased again
 - Allow Python tests to run on any platform

V4:
 - Rebased on top of kwolf/block at the behest of mreitz

V3:
 - Rebased for 4.1+; now based on main branch.

V2:
 - Added all of the other python tests I missed to use script_initialize
 - Refactored the common setup as per Ehabkost's suggestion
 - Added protocol arguments to common initialization,
   but this isn't strictly required.

John Snow (10):
  iotests: do a light delinting
  iotests: don't use 'format' for drive_add
  iotests: ignore import warnings from pylint
  iotests: replace mutable list default args
  iotests: add pylintrc file
  iotests: limit line length to 79 chars
  iotests: add script_initialize
  iotest 258: use script_main
  iotests: Mark verify functions as private
  iotests: use python logging for iotests.log()

 tests/qemu-iotests/030|   4 +-
 tests/qemu-iotests/055|   3 +-
 tests/qemu-iotests/149|   3 +-
 tests/qemu-iotests/194|   4 +-
 tests/qemu-iotests/202|   4 +-
 tests/qemu-iotests/203|   4 +-
 tests/qemu-iotests/206|   2 +-
 tests/qemu-iotests/207|   6 +-
 tests/qemu-iotests/208|   2 +-
 tests/qemu-iotests/209|   2 +-
 tests/qemu-iotests/210|   6 +-
 tests/qemu-iotests/211|   6 +-
 tests/qemu-iotests/212|   6 +-
 tests/qemu-iotests/213|   6 +-
 tests/qemu-iotests/216|   4 +-
 tests/qemu-iotests/218|   2 +-
 tests/qemu-iotests/219|   2 +-
 tests/qemu-iotests/222|   7 +-
 tests/qemu-iotests/224|   4 +-
 tests/qemu-iotests/228|   6 +-
 tests/qemu-iotests/234|   4 +-
 tests/qemu-iotests/235|   4 +-
 tests/qemu-iotests/236|   2 +-
 tests/qemu-iotests/237|   2 +-
 tests/qemu-iotests/238|   2 +
 tests/qemu-iotests/242|   2 +-
 tests/qemu-iotests/245|   1 +
 tests/qemu-iotests/245.out|  24 +--
 tests/qemu-iotests/246|   2 +-
 tests/qemu-iotests/248|   2 +-
 tests/qemu-iotests/254|   2 +-
 tests/qemu-iotests/255|   2 +-
 tests/qemu-iotests/256|   2 +-
 tests/qemu-iotests/258|  10 +-
 tests/qemu-iotests/260|   4 +-
 tests/qemu-iotests/262|   4 +-
 tests/qemu-iotests/264|   4 +-
 tests/qemu-iotests/277|   2 +
 tests/qemu-iotests/280|   8 +-
 tests/qemu-iotests/283|   4 +-
 tests/qemu-iotests/iotests.py | 300 --
 tests/qemu-iotests/pylintrc   |  26 +++
 42 files changed, 300 insertions(+), 196 deletions(-)
 create mode 100644 tests/qemu-iotests/pylintrc

-- 
2.21.1

Re: [PATCH] hw/ide: Remove status register read side effect

[Mark Cave-Ayland]

On 04/03/2020 03:11, jasper.low...@bt.com wrote:

>> cmd646_update_irq() only seems to raise PCI interrupt, should it also
>> have 
>> an option to use INT 14 and 15 in legacy mode similar to what my
>> patch 
>> does for via-ide?
> 
> Looking through /qemu/hw/ide/cmd646.c it doesn't look like QEMU has
> support for legacy mode. At the very least, it looks like we default to
> PCI native mode:
> 
> static void pci_cmd646_ide_realize(PCIDevice *dev, Error **errp)
>   ...
>   pci_conf[PCI_CLASS_PROG] = 0x8f;
>   ...
> 
> To add support for legacy mode it would require changing
> cmd646_update_irq() and maybe cmd646_set_irq() so that interrupts are
> conditionally raised on IRQ14 and/or IRQ15 when the ports are in legacy
> mode.

Yes, that's correct. However I'm quite confident from booting other non-Solaris 
OSs
under qemu-system-sparc64 that PCI native mode is being used, particularly as 
it is
possible to see the related PCI sabre IRQ routing configuration changes.


ATB,

Mark.

Re: [PATCH 2/2] via-ide: Also emulate non 100% native mode

[Mark Cave-Ayland]

On 04/03/2020 00:22, BALATON Zoltan wrote:

 So on that basis the best explanation as to what is happening is the
 comment in the link you provided here:
 https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/powerpc/platforms/chrp/pci.c?h=v4.14.172#n353



 /* Pegasos2 firmware version 20040810 configures the built-in IDE 
 controller
 * in legacy mode, but sets the PCI registers to PCI native mode.
 * The chip can only operate in legacy mode, so force the PCI class into 
 legacy
 * mode as well. The same fixup must be done to the class-code property in
 * the IDE node /pci@8000/ide@C,1
 */
>>>
>>> I'm not sure that it makes much sense that the firmware configures the chip 
>>> one way
>>> then puts info about a different way in the device tree. There could be 
>>> bugs but this
>>> is not likely. Especially because I see in traces that the firmware does 
>>> try to
>>> configure the device in native mode. These are the first few accesses of 
>>> the firmware
>>> to via-ide:
>>
>> But that is exactly what is happening! The comment above clearly indicates 
>> the
>> firmware incorrectly sets the IDE controller in native mode which is in exact
>> agreement with the trace you provide below. And in fact if you look at
> 
> I may be reading the comment wrong but to me that says that "firmware 
> configures IDE
> in _legacy_ mode" whereas the trace shows it actually configures it in 
> _native_ mode
> which is complying to the CHRP doc above. But since it cannot comply to the 
> "native
> devices using OpenPIC" part it probably tries to apply the "ISA devices 
> embedded in
> PCI" part and locks IRQ to 14 and 15. Or it just wants to avoid sharing IRQs 
> as much
> as possible and the designers decided they will use IRQ14 and 15 for IDE.

Interesting. My interpretation of the comment was that the hardware can only 
operate
in legacy mode, even though the firmware configures the PCI registers to enable
native mode (which is why the class-code and IRQ routing are wrong).

>> https://www.powerdeveloper.org/platforms/pegasos/devicetree you can see the 
>> nvramrc
>> hack that was released in order to fix the device tree to boot Linux which 
>> alters the
>> class-code and sets interrupts to 14 (which I believe is invalid, but 
>> seemingly good
>> enough here).
> 
> Isn't this the same fixup that newer Linux kernels already include? Maybe 
> this was
> needed before Linux properly supported Pegasos2 but later kernels will do this
> anyway. This does not give us any new info we did not have before I think 
> maybe just
> easier to see all fixups in one place.
> 
>>> pci_cfg_write via-ide 12:1 @0x9 <- 0xf
>>> pci_cfg_write via-ide 12:1 @0x40 <- 0xb
>>> pci_cfg_write via-ide 12:1 @0x41 <- 0xf2
>>> pci_cfg_write via-ide 12:1 @0x43 <- 0x35
>>> pci_cfg_write via-ide 12:1 @0x44 <- 0x18
>>> pci_cfg_write via-ide 12:1 @0x45 <- 0x1c
>>> pci_cfg_write via-ide 12:1 @0x46 <- 0xc0
>>> pci_cfg_write via-ide 12:1 @0x50 <- 0x17171717
>>> pci_cfg_write via-ide 12:1 @0x54 <- 0x14
>>> pci_cfg_read via-ide 12:1 @0x0 -> 0x5711106
>>> pci_cfg_read via-ide 12:1 @0x0 -> 0x5711106
>>> pci_cfg_read via-ide 12:1 @0x8 -> 0x1018f06
>>> pci_cfg_read via-ide 12:1 @0xc -> 0x0
>>> pci_cfg_read via-ide 12:1 @0x2c -> 0x11001af4
>>> pci_cfg_read via-ide 12:1 @0x3c -> 0x10e
>>> pci_cfg_read via-ide 12:1 @0x4 -> 0x2800080
>>> pci_cfg_read via-ide 12:1 @0x3c -> 0x10e
>>> pci_cfg_write via-ide 12:1 @0x3c <- 0x109
>>>
>>> The very first write is to turn on native mode, so I think it's not about 
>>> what the
>>> firmware does but something about how hardware is wired on Pegasos II or 
>>> the VT8231
>>> chip itself that only allows legacy interrupts instead of 100% native mode 
>>> for IDE.
>>>
 Given that the DT is wrong, then we should assume that all OSs would have 
 to
 compensate for this in the same way as Linux, and therefore this should be 
 handled
 automatically.

 AFAICT this then only leaves the question: why does the firmware set
 PCI_INTERRUPT_LINE to 9, which is presumably why you are seeing problems 
 running
 MorphOS under QEMU.
>>>
>>> Linux does try to handle both true native mode and half-native mode. It 
>>> only uses
>>> half-native mode if finds IRQ14 on Pegasos, otherwise skips Pegasos 
>>> specific fixup
>>> and uses true native mode setup. I don't know what MorphOS does but I think 
>>> it justs
>>> knows that Pegasos2 has this quirk and does not look at the device tree at 
>>> all.

I think this is the other way around? From the code above:

if (viaide->irq != 14)
return;

Doesn't this suggest that chrp_pci_fixup_vt8231_ata() exits without applying 
the fix
if it finds PCI_INTERRUPT_LINE set to 9 from the firmware above?

Do you have a copy of the full DT and the firmware revision number that was 
used to
generate your Linux boot output on real hardware?

>> Again to summarise: this is a 

Re: [PATCH v6 18/18] docs: add Orange Pi PC document

[Niek Linnenbank]

Hello Alex,

On Wed, Mar 4, 2020 at 11:35 AM Alex Bennée  wrote:

>
> Niek Linnenbank  writes:
>
> > The Xunlong Orange Pi PC machine is a functional ARM machine
> > based on the Allwinner H3 System-on-Chip. It supports mainline
> > Linux, U-Boot, NetBSD and is covered by acceptance tests.
> >
> > This commit adds a documentation text file with a description
> > of the machine and instructions for the user.
>
> This is great, thanks for taking the time to include documentation.
>

Sure, I'm happy to contribute it. I tried to include most relevant
information
in there for developers and potential users of the machine. If something is
missing
just let me know and I'll make an update for it.


>
> >
> > Signed-off-by: Niek Linnenbank 
> > ---
> >  docs/orangepi.rst | 226
> > ++
>
> I suspect there is a better place to put this is than the top level. I
> wonder if it should be docs/specs?
>

Yeah I'm fine to move it to a different subdirectory if needed. Right now
its placed in docs,
as it seems that documents for machines are also placed there, for example
docs/microvm.rst


>
> >  MAINTAINERS   |   1 +
> >  2 files changed, 227 insertions(+)
> >  create mode 100644 docs/orangepi.rst
> >
> > diff --git a/docs/orangepi.rst b/docs/orangepi.rst
> > new file mode 100644
> > index 00..a9b46f553c
> > --- /dev/null
> > +++ b/docs/orangepi.rst
> > @@ -0,0 +1,226 @@
> > +=
> > +Orange Pi PC Machine Type
> > +=
> > +
> > +The Xunlong Orange Pi PC is an Allwinner H3 System on Chip
> > +based embedded computer with mainline support in both U-Boot
> > +and Linux. The board comes with a Quad Core Cortex A7 @ 1.3GHz,
> > +1GiB RAM, 100Mbit ethernet, USB, SD/MMC, USB, HDMI and
> > +various other I/O.
>
> When Peter's document PR goes in later this week there will also be a:
>
>   docs/system/target-arm.rst
>
> which would benefit from a section for the Orange Pi in it.
>

Nice, I'll have a look at it when it gets merged and add a section for
Orange Pi.
I'll try to include that in the next version of this series.


>
> > +
> > +Supported devices
> > +-
> > +
> > +The Orange Pi PC machine supports the following devices:
> > +
> > + * SMP (Quad Core Cortex A7)
> > + * Generic Interrupt Controller configuration
> > + * SRAM mappings
> > + * SDRAM controller
> > + * Real Time Clock
> > + * Timer device (re-used from Allwinner A10)
> > + * UART
> > + * SD/MMC storage controller
> > + * EMAC ethernet
>
> Do we ever exercise the ethernet in the acceptance tests? I see we have
> some that boots a full OS but boot console only seems to touch the
> serial console.
>

Good point. Currently in the acceptance tests U-Boot and Linux detect
the ethernet device, but there are no explicit checks for that in the
acceptance tests
nor any actual packet transmission.

I'll try to extend the current tests in the next version of this series to
include some
basic checks for the ethernet device, perhaps with DHCP or a ping.
I'm open to suggestions for the preferred way to do this.

Regards,
Niek






>
> 
>
> --
> Alex Bennée
>


-- 
Niek Linnenbank

Re: [PATCH v3 5/6] hw/misc/mos6522: move timer_new from init() into realize() to avoid memleaks

[Mark Cave-Ayland]

On 03/03/2020 01:36, Pan Nengyuan wrote:

> On 3/3/2020 3:17 AM, Mark Cave-Ayland wrote:
>> On 02/03/2020 13:21, Peter Maydell wrote:
>>
>>> On Thu, 27 Feb 2020 at 02:35, Pan Nengyuan  wrote:

 There are some memleaks when we call 'device_list_properties'. This patch 
 move timer_new from init into realize to fix it.
 Meanwhile, add calls to mos6522_realize() in mac_via_realize to make this 
 move to be valid.

 Reported-by: Euler Robot 
 Signed-off-by: Pan Nengyuan 
 ---
 Cc: Laurent Vivier 
 ---
 v2->v1:
 - no changes in this patch.
 v3->v2:
 - remove null check in reset, and add calls to mos6522_realize() in 
 mac_via_realize to make this move to be valid.
>>>
>>> Hi; this is really fixing two bugs in one patch:
>>>
 ---
  hw/misc/mac_via.c | 5 +
  hw/misc/mos6522.c | 6 ++
  2 files changed, 11 insertions(+)

 diff --git a/hw/misc/mac_via.c b/hw/misc/mac_via.c
 index b7d0012794..1d72d4ef35 100644
 --- a/hw/misc/mac_via.c
 +++ b/hw/misc/mac_via.c
 @@ -879,6 +879,11 @@ static void mac_via_realize(DeviceState *dev, Error 
 **errp)
  sysbus_init_child_obj(OBJECT(dev), "via2", &m->mos6522_via2,
sizeof(m->mos6522_via2), 
 TYPE_MOS6522_Q800_VIA2);

 +object_property_set_bool(OBJECT(&m->mos6522_via1), true, "realized",
 + &error_abort);
 +object_property_set_bool(OBJECT(&m->mos6522_via2), true, "realized",
 + &error_abort);
 +
  /* Pass through mos6522 output IRQs */
  ms = MOS6522(&m->mos6522_via1);
  object_property_add_alias(OBJECT(dev), "irq[0]", OBJECT(ms),
>>>
>>> This is fixing a bug in mac_via where it failed to actually
>>> realize devices it was using. That's a dependency for the bug
>>> you're trying to fix, but it's a separate one and should be
>>> in its own patch.
>>
>> Sigh. Thanks for this - I actually discovered this a little while back and 
>> have some
>> local patches to do the same, but due to lack of time I never managed to 
>> tidy them up
>> for submission.
> 
> Hmm, maybe you can take this other changes(fix memleaks) into your local 
> patches and send it together?
> Or If you have no time, I can help to do it about this device. :)

My patches are part of various q800 branches I have been playing with over the 
past
couple of months, and so are lagging quite far behind master. If you are able to
update them based upon Peter's comments them I'm happy to review them (and 
perhaps
could perhaps take them along with my cmd646 patchset if required).


ATB,

Mark.

Re: [PATCH v2 4/4] qapi: Brush off some (py)lint

[John Snow]

On 3/4/20 10:59 AM, Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster 
> ---
>  scripts/qapi/commands.py   |  2 +-
>  scripts/qapi/expr.py   |  3 +--
>  scripts/qapi/gen.py|  9 ++---
>  scripts/qapi/introspect.py |  2 --
>  scripts/qapi/parser.py |  6 ++
>  scripts/qapi/schema.py | 11 +--
>  6 files changed, 15 insertions(+), 18 deletions(-)
> 

Looks okay. (I don't care as much about no-else-return being there or
not, and this module is your baby.)

Reviewed-by: John Snow 

Re: [PATCH v2 3/4] qapi: Use super() now we have Python 3

[John Snow]

On 3/4/20 10:59 AM, Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster 
> ---
>  scripts/qapi/commands.py   |  4 +--
>  scripts/qapi/error.py  |  4 +--
>  scripts/qapi/events.py |  4 +--
>  scripts/qapi/gen.py| 10 
>  scripts/qapi/introspect.py |  4 +--
>  scripts/qapi/parser.py |  2 +-
>  scripts/qapi/schema.py | 51 +++---
>  scripts/qapi/types.py  |  4 +--
>  scripts/qapi/visit.py  |  4 +--
>  9 files changed, 43 insertions(+), 44 deletions(-)
> 

Reviewed-by: John Snow 

Re: [PATCH v2 2/4] qapi: Drop conditionals for Python 2

[John Snow]

On 3/4/20 10:59 AM, Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster 
> Reviewed-by: Philippe Mathieu-Daudé 
> ---
>  scripts/qapi/common.py | 6 +-
>  scripts/qapi/gen.py| 6 +-
>  scripts/qapi/parser.py | 6 +-
>  tests/qapi-schema/test-qapi.py | 6 +-
>  4 files changed, 4 insertions(+), 20 deletions(-)
> 

Seeyanara.

Reviewed-by: John Snow 

Re: [PATCH v2 1/4] qapi: Inheriting from object is pointless with Python 3, drop

[John Snow]

On 3/4/20 10:59 AM, Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster 
> Reviewed-by: Philippe Mathieu-Daudé 

adios, cowboy

Reviewed-by: John Snow 

Re: [PATCH v5 40/50] multi-process/mig: build migration module in the remote process

[Jag Raman]

On 3/4/2020 2:52 PM, Dr. David Alan Gilbert wrote:

* Jag Raman (jag.ra...@oracle.com) wrote:



On 3/4/2020 10:58 AM, Dr. David Alan Gilbert wrote:

* Jagannathan Raman (jag.ra...@oracle.com) wrote:

Add Makefile support to enable migration in remote process

Signed-off-by: Elena Ufimtseva 
Signed-off-by: John G Johnson 
Signed-off-by: Jagannathan Raman 
---
   Makefile.objs   |  4 +++-
   Makefile.target |  1 +
   migration/Makefile.objs | 13 -
   net/Makefile.objs   |  2 ++
   softmmu/vl.c|  2 --
   stubs/migration.c   | 49 
+
   stubs/net-stub.c| 21 +
   stubs/qapi-misc.c   |  2 ++
   stubs/replay.c  |  8 
   stubs/vl-stub.c | 24 
   vl-parse.c  |  3 +++
   11 files changed, 125 insertions(+), 4 deletions(-)

diff --git a/Makefile.objs b/Makefile.objs
index 4b5db09..65009da 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -74,6 +74,8 @@ common-obj-y += qdev-monitor.o device-hotplug.o
   common-obj-$(CONFIG_WIN32) += os-win32.o
   common-obj-$(CONFIG_POSIX) += os-posix.o
+remote-pci-obj-$(CONFIG_POSIX) += os-posix.o
+
   common-obj-$(CONFIG_LINUX) += fsdev/
   common-obj-y += accel/
@@ -104,11 +106,11 @@ common-obj-y += vl-parse.o
   ###
   # qapi
-
   common-obj-y += qapi/
   endif # CONFIG_SOFTMMU
+remote-pci-obj-$(CONFIG_MPQEMU) += net/
   remote-pci-obj-$(CONFIG_MPQEMU) += qapi/
   remote-pci-obj-$(CONFIG_MPQEMU) += blockdev-nbd.o
   remote-pci-obj-$(CONFIG_MPQEMU) += job-qmp.o
diff --git a/Makefile.target b/Makefile.target
index 4ead5c3..4012ae5 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -240,6 +240,7 @@ all-remote-pci-obj-y += exec.o
   all-remote-pci-obj-y += exec-vary.o
   all-remote-pci-obj-y += ioport.o
   all-remote-pci-obj-y += cpus.o
+all-remote-pci-obj-y += migration/ram.o
   endif
   remote-pci-obj-y :=
diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index e7cdc76..21f9d8d 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -15,4 +15,15 @@ common-obj-$(CONFIG_LIVE_BLOCK_MIGRATION) += block.o
   rdma.o-libs := $(RDMA_LIBS)
-remote-pci-obj-$(CONFIG_MPQEMU) += qemu-file.o vmstate.o qjson.o 
vmstate-types.o
+remote-pci-obj-$(CONFIG_MPQEMU) += migration.o socket.o fd.o exec.o
+remote-pci-obj-$(CONFIG_MPQEMU) += tls.o channel.o savevm.o
+remote-pci-obj-$(CONFIG_MPQEMU) += colo.o colo-failover.o
+remote-pci-obj-$(CONFIG_MPQEMU) += vmstate.o vmstate-types.o page_cache.o
+remote-pci-obj-$(CONFIG_MPQEMU) += qemu-file.o global_state.o
+remote-pci-obj-$(CONFIG_MPQEMU) += qemu-file-channel.o
+remote-pci-obj-$(CONFIG_MPQEMU) += xbzrle.o postcopy-ram.o
+remote-pci-obj-$(CONFIG_MPQEMU) += qjson.o
+remote-pci-obj-$(CONFIG_MPQEMU) += block-dirty-bitmap.o
+remote-pci-obj-$(CONFIG_RDMA) += rdma.o
+remote-pci-obj-$(CONFIG_MPQEMU) += block.o
+remote-pci-obj-$(CONFIG_MPQEMU) += multifd.o


Hmm, are you really going to want all this lot in your remote process?
Assuming it's just devices, I can understand the first line or two, but
it seems odd to have all of this.


Yeah, we ended up needing to compile these in to enable migration. We
are only using "fd" to enable migration. Although we don't use tls,
xbzrle, rdma, multifd, etc... for example, the migration code does
support these protocols and, therefore, we had to compile them in.


But are you even running a migration stream from the remote process?
Aren't you just doing vmstate migration of devices; i.e. do you need
anything relating to incremental RAM migration (e.g. xbzrle, rdma,
postcopy).


We are running a migration stream from the remote process. We are only
doing the vmstate migration of devices, and not anything incremental
related to RAM.

We are using QEMU's existing migration infrastructure (vmstate_save /
qemu_loadvm_section_start_full) to move the vmstate. Based on my limited
experience with the migration code, I get that it comes as a suite.
Without some refactoring, we would need to build all of the files within
the migration folder.

The vmstate functions communicate over QEMUFile, which would be using
fd, tcp, rdma, etc... at the backend. These functions also need other
functions defined in migration.c, which require the building of xbzrle,
postcopy, etc...

Thank you!
--
Jag



Dave


Thank you!
--
Jag



Dave


diff --git a/net/Makefile.objs b/net/Makefile.objs
index c5d076d..a8ad986 100644
--- a/net/Makefile.objs
+++ b/net/Makefile.objs
@@ -30,3 +30,5 @@ common-obj-$(CONFIG_WIN32) += tap-win32.o
   vde.o-libs = $(VDE_LIBS)
   common-obj-$(CONFIG_CAN_BUS) += can/
+
+remote-pci-obj-$(CONFIG_MPQEMU) += announce.o
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 4a4f52c..42d5682 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -128,7 +128,6 @@ const char* keyboard_layout = NULL;
   ram_addr_t ram_size;
   const char *mem_path = NULL;
   int mem_p

Re: [PATCH v5 40/50] multi-process/mig: build migration module in the remote process

[Dr. David Alan Gilbert]

* Jag Raman (jag.ra...@oracle.com) wrote:
> 
> 
> On 3/4/2020 10:58 AM, Dr. David Alan Gilbert wrote:
> > * Jagannathan Raman (jag.ra...@oracle.com) wrote:
> > > Add Makefile support to enable migration in remote process
> > > 
> > > Signed-off-by: Elena Ufimtseva 
> > > Signed-off-by: John G Johnson 
> > > Signed-off-by: Jagannathan Raman 
> > > ---
> > >   Makefile.objs   |  4 +++-
> > >   Makefile.target |  1 +
> > >   migration/Makefile.objs | 13 -
> > >   net/Makefile.objs   |  2 ++
> > >   softmmu/vl.c|  2 --
> > >   stubs/migration.c   | 49 
> > > +
> > >   stubs/net-stub.c| 21 +
> > >   stubs/qapi-misc.c   |  2 ++
> > >   stubs/replay.c  |  8 
> > >   stubs/vl-stub.c | 24 
> > >   vl-parse.c  |  3 +++
> > >   11 files changed, 125 insertions(+), 4 deletions(-)
> > > 
> > > diff --git a/Makefile.objs b/Makefile.objs
> > > index 4b5db09..65009da 100644
> > > --- a/Makefile.objs
> > > +++ b/Makefile.objs
> > > @@ -74,6 +74,8 @@ common-obj-y += qdev-monitor.o device-hotplug.o
> > >   common-obj-$(CONFIG_WIN32) += os-win32.o
> > >   common-obj-$(CONFIG_POSIX) += os-posix.o
> > > +remote-pci-obj-$(CONFIG_POSIX) += os-posix.o
> > > +
> > >   common-obj-$(CONFIG_LINUX) += fsdev/
> > >   common-obj-y += accel/
> > > @@ -104,11 +106,11 @@ common-obj-y += vl-parse.o
> > >   ###
> > >   # qapi
> > > -
> > >   common-obj-y += qapi/
> > >   endif # CONFIG_SOFTMMU
> > > +remote-pci-obj-$(CONFIG_MPQEMU) += net/
> > >   remote-pci-obj-$(CONFIG_MPQEMU) += qapi/
> > >   remote-pci-obj-$(CONFIG_MPQEMU) += blockdev-nbd.o
> > >   remote-pci-obj-$(CONFIG_MPQEMU) += job-qmp.o
> > > diff --git a/Makefile.target b/Makefile.target
> > > index 4ead5c3..4012ae5 100644
> > > --- a/Makefile.target
> > > +++ b/Makefile.target
> > > @@ -240,6 +240,7 @@ all-remote-pci-obj-y += exec.o
> > >   all-remote-pci-obj-y += exec-vary.o
> > >   all-remote-pci-obj-y += ioport.o
> > >   all-remote-pci-obj-y += cpus.o
> > > +all-remote-pci-obj-y += migration/ram.o
> > >   endif
> > >   remote-pci-obj-y :=
> > > diff --git a/migration/Makefile.objs b/migration/Makefile.objs
> > > index e7cdc76..21f9d8d 100644
> > > --- a/migration/Makefile.objs
> > > +++ b/migration/Makefile.objs
> > > @@ -15,4 +15,15 @@ common-obj-$(CONFIG_LIVE_BLOCK_MIGRATION) += block.o
> > >   rdma.o-libs := $(RDMA_LIBS)
> > > -remote-pci-obj-$(CONFIG_MPQEMU) += qemu-file.o vmstate.o qjson.o 
> > > vmstate-types.o
> > > +remote-pci-obj-$(CONFIG_MPQEMU) += migration.o socket.o fd.o exec.o
> > > +remote-pci-obj-$(CONFIG_MPQEMU) += tls.o channel.o savevm.o
> > > +remote-pci-obj-$(CONFIG_MPQEMU) += colo.o colo-failover.o
> > > +remote-pci-obj-$(CONFIG_MPQEMU) += vmstate.o vmstate-types.o page_cache.o
> > > +remote-pci-obj-$(CONFIG_MPQEMU) += qemu-file.o global_state.o
> > > +remote-pci-obj-$(CONFIG_MPQEMU) += qemu-file-channel.o
> > > +remote-pci-obj-$(CONFIG_MPQEMU) += xbzrle.o postcopy-ram.o
> > > +remote-pci-obj-$(CONFIG_MPQEMU) += qjson.o
> > > +remote-pci-obj-$(CONFIG_MPQEMU) += block-dirty-bitmap.o
> > > +remote-pci-obj-$(CONFIG_RDMA) += rdma.o
> > > +remote-pci-obj-$(CONFIG_MPQEMU) += block.o
> > > +remote-pci-obj-$(CONFIG_MPQEMU) += multifd.o
> > 
> > Hmm, are you really going to want all this lot in your remote process?
> > Assuming it's just devices, I can understand the first line or two, but
> > it seems odd to have all of this.
> 
> Yeah, we ended up needing to compile these in to enable migration. We
> are only using "fd" to enable migration. Although we don't use tls,
> xbzrle, rdma, multifd, etc... for example, the migration code does
> support these protocols and, therefore, we had to compile them in.

But are you even running a migration stream from the remote process?
Aren't you just doing vmstate migration of devices; i.e. do you need
anything relating to incremental RAM migration (e.g. xbzrle, rdma,
postcopy).

Dave

> Thank you!
> --
> Jag
> 
> > 
> > Dave
> > 
> > > diff --git a/net/Makefile.objs b/net/Makefile.objs
> > > index c5d076d..a8ad986 100644
> > > --- a/net/Makefile.objs
> > > +++ b/net/Makefile.objs
> > > @@ -30,3 +30,5 @@ common-obj-$(CONFIG_WIN32) += tap-win32.o
> > >   vde.o-libs = $(VDE_LIBS)
> > >   common-obj-$(CONFIG_CAN_BUS) += can/
> > > +
> > > +remote-pci-obj-$(CONFIG_MPQEMU) += announce.o
> > > diff --git a/softmmu/vl.c b/softmmu/vl.c
> > > index 4a4f52c..42d5682 100644
> > > --- a/softmmu/vl.c
> > > +++ b/softmmu/vl.c
> > > @@ -128,7 +128,6 @@ const char* keyboard_layout = NULL;
> > >   ram_addr_t ram_size;
> > >   const char *mem_path = NULL;
> > >   int mem_prealloc = 0; /* force preallocation of physical target memory 
> > > */
> > > -bool enable_mlock = false;
> > >   bool enable_cpu_pm = false;
> > >   int nb_nics;
> > >   NICInfo nd_table[MAX_NICS];
> > > @@ -168,7 +167,6 @@ 

Re: [PATCH v5 08/50] multi-process: add functions to synchronize proxy and remote endpoints

[Dr. David Alan Gilbert]

* Jag Raman (jag.ra...@oracle.com) wrote:
> 
> 
> On 3/3/2020 2:56 PM, Dr. David Alan Gilbert wrote:
> > * Jagannathan Raman (jag.ra...@oracle.com) wrote:
> > > In some cases, for example MMIO read, QEMU has to wait for the remote to
> > > complete a command before proceeding. An eventfd based mechanism is
> > > added to synchronize QEMU & remote process.
> > > 
> > > Signed-off-by: John G Johnson 
> > > Signed-off-by: Jagannathan Raman 
> > > Signed-off-by: Elena Ufimtseva 
> > > ---
> > >   include/io/mpqemu-link.h |  7 +++
> > >   io/mpqemu-link.c | 41 +
> > >   2 files changed, 48 insertions(+)
> > > 
> > > diff --git a/include/io/mpqemu-link.h b/include/io/mpqemu-link.h
> > > index 2f2dd83..ae04fca 100644
> > > --- a/include/io/mpqemu-link.h
> > > +++ b/include/io/mpqemu-link.h
> > > @@ -135,4 +135,11 @@ void mpqemu_link_set_callback(MPQemuLinkState *s,
> > > mpqemu_link_callback callback);
> > >   void mpqemu_start_coms(MPQemuLinkState *s);
> > > +#define GET_REMOTE_WAIT eventfd(0, EFD_CLOEXEC)
> > > +#define PUT_REMOTE_WAIT(wait) close(wait)
> > > +#define PROXY_LINK_WAIT_DONE 1
> > > +
> > > +uint64_t wait_for_remote(int efd);
> > > +void notify_proxy(int fd, uint64_t val);
> > > +
> > >   #endif
> > > diff --git a/io/mpqemu-link.c b/io/mpqemu-link.c
> > > index bac120b..73b7032 100644
> > > --- a/io/mpqemu-link.c
> > > +++ b/io/mpqemu-link.c
> > > @@ -10,6 +10,7 @@
> > >   #include "qemu/osdep.h"
> > >   #include "qemu-common.h"
> > > +#include 
> > >   #include "qemu/module.h"
> > >   #include "io/mpqemu-link.h"
> > > @@ -216,6 +217,46 @@ int mpqemu_msg_recv(MPQemuMsg *msg, MPQemuChannel 
> > > *chan)
> > >   return rc;
> > >   }
> > > +uint64_t wait_for_remote(int efd)
> > > +{
> > > +struct pollfd pfd = { .fd = efd, .events = POLLIN };
> > > +uint64_t val;
> > > +int ret;
> > > +
> > > +ret = poll(&pfd, 1, 1000);
> > > +
> > > +switch (ret) {
> > > +case 0:
> > > +qemu_log_mask(LOG_REMOTE_DEBUG, "Error wait_for_remote: Timed 
> > > out\n");
> > > +/* TODO: Kick-off error recovery */
> > > +return ULLONG_MAX;
> > 
> > Shouldn't these be UINT64_MAX?
> 
> Sure, we'll change these to UINT64_MAX.
> 
> > 
> > > +case -1:
> > > +qemu_log_mask(LOG_REMOTE_DEBUG, "Poll error wait_for_remote: 
> > > %s\n",
> > > +  strerror(errno));
> > > +return ULLONG_MAX;
> > > +default:
> > > +if (read(efd, &val, sizeof(val)) == -1) {
> > > +qemu_log_mask(LOG_REMOTE_DEBUG, "Error wait_for_remote: 
> > > %s\n",
> > > +  strerror(errno));
> > > +return ULLONG_MAX;
> > > +}
> > > +}
> > > +
> > > +val = (val == ULLONG_MAX) ? val : (val - 1);
> > 
> > Can you explain what's going on there??
> 
> This wait_for_remote() function serves two purposes. This first one is
> to synchronize QEMU and the remote process. Secondly, it allows the
> remote process to return a 64-bit value to QEMU. When the remote process
> has completed the task, it uses the notify_proxy() function to return to
> QEMU along with a return value.
> 
> We have implemented this synchronization mechanism using eventfd, as
> it's easy to setup. So the remote process could write a non-zero value
> to the eventfd to wake QEMU up. However, the drawback of using eventfd
> for this purpose is that a return value of zero wouldn't wake QEMU up.
> Therefore, we offset the return value by one at the remote process and
> correct it in the QEMU end.

OK, that needs a big hairy comment to explain what's going on.

Dave

> --
> Jag
> 
> > 
> > > +return val;
> > > +}
> > > +
> > > +void notify_proxy(int efd, uint64_t val)
> > > +{
> > > +val = (val == ULLONG_MAX) ? val : (val + 1);
> > > +
> > > +if (write(efd, &val, sizeof(val)) == -1) {
> > 
> > I'd actually check the write/read's are returning sizeof(val) - they
> > can on a bad day return 0 or send only a few bytes; in theory?
> > 
> > Dave
> > 
> > > +qemu_log_mask(LOG_REMOTE_DEBUG, "Error notify_proxy: %s\n",
> > > +  strerror(errno));
> > > +}
> > > +}
> > > +
> > >   static gboolean mpqemu_link_handler_prepare(GSource *gsrc, gint 
> > > *timeout)
> > >   {
> > >   g_assert(timeout);
> > > -- 
> > > 1.8.3.1
> > > 
> > --
> > Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
> > 
> 
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [PATCH v5 40/50] multi-process/mig: build migration module in the remote process

[Jag Raman]

On 3/4/2020 10:58 AM, Dr. David Alan Gilbert wrote:

* Jagannathan Raman (jag.ra...@oracle.com) wrote:

Add Makefile support to enable migration in remote process

Signed-off-by: Elena Ufimtseva 
Signed-off-by: John G Johnson 
Signed-off-by: Jagannathan Raman 
---
  Makefile.objs   |  4 +++-
  Makefile.target |  1 +
  migration/Makefile.objs | 13 -
  net/Makefile.objs   |  2 ++
  softmmu/vl.c|  2 --
  stubs/migration.c   | 49 +
  stubs/net-stub.c| 21 +
  stubs/qapi-misc.c   |  2 ++
  stubs/replay.c  |  8 
  stubs/vl-stub.c | 24 
  vl-parse.c  |  3 +++
  11 files changed, 125 insertions(+), 4 deletions(-)

diff --git a/Makefile.objs b/Makefile.objs
index 4b5db09..65009da 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -74,6 +74,8 @@ common-obj-y += qdev-monitor.o device-hotplug.o
  common-obj-$(CONFIG_WIN32) += os-win32.o
  common-obj-$(CONFIG_POSIX) += os-posix.o
  
+remote-pci-obj-$(CONFIG_POSIX) += os-posix.o

+
  common-obj-$(CONFIG_LINUX) += fsdev/
  
  common-obj-y += accel/

@@ -104,11 +106,11 @@ common-obj-y += vl-parse.o
  
  ###

  # qapi
-
  common-obj-y += qapi/
  
  endif # CONFIG_SOFTMMU
  
+remote-pci-obj-$(CONFIG_MPQEMU) += net/

  remote-pci-obj-$(CONFIG_MPQEMU) += qapi/
  remote-pci-obj-$(CONFIG_MPQEMU) += blockdev-nbd.o
  remote-pci-obj-$(CONFIG_MPQEMU) += job-qmp.o
diff --git a/Makefile.target b/Makefile.target
index 4ead5c3..4012ae5 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -240,6 +240,7 @@ all-remote-pci-obj-y += exec.o
  all-remote-pci-obj-y += exec-vary.o
  all-remote-pci-obj-y += ioport.o
  all-remote-pci-obj-y += cpus.o
+all-remote-pci-obj-y += migration/ram.o
  endif
  
  remote-pci-obj-y :=

diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index e7cdc76..21f9d8d 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -15,4 +15,15 @@ common-obj-$(CONFIG_LIVE_BLOCK_MIGRATION) += block.o
  
  rdma.o-libs := $(RDMA_LIBS)
  
-remote-pci-obj-$(CONFIG_MPQEMU) += qemu-file.o vmstate.o qjson.o vmstate-types.o

+remote-pci-obj-$(CONFIG_MPQEMU) += migration.o socket.o fd.o exec.o
+remote-pci-obj-$(CONFIG_MPQEMU) += tls.o channel.o savevm.o
+remote-pci-obj-$(CONFIG_MPQEMU) += colo.o colo-failover.o
+remote-pci-obj-$(CONFIG_MPQEMU) += vmstate.o vmstate-types.o page_cache.o
+remote-pci-obj-$(CONFIG_MPQEMU) += qemu-file.o global_state.o
+remote-pci-obj-$(CONFIG_MPQEMU) += qemu-file-channel.o
+remote-pci-obj-$(CONFIG_MPQEMU) += xbzrle.o postcopy-ram.o
+remote-pci-obj-$(CONFIG_MPQEMU) += qjson.o
+remote-pci-obj-$(CONFIG_MPQEMU) += block-dirty-bitmap.o
+remote-pci-obj-$(CONFIG_RDMA) += rdma.o
+remote-pci-obj-$(CONFIG_MPQEMU) += block.o
+remote-pci-obj-$(CONFIG_MPQEMU) += multifd.o


Hmm, are you really going to want all this lot in your remote process?
Assuming it's just devices, I can understand the first line or two, but
it seems odd to have all of this.


Yeah, we ended up needing to compile these in to enable migration. We
are only using "fd" to enable migration. Although we don't use tls,
xbzrle, rdma, multifd, etc... for example, the migration code does
support these protocols and, therefore, we had to compile them in.

Thank you!
--
Jag



Dave


diff --git a/net/Makefile.objs b/net/Makefile.objs
index c5d076d..a8ad986 100644
--- a/net/Makefile.objs
+++ b/net/Makefile.objs
@@ -30,3 +30,5 @@ common-obj-$(CONFIG_WIN32) += tap-win32.o
  vde.o-libs = $(VDE_LIBS)
  
  common-obj-$(CONFIG_CAN_BUS) += can/

+
+remote-pci-obj-$(CONFIG_MPQEMU) += announce.o
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 4a4f52c..42d5682 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -128,7 +128,6 @@ const char* keyboard_layout = NULL;
  ram_addr_t ram_size;
  const char *mem_path = NULL;
  int mem_prealloc = 0; /* force preallocation of physical target memory */
-bool enable_mlock = false;
  bool enable_cpu_pm = false;
  int nb_nics;
  NICInfo nd_table[MAX_NICS];
@@ -168,7 +167,6 @@ const char *prom_envs[MAX_PROM_ENVS];
  int boot_menu;
  bool boot_strict;
  uint8_t *boot_splash_filedata;
-int only_migratable; /* turn it off unless user states otherwise */
  bool wakeup_suspend_enabled;
  
  int icount_align_option;

diff --git a/stubs/migration.c b/stubs/migration.c
index 28ccf80..dbd12db 100644
--- a/stubs/migration.c
+++ b/stubs/migration.c
@@ -6,6 +6,35 @@
  #include "qapi/qapi-types-migration.h"
  #include "qapi/qapi-commands-migration.h"
  #include "qapi/qapi-types-net.h"
+#include "net/filter.h"
+#include "net/colo-compare.h"
+
+#pragma weak qmp_query_migrate_capabilities
+#pragma weak qmp_query_migrate_parameters
+#pragma weak migrate_announce_params
+#pragma weak qmp_query_migrate
+#pragma weak qmp_migrate_set_capabilities
+#pragma weak qmp_migrate_set_parameters
+#pragma weak qmp_migrate_incoming
+#pragma wea

Re: [PATCH v5 16/50] multi-process: Synchronize remote memory

[Jag Raman]

On 3/4/2020 6:53 AM, Dr. David Alan Gilbert wrote:

* Jagannathan Raman (jag.ra...@oracle.com) wrote:

Add memory-listener object which is used to keep the view of the RAM
in sync between QEMU and remote process.
A MemoryListener is registered for system-memory AddressSpace. The
listener sends SYNC_SYSMEM message to the remote process when memory
listener commits the changes to memory, the remote process receives
the message and processes it in the handler for SYNC_SYSMEM message.

TODO: No need to create object for remote memory listener.

Signed-off-by: Jagannathan Raman 
Signed-off-by: John G Johnson 
Signed-off-by: Elena Ufimtseva 
---
  Makefile.target|   3 +
  hw/proxy/memory-sync.c | 212 +
  hw/proxy/qemu-proxy.c  |   5 +
  include/hw/proxy/memory-sync.h |  37 +++
  include/hw/proxy/qemu-proxy.h  |   5 +
  remote/remote-main.c   |  11 +++
  6 files changed, 273 insertions(+)
  create mode 100644 hw/proxy/memory-sync.c
  create mode 100644 include/hw/proxy/memory-sync.h

diff --git a/Makefile.target b/Makefile.target
index cfd36c1..271d883 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -127,6 +127,9 @@ obj-$(CONFIG_TCG) += fpu/softfloat.o
  obj-y += target/$(TARGET_BASE_ARCH)/
  obj-y += disas.o
  obj-$(call notempty,$(TARGET_XML_FILES)) += gdbstub-xml.o
+ifeq ($(TARGET_NAME)-$(CONFIG_MPQEMU)-$(CONFIG_USER_ONLY), x86_64-y-)
+obj-$(CONFIG_MPQEMU) += hw/proxy/memory-sync.o
+endif
  LIBS := $(libs_cpu) $(LIBS)
  
  obj-$(CONFIG_PLUGIN) += plugins/

diff --git a/hw/proxy/memory-sync.c b/hw/proxy/memory-sync.c
new file mode 100644
index 000..3edbb19
--- /dev/null
+++ b/hw/proxy/memory-sync.c
@@ -0,0 +1,212 @@
+/*
+ * Copyright © 2018, 2020 Oracle and/or its affiliates.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include 
+#include 
+#include 
+
+#include "qemu/osdep.h"
+#include "qemu/compiler.h"
+#include "qemu/int128.h"
+#include "qemu/range.h"
+#include "exec/memory.h"
+#include "exec/cpu-common.h"
+#include "cpu.h"
+#include "exec/ram_addr.h"
+#include "exec/address-spaces.h"
+#include "io/mpqemu-link.h"
+#include "hw/proxy/memory-sync.h"
+
+static const TypeInfo remote_mem_sync_type_info = {
+.name  = TYPE_MEMORY_LISTENER,
+.parent= TYPE_OBJECT,
+.instance_size = sizeof(RemoteMemSync),
+};
+
+static void remote_mem_sync_register_types(void)
+{
+type_register_static(&remote_mem_sync_type_info);
+}
+
+type_init(remote_mem_sync_register_types)
+
+static void proxy_ml_begin(MemoryListener *listener)
+{
+RemoteMemSync *sync = container_of(listener, RemoteMemSync, listener);
+int mrs;
+
+for (mrs = 0; mrs < sync->n_mr_sections; mrs++) {
+memory_region_unref(sync->mr_sections[mrs].mr);
+}
+
+g_free(sync->mr_sections);
+sync->mr_sections = NULL;
+sync->n_mr_sections = 0;
+}
+
+static int get_fd_from_hostaddr(uint64_t host, ram_addr_t *offset)
+{
+MemoryRegion *mr;
+ram_addr_t off;
+
+mr = memory_region_from_host((void *)(uintptr_t)host, &off);


Do you need to just check we found an 'mr' ?


OK, we'll add this check.




+if (offset) {
+*offset = off;
+}
+
+return memory_region_get_fd(mr);
+}
+
+static bool proxy_mrs_can_merge(uint64_t host, uint64_t prev_host, size_t size)
+{
+bool merge;
+int fd1, fd2;
+
+fd1 = get_fd_from_hostaddr(host, NULL);
+
+fd2 = get_fd_from_hostaddr(prev_host, NULL);
+
+merge = (fd1 == fd2);
+
+merge &= ((prev_host + size) == host);


It's interesting; I think the vhost code checks that the two mr's are
the same where you are checking for the same underlying fd - but I think
that's OK.
(I wonder if we need to check offset's within the fd's match up when
they're merged - since you added that offset feature in an earlier
patch?
That would also need checking in vhost_region_add_section)


If the fds are the same, and the subsequent check ((prev_host + size) ==
host) passes, then I believe the offsets would match as well.




+return merge;
+}
+
+static void proxy_ml_region_addnop(MemoryListener *listener,
+   MemoryRegionSection *section)
+{
+RemoteMemSync *sync = container_of(listener, RemoteMemSync, listener);
+bool need_add = true;
+uint64_t mrs_size, mrs_gpa, mrs_page;
+uintptr_t mrs_host;
+RAMBlock *mrs_rb;
+MemoryRegionSection *prev_sec;
+
+if (!(memory_region_is_ram(section->mr) &&
+  !memory_region_is_rom(section->mr))) {
+return;
+}
+
+mrs_rb = section->mr->ram_block;
+mrs_page = (uint64_t)qemu_ram_pagesize(mrs_rb);
+mrs_size = int128_get64(section->size);
+mrs_gpa = section->offset_within_address_space;
+mrs_host = (uintptr_t)memory_region_get_ram_ptr(section->mr) +
+   section->offset_within_region;
+
+if (get_fd_from_hostaddr(mrs_host

Re: [PATCH 4/4] qapi: Brush off some (py)lint

[John Snow]

On 3/4/20 3:01 AM, Markus Armbruster wrote:
> John Snow  writes:
> 
>> On 2/27/20 9:45 AM, Markus Armbruster wrote:
>>> Signed-off-by: Markus Armbruster 
>>
>> I wrote some pylint cleanup for iotests recently, too. Are you targeting
>> a subset of pylint errors to clean here?
>>
>> (Do any files pass 100%?)
> 
> Surely you're joking, Mr. Snow!
> 
> I'm chipping away at pylint's gripes.  I ran it with the following
> messages disabled:
> 
> bad-whitespace,
> fixme,
> invalid-name,
> missing-docstring,
> too-few-public-methods,
> too-many-arguments,
> too-many-branches,
> too-many-instance-attributes,
> too-many-lines,
> too-many-locals,
> too-many-statements,
> unused-argument,
> unused-wildcard-import,
> 
> These are not all obviously useless.  They're just not what I want to
> focus on right now.
> 

Yes, understood - so my approach is disable what I don't intend to fix,
commit the pylintrc to prevent backslide, and move on.

I think we have a difference in what a pylintrc means to us (the goal
vs. the current status.)

I didn't mean "100% without caveats", just "100% in some subset of checks".

(I assume the answer is still no.)

> Remaining:
> 
> 1 x C0330: Wrong continued indentation (remove 19 spaces).
> 
> Accident, will fix in v2.
> 
> 8 x R0201: Method could be a function (no-self-use)
> 
> Yes, but the override in a sub-class does use self.
> 
> 2 x W0212: Access to a protected member _body of a client class 
> (protected-access)
> 
> Needs cleanup, but not now.
> 
> 6 x W0401: Wildcard import qapi.common (wildcard-import)
> 
> Not sure I care.  I'd prefer not to have more wildcard imports,
> though.
> 
> 2 x W0603: Using the global statement (global-statement)
> 
> Cleanup is non-trivial.  Not now.
> 
> I also ran pycodestyle-3:
> 
> 1 x E127 continuation line over-indented for visual indent
> 
> Same as pylint's C0330, will fix in v2.
> 
> 3 x E261 at least two spaces before inline comment
> 
> I blame Emacs.  Left for another day.
> 
> 8 x E501 line too long
> 
> Left for another day.
> 
> 1 x E713 test for membership should be 'not in'
> 
> I missed that one, will fix in v2.
> 
>> Consider checking in a pylintrc file that lets others run the same
>> subset of pylint tests as you are doing so that we can prevent future
>> regressions.
> 
> Working towards it, slowly.
> 
>> Take a peek at [PATCH v6 0/9] iotests: use python logging​
>>
>> Thanks for this series. I had a very similar series sitting waiting to
>> go out, but this goes further in a few places.
> 
> Thanks!
> 

Re: [PATCH v2 0/2] misc: Replace zero-length arrays with flexible array member

[John Snow]

On 3/4/20 10:38 AM, Philippe Mathieu-Daudé wrote:
> v2:
> - do not modify qed.h (structure with single member)
> - based on hw/scsi/spapr_vscsi fix series:
>   20200304153311.22959-1-philmd@redhat.com">https://mid.mail-archive.com/20200304153311.22959-1-philmd@redhat.com
> 
> This is a tree-wide cleanup inspired by a Linux kernel commit
> (from Gustavo A. R. Silva).
> 
> --v-- description start --v--
> 
>   The current codebase makes use of the zero-length array language
>   extension to the C90 standard, but the preferred mechanism to
>   declare variable-length types such as these ones is a flexible
>   array member [1], introduced in C99:
> 
>   struct foo {
>   int stuff;
>   struct boo array[];
>   };
> 
>   By making use of the mechanism above, we will get a compiler
>   warning in case the flexible array does not occur last in the
>   structure, which will help us prevent some kind of undefined
>   behavior bugs from being unadvertenly introduced [2] to the
>   Linux codebase from now on.
> 
> --^-- description end --^--
> 
> Do the similar housekeeping in the QEMU codebase (which uses
> C99 since commit 7be41675f7cb).
> 
> The first patch is done with the help of a coccinelle semantic
> patch. However Coccinelle does not recognize:
> 
>   struct foo {
>   int stuff;
>   struct boo array[];
>   } QEMU_PACKED;
> 
> but does recognize:
> 
>   struct QEMU_PACKED foo {
>   int stuff;
>   struct boo array[];
>   };
> 
> I'm not sure why, neither it is worth refactoring all QEMU
> structures to use the attributes before the structure name,
> so I did the 2nd patch manually.
> 
> Anyway this is annoying, because many structures are not handled
> by coccinelle. Maybe this needs to be reported to upstream
> coccinelle?
> 
> I used spatch 1.0.8 with:
> 
>   -I include --include-headers \
>   --macro-file scripts/cocci-macro-file.h \
>   --keep-comments --indent 4
> 
> Regards,
> 
> Phil.
> 
> Based-on: <20200304153311.22959-1-phi...@redhat.com>
> Supersedes: <20200304005105.27454-1-phi...@redhat.com>
> 
> Philippe Mathieu-Daudé (2):
>   misc: Replace zero-length arrays with flexible array member
> (automatic)
>   misc: Replace zero-length arrays with flexible array member (manual)
> 
>  docs/interop/vhost-user.rst   |  4 ++--
>  bsd-user/qemu.h   |  2 +-
>  contrib/libvhost-user/libvhost-user.h |  2 +-
>  hw/m68k/bootinfo.h|  2 +-
>  hw/scsi/srp.h |  6 +++---
>  hw/xen/xen_pt.h   |  2 +-
>  include/hw/acpi/acpi-defs.h   | 16 
>  include/hw/arm/smmu-common.h  |  2 +-
>  include/hw/boards.h   |  2 +-
>  include/hw/i386/intel_iommu.h |  3 ++-
>  include/hw/s390x/event-facility.h |  2 +-
>  include/hw/s390x/sclp.h   |  8 
>  include/hw/virtio/virtio-iommu.h  |  2 +-
>  include/sysemu/cryptodev.h|  2 +-
>  include/tcg/tcg.h |  2 +-
>  pc-bios/s390-ccw/bootmap.h|  2 +-
>  pc-bios/s390-ccw/sclp.h   |  2 +-
>  tests/qtest/libqos/ahci.h |  2 +-
>  block/linux-aio.c |  2 +-
>  block/vmdk.c  |  2 +-
>  hw/acpi/nvdimm.c  |  6 +++---
>  hw/char/sclpconsole-lm.c  |  2 +-
>  hw/char/sclpconsole.c |  2 +-
>  hw/dma/soc_dma.c  |  2 +-
>  hw/i386/x86.c |  2 +-
>  hw/misc/omap_l4.c |  2 +-
>  hw/nvram/eeprom93xx.c |  2 +-
>  hw/rdma/vmw/pvrdma_qp_ops.c   |  4 ++--
>  hw/s390x/virtio-ccw.c |  2 +-
>  hw/usb/dev-network.c  |  2 +-
>  hw/usb/dev-smartcard-reader.c |  4 ++--
>  hw/virtio/virtio.c|  4 ++--
>  net/queue.c   |  2 +-
>  target/s390x/ioinst.c |  2 +-
>  34 files changed, 53 insertions(+), 52 deletions(-)
> 

I'll admit I did not manually verify ALL of this, but instead trust that:

1. The conversion is correct, and this is a desirable change to make.
2. Sample conversions I looked at appear correct.
3. It builds.
4. It passes tests.

So:

Acked-by: John Snow 

Re: [PATCH v6 9/9] iotests: add pylintrc file

[John Snow]

On 3/4/20 2:22 AM, Markus Armbruster wrote:
> John Snow  writes:
> 
>> Repeatable results. run `pylint iotests.py` and you should get a pass.
> 
> Start your sentences with a capital letter, please.
> 

The German complains about the capitalization, but not the sentence
fragment.

>>
>> Signed-off-by: John Snow 
>> ---
>>  tests/qemu-iotests/pylintrc | 20 
>>  1 file changed, 20 insertions(+)
>>  create mode 100644 tests/qemu-iotests/pylintrc
>>
>> diff --git a/tests/qemu-iotests/pylintrc b/tests/qemu-iotests/pylintrc
>> new file mode 100644
>> index 00..feed506f75
>> --- /dev/null
>> +++ b/tests/qemu-iotests/pylintrc
>> @@ -0,0 +1,20 @@
>> +[MESSAGES CONTROL]
>> +
>> +# Disable the message, report, category or checker with the given id(s). You
>> +# can either give multiple identifiers separated by comma (,) or put this
>> +# option multiple times (only on the command line, not in the configuration
>> +# file where it should appear only once). You can also use "--disable=all" 
>> to
>> +# disable everything first and then reenable specific checks. For example, 
>> if
>> +# you want to run only the similarities checker, you can use "--disable=all
>> +# --enable=similarities". If you want to run only the classes checker, but 
>> have
>> +# no Warning level messages displayed, use "--disable=all --enable=classes
>> +# --disable=W".
>> +disable=invalid-name,
>> +missing-docstring,
>> +line-too-long,
>> +too-many-lines,
>> +too-few-public-methods,
>> +too-many-arguments,
>> +too-many-locals,
>> +too-many-branches,
>> +too-many-public-methods,
>> \ No newline at end of file
> 
> Add the newline, please.
> 
> German pejorative for the too-many- and too-few- warnings: "Müsli".
> Implies it's for muesli-knitters / granola-crunchers indulging their
> orthorexia.
> 

They are useful at times as they can suggest when you are going a bit
overboard on "organically grown" design. For cleaning an existing
codebase, it's more of a hindrance to the immediate goal of establishing
a baseline.

(*cough* I try to adhere to them in my own freshly written code, and
disable per-line when I've decided to veto the suggestion. Not
appropriate for a codebase like ours. As Max reminds me, it's just tests
-- don't make them too clever or pretty.)

Regardless. It's not appropriate here and now.

> missing-docstring is not advisable for libraries.  Feels okay here.
> 

Ideally we do start using them, but it's out of scope here. Since I did
some cleanup, I wanted to establish the baseline of what I adhered to.

*not* suggest that it's the destination state.

Adding proper docstrings should be done during mypy conversion once the
types are determined, understood, and enforced. Not before then.

> line-too-long might be worth cleaning up.  How many of them do we have
> now?
> 

Five in iotests.py using the default length of 100. 15 if I limit to 80.

If we agree that 100 is okay, I can tackle this in an amendment patch.
If 80 is okay, I'm going to put it off as one coat of paint too many.

(I will try to clean up the 100+ lines for my next version. I am
hesitant to make a deeper cut because I have the feeling it's the type
of series that will incur a lot of nitpicks on indent style.)

--js

Re: [PATCH v6 17/18] docs: Add protvirt docs

[David Hildenbrand]

On 04.03.20 12:42, Janosch Frank wrote:
> Lets add some documentation for the Protected VM functionality.
> 
> Signed-off-by: Janosch Frank 
> ---
>  docs/system/index.rst|  1 +
>  docs/system/protvirt.rst | 57 
>  2 files changed, 58 insertions(+)
>  create mode 100644 docs/system/protvirt.rst
> 
> diff --git a/docs/system/index.rst b/docs/system/index.rst
> index 1a4b2c82ac..d2dc63b973 100644
> --- a/docs/system/index.rst
> +++ b/docs/system/index.rst
> @@ -16,3 +16,4 @@ Contents:
>  
> qemu-block-drivers
> vfio-ap
> +   protvirt
> diff --git a/docs/system/protvirt.rst b/docs/system/protvirt.rst
> new file mode 100644
> index 00..a1902cc47c
> --- /dev/null
> +++ b/docs/system/protvirt.rst
> @@ -0,0 +1,57 @@
> +Protected Virtualization on s390x
> +=
> +
> +The memory and most of the register contents of Protected Virtual

s/register contents/registers/

> +Machines (PVMs) are inaccessible to the hypervisor, effectively

s/inaccessible/encrypted or even inaccessible/ ?

> +prohibiting VM introspection when the VM is running. At rest, PVMs are
> +encrypted and can only be decrypted by the firmware of specific IBM Z
> +machines.

maybe "(a.k.a. the Ultravisor)"

> +
> +
> +Prerequisites
> +-
> +
> +To run PVMs, you need to have a machine with the Protected

"a machine with the Protected Virtualization feature is required"

> +Virtualization feature, which is indicated by the Ultravisor Call
> +facility (stfle bit 158). This is a KVM only feature, therefore you

", therefore, "

I don't understand the "KVM only" feature part. Just say that an updated
KVM + right HW is required and how it is to be updated.

> +need a KVM which is able to support PVMs and activate the Ultravisor

"a KVM version"

> +initialization by setting `prot_virt=1` on the kernel command line.
> +
> +If those requirements are met, the capability `KVM_CAP_S390_PROTECTED`
> +will indicate that KVM can support PVMs on that LPAR.
> +
> +
> +QEMU Settings
> +-
> +
> +To indicate to the VM that it can move into protected mode, the

s/move/transition/ ?

> +`Unpack facility` (stfle bit 161) needs to be part of the cpu model of
> +the VM.

Maybe mention the CPU feature name here.

> +
> +All I/O devices need to use the IOMMU.
> +Passthrough (vfio) devices are currently not supported.
> +
> +Host huge page backings are not supported. The guest however can use

"However, the guest can ..."

> +huge pages as indicated by its facilities.
> +
> +
> +Boot Process
> +
> +
> +A secure guest image can be both booted from disk and using the QEMU

"either be loaded from disk or supplied on the QEMU command line" ?

> +command line. Booting from disk is done by the unmodified s390-ccw
> +BIOS. I.e., the bootmap is interpreted and a number of components is

"interpreted, multiple components are"

> +read into memory and control is transferred to one of the components
> +(zipl stage3), which does some fixups and then transfers control to
> +some program residing in guest memory, which is normally the OS

to many ", which". Better split that up for readability.

> +kernel. The secure image has another component prepended (stage3a)
> +which uses the new diag308 subcodes 8 and 10 to trigger the transition
> +into secure mode.
> +
> +Booting from the command line requires that the file passed

"from the image supplied on the QEMU command line" ?

> +via -kernel has the same memory layout as would result from the disk
> +boot. This memory layout includes the encrypted components (kernel,
> +initrd, cmdline), the stage3a loader and metadata. In case this boot
> +method is used, the command line options -initrd and -cmdline are
> +ineffective.  The preparation of secure guest image is done by a

s/of secure/of a PMV image/

> +program (name tbd) of the s390-tools package.
> 


General: secure guest -> PMV

-- 
Thanks,

David / dhildenb

Re: [PATCH] configure: change a typo in zstd config

[Philippe Mathieu-Daudé]

On 3/3/20 1:56 PM, Juan Quintela wrote:

Denis Plotnikov  wrote:

Package manager --exist flag is used instead of --exists.
Fix it.

Signed-off-by: Denis Plotnikov 


Reviewed-by: Juan Quintela 

queued.


It doesn't seem that trivial:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg684754.html



thanks.


---
  configure | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure b/configure
index 7b373bc0bb..caa65f5883 100755
--- a/configure
+++ b/configure
@@ -2464,7 +2464,7 @@ fi
  # zstd check
  
  if test "$zstd" != "no" ; then

-if $pkg_config --exist libzstd ; then
+if $pkg_config --exists libzstd ; then
  zstd_cflags="$($pkg_config --cflags libzstd)"
  zstd_libs="$($pkg_config --libs libzstd)"
  LIBS="$zstd_libs $LIBS"

Re: [PATCH] configure: Fix pkg_config --exists parameter

[Philippe Mathieu-Daudé]

On 3/4/20 6:42 PM, Dr. David Alan Gilbert wrote:

* Liran Alon (liran.a...@oracle.com) wrote:

pkg_config parameter should be "--exists", not "--exist".
This is probably a typo.

Fixes: 3a67848134d0 ("configure: Enable test and libs for zstd")
Reviewed-by: Bhavesh Davda 
Signed-off-by: Liran Alon 


That's the same as 20200303124925.28079-1-dplotni...@virtuozzo.com from
Denis yesterday.


And Alex 2 days ago:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg684754.html



Dave


---
  configure | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure b/configure
index 50c4d96e2a8c..280f3654f786 100755
--- a/configure
+++ b/configure
@@ -2464,7 +2464,7 @@ fi
  # zstd check
  
  if test "$zstd" != "no" ; then

-if $pkg_config --exist libzstd ; then
+if $pkg_config --exists libzstd ; then
  zstd_cflags="$($pkg_config --cflags libzstd)"
  zstd_libs="$($pkg_config --libs libzstd)"
  LIBS="$zstd_libs $LIBS"
--
2.20.1


--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK