Re: [Qemu-devel] ppc: regression in 2.7.0-rc3 on rfi/rfid/hrfi instructions

[Benjamin Herrenschmidt]

On Thu, 2016-08-25 at 11:31 +1000, Benjamin Herrenschmidt wrote:
> 
> Interesting, the problem is only with 603, not 604. I think I might
> have broken something with the 603 SW TLB loading stuff which is
> a bit weird, I'll have a look. Thanks.

This fixes it to some extent... It gets to "press enter to activate
this console" but from there things don't work properly unless I
emulate a 604. I suspect there are still issues with the 603 SW
TLB mode.

--- a/target-ppc/excp_helper.c
+++ b/target-ppc/excp_helper.c
@@ -959,8 +959,13 @@ static inline void do_rfi(CPUPPCState *env, target_ulong 
nip, target_ulong msr)
 {
 CPUState *cs = CPU(ppc_env_get_cpu(env));
 
-/* MSR:POW cannot be set by any form of rfi */
-msr &= ~(1ULL << MSR_POW);
+/* These bits cannot be set by RFI on non-BookE systems and so must
+ * be filtered out. 6xx and 7xxx with SW TLB management will put
+ * TLB related junk in there among other things.
+ */
+if (env->excp_model &= POWERPC_EXCP_BOOKE) {
+msr &= ~(target_ulong)0xf;
+}
 
 #if defined(TARGET_PPC64)
 /* Switching to 32-bit ? Crop the nip */
@@ -990,7 +995,6 @@ void helper_rfi(CPUPPCState *env)
 do_rfi(env, env->spr[SPR_SRR0], env->spr[SPR_SRR1] & 0xul);
 }
 
-#define MSR_BOOK3S_MASK
 #if defined(TARGET_PPC64)
 void helper_rfid(CPUPPCState *env)
 {

Re: [Qemu-devel] ppc: regression in 2.7.0-rc3 on rfi/rfid/hrfi instructions

[Benjamin Herrenschmidt]

On Thu, 2016-08-25 at 11:31 +1000, Benjamin Herrenschmidt wrote:
> On Wed, 2016-08-24 at 20:39 +0200, Hervé Poussineau wrote:
> > 
> > Hi,
> > 
> > Following patch regresses Linux boot on PReP machine:
> > 
> > commit a2e71b28e832346409efc795ecd1f0a2bcb705a3
> > Author: Benjamin Herrenschmidt 
> > Date:   Tue Jun 21 23:48:46 2016 +0200
> > 
> >  ppc: Fix rfi/rfid/hrfi/... emulation
> > 
> >  This reworks emulation of the various "rfi" variants. I
> > removed
> >  some masking bits that I couldn't make sense of, the only bit
> > that
> 
> Interesting, the problem is only with 603, not 604. I think I might
> have broken something with the 603 SW TLB loading stuff which is
> a bit weird, I'll have a look. Thanks.

Actually it's trickier than that...

PReP seems to default to 602 ! Linux never supported the 602 so that's
really strange.

If I use 603 instead, I quickly hit a bad SPR, looks like we are trying
to hit the DABR which is ... odd. Probably getting into xmon but your
kernel is trying to get to the video rather than the serial port so I
can't see it.

I need that kernel source, we dropped PReP support from Linux ages
ago...

Cheers,
Ben.

Re: [Qemu-devel] [PATCH V12 00/10] Introduce COLO-compare and filter-rewriter

[Jason Wang]

On 2016年08月25日 11:44, Zhang Chen wrote:

Hi~~ Jason.

If you have time, can you give me some feedback for this series?


Thanks

Zhang Chen


Yes, a little busy this week, will do it next week.

Thanks




On 08/17/2016 04:10 PM, Zhang Chen wrote:

COLO-compare is a part of COLO project. It is used
to compare the network package to help COLO decide
whether to do checkpoint.

Filter-rewriter is a part of COLO project too.
It will rewrite some of secondary packet to make
secondary guest's connection established successfully.
In this module we will rewrite tcp packet's ack to the secondary
from primary,and rewrite tcp packet's seq to the primary from
secondary.

The full version in this github:
https://github.com/zhangckid/qemu/tree/colo-v2.7-proxy-mode-compare-and-rewriter-aug16 




v12:
   - add qemu-char: Add qemu_chr_add_handlers_full() for GMaincontext
 to this series as the first patch.
   - update COLO net ascii figure.
   - add chardev socket check.
   - fix some typo.
   - add some comments.
   - rename net/colo-base.c to net/colo.c
   - rename network/transport_layer to network/transport_header.
   - move the job that clear coon_list when hashtable_size oversize
 to connection_get.
   - reuse connection_destroy() do colo_rm_connection().
   - fix pkt mem leak in colo_compare_connection().
 (result be released in g_queue_remove(), so it were not leak)
   - rename thread_name "compare" to "colo-compare".
   - change icmp compare to memcmp().

v11:
   - Make patch 5 to a independent patch series.
 [PATCH V3] qemu-char: Add qemu_chr_add_handlers_full() for 
GMaincontext

   - For Jason's comments, merge filter-rewriter to this series.
 (patch 7,8,9)
   - Add reverse_connection_key()
   - remove conn_list in filter-rewriter
   - remove unprocessed_connections
   - add some comments

v10:
   - fix typo
   - Should we make patch 5 independent with this series?
 This patch just add a API for qemu-char.

v9:
  p5:
   - use chr_update_read_handler_full() replace
 the chr_update_read_handler()
   - use io_watch_poll_prepare_full() replace
 the io_watch_poll_prepare()
   - use io_watch_poll_funcs_full replace
 the io_watch_poll_funcs
   - avoid code duplication

v8:
  p5:
   - add new patch:
 qemu-char: Add qemu_chr_add_handlers_full() for GMaincontext

v7:
  p5:
- add [PATCH]qemu-char: Fix context for g_source_attach()
  in this patch series.

v6:
  p6:
- add more commit log.
- fix icmp comparison to compare all packet.

  p5:
- add more cpmments in commit log.
- change REGULAR_CHECK_MS to REGULAR_PACKET_CHECK_MS
- make check old packet independent to compare thread
- remove thread_status

  p4:
- change this patch only about
  Connection and ConnectionKey.
- add some comments in commit log.
- remove mode in fill_connection_key().
- fix some comments and bug.
- move colo_conn_state to patch of
  "work with colo-frame"
- remove conn_list_lock.
- add MAX_QUEUE_SIZE, if primary_list or
  secondary_list biger than MAX_QUEUE_SIZE
  we will drop packet.

  p3:
- add new independent kernel jhash patch.

  p2:
- add new independent colo-base patch.

  p1:
- add a ascii figure and some comments to explain it
- move trace.h to p2
- move QTAILQ_HEAD(, CompareState) net_compares to
  patch of "work with colo-frame"
- add some comments in qemu-option.hx


v5:
  p3:
 - comments from Jason
   we poll and handle chardev in comapre thread,
   Through this way, there's no need for extra
   synchronization with main loop
   this depend on another patch:
   qemu-char: Fix context for g_source_attach()
 - remove QemuEvent
  p2:
 - remove conn->list_lock
  p1:
 - move compare_pri/sec_chr_in to p3
 - move compare_chr_send to p2

v4:
  p4:
 - add some comments
 - fix some trace-events
 - fix tcp compare error
  p3:
 - add rcu_read_lock().
 - fix trace name
 - fix jason's other comments
 - rebase some Dave's branch function
  p2:
 - colo_compare_connection() change g_queue_push_head() to
 - g_queue_push_tail() match to sorted order.
 - remove pkt->s
 - move data structure to colo-base.h
 - add colo-base.c reuse codes for filter-rewriter
 - add some filter-rewriter needs struct
 - depends on previous SocketReadState patch
  p1:
 - except move qemu_chr_add_handlers()
   to colo thread
 - remove class_finalize
 - remove secondary arp codes
 - depends on previous SocketReadState patch

v3:
   - rebase colo-compare to colo-frame v2.7
   - fix most of Dave's comments
 (except RCU)
   - add TCP,UDP,ICMP and other packet comparison
   - add trace-event
   - add some comments
   - other bug fix
   - add RFC index
   - add usage in patch 1/4

v2:
   - add jhash.h

v1:
   - initial patch


Zhang Chen (10):
   qemu-char: Add qemu_chr_add_handlers_full() for GMaincontext
   

[Qemu-devel] [PATCH v7 3/4] vfio iommu: Add support for mediated devices

[Kirti Wankhede]

VFIO IOMMU drivers are designed for the devices which are IOMMU capable.
Mediated device only uses IOMMU APIs, the underlying hardware can be
managed by an IOMMU domain.

Aim of this change is:
- To use most of the code of TYPE1 IOMMU driver for mediated devices
- To support direct assigned device and mediated device in single module

Added two new callback functions to struct vfio_iommu_driver_ops. Backend
IOMMU module that supports pining and unpinning pages for mdev devices
should provide these functions.
Added APIs for pining and unpining pages to VFIO module. These calls back
into backend iommu module to actually pin and unpin pages.

This change adds pin and unpin support for mediated device to TYPE1 IOMMU
backend module. More details:
- When iommu_group of mediated devices is attached, task structure is
  cached which is used later to pin pages and page accounting.
- It keeps track of pinned pages for mediated domain. This data is used to
  verify unpinning request and to unpin remaining pages while detaching, if
  there are any.
- Used existing mechanism for page accounting. If iommu capable domain
  exist in the container then all pages are already pinned and accounted.
  Accouting for mdev device is only done if there is no iommu capable
  domain in the container.

Tested by assigning below combinations of devices to a single VM:
- GPU pass through only
- vGPU device only
- One GPU pass through and one vGPU device
- two GPU pass through

Signed-off-by: Kirti Wankhede 
Signed-off-by: Neo Jia 
Change-Id: I295d6f0f2e0579b8d9882bfd8fd5a4194b97bd9a
Reviewed-on: http://git-master/r/1175707
Reviewed-by: Automatic_Commit_Validation_User
---
 drivers/vfio/vfio.c | 117 ++
 drivers/vfio/vfio_iommu_type1.c | 498 
 include/linux/vfio.h|  13 +-
 3 files changed, 580 insertions(+), 48 deletions(-)

diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index 6fd6fa5469de..e3e342861e04 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -1782,6 +1782,123 @@ void vfio_info_cap_shift(struct vfio_info_cap *caps, 
size_t offset)
 }
 EXPORT_SYMBOL_GPL(vfio_info_cap_shift);
 
+static struct vfio_group *vfio_group_from_dev(struct device *dev)
+{
+   struct vfio_device *device;
+   struct vfio_group *group;
+   int ret;
+
+   device = vfio_device_get_from_dev(dev);
+   if (!device)
+   return ERR_PTR(-EINVAL);
+
+   group = device->group;
+   if (!atomic_inc_not_zero(>container_users)) {
+   ret = -EINVAL;
+   goto err_ret;
+   }
+
+   if (group->noiommu) {
+   atomic_dec(>container_users);
+   ret = -EPERM;
+   goto err_ret;
+   }
+
+   if (!group->container->iommu_driver ||
+   !vfio_group_viable(group)) {
+   atomic_dec(>container_users);
+   ret = -EINVAL;
+   goto err_ret;
+   }
+
+   vfio_device_put(device);
+   return group;
+
+err_ret:
+   vfio_device_put(device);
+   return ERR_PTR(ret);
+}
+
+/*
+ * Pin a set of guest PFNs and return their associated host PFNs for local
+ * domain only.
+ * @dev [in] : device
+ * @user_pfn [in]: array of user/guest PFNs
+ * @npage [in]: count of array elements
+ * @prot [in] : protection flags
+ * @phys_pfn[out] : array of host PFNs
+ */
+long vfio_pin_pages(struct device *dev, unsigned long *user_pfn,
+   long npage, int prot, unsigned long *phys_pfn)
+{
+   struct vfio_container *container;
+   struct vfio_group *group;
+   struct vfio_iommu_driver *driver;
+   ssize_t ret = -EINVAL;
+
+   if (!dev || !user_pfn || !phys_pfn)
+   return -EINVAL;
+
+   group = vfio_group_from_dev(dev);
+   if (IS_ERR(group))
+   return PTR_ERR(group);
+
+   container = group->container;
+   if (IS_ERR(container))
+   return PTR_ERR(container);
+
+   down_read(>group_lock);
+
+   driver = container->iommu_driver;
+   if (likely(driver && driver->ops->pin_pages))
+   ret = driver->ops->pin_pages(container->iommu_data, user_pfn,
+npage, prot, phys_pfn);
+
+   up_read(>group_lock);
+   vfio_group_try_dissolve_container(group);
+
+   return ret;
+
+}
+EXPORT_SYMBOL(vfio_pin_pages);
+
+/*
+ * Unpin set of host PFNs for local domain only.
+ * @dev [in] : device
+ * @pfn [in] : array of host PFNs to be unpinned.
+ * @npage [in] :count of elements in array, that is number of pages.
+ */
+long vfio_unpin_pages(struct device *dev, unsigned long *pfn, long npage)
+{
+   struct vfio_container *container;
+   struct vfio_group *group;
+   struct vfio_iommu_driver *driver;
+   ssize_t ret = -EINVAL;
+
+   if (!dev || !pfn)
+   return -EINVAL;
+
+   group = vfio_group_from_dev(dev);
+   if (IS_ERR(group))
+   

[Qemu-devel] [PATCH v7 4/4] docs: Add Documentation for Mediated devices

[Kirti Wankhede]

Add file Documentation/vfio-mediated-device.txt that include details of
mediated device framework.

Signed-off-by: Kirti Wankhede 
Signed-off-by: Neo Jia 
Change-Id: I137dd646442936090d92008b115908b7b2c7bc5d
Reviewed-on: http://git-master/r/1182512
Reviewed-by: Automatic_Commit_Validation_User
---
 Documentation/vfio-mediated-device.txt | 203 +
 1 file changed, 203 insertions(+)
 create mode 100644 Documentation/vfio-mediated-device.txt

diff --git a/Documentation/vfio-mediated-device.txt 
b/Documentation/vfio-mediated-device.txt
new file mode 100644
index ..237d8eb630b7
--- /dev/null
+++ b/Documentation/vfio-mediated-device.txt
@@ -0,0 +1,203 @@
+VFIO Mediated devices [1]
+---
+
+There are more and more use cases/demands to virtualize the DMA devices which
+doesn't have SR_IOV capability built-in. To do this, drivers of different
+devices had to develop their own management interface and set of APIs and then
+integrate it to user space software. We've identified common requirements and
+unified management interface for such devices to make user space software
+integration easier.
+
+The VFIO driver framework provides unified APIs for direct device access. It is
+an IOMMU/device agnostic framework for exposing direct device access to
+user space, in a secure, IOMMU protected environment. This framework is
+used for multiple devices like GPUs, network adapters and compute accelerators.
+With direct device access, virtual machines or user space applications have
+direct access of physical device. This framework is reused for mediated 
devices.
+
+Mediated core driver provides a common interface for mediated device management
+that can be used by drivers of different devices. This module provides a 
generic
+interface to create/destroy mediated device, add/remove it to mediated bus
+driver, add/remove device to IOMMU group. It also provides an interface to
+register bus driver, for example, Mediated VFIO mdev driver is designed for
+mediated devices and supports VFIO APIs. Mediated bus driver add/delete 
mediated
+device to VFIO Group.
+
+Below is the high Level block diagram, with NVIDIA, Intel and IBM devices
+as example, since these are the devices which are going to actively use
+this module as of now.
+
+ +---+
+ |   |
+ | +---+ |  mdev_register_driver() +--+
+ | |   | +<+  |
+ | |  mdev | | |  |
+ | |  bus  | +>+ vfio_mdev.ko |<-> VFIO user
+ | |  driver   | | probe()/remove()|  |APIs
+ | |   | | +--+
+ | +---+ |
+ |   |
+ |  MDEV CORE|
+ |   MODULE  |
+ |   mdev.ko |
+ | +---+ |  mdev_register_device() +--+
+ | |   | +<+  |
+ | |   | | |  nvidia.ko   |<-> physical
+ | |   | +>+  |device
+ | |   | |callbacks+--+
+ | | Physical  | |
+ | |  device   | |  mdev_register_device() +--+
+ | | interface | |<+  |
+ | |   | | |  i915.ko |<-> physical
+ | |   | +>+  |device
+ | |   | |callbacks+--+
+ | |   | |
+ | |   | |  mdev_register_device() +--+
+ | |   | +<+  |
+ | |   | | | ccw_device.ko|<-> physical
+ | |   | +>+  |device
+ | |   | |callbacks+--+
+ | +---+ |
+ +---+
+
+
+Registration Interfaces
+---
+
+Mediated core driver provides two types of registration interfaces:
+
+1. Registration interface for mediated bus driver:
+-
+ /*
+  * struct mdev_driver [2] - Mediated device's driver
+  * @name: driver name
+  * @probe: called when new device created
+  * @remove: called when device removed
+  * @driver: device driver structure
+  */
+ struct mdev_driver {
+const char *name;
+int  (*probe)  (struct device *dev);
+void (*remove) (struct device *dev);
+struct device_driverdriver;
+ };
+
+Mediated bus driver for mdev should use this interface to register and
+unregister with core driver respectively:
+
+extern int  

[Qemu-devel] [PATCH v7 2/4] vfio: VFIO driver for mediated devices

[Kirti Wankhede]

VFIO MDEV driver registers with MDEV core driver. MDEV core driver creates
mediated device and calls probe routine of MPCI VFIO driver. This driver
adds mediated device to VFIO core module.
Main aim of this module is to manage all VFIO APIs for each mediated
device. Those are:
- get VFIO device information about type of device, maximum number of
  regions and maximum number of interrupts supported.
- get region information from vendor driver.
- Get interrupt information and send interrupt configuration information to
  vendor driver.
- Device reset
- Trap and forward read/write for emulated regions.

Signed-off-by: Kirti Wankhede 
Signed-off-by: Neo Jia 
Change-Id: I583f4734752971d3d112324d69e2508c88f359ec
Reviewed-on: http://git-master/r/1175706
Reviewed-by: Automatic_Commit_Validation_User
---
 drivers/vfio/mdev/Kconfig   |   6 +
 drivers/vfio/mdev/Makefile  |   1 +
 drivers/vfio/mdev/vfio_mdev.c   | 467 
 drivers/vfio/pci/vfio_pci_private.h |   6 +-
 4 files changed, 477 insertions(+), 3 deletions(-)
 create mode 100644 drivers/vfio/mdev/vfio_mdev.c

diff --git a/drivers/vfio/mdev/Kconfig b/drivers/vfio/mdev/Kconfig
index a34fbc66f92f..703abd0a9bff 100644
--- a/drivers/vfio/mdev/Kconfig
+++ b/drivers/vfio/mdev/Kconfig
@@ -9,4 +9,10 @@ config VFIO_MDEV
 
 If you don't know what do here, say N.
 
+config VFIO_MDEV_DEVICE
+tristate "VFIO support for Mediated devices"
+depends on VFIO && VFIO_MDEV
+default n
+help
+VFIO based driver for mediated devices.
 
diff --git a/drivers/vfio/mdev/Makefile b/drivers/vfio/mdev/Makefile
index 56a75e689582..e5087ed83a34 100644
--- a/drivers/vfio/mdev/Makefile
+++ b/drivers/vfio/mdev/Makefile
@@ -2,4 +2,5 @@
 mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o
 
 obj-$(CONFIG_VFIO_MDEV) += mdev.o
+obj-$(CONFIG_VFIO_MDEV_DEVICE) += vfio_mdev.o
 
diff --git a/drivers/vfio/mdev/vfio_mdev.c b/drivers/vfio/mdev/vfio_mdev.c
new file mode 100644
index ..28f13aeaa46b
--- /dev/null
+++ b/drivers/vfio/mdev/vfio_mdev.c
@@ -0,0 +1,467 @@
+/*
+ * VFIO based Mediated PCI device driver
+ *
+ * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+ * Author: Neo Jia 
+ *Kirti Wankhede 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "mdev_private.h"
+
+#define DRIVER_VERSION  "0.1"
+#define DRIVER_AUTHOR   "NVIDIA Corporation"
+#define DRIVER_DESC "VFIO based Mediated PCI device driver"
+
+struct vfio_mdev {
+   struct iommu_group *group;
+   struct mdev_device *mdev;
+   struct vfio_device_info dev_info;
+};
+
+static int vfio_mdev_open(void *device_data)
+{
+   int ret = 0;
+
+   if (!try_module_get(THIS_MODULE))
+   return -ENODEV;
+
+   return ret;
+}
+
+static void vfio_mdev_close(void *device_data)
+{
+   module_put(THIS_MODULE);
+}
+
+static int sparse_mmap_cap(struct vfio_info_cap *caps, void *cap_type)
+{
+   struct vfio_info_cap_header *header;
+   struct vfio_region_info_cap_sparse_mmap *sparse_cap, *sparse = cap_type;
+   size_t size;
+
+   size = sizeof(*sparse) + sparse->nr_areas *  sizeof(*sparse->areas);
+   header = vfio_info_cap_add(caps, size,
+  VFIO_REGION_INFO_CAP_SPARSE_MMAP, 1);
+   if (IS_ERR(header))
+   return PTR_ERR(header);
+
+   sparse_cap = container_of(header,
+   struct vfio_region_info_cap_sparse_mmap, header);
+   sparse_cap->nr_areas = sparse->nr_areas;
+   memcpy(sparse_cap->areas, sparse->areas,
+  sparse->nr_areas * sizeof(*sparse->areas));
+   return 0;
+}
+
+static int region_type_cap(struct vfio_info_cap *caps, void *cap_type)
+{
+   struct vfio_info_cap_header *header;
+   struct vfio_region_info_cap_type *type_cap, *cap = cap_type;
+
+   header = vfio_info_cap_add(caps, sizeof(*cap),
+  VFIO_REGION_INFO_CAP_TYPE, 1);
+   if (IS_ERR(header))
+   return PTR_ERR(header);
+
+   type_cap = container_of(header, struct vfio_region_info_cap_type,
+   header);
+   type_cap->type = cap->type;
+   type_cap->subtype = cap->type;
+   return 0;
+}
+
+static long vfio_mdev_unlocked_ioctl(void *device_data,
+unsigned int cmd, unsigned long arg)
+{
+   int ret = 0;
+   struct vfio_mdev *vmdev = device_data;
+   struct parent_device *parent = vmdev->mdev->parent;
+   unsigned long minsz;
+
+   switch (cmd) {
+   case VFIO_DEVICE_GET_INFO:
+   {
+   struct vfio_device_info 

[Qemu-devel] [PATCH v7 1/4] vfio: Mediated device Core driver

[Kirti Wankhede]

Design for Mediated Device Driver:
Main purpose of this driver is to provide a common interface for mediated
device management that can be used by different drivers of different
devices.

This module provides a generic interface to create the device, add it to
mediated bus, add device to IOMMU group and then add it to vfio group.

Below is the high Level block diagram, with Nvidia, Intel and IBM devices
as example, since these are the devices which are going to actively use
this module as of now.

 +---+
 |   |
 | +---+ |  mdev_register_driver() +--+
 | |   | +<+ __init() |
 | |  mdev | | |  |
 | |  bus  | +>+  |<-> VFIO user
 | |  driver   | | probe()/remove()| vfio_mdev.ko |APIs
 | |   | | |  |
 | +---+ | +--+
 |   |
 |  MDEV CORE|
 |   MODULE  |
 |   mdev.ko |
 | +---+ |  mdev_register_device() +--+
 | |   | +<+  |
 | |   | | |  nvidia.ko   |<-> physical
 | |   | +>+  |device
 | |   | |callback +--+
 | | Physical  | |
 | |  device   | |  mdev_register_device() +--+
 | | interface | |<+  |
 | |   | | |  i915.ko |<-> physical
 | |   | +>+  |device
 | |   | |callback +--+
 | |   | |
 | |   | |  mdev_register_device() +--+
 | |   | +<+  |
 | |   | | | ccw_device.ko|<-> physical
 | |   | +>+  |device
 | |   | |callback +--+
 | +---+ |
 +---+

Core driver provides two types of registration interfaces:
1. Registration interface for mediated bus driver:

/**
  * struct mdev_driver - Mediated device's driver
  * @name: driver name
  * @probe: called when new device created
  * @remove:called when device removed
  * @driver:device driver structure
  *
  **/
struct mdev_driver {
 const char *name;
 int  (*probe)  (struct device *dev);
 void (*remove) (struct device *dev);
 struct device_driverdriver;
};

int  mdev_register_driver(struct mdev_driver *drv, struct module *owner);
void mdev_unregister_driver(struct mdev_driver *drv);

Mediated device's driver for mdev, vfio_mdev, uses this interface to
register with Core driver. vfio_mdev module adds mediated device to VFIO
group.

2. Physical device driver interface
This interface provides vendor driver the set APIs to manage physical
device related work in their own driver. APIs are :
- supported_config: provide supported configuration list by the vendor
driver
- create: to allocate basic resources in vendor driver for a mediated
  device.
- destroy: to free resources in vendor driver when mediated device is
   destroyed.
- reset: to free and reallocate resources in vendor driver during device
 reset.
- set_online_status: to change online status of mediated device.
- get_online_status: to get current (online/offline) status of mediated
 device.
- read : read emulation callback.
- write: write emulation callback.
- mmap: mmap emulation callback.
- get_irq_info: to retrieve information about mediated device's IRQ.
- set_irqs: send interrupt configuration information that VMM sets.
- get_device_info: to retrieve VFIO device related flags, number of regions
   and number of IRQs supported.
- get_region_info: to provide region size and its flags for the mediated
   device.

This registration interface should be used by vendor drivers to register
each physical device to mdev core driver.
Locks to serialize above callbacks are removed. If required, vendor driver
can have locks to serialize above APIs in their driver.

Signed-off-by: Kirti Wankhede 
Signed-off-by: Neo Jia 
Change-Id: I73a5084574270b14541c529461ea2f03c292d510
Reviewed-on: http://git-master/r/1175705
Reviewed-by: Automatic_Commit_Validation_User
---
 drivers/vfio/Kconfig |   1 +
 drivers/vfio/Makefile|   1 +
 drivers/vfio/mdev/Kconfig|  12 +
 drivers/vfio/mdev/Makefile   |   5 +
 drivers/vfio/mdev/mdev_core.c| 509 +++
 drivers/vfio/mdev/mdev_driver.c  | 131 ++
 drivers/vfio/mdev/mdev_private.h |  36 +++
 drivers/vfio/mdev/mdev_sysfs.c   | 240 ++
 include/linux/mdev.h | 212 
 9 files changed, 

[Qemu-devel] [PATCH v7 0/4] Add Mediated device support

[Kirti Wankhede]

This series adds Mediated device support to Linux host kernel. Purpose
of this series is to provide a common interface for mediated device
management that can be used by different devices. This series introduces
Mdev core module that create and manage mediated devices, VFIO based driver
for mediated devices that are created by mdev core module and update VFIO type1
IOMMU module to support pinning & unpinning for mediated devices.

This change uses uuid_le_to_bin() to parse UUID string and convert to bin.
This requires following commits from linux master branch:
* commit bc9dc9d5eec908806f1b15c9ec2253d44dcf7835 :
lib/uuid.c: use correct offset in uuid parser
* commit 2b1b0d66704a8cafe83be7114ec4c15ab3a314ad :
lib/uuid.c: introduce a few more generic helpers

Requires below commits from linux master branch for mmap region fault handler
that uses remap_pfn_range() to setup EPT properly.
* commit add6a0cd1c5ba51b201e1361b05a5df817083618
KVM: MMU: try to fix up page faults before giving up
* commit 92176a8ede577d0ff78ab3298e06701f67ad5f51 :
KVM: MMU: prepare to support mapping of VM_IO and VM_PFNMAP frames

What's new in v7?
- Removed 'instance' field from mdev_device structure.
- Replaced 'start' and 'stop' with 'online' interface which is per mdev device
  and takes 1 or 0 as argument.
- Removed validate_mmap_request() callback and added mmap() callback to
  parent_ops.
- With above change, removed mapping tracking logic and invalidation function
  from mdev core module. Vendor driver should have this in their module.
- Added get_device_info() callback so that vendor driver can define the device
  type, number of regions and number of IRQs supported.
- Added get_irq_info() callback for vendor driver to define the flags for irqs.
- Updated get_region_info() callback so that vendor driver can specify the
  capabilities.
- With all the above changes, VFIO driver is no more PCI driver. It can be used
  for any type of device. Hence, renamed vfio_mpci module to vfio_mdev and
  removed match() from driver interface structure.

Yet TODO:
  Need to handle the case in vfio_type1_iommu module that Alex pointed out in v6
review, that is, if the devices attached to the normal IOMMU API domain go away,
need to re-establish accounting for local domain.


Kirti Wankhede (4):
  vfio: Mediated device Core driver
  vfio: VFIO driver for mediated devices
  vfio iommu: Add support for mediated devices
  docs: Add Documentation for Mediated devices

 Documentation/vfio-mediated-device.txt | 203 +
 drivers/vfio/Kconfig   |   1 +
 drivers/vfio/Makefile  |   1 +
 drivers/vfio/mdev/Kconfig  |  18 ++
 drivers/vfio/mdev/Makefile |   6 +
 drivers/vfio/mdev/mdev_core.c  | 509 +
 drivers/vfio/mdev/mdev_driver.c| 131 +
 drivers/vfio/mdev/mdev_private.h   |  36 +++
 drivers/vfio/mdev/mdev_sysfs.c | 240 
 drivers/vfio/mdev/vfio_mdev.c  | 467 ++
 drivers/vfio/pci/vfio_pci_private.h|   6 +-
 drivers/vfio/vfio.c| 117 
 drivers/vfio/vfio_iommu_type1.c| 499 +---
 include/linux/mdev.h   | 212 ++
 include/linux/vfio.h   |  13 +-
 15 files changed, 2408 insertions(+), 51 deletions(-)
 create mode 100644 Documentation/vfio-mediated-device.txt
 create mode 100644 drivers/vfio/mdev/Kconfig
 create mode 100644 drivers/vfio/mdev/Makefile
 create mode 100644 drivers/vfio/mdev/mdev_core.c
 create mode 100644 drivers/vfio/mdev/mdev_driver.c
 create mode 100644 drivers/vfio/mdev/mdev_private.h
 create mode 100644 drivers/vfio/mdev/mdev_sysfs.c
 create mode 100644 drivers/vfio/mdev/vfio_mdev.c
 create mode 100644 include/linux/mdev.h

-- 
2.7.0

Re: [Qemu-devel] [PATCH V12 00/10] Introduce COLO-compare and filter-rewriter

[Zhang Chen]

Hi~~ Jason.

If you have time, can you give me some feedback for this series?


Thanks

Zhang Chen


On 08/17/2016 04:10 PM, Zhang Chen wrote:

COLO-compare is a part of COLO project. It is used
to compare the network package to help COLO decide
whether to do checkpoint.

Filter-rewriter is a part of COLO project too.
It will rewrite some of secondary packet to make
secondary guest's connection established successfully.
In this module we will rewrite tcp packet's ack to the secondary
from primary,and rewrite tcp packet's seq to the primary from
secondary.

The full version in this github:
https://github.com/zhangckid/qemu/tree/colo-v2.7-proxy-mode-compare-and-rewriter-aug16


v12:
   - add qemu-char: Add qemu_chr_add_handlers_full() for GMaincontext
 to this series as the first patch.
   - update COLO net ascii figure.
   - add chardev socket check.
   - fix some typo.
   - add some comments.
   - rename net/colo-base.c to net/colo.c
   - rename network/transport_layer to network/transport_header.
   - move the job that clear coon_list when hashtable_size oversize
 to connection_get.
   - reuse connection_destroy() do colo_rm_connection().
   - fix pkt mem leak in colo_compare_connection().
 (result be released in g_queue_remove(), so it were not leak)
   - rename thread_name "compare" to "colo-compare".
   - change icmp compare to memcmp().

v11:
   - Make patch 5 to a independent patch series.
 [PATCH V3] qemu-char: Add qemu_chr_add_handlers_full() for GMaincontext
   - For Jason's comments, merge filter-rewriter to this series.
 (patch 7,8,9)
   - Add reverse_connection_key()
   - remove conn_list in filter-rewriter
   - remove unprocessed_connections
   - add some comments

v10:
   - fix typo
   - Should we make patch 5 independent with this series?
 This patch just add a API for qemu-char.

v9:
  p5:
   - use chr_update_read_handler_full() replace
 the chr_update_read_handler()
   - use io_watch_poll_prepare_full() replace
 the io_watch_poll_prepare()
   - use io_watch_poll_funcs_full replace
 the io_watch_poll_funcs
   - avoid code duplication

v8:
  p5:
   - add new patch:
 qemu-char: Add qemu_chr_add_handlers_full() for GMaincontext

v7:
  p5:
- add [PATCH]qemu-char: Fix context for g_source_attach()
  in this patch series.

v6:
  p6:
- add more commit log.
- fix icmp comparison to compare all packet.

  p5:
- add more cpmments in commit log.
- change REGULAR_CHECK_MS to REGULAR_PACKET_CHECK_MS
- make check old packet independent to compare thread
- remove thread_status

  p4:
- change this patch only about
  Connection and ConnectionKey.
- add some comments in commit log.
- remove mode in fill_connection_key().
- fix some comments and bug.
- move colo_conn_state to patch of
  "work with colo-frame"
- remove conn_list_lock.
- add MAX_QUEUE_SIZE, if primary_list or
  secondary_list biger than MAX_QUEUE_SIZE
  we will drop packet.

  p3:
- add new independent kernel jhash patch.

  p2:
- add new independent colo-base patch.

  p1:
- add a ascii figure and some comments to explain it
- move trace.h to p2
- move QTAILQ_HEAD(, CompareState) net_compares to
  patch of "work with colo-frame"
- add some comments in qemu-option.hx


v5:
  p3:
 - comments from Jason
   we poll and handle chardev in comapre thread,
   Through this way, there's no need for extra
   synchronization with main loop
   this depend on another patch:
   qemu-char: Fix context for g_source_attach()
 - remove QemuEvent
  p2:
 - remove conn->list_lock
  p1:
 - move compare_pri/sec_chr_in to p3
 - move compare_chr_send to p2

v4:
  p4:
 - add some comments
 - fix some trace-events
 - fix tcp compare error
  p3:
 - add rcu_read_lock().
 - fix trace name
 - fix jason's other comments
 - rebase some Dave's branch function
  p2:
 - colo_compare_connection() change g_queue_push_head() to
 - g_queue_push_tail() match to sorted order.
 - remove pkt->s
 - move data structure to colo-base.h
 - add colo-base.c reuse codes for filter-rewriter
 - add some filter-rewriter needs struct
 - depends on previous SocketReadState patch
  p1:
 - except move qemu_chr_add_handlers()
   to colo thread
 - remove class_finalize
 - remove secondary arp codes
 - depends on previous SocketReadState patch

v3:
   - rebase colo-compare to colo-frame v2.7
   - fix most of Dave's comments
 (except RCU)
   - add TCP,UDP,ICMP and other packet comparison
   - add trace-event
   - add some comments
   - other bug fix
   - add RFC index
   - add usage in patch 1/4

v2:
   - add jhash.h

v1:
   - initial patch


Zhang Chen (10):
   qemu-char: Add qemu_chr_add_handlers_full() for GMaincontext
   colo-compare: introduce colo compare initialization
   net/colo.c: add colo.c to define and handle packet
   Jhash: add 

[Qemu-devel] [Bug 1616706] [NEW] watchdog doesn't bring down the VM

[Amit Chandak]

Public bug reported:

Qemu-kvm version : QEMU emulator version 1.5.3 (qemu-kvm-1.5.3-105.el7),
Copyright (c) 2003-2008 Fabrice Billard

Qemu version: Virsh command line tool of libvirt 1.2.17

I have the VM stuck in bios (efi shell), but i don't see the watchdog in
the host bringing it down?

Couple of questions:

1. Does the watchdog functionality requires the driver in adminvm to
trigger the reload? or qemu detects it in the host and causes the
reload.

2. Does this work reliably? I have seen cases where in i have the
watchdog daemon in the VM shut, still don't see the VM going down (I put
the action in the XML file as power off)

Thanks
Amit

** Affects: qemu
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1616706

Title:
  watchdog doesn't bring down the VM

Status in QEMU:
  New

Bug description:
  Qemu-kvm version : QEMU emulator version 1.5.3 (qemu-
  kvm-1.5.3-105.el7), Copyright (c) 2003-2008 Fabrice Billard

  Qemu version: Virsh command line tool of libvirt 1.2.17

  I have the VM stuck in bios (efi shell), but i don't see the watchdog
  in the host bringing it down?

  Couple of questions:

  1. Does the watchdog functionality requires the driver in adminvm to
  trigger the reload? or qemu detects it in the host and causes the
  reload.

  2. Does this work reliably? I have seen cases where in i have the
  watchdog daemon in the VM shut, still don't see the VM going down (I
  put the action in the XML file as power off)

  Thanks
  Amit

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1616706/+subscriptions

Re: [Qemu-devel] A question about this commit 9894dc0cdcc397ee5b26370bc53da6d360a363c2

[Daniel P. Berrange]

On Tue, Aug 23, 2016 at 08:57:44AM +, Gaohaifeng (A) wrote:
> Hi Daniel & Paolo,
> 
> Commit 9894dc0c "char: convert from GIOChannel to QIOChannel", about
> 
> the below code segment:
> 
> -static gboolean tcp_chr_read(GIOChannel *chan, GIOCondition cond, void 
> *opaque)
> +static gboolean tcp_chr_read(QIOChannel *chan, GIOCondition cond, void 
> *opaque)
> {
>  CharDriverState *chr = opaque;
>  TCPCharDriver *s = chr->opaque;
> @@ -2938,9 +2801,7 @@ static gboolean tcp_chr_read(GIOChannel *chan, 
> GIOCondition cond, void *opaque)
>  if (len > s->max_size)
>  len = s->max_size;
>  size = tcp_chr_recv(chr, (void *)buf, len);
> -if (size == 0 ||
> -(size < 0 &&
> - socket_error() != EAGAIN && socket_error() != EWOULDBLOCK)) {
> +if (size == 0 || size == -1) {
>  /* connection closed */
>  tcp_chr_disconnect(chr);
>  } else if (size > 0) {
> 
> The old version will not call tcp_chr_disconnect when error is not
> EAGAIN.
> The new version will call tcp_chr_disconnect when size==-1. From the
> tcp_chr_recv function we see EAGIN will return -1, so EAGIN will call
> tcp_chr_disconnect.
> 
> We meet an issue when Guest VM use DPDK(1.6.0) l2fwd, it may exit since
> link status is not up. The link is down because tcp_chr_disconnect will
> set it.
> 
> Can you explain it why EAGIN here changes the behavior ?

tcp_chr_read is used in a call to io_add_watch_poll() which sets up an
event loop watch so that tcp_chr_read is called when there is incoming
data to be read.  IOW, when tcp_chr_read is invoked, I would always
expect that at least 1 byte is available, and so EAGAIN should not
occurr.

So I'm puzzled why you would get EAGAIN at all, unless there is some
kind of race with other code also consuming bytes from the same
socket.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] ppc: regression in 2.7.0-rc3 on rfi/rfid/hrfi instructions

[Benjamin Herrenschmidt]

On Wed, 2016-08-24 at 20:39 +0200, Hervé Poussineau wrote:
> Hi,
> 
> Following patch regresses Linux boot on PReP machine:
> 
> commit a2e71b28e832346409efc795ecd1f0a2bcb705a3
> Author: Benjamin Herrenschmidt 
> Date:   Tue Jun 21 23:48:46 2016 +0200
> 
>  ppc: Fix rfi/rfid/hrfi/... emulation
> 
>  This reworks emulation of the various "rfi" variants. I removed
>  some masking bits that I couldn't make sense of, the only bit
> that

Interesting, the problem is only with 603, not 604. I think I might
have broken something with the 603 SW TLB loading stuff which is
a bit weird, I'll have a look. Thanks.

Cheers,
Ben.

Re: [Qemu-devel] transient failure in the test-qht tests

[Peter Maydell]

On 25 August 2016 at 00:44, Emilio G. Cota  wrote:
> On Wed, Aug 24, 2016 at 21:39:01 +0100, Peter Maydell wrote:
>> So I encountered this test failure running 'make check' on
>> 32-bit ARM:
>>
>> MALLOC_PERTURB_=${MALLOC_PERTURB_:-$((RANDOM % 255 + 1))} gtester -k
>> --verbose -m=quick tests/test-qht
>> TEST: tests/test-qht... (pid=15763)
>>   /qht/mode/default:   OK
>>   /qht/mode/resize:FAIL
>> GTester: last random seed: R02S08efd89fe4d862dd0191c13d5ce4d76e
>> (pid=16462)
>> FAIL: tests/test-qht
>>
>> The test suite passed on a rerun.
>>
>> Any ideas?
>
> I wonder whether malloc perturb had to do with the failure, because
> -ENOMEM is unlikely (I only see a few MB of peak mem usage for qht-test)
>
> However, I just ran qht-test under valgrind on an i686 machine, and it comes
> clean.
>
> I also brute-forced this to see if a particular perturb value would
> make it fail:
>   for i in $(seq 0 255); do \
> echo $i && \
> MALLOC_PERTURB_=$i gtester -k --verbose -m=quick tests/test-qht \
> --seed=R02S08efd89fe4d862dd0191c13d5ce4d76e || break; \
>   done
>
> I get no failures on both i686 and x86_64, with and without that --seed flag.
>
> Is there any chance of getting a core dump for the failure you encountered?

Unfortunately not, the test config doesn't save core dumps. In any case
I assume from the output that the test didn't actually dump core, it
just failed (without saying anything about why).

thanks
-- PMM

Re: [Qemu-devel] transient failure in the test-qht tests

[Emilio G. Cota]

On Wed, Aug 24, 2016 at 21:39:01 +0100, Peter Maydell wrote:
> So I encountered this test failure running 'make check' on
> 32-bit ARM:
> 
> MALLOC_PERTURB_=${MALLOC_PERTURB_:-$((RANDOM % 255 + 1))} gtester -k
> --verbose -m=quick tests/test-qht
> TEST: tests/test-qht... (pid=15763)
>   /qht/mode/default:   OK
>   /qht/mode/resize:FAIL
> GTester: last random seed: R02S08efd89fe4d862dd0191c13d5ce4d76e
> (pid=16462)
> FAIL: tests/test-qht
> 
> The test suite passed on a rerun.
> 
> Any ideas?

I wonder whether malloc perturb had to do with the failure, because
-ENOMEM is unlikely (I only see a few MB of peak mem usage for qht-test)

However, I just ran qht-test under valgrind on an i686 machine, and it comes
clean.

I also brute-forced this to see if a particular perturb value would
make it fail:
  for i in $(seq 0 255); do \
echo $i && \
MALLOC_PERTURB_=$i gtester -k --verbose -m=quick tests/test-qht \
--seed=R02S08efd89fe4d862dd0191c13d5ce4d76e || break; \
  done

I get no failures on both i686 and x86_64, with and without that --seed flag.

Is there any chance of getting a core dump for the failure you encountered?

Thanks,

Emilio

Re: [Qemu-devel] [PATCH v6 4/4] docs: Add Documentation for Mediated devices

[Daniel P. Berrange]

On Thu, Aug 04, 2016 at 12:33:54AM +0530, Kirti Wankhede wrote:
> diff --git a/Documentation/vfio-mediated-device.txt 
> b/Documentation/vfio-mediated-device.txt
> new file mode 100644
> index ..029152670141
> --- /dev/null
> +++ b/Documentation/vfio-mediated-device.txt
> @@ -0,0 +1,235 @@
> +Mediated device management interface via sysfs
> +---
> +This is the interface that allows user space software, like libvirt, to query
> +and configure mediated device in a HW agnostic fashion. This management
> +interface provide flexibility to underlying physical device's driver to 
> support
> +mediated device hotplug, multiple mediated devices per virtual machine, 
> multiple
> +mediated devices from different physical devices, etc.

A key point from the libvirt POV is that we want to be able to use the
sysfs interfaces without having to write vendor specific custom code for
each vendor's hardware.

> +Under per-physical device sysfs:
> +
> +
> +* mdev_supported_types: (read only)
> +List the current supported mediated device types and its details.

This really ought to describe the data format that is to be reported,
as from libvirt POV we don't want to see every vendor's driver reporting
arbitrarily different information here.

> +* mdev_create: (write only)
> + Create a mediated device on target physical device.
> + Input syntax: 
> + where,
> + UUID: mediated device's UUID
> + idx: mediated device index inside a VM
> + params: extra parameters required by driver

There's no specification about what 'params' is - it just looks like
an arbitrary vendor specific blob, which is not something that's
particularly pleasant to use. How would a userspace application
discover what parameters exist, and whether they are required to be
passed, vs optional, and standardization of those parameters across
different vendors's vGPU drivers so we don't have each vendor doing
something different.

> + Example:
> + # echo "12345678-1234-1234-1234-123456789abc:0:0" >
> +  /sys/bus/pci/devices/\:05\:00.0/mdev_create
> +
> +* mdev_destroy: (write only)
> + Destroy a mediated device on a target physical device.
> + Input syntax: 
> + where,
> + UUID: mediated device's UUID
> + idx: mediated device index inside a VM
> + Example:
> + # echo "12345678-1234-1234-1234-123456789abc:0" >
> +/sys/bus/pci/devices/\:05\:00.0/mdev_destroy

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [RFC] libvirt vGPU QEMU integration

[Daniel P. Berrange]

On Thu, Aug 18, 2016 at 09:41:59AM -0700, Neo Jia wrote:
> Hi libvirt experts,
> 
> I am starting this email thread to discuss the potential solution / proposal 
> of
> integrating vGPU support into libvirt for QEMU.
> 
> Some quick background, NVIDIA is implementing a VFIO based mediated device
> framework to allow people to virtualize their devices without SR-IOV, for
> example NVIDIA vGPU, and Intel KVMGT. Within this framework, we are reusing 
> the
> VFIO API to process the memory / interrupt as what QEMU does today with 
> passthru
> device.
> 
> The difference here is that we are introducing a set of new sysfs file for
> virtual device discovery and life cycle management due to its virtual nature.
> 
> Here is the summary of the sysfs, when they will be created and how they 
> should
> be used:
> 
> 1. Discover mediated device
> 
> As part of physical device initialization process, vendor driver will register
> their physical devices, which will be used to create virtual device (mediated
> device, aka mdev) to the mediated framework.
> 
> Then, the sysfs file "mdev_supported_types" will be available under the 
> physical
> device sysfs, and it will indicate the supported mdev and configuration for 
> this 
> particular physical device, and the content may change dynamically based on 
> the
> system's current configurations, so libvirt needs to query this file every 
> time
> before create a mdev.
> 
> Note: different vendors might have their own specific configuration sysfs as
> well, if they don't have pre-defined types.
> 
> For example, we have a NVIDIA Tesla M60 on 86:00.0 here registered, and here 
> is
> NVIDIA specific configuration on an idle system.
> 
> For example, to query the "mdev_supported_types" on this Tesla M60:
> 
> cat /sys/bus/pci/devices/:86:00.0/mdev_supported_types
> # vgpu_type_id, vgpu_type, max_instance, num_heads, frl_config, framebuffer,
> max_resolution
> 11  ,"GRID M60-0B",  16,   2,  45, 512M,2560x1600
> 12  ,"GRID M60-0Q",  16,   2,  60, 512M,2560x1600
> 13  ,"GRID M60-1B",   8,   2,  45,1024M,2560x1600
> 14  ,"GRID M60-1Q",   8,   2,  60,1024M,2560x1600
> 15  ,"GRID M60-2B",   4,   2,  45,2048M,2560x1600
> 16  ,"GRID M60-2Q",   4,   4,  60,2048M,2560x1600
> 17  ,"GRID M60-4Q",   2,   4,  60,4096M,3840x2160
> 18  ,"GRID M60-8Q",   1,   4,  60,8192M,3840x2160

I'm unclear on the requirements about data format for this file.
Looking at the docs:

  http://www.spinics.net/lists/kvm/msg136476.html

the format is completely unspecified.

> 
> 2. Create/destroy mediated device
> 
> Two sysfs files are available under the physical device sysfs path : 
> mdev_create
> and mdev_destroy
> 
> The syntax of creating a mdev is:
> 
> echo "$mdev_UUID:vendor_specific_argument_list" >
> /sys/bus/pci/devices/.../mdev_create

I'm not really a fan of the idea of having to provide arbitrary vendor
specific arguments to the mdev_create call, as I don't really want to
have to create vendor specific code for each vendor's vGPU hardware in
libvirt.

What is the relationship between the mdev_supported_types data and
the vendor_specific_argument_list requirements ?


> The syntax of destroying a mdev is:
> 
> echo "$mdev_UUID:vendor_specific_argument_list" >
> /sys/bus/pci/devices/.../mdev_destroy
> 
> The $mdev_UUID is a unique identifier for this mdev device to be created, and 
> it
> is unique per system.
> 
> For NVIDIA vGPU, we require a vGPU type identifier (shown as vgpu_type_id in
> above Tesla M60 output), and a VM UUID to be passed as
> "vendor_specific_argument_list".
> 
> If there is no vendor specific arguments required, either "$mdev_UUID" or
> "$mdev_UUID:" will be acceptable as input syntax for the above two commands.

This raises the question of how an application discovers what
vendor specific arguments are required or not, and what they
might mean.

> To create a M60-4Q device, libvirt needs to do:
> 
> echo "$mdev_UUID:vgpu_type_id=20,vm_uuid=$VM_UUID" >
> /sys/bus/pci/devices/\:86\:00.0/mdev_create

Overall it doesn't seem like the proposed kernel interfaces provide
enough vendor abstraction to be able to use this functionality without
having to create vendor specific code in libvirt, which is something
I want to avoid us doing.



Ignoring the details though, in terms of libvirt integration, I think I'd
see us primarily doing work in the node device APIs / XML. Specifically
for physical devices, we'd have to report whether they support the
mediated device feature and some way to enumerate the validate device
types that can be created. The node device creation API would have to
support create/deletion of the devices (mapping to mdev_create/destroy)


When configuring a guest VM, we'd use the  XML to point to one
or more mediated devices that have been created via 

[Qemu-devel] [PATCH 5/8] linux-user: add stop-the-world to be called from CPU loop

[Emilio G. Cota]

Signed-off-by: Emilio G. Cota 
---
 cpu-exec.c  |  1 +
 include/exec/exec-all.h |  5 +++
 linux-user/main.c   | 89 +
 linux-user/syscall.c|  1 +
 4 files changed, 96 insertions(+)

diff --git a/cpu-exec.c b/cpu-exec.c
index 63d739a..8f1adc4 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -649,6 +649,7 @@ int cpu_exec(CPUState *cpu)
 g_assert(cc == CPU_GET_CLASS(cpu));
 #endif /* buggy compiler */
 cpu->can_do_io = 1;
+stop_the_world_reset();
 tb_lock_reset();
 }
 } /* for(;;) */
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index ec72c5a..c483d80 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -61,6 +61,11 @@ void QEMU_NORETURN cpu_loop_exit(CPUState *cpu);
 void QEMU_NORETURN cpu_loop_exit_restore(CPUState *cpu, uintptr_t pc);
 void QEMU_NORETURN cpu_loop_exit_atomic(CPUState *cpu, uintptr_t pc);
 
+void stop_the_world_lock(CPUState *cpu);
+void stop_the_world_unlock(void);
+void stop_the_world_reset(void);
+extern __thread bool stw_held;
+
 #if !defined(CONFIG_USER_ONLY)
 void cpu_reloading_memory_map(void);
 /**
diff --git a/linux-user/main.c b/linux-user/main.c
index 9880505..94c6625 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -114,11 +114,19 @@ static pthread_cond_t exclusive_cond = 
PTHREAD_COND_INITIALIZER;
 static pthread_cond_t exclusive_resume = PTHREAD_COND_INITIALIZER;
 static int pending_cpus;
 
+static pthread_cond_t stw_sleep_cond   = PTHREAD_COND_INITIALIZER;
+static pthread_cond_t stw_request_cond = PTHREAD_COND_INITIALIZER;
+static pthread_mutex_t stw_lock = PTHREAD_MUTEX_INITIALIZER;
+static int stw_requests;
+static bool stw_ongoing;
+__thread bool stw_held;
+
 /* Make sure everything is in a consistent state for calling fork().  */
 void fork_start(void)
 {
 qemu_mutex_lock(_ctx.tb_ctx.tb_lock);
 pthread_mutex_lock(_lock);
+pthread_mutex_lock(_lock);
 mmap_fork_start();
 }
 
@@ -137,11 +145,17 @@ void fork_end(int child)
 pending_cpus = 0;
 pthread_mutex_init(_lock, NULL);
 pthread_mutex_init(_list_mutex, NULL);
+pthread_mutex_init(_lock, NULL);
+stw_held = false;
+stw_ongoing = false;
 pthread_cond_init(_cond, NULL);
 pthread_cond_init(_resume, NULL);
+pthread_cond_init(_sleep_cond, NULL);
+pthread_cond_init(_request_cond, NULL);
 qemu_mutex_init(_ctx.tb_ctx.tb_lock);
 gdbserver_fork(thread_cpu);
 } else {
+pthread_mutex_unlock(_lock);
 pthread_mutex_unlock(_lock);
 qemu_mutex_unlock(_ctx.tb_ctx.tb_lock);
 }
@@ -198,6 +212,79 @@ static void step_atomic(CPUState *cpu)
 end_exclusive();
 }
 
+void stop_the_world_lock(CPUState *cpu)
+{
+CPUState *other;
+
+if (stw_held) {
+return;
+}
+rcu_read_unlock();
+assert(!rcu_read_lock_held());
+
+pthread_mutex_lock(_lock);
+if (stw_ongoing) {
+stw_requests++;
+/* wait for ongoing stops to occur */
+while (stw_ongoing) {
+pthread_cond_wait(_request_cond, _lock);
+}
+stw_requests--;
+}
+
+/* it's our turn! */
+stw_ongoing = true;
+stw_held = true;
+CPU_FOREACH(other) {
+if (other != cpu) {
+cpu_exit(other);
+}
+}
+synchronize_rcu();
+}
+
+void stop_the_world_unlock(void)
+{
+if (!stw_held) {
+return;
+}
+assert(stw_ongoing);
+assert(!rcu_read_lock_held());
+
+if (stw_requests) {
+pthread_cond_signal(_request_cond);
+} else {
+pthread_cond_broadcast(_sleep_cond);
+}
+/*
+ * Make sure the next STW requester (if any) will perceive that we're
+ * in an RCU read critical section
+ */
+rcu_read_lock();
+stw_ongoing = false;
+stw_held = false;
+pthread_mutex_unlock(_lock);
+}
+
+void stop_the_world_reset(void)
+{
+if (likely(!stw_held)) {
+return;
+}
+stop_the_world_unlock();
+}
+
+static inline void stop_the_world_sleep(void)
+{
+pthread_mutex_lock(_lock);
+if (unlikely(stw_ongoing)) {
+while (stw_ongoing) {
+pthread_cond_wait(_sleep_cond, _lock);
+}
+}
+pthread_mutex_unlock(_lock);
+}
+
 /* Wait for exclusive ops to finish, and begin cpu execution.  */
 static inline void cpu_exec_start(CPUState *cpu)
 {
@@ -205,6 +292,8 @@ static inline void cpu_exec_start(CPUState *cpu)
 exclusive_idle();
 cpu->running = true;
 pthread_mutex_unlock(_lock);
+
+stop_the_world_sleep();
 }
 
 /* Mark cpu as not executing, and release pending exclusive ops.  */
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 2911319..740af23 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -5403,6 +5403,7 @@ static void *clone_func(void *arg)
 /* Wait until the parent has finshed initializing the tls state.  */
 

Re: [Qemu-devel] ppc: regression in 2.7.0-rc3 on rfi/rfid/hrfi instructions

[Benjamin Herrenschmidt]

On Wed, 2016-08-24 at 20:39 +0200, Hervé Poussineau wrote:
> Hi,
> 
> Following patch regresses Linux boot on PReP machine:

Where do I find the source for that kernel ?

Cheers,
Ben.

> commit a2e71b28e832346409efc795ecd1f0a2bcb705a3
> Author: Benjamin Herrenschmidt 
> Date:   Tue Jun 21 23:48:46 2016 +0200
> 
>  ppc: Fix rfi/rfid/hrfi/... emulation
> 
>  This reworks emulation of the various "rfi" variants. I removed
>  some masking bits that I couldn't make sense of, the only bit
> that
>  I am aware we should mask here is POW, the CPU's MSR mask should
>  take care of the rest.
> 
>  This also fixes some problems when running 32-bit userspace
> under
>  a 64-bit kernel.
> 
>  This patch broke 32bit OpenBIOS when run under a 970 cpu. A fix
> was
>  proposed here :
> 
>  https://www.coreboot.org/pipermail/openbios/2016-June/009452
> .html
> 
>  Signed-off-by: Benjamin Herrenschmidt 
>  Reviewed-by: David Gibson 
>  [clg: updated the commit log with the reference of the openbios
> fix ]
>  Signed-off-by: Cédric Le Goater 
>  [dwg: Remove hunk which disabled rfi on 64-bit CPUS.  The change
> was
>   correct, but we need to fix OpenBIOS before applying it]
>  Signed-off-by: David Gibson 
> 
> Test case:
> - Download: http://www.juneau-lug.org/zImage.initrd.sandalfoot
> - Run: qemu-system-ppc -M prep -kernel zImage.initrd.sandalfoot
> 
> Firmware starts, loads kernel, but seems to error out just after PS/2
> detection.
> 
> Reverting the commit and fixing the conflict makes it work again, up
> to shell.
> 
> Hervé

[Qemu-devel] [PATCH 2/8] cpu-exec: remove tb_lock from hot path

[Emilio G. Cota]

Signed-off-by: Emilio G. Cota 
---
 cpu-exec.c | 38 +++---
 1 file changed, 11 insertions(+), 27 deletions(-)

diff --git a/cpu-exec.c b/cpu-exec.c
index 041f8b7..63d739a 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -309,34 +309,18 @@ static TranslationBlock *tb_find_slow(CPUState *cpu,
 TranslationBlock *tb;
 
 tb = tb_find_physical(cpu, pc, cs_base, flags);
-if (tb) {
-goto found;
-}
-
-#ifdef CONFIG_USER_ONLY
-/* mmap_lock is needed by tb_gen_code, and mmap_lock must be
- * taken outside tb_lock.  Since we're momentarily dropping
- * tb_lock, there's a chance that our desired tb has been
- * translated.
- */
-tb_unlock();
-mmap_lock();
-tb_lock();
-tb = tb_find_physical(cpu, pc, cs_base, flags);
-if (tb) {
+if (!tb) {
+mmap_lock();
+tb_lock();
+tb = tb_find_physical(cpu, pc, cs_base, flags);
+if (!tb) {
+/* if no translated code available, then translate it now */
+tb = tb_gen_code(cpu, pc, cs_base, flags, 0);
+}
+tb_unlock();
 mmap_unlock();
-goto found;
 }
-#endif
-
-/* if no translated code available, then translate it now */
-tb = tb_gen_code(cpu, pc, cs_base, flags, 0);
-
-#ifdef CONFIG_USER_ONLY
-mmap_unlock();
-#endif
 
-found:
 /* we add the TB in the virtual pc hash table */
 cpu->tb_jmp_cache[tb_jmp_cache_hash_func(pc)] = tb;
 return tb;
@@ -355,7 +339,6 @@ static inline TranslationBlock *tb_find_fast(CPUState *cpu,
always be the same before a given translated block
is executed. */
 cpu_get_tb_cpu_state(env, , _base, );
-tb_lock();
 tb = cpu->tb_jmp_cache[tb_jmp_cache_hash_func(pc)];
 if (unlikely(!tb || tb->pc != pc || tb->cs_base != cs_base ||
  tb->flags != flags)) {
@@ -379,9 +362,10 @@ static inline TranslationBlock *tb_find_fast(CPUState *cpu,
 #endif
 /* See if we can patch the calling TB. */
 if (*last_tb && !qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN)) {
+tb_lock();
 tb_add_jump(*last_tb, tb_exit, tb);
+tb_unlock();
 }
-tb_unlock();
 return tb;
 }
 
-- 
2.5.0

[Qemu-devel] [PATCH 8/8] target-arm/a64: use HTM with stop-the-world fall-back path

[Emilio G. Cota]

TODO: convert paired atomics as well.

Signed-off-by: Emilio G. Cota 
---
 cpu-exec.c |  4 
 target-arm/helper-a64.c| 31 +++
 target-arm/helper-a64.h|  4 
 target-arm/op_helper.c |  4 
 target-arm/translate-a64.c | 16 ++--
 5 files changed, 53 insertions(+), 6 deletions(-)

diff --git a/cpu-exec.c b/cpu-exec.c
index 8f1adc4..6e2531f 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -26,6 +26,7 @@
 #include "sysemu/qtest.h"
 #include "qemu/timer.h"
 #include "exec/address-spaces.h"
+#include "qemu/htm.h"
 #include "qemu/rcu.h"
 #include "exec/tb-hash.h"
 #include "exec/log.h"
@@ -651,6 +652,9 @@ int cpu_exec(CPUState *cpu)
 cpu->can_do_io = 1;
 stop_the_world_reset();
 tb_lock_reset();
+if (unlikely(htm_test())) {
+htm_end();
+}
 }
 } /* for(;;) */
 
diff --git a/target-arm/helper-a64.c b/target-arm/helper-a64.c
index 6f3fd17..741e6de 100644
--- a/target-arm/helper-a64.c
+++ b/target-arm/helper-a64.c
@@ -25,6 +25,7 @@
 #include "qemu/log.h"
 #include "sysemu/sysemu.h"
 #include "qemu/bitops.h"
+#include "qemu/htm.h"
 #include "internals.h"
 #include "qemu/crc32c.h"
 #include "exec/exec-all.h"
@@ -579,3 +580,33 @@ uint64_t HELPER(paired_cmpxchg64_be)(CPUARMState *env, 
uint64_t addr,
 
 return !success;
 }
+
+void HELPER(xbegin)(CPUARMState *env)
+{
+int status;
+int retries = 100;
+
+ retry:
+status = htm_begin();
+if (unlikely(status != HTM_OK)) {
+if ((status & HTM_ABORT_RETRY) && retries) {
+retries--;
+goto retry;
+}
+stop_the_world_lock(ENV_GET_CPU(env));
+}
+}
+
+void HELPER(xend)(void)
+{
+if (likely(htm_test())) {
+htm_end();
+} else {
+stop_the_world_unlock();
+}
+}
+
+uint64_t HELPER(x_ok)(void)
+{
+return likely(htm_test()) || stw_held;
+}
diff --git a/target-arm/helper-a64.h b/target-arm/helper-a64.h
index dd32000..e7ede43 100644
--- a/target-arm/helper-a64.h
+++ b/target-arm/helper-a64.h
@@ -48,3 +48,7 @@ DEF_HELPER_FLAGS_3(crc32_64, TCG_CALL_NO_RWG_SE, i64, i64, 
i64, i32)
 DEF_HELPER_FLAGS_3(crc32c_64, TCG_CALL_NO_RWG_SE, i64, i64, i64, i32)
 DEF_HELPER_FLAGS_4(paired_cmpxchg64_le, TCG_CALL_NO_WG, i64, env, i64, i64, 
i64)
 DEF_HELPER_FLAGS_4(paired_cmpxchg64_be, TCG_CALL_NO_WG, i64, env, i64, i64, 
i64)
+
+DEF_HELPER_1(xbegin, void, env)
+DEF_HELPER_0(x_ok, i64)
+DEF_HELPER_0(xend, void)
diff --git a/target-arm/op_helper.c b/target-arm/op_helper.c
index 73da759..91b1413 100644
--- a/target-arm/op_helper.c
+++ b/target-arm/op_helper.c
@@ -17,6 +17,7 @@
  * License along with this library; if not, see .
  */
 #include "qemu/osdep.h"
+#include "qemu/htm.h"
 #include "cpu.h"
 #include "exec/helper-proto.h"
 #include "internals.h"
@@ -31,6 +32,9 @@ static void raise_exception(CPUARMState *env, uint32_t excp,
 {
 CPUState *cs = CPU(arm_env_get_cpu(env));
 
+if (unlikely(htm_test())) {
+htm_end();
+}
 assert(!excp_is_internal(excp));
 cs->exception_index = excp;
 env->exception.syndrome = syndrome;
diff --git a/target-arm/translate-a64.c b/target-arm/translate-a64.c
index 450c359..cc3baa0 100644
--- a/target-arm/translate-a64.c
+++ b/target-arm/translate-a64.c
@@ -1760,6 +1760,8 @@ static void gen_load_exclusive(DisasContext *s, int rt, 
int rt2,
 TCGv_i64 tmp = tcg_temp_new_i64();
 TCGMemOp be = s->be_data;
 
+gen_helper_xbegin(cpu_env);
+
 g_assert(size <= 3);
 if (is_pair) {
 TCGv_i64 hitmp = tcg_temp_new_i64();
@@ -1825,6 +1827,10 @@ static void gen_store_exclusive(DisasContext *s, int rd, 
int rt, int rt2,
 tcg_gen_brcond_i64(TCG_COND_NE, addr, cpu_exclusive_addr, fail_label);
 
 tmp = tcg_temp_new_i64();
+/* strex without a prior ldrex should just fail */
+gen_helper_x_ok(tmp);
+tcg_gen_brcondi_i64(TCG_COND_EQ, tmp, 0, fail_label);
+
 if (is_pair) {
 if (size == 2) {
 TCGv_i64 val = tcg_temp_new_i64();
@@ -1844,16 +1850,14 @@ static void gen_store_exclusive(DisasContext *s, int 
rd, int rt, int rt2,
 }
 } else {
 TCGv_i64 val = cpu_reg(s, rt);
-tcg_gen_atomic_cmpxchg_i64(tmp, addr, cpu_exclusive_val, val,
-   get_mem_index(s),
-   size | MO_ALIGN | s->be_data);
-tcg_gen_setcond_i64(TCG_COND_NE, tmp, tmp, cpu_exclusive_val);
+tcg_gen_qemu_st_i64(val, addr, get_mem_index(s), s->be_data + size);
 }
 
 tcg_temp_free_i64(addr);
-
-tcg_gen_mov_i64(cpu_reg(s, rd), tmp);
 tcg_temp_free_i64(tmp);
+
+tcg_gen_movi_i64(cpu_reg(s, rd), 0);
+gen_helper_xend();
 tcg_gen_br(done_label);
 
 gen_set_label(fail_label);
-- 
2.5.0

[Qemu-devel] [PATCH 4/8] target-arm: helper fixup for paired atomics

[Emilio G. Cota]

Signed-off-by: Emilio G. Cota 
---
 target-arm/helper-a64.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target-arm/helper-a64.c b/target-arm/helper-a64.c
index 8ce518b..6f3fd17 100644
--- a/target-arm/helper-a64.c
+++ b/target-arm/helper-a64.c
@@ -453,7 +453,7 @@ uint64_t HELPER(crc32c_64)(uint64_t acc, uint64_t val, 
uint32_t bytes)
 uint64_t HELPER(paired_cmpxchg64_le)(CPUARMState *env, uint64_t addr,
  uint64_t new_lo, uint64_t new_hi)
 {
-#ifndef CONFIG_USER_ONLY
+#if !defined(CONFIG_USER_ONLY) || !defined(CONFIG_ATOMIC128)
 uintptr_t ra = GETPC();
 #endif
 Int128 oldv, cmpv, newv;
@@ -518,7 +518,7 @@ uint64_t HELPER(paired_cmpxchg64_le)(CPUARMState *env, 
uint64_t addr,
 uint64_t HELPER(paired_cmpxchg64_be)(CPUARMState *env, uint64_t addr,
  uint64_t new_lo, uint64_t new_hi)
 {
-#ifndef CONFIG_USER_ONLY
+#if !defined(CONFIG_USER_ONLY) || !defined(CONFIG_ATOMIC128)
 uintptr_t ra = GETPC();
 #endif
 Int128 oldv, cmpv, newv;
-- 
2.5.0

[Qemu-devel] [PATCH 1/8] cpu list: convert to RCU QLIST

[Emilio G. Cota]

This avoids the chance of reading a corrupted list of CPUs in usermode.

Note: this breaks hw/ppc/spapr due to the removal of CPU_FOREACH_REVERSE.

Signed-off-by: Emilio G. Cota 
---
 cpus.c   |  2 +-
 exec.c   | 18 +++---
 include/qom/cpu.h| 16 +++-
 linux-user/main.c|  2 +-
 linux-user/syscall.c |  2 +-
 5 files changed, 25 insertions(+), 15 deletions(-)

diff --git a/cpus.c b/cpus.c
index a01bbbd..bc573be 100644
--- a/cpus.c
+++ b/cpus.c
@@ -1177,7 +1177,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
 qemu_clock_notify(QEMU_CLOCK_VIRTUAL);
 }
 }
-qemu_tcg_wait_io_event(QTAILQ_FIRST());
+qemu_tcg_wait_io_event(first_cpu);
 CPU_FOREACH(cpu) {
 if (cpu->unplug && !cpu_can_run(cpu)) {
 remove_cpu = cpu;
diff --git a/exec.c b/exec.c
index 806e2fe..70dd869 100644
--- a/exec.c
+++ b/exec.c
@@ -93,7 +93,7 @@ static MemoryRegion io_mem_unassigned;
 
 #endif
 
-struct CPUTailQ cpus = QTAILQ_HEAD_INITIALIZER(cpus);
+struct CPUTailQ cpus = QLIST_HEAD_INITIALIZER(cpus);
 /* current CPU in the current thread. It is only valid inside
cpu_exec() */
 __thread CPUState *current_cpu;
@@ -651,7 +651,7 @@ void cpu_exec_exit(CPUState *cpu)
 return;
 }
 
-QTAILQ_REMOVE(, cpu, node);
+QLIST_REMOVE_RCU(cpu, node);
 cpu_release_index(cpu);
 cpu->cpu_index = -1;
 #if defined(CONFIG_USER_ONLY)
@@ -703,7 +703,19 @@ void cpu_exec_init(CPUState *cpu, Error **errp)
 #endif
 return;
 }
-QTAILQ_INSERT_TAIL(, cpu, node);
+/* poor man's QLIST_INSERT_TAIL_RCU */
+if (QLIST_EMPTY_RCU()) {
+QLIST_INSERT_HEAD_RCU(, cpu, node);
+} else {
+CPUState *some_cpu;
+
+CPU_FOREACH(some_cpu) {
+if (QLIST_NEXT_RCU(some_cpu, node) == NULL) {
+QLIST_INSERT_AFTER_RCU(some_cpu, cpu, node);
+break;
+}
+}
+}
 #if defined(CONFIG_USER_ONLY)
 (void) cc;
 cpu_list_unlock();
diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index 32f3af3..eba48ed 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -24,7 +24,7 @@
 #include "disas/bfd.h"
 #include "exec/hwaddr.h"
 #include "exec/memattrs.h"
-#include "qemu/queue.h"
+#include "qemu/rcu_queue.h"
 #include "qemu/thread.h"
 
 typedef int (*WriteCoreDumpFunction)(const void *buf, size_t size,
@@ -319,7 +319,7 @@ struct CPUState {
 struct GDBRegisterState *gdb_regs;
 int gdb_num_regs;
 int gdb_num_g_regs;
-QTAILQ_ENTRY(CPUState) node;
+QLIST_ENTRY(CPUState) node;
 
 /* ice debug support */
 QTAILQ_HEAD(breakpoints_head, CPUBreakpoint) breakpoints;
@@ -362,15 +362,13 @@ struct CPUState {
 uint32_t tcg_exit_req;
 };
 
-QTAILQ_HEAD(CPUTailQ, CPUState);
+QLIST_HEAD(CPUTailQ, CPUState);
 extern struct CPUTailQ cpus;
-#define CPU_NEXT(cpu) QTAILQ_NEXT(cpu, node)
-#define CPU_FOREACH(cpu) QTAILQ_FOREACH(cpu, , node)
+#define CPU_NEXT(cpu) QLIST_NEXT_RCU(cpu, node)
+#define CPU_FOREACH(cpu) QLIST_FOREACH_RCU(cpu, , node)
 #define CPU_FOREACH_SAFE(cpu, next_cpu) \
-QTAILQ_FOREACH_SAFE(cpu, , node, next_cpu)
-#define CPU_FOREACH_REVERSE(cpu) \
-QTAILQ_FOREACH_REVERSE(cpu, , CPUTailQ, node)
-#define first_cpu QTAILQ_FIRST()
+QLIST_FOREACH_SAFE_RCU(cpu, , node, next_cpu)
+#define first_cpu QLIST_FIRST_RCU()
 
 extern __thread CPUState *current_cpu;
 
diff --git a/linux-user/main.c b/linux-user/main.c
index f2f7422..9880505 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -131,7 +131,7 @@ void fork_end(int child)
Discard information about the parent threads.  */
 CPU_FOREACH_SAFE(cpu, next_cpu) {
 if (cpu != thread_cpu) {
-QTAILQ_REMOVE(, cpu, node);
+QLIST_REMOVE_RCU(cpu, node);
 }
 }
 pending_cpus = 0;
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 1c17b74..2911319 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -6710,7 +6710,7 @@ abi_long do_syscall(void *cpu_env, int num, abi_long arg1,
 
 cpu_list_lock();
 /* Remove the CPU from the list.  */
-QTAILQ_REMOVE(, cpu, node);
+QLIST_REMOVE_RCU(cpu, node);
 cpu_list_unlock();
 ts = cpu->opaque;
 if (ts->child_tidptr) {
-- 
2.5.0

[Qemu-devel] [PATCH 7/8] htm: add powerpc64 intrinsics

[Emilio G. Cota]

Signed-off-by: Emilio G. Cota 
---
 include/qemu/htm.h | 39 +++
 1 file changed, 39 insertions(+)

diff --git a/include/qemu/htm.h b/include/qemu/htm.h
index dc84bc1..f367ee4 100644
--- a/include/qemu/htm.h
+++ b/include/qemu/htm.h
@@ -39,5 +39,44 @@ static inline void htm_abort(void)
 {
 _xabort(0);
 }
+
+#elif defined(__powerpc64__)
+/* compile with -mhtm */
+#include 
+
+static inline int htm_begin(void)
+{
+unsigned int status;
+
+status = __builtin_tbegin(0);
+if (likely(status)) {
+return HTM_OK;
+}
+if (_TEXASRU_FAILURE_PERSISTENT(__builtin_get_texasru())) {
+return HTM_ABORT_NORETRY;
+}
+return HTM_ABORT_RETRY;
+}
+
+static inline void htm_end(void)
+{
+__builtin_tend(0);
+}
+
+static inline int htm_test(void)
+{
+unsigned char state = _HTM_STATE(__builtin_ttest());
+
+if (likely(state == _HTM_TRANSACTIONAL)) {
+return 1;
+}
+return 0;
+}
+
+static inline void htm_abort(void)
+{
+__builtin_tabort(0);
+}
+
 #endif /* ISA */
 #endif /* HTM_H */
-- 
2.5.0

[Qemu-devel] [PATCH 6/8] htm: add header to abstract Hardware Transactional Memory intrinsics

[Emilio G. Cota]

Signed-off-by: Emilio G. Cota 
---
 include/qemu/htm.h | 43 +++
 1 file changed, 43 insertions(+)
 create mode 100644 include/qemu/htm.h

diff --git a/include/qemu/htm.h b/include/qemu/htm.h
new file mode 100644
index 000..dc84bc1
--- /dev/null
+++ b/include/qemu/htm.h
@@ -0,0 +1,43 @@
+#ifndef HTM_H
+#define HTM_H
+
+enum htm {
+HTM_OK,
+HTM_ABORT_RETRY,
+HTM_ABORT_NORETRY,
+};
+
+#if defined(__x86_64__)
+/* compile with -mrtm */
+#include 
+
+static inline enum htm htm_begin(void)
+{
+int status;
+
+status = _xbegin();
+if (unlikely(status != _XBEGIN_STARTED)) {
+if (status & _XABORT_RETRY) {
+return HTM_ABORT_RETRY;
+}
+return HTM_ABORT_NORETRY;
+}
+return HTM_OK;
+}
+
+static inline void htm_end(void)
+{
+_xend();
+}
+
+static inline bool htm_test(void)
+{
+return _xtest();
+}
+
+static inline void htm_abort(void)
+{
+_xabort(0);
+}
+#endif /* ISA */
+#endif /* HTM_H */
-- 
2.5.0

[Qemu-devel] [PATCH 3/8] rcu: add rcu_read_lock_held()

[Emilio G. Cota]

Signed-off-by: Emilio G. Cota 
---
 include/qemu/rcu.h | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h
index 83ae280..0f6e467 100644
--- a/include/qemu/rcu.h
+++ b/include/qemu/rcu.h
@@ -98,6 +98,13 @@ static inline void rcu_read_unlock(void)
 }
 }
 
+static inline bool rcu_read_lock_held(void)
+{
+struct rcu_reader_data *p_rcu_reader = _reader;
+
+return p_rcu_reader->depth > 0;
+}
+
 extern void synchronize_rcu(void);
 
 /*
-- 
2.5.0

Re: [Qemu-devel] [PATCH 2/3] qemu: Implement virtio-pstore device

[Daniel P. Berrange]

> diff --git a/hw/virtio/virtio-pstore.c b/hw/virtio/virtio-pstore.c
> new file mode 100644
> index 000..b8fb4be
> --- /dev/null
> +++ b/hw/virtio/virtio-pstore.c
> @@ -0,0 +1,699 @@
> +/*
> + * Virtio Pstore Device
> + *
> + * Copyright (C) 2016  LG Electronics
> + *
> + * Authors:
> + *  Namhyung Kim  
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
> +
> +#include 
> +
> +#include "qemu/osdep.h"
> +#include "qemu/iov.h"
> +#include "qemu-common.h"
> +#include "qemu/cutils.h"
> +#include "qemu/error-report.h"
> +#include "sysemu/kvm.h"
> +#include "qapi/visitor.h"
> +#include "qapi-event.h"
> +#include "io/channel-util.h"
> +#include "trace.h"
> +
> +#include "hw/virtio/virtio.h"
> +#include "hw/virtio/virtio-bus.h"
> +#include "hw/virtio/virtio-access.h"
> +#include "hw/virtio/virtio-pstore.h"
> +
> +#define PSTORE_DEFAULT_BUFSIZE   (16 * 1024)
> +#define PSTORE_DEFAULT_FILE_MAX  5
> +
> +/* the index should match to the type value */
> +static const char *virtio_pstore_file_prefix[] = {
> +"unknown-",  /* VIRTIO_PSTORE_TYPE_UNKNOWN */
> +"dmesg-",/* VIRTIO_PSTORE_TYPE_DMESG */
> +};
> +
> +static char *virtio_pstore_to_filename(VirtIOPstore *s,
> +   struct virtio_pstore_req *req)
> +{
> +const char *basename;
> +unsigned long long id;
> +unsigned int type = le16_to_cpu(req->type);
> +unsigned int flags = le32_to_cpu(req->flags);
> +
> +if (type < ARRAY_SIZE(virtio_pstore_file_prefix)) {
> +basename = virtio_pstore_file_prefix[type];
> +} else {
> +basename = "unknown-";
> +}
> +
> +id = s->id++;
> +return g_strdup_printf("%s/%s%llu%s", s->directory, basename, id,
> +flags & VIRTIO_PSTORE_FL_COMPRESSED ? ".enc.z" : 
> "");
> +}
> +
> +static char *virtio_pstore_from_filename(VirtIOPstore *s, char *name,
> + struct virtio_pstore_fileinfo *info)
> +{
> +char *filename;
> +unsigned int idx;
> +
> +filename = g_strdup_printf("%s/%s", s->directory, name);
> +if (filename == NULL)
> +return NULL;
> +
> +for (idx = 0; idx < ARRAY_SIZE(virtio_pstore_file_prefix); idx++) {
> +if (g_str_has_prefix(name, virtio_pstore_file_prefix[idx])) {
> +info->type = idx;
> +name += strlen(virtio_pstore_file_prefix[idx]);
> +break;
> +}
> +}
> +
> +if (idx == ARRAY_SIZE(virtio_pstore_file_prefix)) {
> +g_free(filename);
> +return NULL;
> +}
> +
> +qemu_strtoull(name, NULL, 0, >id);
> +
> +info->flags = 0;
> +if (g_str_has_suffix(name, ".enc.z")) {
> +info->flags |= VIRTIO_PSTORE_FL_COMPRESSED;
> +}
> +
> +return filename;
> +}
> +
> +static int prefix_idx;
> +static int prefix_count;
> +static int prefix_len;
> +
> +static int filter_pstore(const struct dirent *de)
> +{
> +int i;
> +
> +for (i = 0; i < prefix_count; i++) {
> +const char *prefix = virtio_pstore_file_prefix[prefix_idx + i];
> +
> +if (g_str_has_prefix(de->d_name, prefix)) {
> +return 1;
> +}
> +}
> +return 0;
> +}
> +
> +static int sort_pstore(const struct dirent **a, const struct dirent **b)
> +{
> +uint64_t id_a, id_b;
> +
> +qemu_strtoull((*a)->d_name + prefix_len, NULL, 0, _a);
> +qemu_strtoull((*b)->d_name + prefix_len, NULL, 0, _b);
> +
> +return id_a - id_b;
> +}
> +
> +static int rotate_pstore_file(VirtIOPstore *s, unsigned short type)

AFAIK you're not actually doing file rotation here - that implies a
fixed base filename, with .0, .1, .2, etc suffixes where we rename
files each time. It looks like you are assuming separate filenames,
and are merely deleting the oldest each time.

> +{
> +int ret = 0;
> +int i, num;
> +char *filename;
> +struct dirent **files;
> +
> +if (type >= ARRAY_SIZE(virtio_pstore_file_prefix)) {
> +type = VIRTIO_PSTORE_TYPE_UNKNOWN;
> +}
> +
> +prefix_idx = type;
> +prefix_len = strlen(virtio_pstore_file_prefix[type]);
> +prefix_count = 1;  /* only scan current type */
> +
> +/* delete the oldest file in the same type */
> +num = scandir(s->directory, , filter_pstore, sort_pstore);
> +if (num < 0)
> +return num;
> +if (num < (int)s->file_max)
> +goto out;
> +
> +filename = g_strdup_printf("%s/%s", s->directory, files[0]->d_name);
> +if (filename == NULL) {
> +ret = -1;
> +goto out;
> +}
> +
> +ret = unlink(filename);





> +static gboolean pstore_async_read_fn(QIOChannel *ioc, GIOCondition condition,
> + gpointer data)
> +{
> +struct pstore_read_arg *rarg = data;
> +struct virtio_pstore_fileinfo *info = >info;
> +VirtIOPstore *vps = rarg->vps;
> +

Re: [Qemu-devel] ppc: regression in 2.7.0-rc3 on rfi/rfid/hrfi instructions

[Mark Cave-Ayland]

On 24/08/16 19:39, Hervé Poussineau wrote:

> Hi,
> 
> Following patch regresses Linux boot on PReP machine:
> 
> commit a2e71b28e832346409efc795ecd1f0a2bcb705a3
> Author: Benjamin Herrenschmidt 
> Date:   Tue Jun 21 23:48:46 2016 +0200
> 
> ppc: Fix rfi/rfid/hrfi/... emulation
> 
> This reworks emulation of the various "rfi" variants. I removed
> some masking bits that I couldn't make sense of, the only bit that
> I am aware we should mask here is POW, the CPU's MSR mask should
> take care of the rest.
> 
> This also fixes some problems when running 32-bit userspace under
> a 64-bit kernel.
> 
> This patch broke 32bit OpenBIOS when run under a 970 cpu. A fix was
> proposed here :
> 
> https://www.coreboot.org/pipermail/openbios/2016-June/009452.html
> 
> Signed-off-by: Benjamin Herrenschmidt 
> Reviewed-by: David Gibson 
> [clg: updated the commit log with the reference of the openbios fix ]
> Signed-off-by: Cédric Le Goater 
> [dwg: Remove hunk which disabled rfi on 64-bit CPUS.  The change was
>  correct, but we need to fix OpenBIOS before applying it]
> Signed-off-by: David Gibson 
> 
> Test case:
> - Download: http://www.juneau-lug.org/zImage.initrd.sandalfoot
> - Run: qemu-system-ppc -M prep -kernel zImage.initrd.sandalfoot
> 
> Firmware starts, loads kernel, but seems to error out just after PS/2
> detection.
> 
> Reverting the commit and fixing the conflict makes it work again, up to
> shell.

I can at least confirm that the v4 patch was applied to OpenBIOS for the
last update:
http://git.qemu.org/?p=openbios.git;a=commit;h=b747b6acc272f6ab839728193042455c9b36e26a.


ATB,

Mark.

Re: [Qemu-devel] [PATCH] aarch64: use TSX for ldrex/strex

[Emilio G. Cota]

On Thu, Aug 18, 2016 at 08:38:47 -0700, Richard Henderson wrote:
> A couple of other notes, as I've thought about this some more.

Thanks for spending time on this.

I have a new patchset (will send as a reply to this e-mail in a few
minutes) that has good performance. Its main ideas:

- Use transactions that start on ldrex and finish on strex. On
  an exception, end (instead of abort) the ongoing transaction,
  if any. There's little point in aborting, since the subsequent
  retries will end up in the same exception anyway. This means
  the translation of the corresponding blocks might happen via
  the fallback path. That's OK, given that subsequent executions
  of the TBs will (likely) complete via HTM.

- For the fallback path, add a stop-the-world primitive that stops
  all other CPUs, without requiring the calling CPU to exit the CPU loop.
  Not breaking from the loop keeps the code simple--we can just
  keep translating/executing normally, with the guarantee that
  no other CPU can run until we're done.

- The fallback path of the transaction stops the world and then
  continues execution (from ldrex) as the only running CPU.

- Only retry when the hardware hints that we may do so. This
  ends up being rare (I can only get dozens of retries under
  heavy contention, for instance with 'atomic_add-bench -r 1')

Limitations: for now user-mode only, and I have paid no attention
to paired atomics. Also, I'm making no checks for unusual (undefined?)
guest code, such as stray ldrex/strex thrown in there.

Performance optimizations like you suggest (e.g. starting a TB
on ldrex, or using TCG ops for beginning/ending the transaction)
could be implemented, but at least on Intel TSX (the only one I've
tried so far[*]), the transaction buffer seems big enough to not
make these optimizations a necessity.

[*] I tried running HTM primitives on the gcc compile farm's Power8,
  but I get an illegal instruction fault on tbegin. I've filed
  an issue here to report it: https://gna.org/support/?3369 ]

Some observations:

- The peak number of retries I see is for atomic_add-bench -r 1 -n 16
  (on an 8-thread machine) at about ~90 retries. So I set the limit
  to 100.

- The lowest success rate I've seen is ~98%, again for atomic_add-bench
  under high contention.

Some numbers:

- atomic_add's performance is lower for HTM vs cmpxchg, although under
  contention performance gets very similar. The reason for the perf
  gap is that xbegin/xend takes more cycles than cmpxchg, especially
  under little or no contention; this explains the large difference
  for threads=1.
  http://imgur.com/5kiT027
  As a side note, contended transactions seem to scale worse than contended
  cmpxchg when exploiting SMT. But anyway I wouldn't read much into
  that.

- For more realistic workloads that gap goes away, as the relative impact
  of cmpxchg or transaction delays is lower. For QHT, 1000 keys:
  http://imgur.com/l6vcowu
  And for SPEC (note that despite being single-threaded, SPEC executes
  a lot of atomics, e.g. from mutexes and from forking):
  http://imgur.com/W49YMhJ
  Performance is essentially identical to that of cmpxchg, but of course
  with HTM we get correct emulation.

Thanks for reading this far!

Emilio

Re: [Qemu-devel] [Patch v3 01/30] qmp: details about CPU definitions in query-cpu-definitions

[David Hildenbrand]

> On 08/24/2016 01:10 PM, David Hildenbrand wrote:
> > It might be of interest for tooling whether a CPU definition can be safely
> > used when migrating, or if e.g. CPU features might get lost during
> > migration when migrationg from/to a different QEMU version or host, even if
> > the same compatibility machine is used.
> > 
> > Also, we want to know if a CPU definition is static and will never change.
> > Beause these definitions can then be used independantly of a compatibility
> > machine and will always have the same feature set, they can e.g. be used
> > to indicate the "host" model in libvirt later on.
> > 
> > Let's add two return values to query-cpu-definitions, stating for each
> > returned CPU definition, if it is migration-safe and if it is static.
> > 
> > While "migration-safe" is optional, "static" will be set to "false"
> > automatically by all implementing architectures. If a model really was
> > static all the time and will be in the future, this can simply be changed
> > later.
> > 
> > Signed-off-by: David Hildenbrand 
> > ---
> >  qapi-schema.json | 14 +-
> >  1 file changed, 13 insertions(+), 1 deletion(-)
> > 
> > diff --git a/qapi-schema.json b/qapi-schema.json
> > index 5658723..0d9ae50 100644
> > --- a/qapi-schema.json
> > +++ b/qapi-schema.json
> > @@ -3038,10 +3038,22 @@
> >  #
> >  # @name: the name of the CPU definition
> >  #
> > +# @migration-safe: #optional whether a CPU definition can be safely used 
> > for
> > +#  migration in combination with a QEMU compatibility 
> > machine
> > +#  when migrating between different QMU versions and 
> > between
> > +#  hosts  with different sets of (hardware or software)  
> 
> Why two spaces after 'hosts'?
> 
> > +#  capabilities. If not provided, information is not 
> > available
> > +#  and callers should not assume the CPU definition to be
> > +#  migration-safe.  
> 
> Missing a '(since 2.8)' designation (at least, I'm assuming this is 2.8
> material).
> 
> > +#
> > +# @static: whether a CPU definition is static and will not change 
> > depending on
> > +#  QEMU version, machine type, machine options and accelerator 
> > options.
> > +#  A static model is always migration-safe.  
> 
> Likewise missing a since tag.
> 
> > +#
> >  # Since: 1.2.0
> >  ##
> >  { 'struct': 'CpuDefinitionInfo',
> > -  'data': { 'name': 'str' } }
> > +  'data': { 'name': 'str', '*migration-safe' : 'bool', 'static' : 'bool' } 
> > }  
> 
> We aren't consistent on whether to use space before ':' or to put it
> flush against the key, but it's at least nice to be consistent within a
> single line.
> 

All valid and fixed!

Thanks for having a look!

David

Re: [Qemu-devel] [Patch v3 01/30] qmp: details about CPU definitions in query-cpu-definitions

[Eric Blake]

On 08/24/2016 01:10 PM, David Hildenbrand wrote:
> It might be of interest for tooling whether a CPU definition can be safely
> used when migrating, or if e.g. CPU features might get lost during
> migration when migrationg from/to a different QEMU version or host, even if
> the same compatibility machine is used.
> 
> Also, we want to know if a CPU definition is static and will never change.
> Beause these definitions can then be used independantly of a compatibility
> machine and will always have the same feature set, they can e.g. be used
> to indicate the "host" model in libvirt later on.
> 
> Let's add two return values to query-cpu-definitions, stating for each
> returned CPU definition, if it is migration-safe and if it is static.
> 
> While "migration-safe" is optional, "static" will be set to "false"
> automatically by all implementing architectures. If a model really was
> static all the time and will be in the future, this can simply be changed
> later.
> 
> Signed-off-by: David Hildenbrand 
> ---
>  qapi-schema.json | 14 +-
>  1 file changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/qapi-schema.json b/qapi-schema.json
> index 5658723..0d9ae50 100644
> --- a/qapi-schema.json
> +++ b/qapi-schema.json
> @@ -3038,10 +3038,22 @@
>  #
>  # @name: the name of the CPU definition
>  #
> +# @migration-safe: #optional whether a CPU definition can be safely used for
> +#  migration in combination with a QEMU compatibility machine
> +#  when migrating between different QMU versions and between
> +#  hosts  with different sets of (hardware or software)

Why two spaces after 'hosts'?

> +#  capabilities. If not provided, information is not 
> available
> +#  and callers should not assume the CPU definition to be
> +#  migration-safe.

Missing a '(since 2.8)' designation (at least, I'm assuming this is 2.8
material).

> +#
> +# @static: whether a CPU definition is static and will not change depending 
> on
> +#  QEMU version, machine type, machine options and accelerator 
> options.
> +#  A static model is always migration-safe.

Likewise missing a since tag.

> +#
>  # Since: 1.2.0
>  ##
>  { 'struct': 'CpuDefinitionInfo',
> -  'data': { 'name': 'str' } }
> +  'data': { 'name': 'str', '*migration-safe' : 'bool', 'static' : 'bool' } }

We aren't consistent on whether to use space before ':' or to put it
flush against the key, but it's at least nice to be consistent within a
single line.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PATCH 3/3] atomics: Remove redundant barrier()'s

[Pranith Kumar]

Remove the redundant barrier() after the fence as agreed in previous
discussion here:
https://lists.gnu.org/archive/html/qemu-devel/2016-04/msg00489.html

Signed-off-by: Pranith Kumar 
---
 include/qemu/atomic.h | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/include/qemu/atomic.h b/include/qemu/atomic.h
index fc2309f..7e557d5 100644
--- a/include/qemu/atomic.h
+++ b/include/qemu/atomic.h
@@ -72,16 +72,16 @@
  * Add one here, and similarly in smp_rmb() and smp_read_barrier_depends().
  */
 
-#define smp_mb()({ barrier(); __atomic_thread_fence(__ATOMIC_SEQ_CST); 
barrier(); })
-#define smp_wmb()   ({ barrier(); __atomic_thread_fence(__ATOMIC_RELEASE); 
barrier(); })
-#define smp_rmb()   ({ barrier(); __atomic_thread_fence(__ATOMIC_ACQUIRE); 
barrier(); })
+#define smp_mb()({ barrier(); __atomic_thread_fence(__ATOMIC_SEQ_CST); })
+#define smp_wmb()   ({ barrier(); __atomic_thread_fence(__ATOMIC_RELEASE); })
+#define smp_rmb()   ({ barrier(); __atomic_thread_fence(__ATOMIC_ACQUIRE); })
 
 /* Most compilers currently treat consume and acquire the same, but really
  * no processors except Alpha need a barrier here.  Leave it in if
  * using Thread Sanitizer to avoid warnings, otherwise optimize it away.
  */
 #if defined(__SANITIZE_THREAD__)
-#define smp_read_barrier_depends() ({ barrier(); 
__atomic_thread_fence(__ATOMIC_CONSUME); barrier(); })
+#define smp_read_barrier_depends() ({ barrier(); 
__atomic_thread_fence(__ATOMIC_CONSUME); })
 #elsif defined(__alpha__)
 #define smp_read_barrier_depends()   asm volatile("mb":::"memory")
 #else
-- 
2.9.3

[Qemu-devel] [PATCH 2/3] atomics: Use __atomic_*_n() variant primitives

[Pranith Kumar]

Use the __atomic_*_n() primitives which take the value as argument. It
is not necessary to store the value locally before calling the
primitive, hence saving us a stack store and load.

Signed-off-by: Pranith Kumar 
---
 include/qemu/atomic.h | 16 +---
 1 file changed, 5 insertions(+), 11 deletions(-)

diff --git a/include/qemu/atomic.h b/include/qemu/atomic.h
index d313e6b..fc2309f 100644
--- a/include/qemu/atomic.h
+++ b/include/qemu/atomic.h
@@ -103,8 +103,7 @@
 
 #define atomic_set(ptr, i)  do {  \
 QEMU_BUILD_BUG_ON(sizeof(*ptr) > sizeof(void *)); \
-typeof(*ptr) _val = (i);  \
-__atomic_store(ptr, &_val, __ATOMIC_RELAXED); \
+__atomic_store_n(ptr, i, __ATOMIC_RELAXED);   \
 } while(0)
 
 /* See above: most compilers currently treat consume and acquire the
@@ -129,8 +128,7 @@
 
 #define atomic_rcu_set(ptr, i) do {   \
 QEMU_BUILD_BUG_ON(sizeof(*ptr) > sizeof(void *)); \
-typeof(*ptr) _val = (i);  \
-__atomic_store(ptr, &_val, __ATOMIC_RELEASE); \
+__atomic_store_n(ptr, i, __ATOMIC_RELEASE);   \
 } while(0)
 
 /* atomic_mb_read/set semantics map Java volatile variables. They are
@@ -153,9 +151,8 @@
 
 #define atomic_mb_set(ptr, i)  do { \
 QEMU_BUILD_BUG_ON(sizeof(*ptr) > sizeof(void *));   \
-typeof(*ptr) _val = (i);\
 smp_wmb();  \
-__atomic_store(ptr, &_val, __ATOMIC_RELAXED);   \
+__atomic_store_n(ptr, i, __ATOMIC_RELAXED); \
 smp_mb();   \
 } while(0)
 #else
@@ -169,8 +166,7 @@
 
 #define atomic_mb_set(ptr, i)  do { \
 QEMU_BUILD_BUG_ON(sizeof(*ptr) > sizeof(void *));   \
-typeof(*ptr) _val = (i);\
-__atomic_store(ptr, &_val, __ATOMIC_SEQ_CST);   \
+__atomic_store_n(ptr, i, __ATOMIC_SEQ_CST); \
 } while(0)
 #endif
 
@@ -179,9 +175,7 @@
 
 #define atomic_xchg(ptr, i)({   \
 QEMU_BUILD_BUG_ON(sizeof(*ptr) > sizeof(void *));   \
-typeof_strip_qual(*ptr) _new = (i), _old;   \
-__atomic_exchange(ptr, &_new, &_old, __ATOMIC_SEQ_CST); \
-_old;   \
+__atomic_exchange_n(ptr, i, __ATOMIC_SEQ_CST);  \
 })
 
 /* Returns the eventual value, failed or not */
-- 
2.9.3

[Qemu-devel] [PATCH 1/3] atomics: Test __STDC_VERSION__ for C11 compat

[Pranith Kumar]

This patch tries to do the Right Thing™ to test for C11 features,
which is to test __STDC_VERSION__.

Signed-off-by: Pranith Kumar 
---
 include/qemu/atomic.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/include/qemu/atomic.h b/include/qemu/atomic.h
index 43b0645..d313e6b 100644
--- a/include/qemu/atomic.h
+++ b/include/qemu/atomic.h
@@ -60,7 +60,7 @@
 (unsigned short)1, 
\
   (expr)+0))
 
-#ifdef __ATOMIC_RELAXED
+#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 201112L)
 /* For C11 atomic ops */
 
 /* Manual memory barriers
@@ -210,7 +210,7 @@
 #define atomic_and(ptr, n) ((void) __atomic_fetch_and(ptr, n, 
__ATOMIC_SEQ_CST))
 #define atomic_or(ptr, n)  ((void) __atomic_fetch_or(ptr, n, __ATOMIC_SEQ_CST))
 
-#else /* __ATOMIC_RELAXED */
+#else /* __STDC_VERSION__ */
 
 /*
  * We use GCC builtin if it's available, as that can use mfence on
@@ -405,5 +405,5 @@
 #define atomic_and(ptr, n) ((void) __sync_fetch_and_and(ptr, n))
 #define atomic_or(ptr, n)  ((void) __sync_fetch_and_or(ptr, n))
 
-#endif /* __ATOMIC_RELAXED */
+#endif /* __STDC_VERSION__ */
 #endif /* QEMU_ATOMIC_H */
-- 
2.9.3

Re: [Qemu-devel] Patch to add support for building in MSYS2 on Windows

[Peter Maydell]

On 24 August 2016 at 20:58, Stefan Weil  wrote:
> Maybe you could start with documenting the MSYS2 based build process on
> http://wiki.qemu.org/Hosts/W32 and refer to a QEMU git clone which includes
> the necessary modifications.
>
> More feedback also from other contributors whether MSYS2 support is wanted
> or not would be helpful.

You could reduce the delta between stock QEMU and one which builds
on MSYS2 by converting some of these "ifdef MSYS2" changes to
instead by "ifdef ". For instance, instead
of guarding a #include of  with "#if defined(__GLIBC__) ||
defined(__MSYS__)"
we could guard it with "#ifdef CONFIG_HAVE_PTY_H".

In general I think that per-host-OS-ifdefs are not a great idea, though they
are the easy expedient way to handle things and so we have a fair
number at the moment. Converting a per-host-ifdef setup to a "check for
existence of feature" setup is a cleanup I think we'd be happy to
accept even if we don't want to care about MSYS2 build support.

thanks
-- PMM

[Qemu-devel] transient failure in the test-qht tests

[Peter Maydell]

So I encountered this test failure running 'make check' on
32-bit ARM:

MALLOC_PERTURB_=${MALLOC_PERTURB_:-$((RANDOM % 255 + 1))} gtester -k
--verbose -m=quick tests/test-qht
TEST: tests/test-qht... (pid=15763)
  /qht/mode/default:   OK
  /qht/mode/resize:FAIL
GTester: last random seed: R02S08efd89fe4d862dd0191c13d5ce4d76e
(pid=16462)
FAIL: tests/test-qht

The test suite passed on a rerun.

Any ideas?

thanks
-- PMM

Re: [Qemu-devel] [PATCH v2 0/8] Improve buffer_is_zero

[Richard Henderson]

On 08/24/2016 12:18 PM, Eric Blake wrote:

On 08/24/2016 12:48 PM, Richard Henderson wrote:

Patches 1-4 remove the use of ifunc from the implementation.

Patch 6 adjusts the x86 implementation a bit more to take
advantage of ptest (in sse4.1) and unaligned accesses (in avx1).


Do we really care about unaligned access?  Or can we guarantee that all
our calls to buffer_is_zero are already aligned, and make optimizations
along those lines?


The old code asserted alignment of at least sizeof(long), although a survey of 
call sites doesn't make this obvious.  I could imagine that we get alignment 
consistent with that of malloc, but can't prove it.


However, we're certainly not going to be able to assert arbitrary alignment, 
such as the 32-byte for AVX2, or the 64-byte for AVX512 (when that comes along).


Thankfully, at least AVX capable cpus are very efficient with unaligned 
accesses.


r~

Re: [Qemu-devel] Patch to add support for building in MSYS2 on Windows

[Stefan Weil]

On 08/24/16 09:44, Evan Hunter wrote:

Hi,

I noticed that QEMU would not compile on Windows in the MSYS2 environment.
Here is a patch that adds a check for it to the configure script and
amended some ifdefs so it now builds.

At the bottom of this email is a build script I made for building under
MSYS2, however I am
not sure if it is suitable merging nor where it should go.

Regards,

Evan Hunter


Hi Evan,

I'm not sure whether adding support for MSYS2 builds is a good idea or 
not. Supporting several build environments for QEMU for Windows requires 
a lot of time. Currently we only support MinGW-w64. It has several 
advantages:


* 32 and 64 bit builds supported
* cross builds on Linux (Redhat, Debian, others) supported
* cross builds on Windows with Cygwin supported
* native builds on Windows supported

AFAIK Redhat includes all packages needed for the cross builds.
Cygwin also has excellent package support.
Debian only includes the build tools, but can use Cygwin packages.

In most cases it is sufficient to test QEMU builds with MinGW-w64 on one 
platform, and all others will work, too. Usually QEMU developers will 
run their tests using Linux (me too).


For MSYS2, we'd need additional build tests running on Windows (please 
correct me if I am wrong - I have no own experience with MSYS2). What 
would be the advantages of supporting MSYS2?


Maybe you could start with documenting the MSYS2 based build process on 
http://wiki.qemu.org/Hosts/W32 and refer to a QEMU git clone which 
includes the necessary modifications.


More feedback also from other contributors whether MSYS2 support is 
wanted or not would be helpful.


Kind regards,
Stefan Weil

Re: [Qemu-devel] [Patch v3 27/30] qmp: add QMP interface "query-cpu-model-baseline"

[Eduardo Habkost]

On Wed, Aug 24, 2016 at 08:11:03PM +0200, David Hildenbrand wrote:
> Let's provide a standardized interface to baseline two CPU models, to
> create a third, compatible one. This is especially helpful when two
> CPU models are not identical, but a CPU model is required that is
> guaranteed to run under both configurations, where the original models run.
> 
> "query-cpu-model-baseline" takes two CPU models and returns a third,
> compatible model. The result will always be a static CPU model.
> 
> Acked-by: Cornelia Huck 
> Signed-off-by: David Hildenbrand 

Reviewed-by: Eduardo Habkost 

-- 
Eduardo

Re: [Qemu-devel] [Patch v3 26/30] qmp: add QMP interface "query-cpu-model-comparison"

[Eduardo Habkost]

On Wed, Aug 24, 2016 at 08:11:02PM +0200, David Hildenbrand wrote:
> Let's provide a standardized interface to compare two CPU models.
> "query-cpu-model-compare" takes two models and returns  how they compare
> in a specific configuration.
> 
> The result will give guarantees about runnability. E.g. if a CPU model A
> is a subset of CPU model B, model A is guaranteed to run in configurations
> where model B runs, but not the other way around (might or might not run).
> 
> Usually, CPU features or CPU generations are used to calculate the result.
> If a model is not guaranteed to run in a certain environment (e.g.
> incompatible), a  compatible one can be created by "baselining" both models
> (follow up patch).
> 
> Acked-by: Cornelia Huck 
> Signed-off-by: David Hildenbrand 

Reviewed-by: Eduardo Habkost 

-- 
Eduardo

Re: [Qemu-devel] [Patch v3 25/30] qmp: add QMP interface "query-cpu-model-expansion"

[Eduardo Habkost]

On Wed, Aug 24, 2016 at 08:11:01PM +0200, David Hildenbrand wrote:
> Let's provide a standardized interface to expand CPU models. This interface
> can be used by tooling to get details about a specific CPU model in a
> certain configuration, e.g. about the "host" model.
> 
> To take care of all architectures, two detail levels for an expansion
> are introduced. Certain architectures might not support all detail levels.
> While "full" will expand and indicate all relevant properties/features
> of a CPU model, "static" expands to a static base CPU model, that will
> never change between QEMU versions and therefore have the same features
> when used under different compatibility machines.
> 
> Acked-by: Cornelia Huck 
> Signed-off-by: David Hildenbrand 

Reviewed-by: Eduardo Habkost 

-- 
Eduardo

Re: [Qemu-devel] [PATCH] 9pfs: disallow / in path components

[Peter Maydell]

On 24 August 2016 at 17:40, Greg Kurz  wrote:
> On Wed, 24 Aug 2016 16:00:24 +0100
> Peter Maydell  wrote:
>> Do we also need ".." and "." to be illegal names (for at least most
>> operations)?

> I understand how ".." could be an issue, but I don't for "."... can you
> please elaborate ?

If you try to create, open, etc a file named "." then it won't create a
file named "."; it'll do something wrong instead. So we shouldn't
permit attempts to treat "." as an ordinary filename.

(Basically the rationale is that for Linux the constraints on
file and pathnames are only
 * no NULs
 * no /
 * "." is special
 * ".." is special
 * can't be the empty string
We should reflect that in our error checking.)

> The spec says:
>
>   Directories are created by create with DMDIR set in the per-
>   missions argument (see stat(5)). The members of a directory
>   can be found with read(5). All directories must support
>   walks to the directory .. (dot-dot) meaning parent direc-
>   tory, although by convention directories contain no explicit
>   entry for .. or . (dot).  The parent of the root directory
>   of a server's tree is itself.

Yes, walk is the one special case that needs to handle "." and ".."
(because for this operation they have a defined meaning that doesn't
mean "just pass them through to the libc functions").

> So I don't think we should boldly make ".." an illegal name, but
> rather ignore it. Pretty much like doing chdir("..") when the current
> directory is /.
>
> All operations except walk take an existing fid and a single path component.
> A possible fix would be to convert ".." to "" when the fid points to the root
> of the export path. Makes sense ?

What did you want the empty string to mean? (We should probably
also define the empty string as an illegal name).

thanks
-- PMM

Re: [Qemu-devel] [PATCH v2 0/8] Improve buffer_is_zero

[Eric Blake]

On 08/24/2016 12:48 PM, Richard Henderson wrote:
> Patches 1-4 remove the use of ifunc from the implementation.
> 
> Patch 6 adjusts the x86 implementation a bit more to take
> advantage of ptest (in sse4.1) and unaligned accesses (in avx1).

Do we really care about unaligned access?  Or can we guarantee that all
our calls to buffer_is_zero are already aligned, and make optimizations
along those lines?



-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PULL 0/2] virtio: fixes

[Peter Maydell]

On 23 August 2016 at 17:34, Michael S. Tsirkin  wrote:
> The following changes since commit 60ac136102098a70b97ab0c07bc7bf53131c:
>
>   target-arm: Fix warn about implicit conversion (2016-08-12 11:12:24 +0100)
>
> are available in the git repository at:
>
>   git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream
>
> for you to fetch changes up to 58a83c61496eeb0d31571a07a51bc1947e3379ac:
>
>   virtio: decrement vq->inuse in virtqueue_discard() (2016-08-23 19:20:24 
> +0300)
>
> 
> virtio: fixes
>
> some bugfixes for virtio
> balloon is still broken wrt migration

Is this a bug we need to fix for the release?


> Signed-off-by: Michael S. Tsirkin 
>
> 
> Stefan Hajnoczi (2):
>   virtio: recalculate vq->inuse after migration
>   virtio: decrement vq->inuse in virtqueue_discard()
>
>  hw/virtio/virtio.c | 16 
>  1 file changed, 16 insertions(+)

Applied to master, thanks.

-- PMM

Re: [Qemu-devel] [PULL 2/2] e1000e: remove internal interrupt flag

[Amit Shah]

On (Tue) 23 Aug 2016 [23:12:21], Markus Armbruster wrote:
> Amit Shah  writes:
> 
> > On (Mon) 22 Aug 2016 [16:09:27], Jason Wang wrote:
> >> From: Cao jin 
> >> 
> >> Commit 66bf7d58 removed internal msi state flag E1000E_USE_MSI, 
> >> E1000E_USE_MSIX
> >> is not necessary too, remove it now. And interrupt flag field intr_state 
> >> also
> >> can be removed now.
> >> 
> >> CC: Dmitry Fleytman 
> >> CC: Jason Wang 
> >> CC: Markus Armbruster 
> >> CC: Marcel Apfelbaum 
> >> CC: Michael S. Tsirkin 
> >> CC: Paolo Bonzini 
> >> Signed-off-by: Cao jin 
> >> Reviewed-by: Markus Armbruster 
> >> Acked-by: Dmitry Fleytman 
> >> Reviewed-by: Paolo Bonzini 
> >> Signed-off-by: Jason Wang 
> >> ---
> >>  hw/net/e1000e.c | 8 +---
> >>  1 file changed, 1 insertion(+), 7 deletions(-)
> >
> >
> >> @@ -601,7 +596,6 @@ static const VMStateDescription e1000e_vmstate = {
> >>  VMSTATE_MSIX(parent_obj, E1000EState),
> >>  
> >>  VMSTATE_UINT32(ioaddr, E1000EState),
> >> -VMSTATE_UINT32(intr_state, E1000EState),
> >>  VMSTATE_UINT32(core.rxbuf_min_shift, E1000EState),
> >>  VMSTATE_UINT8(core.rx_desc_len, E1000EState),
> >>  VMSTATE_UINT32_ARRAY(core.rxbuf_sizes, E1000EState,
> >
> > This breaks migration - please mark that field 'unused'.
> 
> Does it matter?  The device hasn't been in any released version of
> QEMU...

Oh, OK then.  The nightly script doesn't know about it and I just
forward its objections to the list.

[Qemu-devel] ppc: regression in 2.7.0-rc3 on rfi/rfid/hrfi instructions

[Hervé Poussineau]

Hi,

Following patch regresses Linux boot on PReP machine:

commit a2e71b28e832346409efc795ecd1f0a2bcb705a3
Author: Benjamin Herrenschmidt 
Date:   Tue Jun 21 23:48:46 2016 +0200

ppc: Fix rfi/rfid/hrfi/... emulation

This reworks emulation of the various "rfi" variants. I removed
some masking bits that I couldn't make sense of, the only bit that
I am aware we should mask here is POW, the CPU's MSR mask should
take care of the rest.

This also fixes some problems when running 32-bit userspace under
a 64-bit kernel.

This patch broke 32bit OpenBIOS when run under a 970 cpu. A fix was
proposed here :

https://www.coreboot.org/pipermail/openbios/2016-June/009452.html

Signed-off-by: Benjamin Herrenschmidt 
Reviewed-by: David Gibson 
[clg: updated the commit log with the reference of the openbios fix ]
Signed-off-by: Cédric Le Goater 
[dwg: Remove hunk which disabled rfi on 64-bit CPUS.  The change was
 correct, but we need to fix OpenBIOS before applying it]
Signed-off-by: David Gibson 

Test case:
- Download: http://www.juneau-lug.org/zImage.initrd.sandalfoot
- Run: qemu-system-ppc -M prep -kernel zImage.initrd.sandalfoot

Firmware starts, loads kernel, but seems to error out just after PS/2 detection.

Reverting the commit and fixing the conflict makes it work again, up to shell.

Hervé

[Qemu-devel] [Patch v3 27/30] qmp: add QMP interface "query-cpu-model-baseline"

[David Hildenbrand]

Let's provide a standardized interface to baseline two CPU models, to
create a third, compatible one. This is especially helpful when two
CPU models are not identical, but a CPU model is required that is
guaranteed to run under both configurations, where the original models run.

"query-cpu-model-baseline" takes two CPU models and returns a third,
compatible model. The result will always be a static CPU model.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 include/sysemu/arch_init.h|  3 +++
 qapi-schema.json  | 51 +++
 qmp-commands.hx   |  6 +
 qmp.c |  7 +
 stubs/Makefile.objs   |  1 +
 stubs/arch-query-cpu-model-baseline.c | 12 +
 6 files changed, 80 insertions(+)
 create mode 100644 stubs/arch-query-cpu-model-baseline.c

diff --git a/include/sysemu/arch_init.h b/include/sysemu/arch_init.h
index 96d47c0..1c9dad1 100644
--- a/include/sysemu/arch_init.h
+++ b/include/sysemu/arch_init.h
@@ -41,5 +41,8 @@ CpuModelExpansionInfo 
*arch_query_cpu_model_expansion(CpuModelExpansionType type
 CpuModelCompareInfo *arch_query_cpu_model_comparison(CpuModelInfo *modela,
  CpuModelInfo *modelb,
  Error **errp);
+CpuModelBaselineInfo *arch_query_cpu_model_baseline(CpuModelInfo *modela,
+CpuModelInfo *modelb,
+Error **errp);
 
 #endif
diff --git a/qapi-schema.json b/qapi-schema.json
index 4f1b9ec..7f469d0 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3244,6 +3244,57 @@
   'data': { 'modela': 'CpuModelInfo', 'modelb': 'CpuModelInfo' },
   'returns': 'CpuModelCompareInfo' }
 
+##
+# @CpuModelBaselineInfo
+#
+# The result of a CPU model baseline.
+#
+# @model: the baselined CpuModelInfo.
+#
+# Since: 2.8.0
+##
+{ 'struct': 'CpuModelBaselineInfo',
+  'data': { 'model': 'CpuModelInfo' } }
+
+##
+# @query-cpu-model-baseline:
+#
+# Baseline two CPU models, creating a compatible third model. The created
+# model will always be a static, migration-safe CPU model (see "static"
+# CPU model expansion for details).
+#
+# This interface can be used by tooling to create a compatible CPU model out
+# two CPU models. The created CPU model will be identical to or a subset of
+# both CPU models when comparing them. Therefore, the created CPU model is
+# guaranteed to run where the given CPU models run.
+#
+# The result returned by this command may be affected by:
+#
+# * QEMU version: CPU models may look different depending on the QEMU version.
+#   (Except for CPU models reported as "static" in query-cpu-definitions.)
+# * machine-type: CPU model  may look different depending on the machine-type.
+#   (Except for CPU models reported as "static" in query-cpu-definitions.)
+# * machine options (including accelerator): in some architectures, CPU models
+#   may look different depending on machine and accelerator options. (Except 
for
+#   CPU models reported as "static" in query-cpu-definitions.)
+# * "-cpu" arguments and global properties: arguments to the -cpu option and
+#   global properties may affect expansion of CPU models. Using
+#   query-cpu-model-expansion while using these is not advised.
+#
+# Some architectures may not support baselining CPU models.
+#
+# Returns: a CpuModelBaselineInfo. Returns an error if baselining CPU models is
+#  not supported, if a model cannot be used, if a model contains
+#  an unknown cpu definition name, unknown properties or properties
+#  with wrong types.
+#
+# Since: 2.8.0
+##
+{ 'command': 'query-cpu-model-baseline',
+  'data': { 'modela': 'CpuModelInfo',
+'modelb': 'CpuModelInfo' },
+  'returns': 'CpuModelBaselineInfo' }
+
 # @AddfdInfo:
 #
 # Information about a file descriptor that was added to an fd set.
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 70ad681..5c8d1d5 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -3954,6 +3954,12 @@ EQMP
 },
 
 {
+.name   = "query-cpu-model-baseline",
+.args_type  = "modela:q,modelb:q",
+.mhandler.cmd_new = qmp_marshal_query_cpu_model_baseline,
+},
+
+{
 .name   = "query-target",
 .args_type  = "",
 .mhandler.cmd_new = qmp_marshal_query_target,
diff --git a/qmp.c b/qmp.c
index f551019..dea8f81 100644
--- a/qmp.c
+++ b/qmp.c
@@ -621,6 +621,13 @@ CpuModelCompareInfo 
*qmp_query_cpu_model_comparison(CpuModelInfo *modela,
 return arch_query_cpu_model_comparison(modela, modelb, errp);
 }
 
+CpuModelBaselineInfo *qmp_query_cpu_model_baseline(CpuModelInfo *modela,
+   CpuModelInfo *modelb,
+   Error **errp)
+{
+

Re: [Qemu-devel] [Patch v3 00/30] s390x CPU models: exposing features

[no-reply]

Hi,

Your series seems to have some coding style problems. See output below for
more information:

Message-id: 1472062266-53206-1-git-send-email-d...@linux.vnet.ibm.com
Subject: [Qemu-devel] [Patch v3 00/30] s390x CPU models: exposing features
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

# Useful git options
git config --local diff.renamelimit 0
git config --local diff.renames True

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
echo "Checking PATCH $n/$total: $(git show --no-patch --format=%s $c)..."
if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
failed=1
echo
fi
n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag] 
patchew/1472062266-53206-1-git-send-email-d...@linux.vnet.ibm.com -> 
patchew/1472062266-53206-1-git-send-email-d...@linux.vnet.ibm.com
Switched to a new branch 'test'
421b6e1 s390x/cpumodel: implement QMP interface "query-cpu-model-baseline"
42e8151 s390x/cpumodel: implement QMP interface "query-cpu-model-comparison"
efcb308 s390x/cpumodel: implement QMP interface "query-cpu-model-expansion"
a982bbe qmp: add QMP interface "query-cpu-model-baseline"
29d4b50 qmp: add QMP interface "query-cpu-model-comparison"
df10307 qmp: add QMP interface "query-cpu-model-expansion"
c7b6934 s390x/kvm: don't enable key wrapping if msa3 is disabled
3760e3c s390x/kvm: let the CPU model control CMM(A)
2590fcd s390x/kvm: disable host model for problematic compat machines
51c1b5f s390x/kvm: implement CPU model support
ba9852e s390x/kvm: allow runtime-instrumentation for "none" machine
626f7ad linux-headers: update against kvm/next
a31ce26 s390x/sclp: propagate hmfai
c487957 s390x/sclp: propagate the mha via sclp
cdd22b8 s390x/sclp: propagate the ibc val(lowest and unblocked ibc)
5c83927 s390x/sclp: indicate sclp features
b6d817e s390x/sclp: introduce sclp feature blocks
a7124ab s390x/sclp: factor out preparation of cpu entries
59cb0b0 s390x/cpumodel: check and apply the CPU model
aeac742 s390x/cpumodel: let the CPU model handle feature checks
0c18ada s390x/cpumodel: expose features and feature groups as properties
106a6eb s390x/cpumodel: store the CPU model in the CPU instance
9cf5a9c s390x/cpumodel: register defined CPU models as subclasses
8cd2e12 s390x/cpumodel: introduce CPU feature group definitions
6fdcaf6 s390x/cpumodel: generate CPU feature group lists
24695d8 s390x/cpumodel: generate CPU feature lists for CPU models
af645fe s390x/cpumodel: introduce CPU features
1ea1411 s390x/cpumodel: expose CPU class properties
de1d555 s390x/cpumodel: "host" and "qemu" as CPU subclasses
a061875 qmp: details about CPU definitions in query-cpu-definitions

=== OUTPUT BEGIN ===
Checking PATCH 1/30: qmp: details about CPU definitions in 
query-cpu-definitions...
Checking PATCH 2/30: s390x/cpumodel: "host" and "qemu" as CPU subclasses...
Checking PATCH 3/30: s390x/cpumodel: expose CPU class properties...
Checking PATCH 4/30: s390x/cpumodel: introduce CPU features...
WARNING: line over 80 characters
#65: FILE: target-s390x/cpu_features.c:27:
+FEAT_INIT("zarch", S390_FEAT_TYPE_STFL, 1, "z/Architecture architectural 
mode"),

ERROR: line over 90 characters
#67: FILE: target-s390x/cpu_features.c:29:
+FEAT_INIT("idtes", S390_FEAT_TYPE_STFL, 4, "IDTE selective TLB 
segment-table clearing"),

ERROR: line over 90 characters
#68: FILE: target-s390x/cpu_features.c:30:
+FEAT_INIT("idter", S390_FEAT_TYPE_STFL, 5, "IDTE selective TLB 
region-table clearing"),

WARNING: line over 80 characters
#70: FILE: target-s390x/cpu_features.c:32:
+FEAT_INIT("stfle", S390_FEAT_TYPE_STFL, 7, "Store-facility-list-extended 
facility"),

WARNING: line over 80 characters
#74: FILE: target-s390x/cpu_features.c:36:
+FEAT_INIT("ctop", S390_FEAT_TYPE_STFL, 11, "Configuration-topology 
facility"),

WARNING: line over 80 characters
#76: FILE: target-s390x/cpu_features.c:38:
+FEAT_INIT("nonqks", S390_FEAT_TYPE_STFL, 14, "Nonquiescing key-setting 
facility"),

WARNING: line over 80 characters
#77: FILE: target-s390x/cpu_features.c:39:
+FEAT_INIT("etf2", S390_FEAT_TYPE_STFL, 16, "Extended-translation facility 
2"),

ERROR: line over 90 characters
#78: FILE: target-s390x/cpu_features.c:40:
+FEAT_INIT("msa-base", S390_FEAT_TYPE_STFL, 17, "Message-security-assist 
facility (excluding subfunctions)"),

ERROR: line over 90 characters
#80: FILE: target-s390x/cpu_features.c:42:
+FEAT_INIT("ldisphp", S390_FEAT_TYPE_STFL, 19, "Long-displacement facility 
has high performance"),

WARNING: line over 80 characters
#81: FILE: target-s390x/cpu_features.c:43:
+FEAT_INIT("hfpm", S390_FEAT_TYPE_STFL, 20, "HFP-multiply-add/subtract 
facility"),

WARNING: line over 80 characters
#83: FILE: target-s390x/cpu_features.c:45:
+FEAT_INIT("etf3", S390_FEAT_TYPE_STFL, 22, 

[Qemu-devel] [PATCH v2 5/7] docker: make sure debootstrap is at least 1.0.67

[Sascha Silbe]

debootstrap prior to 1.0.67 generated an empty sources.list during
foreign bootstraps (Debian#732255 [1]). Fall back to the git checkout
if the installed debootstrap version is too old.

[1] https://bugs.debian.org/732255

Signed-off-by: Sascha Silbe 
---
Not sure if this used to work in even older debootstrap versions; I
don't remember running into this before. But then I needed to set up
several config files manually after a debootstrap anyway (both foreign
and native), so sources.list might just have been another one to fix
up manually.

 tests/docker/dockerfiles/debian-bootstrap.pre | 9 +
 1 file changed, 9 insertions(+)

diff --git a/tests/docker/dockerfiles/debian-bootstrap.pre 
b/tests/docker/dockerfiles/debian-bootstrap.pre
index 2ae363f..3c3e781 100755
--- a/tests/docker/dockerfiles/debian-bootstrap.pre
+++ b/tests/docker/dockerfiles/debian-bootstrap.pre
@@ -3,6 +3,8 @@
 # Simple wrapper for debootstrap, run in the docker build context
 #
 FAKEROOT=`which fakeroot 2> /dev/null`
+# debootstrap < 1.0.67 generates empty sources.list, see Debian#732255
+MIN_DEBOOTSTRAP_VERSION=1.0.67
 
 exit_and_skip()
 {
@@ -40,9 +42,16 @@ fi
 #
 
 if [ -z $DEBOOTSTRAP_DIR ]; then
+NEED_DEBOOTSTRAP=false
 DEBOOTSTRAP=`which debootstrap 2> /dev/null`
 if [ -z $DEBOOTSTRAP ]; then
 echo "No debootstrap installed, attempting to install from SCM"
+NEED_DEBOOTSTRAP=true
+elif ! (echo "${MIN_DEBOOTSTRAP_VERSION}" ; "${DEBOOTSTRAP}" --version 
|cut -d ' ' -f 2) |sort -VC; then
+echo "debootstrap too old, attempting to install from SCM"
+NEED_DEBOOTSTRAP=true
+fi
+if $NEED_DEBOOTSTRAP; then
 DEBOOTSTRAP_SOURCE=https://anonscm.debian.org/git/d-i/debootstrap.git
 git clone ${DEBOOTSTRAP_SOURCE} ./debootstrap.git
 export DEBOOTSTRAP_DIR=./debootstrap.git
-- 
1.9.1

[Qemu-devel] [PATCH v2 2/7] docker: avoid dependency on 'realpath' package

[Sascha Silbe]

The 'realpath' executable is shipped in a separate package that isn't
installed by default on some distros.

We already use 'readlink -e' (provided by GNU coreutils) in some other
part of the code, so let's settle for that instead.

Signed-off-by: Sascha Silbe 
---
Too bad there isn't a POSIX equivalent of this.

 tests/docker/Makefile.include | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 4f4707d..1b20db0 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -116,7 +116,7 @@ docker-run-%: docker-qemu-src
-e EXTRA_CONFIGURE_OPTS=$(EXTRA_CONFIGURE_OPTS) 
\
-e V=$V -e J=$J -e DEBUG=$(DEBUG)\
-e CCACHE_DIR=/var/tmp/ccache \
-   -v $$(realpath 
$(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
+   -v $$(readlink -e 
$(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \
qemu:$(IMAGE) \
/var/tmp/qemu/run \
-- 
1.9.1

[Qemu-devel] [PATCH v2 6/7] docker: build debootstrap after cloning

[Sascha Silbe]

When using the git version of debootstrap (because no usable version
of debootstrap was installed on the host), we need to run 'make' so
that devices.tar.gz gets built. Otherwise the first debootstrap stage
will fail without printing any error message.

Signed-off-by: Sascha Silbe 
---
 tests/docker/dockerfiles/debian-bootstrap.pre | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/docker/dockerfiles/debian-bootstrap.pre 
b/tests/docker/dockerfiles/debian-bootstrap.pre
index 3c3e781..a633224 100755
--- a/tests/docker/dockerfiles/debian-bootstrap.pre
+++ b/tests/docker/dockerfiles/debian-bootstrap.pre
@@ -56,6 +56,7 @@ if [ -z $DEBOOTSTRAP_DIR ]; then
 git clone ${DEBOOTSTRAP_SOURCE} ./debootstrap.git
 export DEBOOTSTRAP_DIR=./debootstrap.git
 DEBOOTSTRAP=./debootstrap.git/debootstrap
+(cd "${DEBOOTSTRAP_DIR}" && "${FAKEROOT}" make )
 fi
 else
 DEBOOTSTRAP=${DEBOOTSTRAP_DIR}/debootstrap
-- 
1.9.1

[Qemu-devel] [Patch v3 17/30] s390x/sclp: propagate the mha via sclp

[David Hildenbrand]

The mha is provided in the CPU model, so get any CPU and extract the value.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 hw/s390x/sclp.c   |  1 +
 include/hw/s390x/sclp.h   |  3 ++-
 target-s390x/cpu_models.c | 14 ++
 target-s390x/cpu_models.h |  1 +
 4 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index 52f8bb9..5ffcf51 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -105,6 +105,7 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
 
 read_info->facilities |= cpu_to_be64(SCLP_FC_ASSIGN_ATTACH_READ_STOR);
 }
+read_info->mha_pow = s390_get_mha_pow();
 
 rnsize = 1 << (sclp->increment_size - 20);
 if (rnsize <= 128) {
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index 664be9b..dab3c0f 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -124,7 +124,8 @@ typedef struct ReadInfo {
 uint8_t  _reserved0[76 - 56];   /* 56-75 */
 uint32_t ibc_val;
 uint8_t  conf_char[96 - 80];/* 80-95 */
-uint8_t  _reserved4[100 - 96];  /* 96-99 */
+uint8_t  _reserved4[99 - 96];   /* 96-98 */
+uint8_t mha_pow;
 uint32_t rnsize2;
 uint64_t rnmax2;
 uint8_t  _reserved6[116 - 112]; /* 112-115 */
diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index 70337b8..bda67a5 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -74,6 +74,20 @@ static const S390CPUDef s390_cpu_defs[] = {
 CPUDEF_INIT(0x2965, 13, 2, 47, 0x0800U, "z13s", "IBM z13s GA1"),
 };
 
+uint8_t s390_get_mha_pow(void)
+{
+static S390CPU *cpu;
+
+if (!cpu) {
+cpu = S390_CPU(qemu_get_cpu(0));
+}
+
+if (!cpu || !cpu->model) {
+return 0;
+}
+return cpu->model->def->mha_pow;
+}
+
 uint32_t s390_get_ibc_val(void)
 {
 uint16_t unblocked_ibc, lowest_ibc;
diff --git a/target-s390x/cpu_models.h b/target-s390x/cpu_models.h
index bbb85ac..ee019b4 100644
--- a/target-s390x/cpu_models.h
+++ b/target-s390x/cpu_models.h
@@ -45,6 +45,7 @@ typedef struct S390CPUModel {
 
 #define S390_GEN_Z10 0xa
 
+uint8_t s390_get_mha_pow(void);
 uint32_t s390_get_ibc_val(void);
 static inline uint16_t s390_ibc_from_cpu_model(const S390CPUModel *model)
 {
-- 
2.6.6

[Qemu-devel] [PATCH v2 1/7] docker.py: don't hang on large docker output

[Sascha Silbe]

Unlike Popen.communicate(), subprocess.call() doesn't read from the
stdout file descriptor. If the child process produces more output than
fits into the pipe buffer, it will block indefinitely.

If we don't intend to consume the output, just send it straight to
/dev/null to avoid this issue.

Signed-off-by: Sascha Silbe 
Reviewed-by: Janosch Frank 
---
This fixes a hang for me when building the Ubuntu docker image (empty
docker image cache).

 tests/docker/docker.py | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 222a105..efb2bf4 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -25,6 +25,10 @@ from tarfile import TarFile, TarInfo
 from StringIO import StringIO
 from shutil import copy, rmtree
 
+
+DEVNULL = open(os.devnull, 'wb')
+
+
 def _text_checksum(text):
 """Calculate a digest string unique to the text content"""
 return hashlib.sha1(text).hexdigest()
@@ -34,8 +38,7 @@ def _guess_docker_command():
 commands = [["docker"], ["sudo", "-n", "docker"]]
 for cmd in commands:
 if subprocess.call(cmd + ["images"],
-   stdout=subprocess.PIPE,
-   stderr=subprocess.PIPE) == 0:
+   stdout=DEVNULL, stderr=DEVNULL) == 0:
 return cmd
 commands_txt = "\n".join(["  " + " ".join(x) for x in commands])
 raise Exception("Cannot find working docker command. Tried:\n%s" % \
@@ -98,7 +101,7 @@ class Docker(object):
 
 def _do(self, cmd, quiet=True, infile=None, **kwargs):
 if quiet:
-kwargs["stdout"] = subprocess.PIPE
+kwargs["stdout"] = DEVNULL
 if infile:
 kwargs["stdin"] = infile
 return subprocess.call(self._command + cmd, **kwargs)
-- 
1.9.1

[Qemu-devel] [Patch v3 09/30] s390x/cpumodel: store the CPU model in the CPU instance

[David Hildenbrand]

A CPU model consists of a CPU definition, to which delta changes are
applied - features added or removed (e.g. z13-base,vx=on). In addition,
certain properties (e.g. cpu id) can later on change during migration
but belong into the CPU model. This data will later be filled from the
host model in the KVM case.

Therefore, store the configured CPU model inside the CPU instance, so
we can later on perform delta changes using properties.

For the "qemu" model, we emulate in TCG a z900. "host" will be
uninitialized (cpu->model == NULL) unless we have CPU model support in KVM
later on. The other models are all initialized from their definitions.
Only the "host" model can have a cpu->model == NULL.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 target-s390x/cpu.h|  1 +
 target-s390x/cpu_models.c | 26 ++
 target-s390x/cpu_models.h | 10 ++
 3 files changed, 37 insertions(+)

diff --git a/target-s390x/cpu.h b/target-s390x/cpu.h
index 075bb37..3b76654 100644
--- a/target-s390x/cpu.h
+++ b/target-s390x/cpu.h
@@ -188,6 +188,7 @@ struct S390CPU {
 
 CPUS390XState env;
 int64_t id;
+S390CPUModel *model;
 /* needed for live migration */
 void *irqstate;
 uint32_t irqstate_saved_size;
diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index b59765c..aaf53f6 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -153,6 +153,21 @@ void s390_realize_cpu_model(CPUState *cs, Error **errp)
 
 static void s390_cpu_model_initfn(Object *obj)
 {
+S390CPU *cpu = S390_CPU(obj);
+S390CPUClass *xcc = S390_CPU_GET_CLASS(cpu);
+
+cpu->model = g_malloc0(sizeof(*cpu->model));
+/* copy the model, so we can modify it */
+cpu->model->def = xcc->cpu_def;
+if (xcc->is_static) {
+/* base model - features will never change */
+bitmap_copy(cpu->model->features, cpu->model->def->base_feat,
+S390_FEAT_MAX);
+} else {
+/* latest model - features can change */
+bitmap_copy(cpu->model->features,
+cpu->model->def->default_feat, S390_FEAT_MAX);
+}
 }
 
 #ifdef CONFIG_KVM
@@ -163,10 +178,21 @@ static void s390_host_cpu_model_initfn(Object *obj)
 
 static void s390_qemu_cpu_model_initfn(Object *obj)
 {
+S390CPU *cpu = S390_CPU(obj);
+
+cpu->model = g_malloc0(sizeof(*cpu->model));
+/* TCG emulates a z900 */
+cpu->model->def = _cpu_defs[0];
+bitmap_copy(cpu->model->features, cpu->model->def->default_feat,
+S390_FEAT_MAX);
 }
 
 static void s390_cpu_model_finalize(Object *obj)
 {
+S390CPU *cpu = S390_CPU(obj);
+
+g_free(cpu->model);
+cpu->model = NULL;
 }
 
 static bool get_is_migration_safe(Object *obj, Error **errp)
diff --git a/target-s390x/cpu_models.h b/target-s390x/cpu_models.h
index 13f7217..244256b 100644
--- a/target-s390x/cpu_models.h
+++ b/target-s390x/cpu_models.h
@@ -33,4 +33,14 @@ typedef struct S390CPUDef {
 S390FeatBitmap full_feat;
 } S390CPUDef;
 
+/* CPU model based on a CPU definition */
+typedef struct S390CPUModel {
+const S390CPUDef *def;
+S390FeatBitmap features;
+/* values copied from the "host" model, can change during migration */
+uint16_t lowest_ibc;/* lowest IBC that the hardware supports */
+uint32_t cpu_id;/* CPU id */
+uint8_t cpu_ver;/* CPU version, usually "ff" for kvm */
+} S390CPUModel;
+
 #endif /* TARGET_S390X_CPU_MODELS_H */
-- 
2.6.6

Re: [Qemu-devel] [PATCH] 9pfs: disallow / in path components

[Michael S. Tsirkin]

On Wed, Aug 24, 2016 at 04:29:07PM +0200, Greg Kurz wrote:
> At various places in 9pfs, full paths are created by concatenating a guest
> originated string to the export path. A malicious guest could forge a
> relative path and access files outside the export path.
> 
> A tentative fix was sent recently by Prasad J Pandit, but it was only
> focused on the local backend and did not get a positive review. This patch
> tries to address the issue more globally, based on the official 9P spec.
> 
> The walk request described in the 9P spec [1] clearly shows that the client
> is supposed to send individual path components: the official linux client
> never sends portions of path containing the / character for example.
> 
> Moreover, the 9P spec [2] also states that a system can decide to restrict
> the set of supported characters used in path components, with an explicit
> mention "to remove slashes from name components".
> 
> This patch introduces a new name_has_illegal_characters() helper that looks
> for such unwanted characters in strings sent by the client. Since 9pfs is
> only supported on linux hosts, only the / character is checked at the
> moment. When support for other hosts (AKA. win32) is added, other chars
> may need to be blacklisted as well.
> 
> If a client sends a path component with an illegal character, the request
> will fail and EINVAL is returned to the client.
> 
> For the sake of simplicity and consistency, the check is done at top-level
> for all affected 9P requests:
> 
> - xattrwalk
> - xattrcreate
> - mknod
> - rename
> - renameat
> - unlinkat
> - mkdir
> - walk
> - link
> - symlink
> - create
> - lcreate
> 
> [1] http://man.cat-v.org/plan_9/5/walk
> [2] http://man.cat-v.org/plan_9/5/intro
> 
> Reported-by: Felix Wilhelm 
> Suggested-by: Peter Maydell 
> Signed-off-by: Greg Kurz 


Acked-by: Michael S. Tsirkin 

> ---
> 
> Since the linux client does not send / in path components and I don't
> have enough time to write an appropriate qtest, I choosed to do manual
> testing of the mkdir request with GDB:
> 
> [greg@vm66 host]$ mkdir ...foo
> (then turning ...foo into ../foo in QEMU with GDB)
> mkdir: cannot create directory ‘...foo’: Invalid argument
> 
> I also could run the POSIX file system test suite from TUXERA:
> 
> http://www.tuxera.com/community/open-source-posix/
> 
> and did not observe any regression with this patch.
> 
>  hw/9pfs/9p.c |   67 
> ++
>  1 file changed, 67 insertions(+)
> 
> diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c
> index b6b02b46a9da..1c008814509c 100644
> --- a/hw/9pfs/9p.c
> +++ b/hw/9pfs/9p.c
> @@ -1256,6 +1256,11 @@ static int v9fs_walk_marshal(V9fsPDU *pdu, uint16_t 
> nwnames, V9fsQID *qids)
>  return offset;
>  }
>  
> +static bool name_has_illegal_characters(const char *name)
> +{
> +return strchr(name, '/') != NULL;
> +}
> +
>  static void v9fs_walk(void *opaque)
>  {
>  int name_idx;
> @@ -1289,6 +1294,10 @@ static void v9fs_walk(void *opaque)
>  if (err < 0) {
>  goto out_nofid;
>  }
> +if (name_has_illegal_characters(wnames[i].data)) {
> +err = -EINVAL;
> +goto out_nofid;
> +}
>  offset += err;
>  }
>  } else if (nwnames > P9_MAXWELEM) {
> @@ -1483,6 +1492,11 @@ static void v9fs_lcreate(void *opaque)
>  }
>  trace_v9fs_lcreate(pdu->tag, pdu->id, dfid, flags, mode, gid);
>  
> +if (name_has_illegal_characters(name.data)) {
> +err = -EINVAL;
> +goto out_nofid;
> +}
> +
>  fidp = get_fid(pdu, dfid);
>  if (fidp == NULL) {
>  err = -ENOENT;
> @@ -2077,6 +2091,11 @@ static void v9fs_create(void *opaque)
>  }
>  trace_v9fs_create(pdu->tag, pdu->id, fid, name.data, perm, mode);
>  
> +if (name_has_illegal_characters(name.data)) {
> +err = -EINVAL;
> +goto out_nofid;
> +}
> +
>  fidp = get_fid(pdu, fid);
>  if (fidp == NULL) {
>  err = -EINVAL;
> @@ -2242,6 +2261,11 @@ static void v9fs_symlink(void *opaque)
>  }
>  trace_v9fs_symlink(pdu->tag, pdu->id, dfid, name.data, symname.data, 
> gid);
>  
> +if (name_has_illegal_characters(symname.data)) {
> +err = -EINVAL;
> +goto out_nofid;
> +}
> +
>  dfidp = get_fid(pdu, dfid);
>  if (dfidp == NULL) {
>  err = -EINVAL;
> @@ -2316,6 +2340,11 @@ static void v9fs_link(void *opaque)
>  }
>  trace_v9fs_link(pdu->tag, pdu->id, dfid, oldfid, name.data);
>  
> +if (name_has_illegal_characters(name.data)) {
> +err = -EINVAL;
> +goto out_nofid;
> +}
> +
>  dfidp = get_fid(pdu, dfid);
>  if (dfidp == NULL) {
>  err = -ENOENT;
> @@ -2398,6 +2427,12 @@ static void v9fs_unlinkat(void *opaque)
>  if (err < 0) {
>  goto out_nofid;
>  }
> +
> +if 

[Qemu-devel] [PATCH v2 4/7] docker: print warning if EXECUTABLE is not set when building debootstrap image

[Sascha Silbe]

Building the debian-debootstrap image will usually fail if EXECUTABLE
isn't set (when using the Makefile). Warn the user in this case so
they know why it's failing.

Signed-off-by: Sascha Silbe 
---
 tests/docker/Makefile.include | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 1b20db0..19d4cc7 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -44,6 +44,9 @@ docker-image: ${DOCKER_TARGETS}
 
 # General rule for building docker images
 docker-image-%: $(DOCKER_FILES_DIR)/%.docker
+   @if test "$@" = docker-image-debian-bootstrap -a -z "$(EXECUTABLE)"; 
then \
+   echo WARNING: EXECUTABLE is not set, debootstrap may fail. 2>&1 
; \
+   fi
$(call quiet-command,\
$(SRC_PATH)/tests/docker/docker.py build qemu:$* $< \
$(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \
-- 
1.9.1

[Qemu-devel] [PATCH v2 7/7] docker: silence debootstrap when --quiet is given

[Sascha Silbe]

If we silence docker when --quiet is given, we should also silence the
.pre script (i.e. debootstrap).

Only discards stdout, so some diagnostics (e.g. from git clone) are
still printed. Most of the verbose output is gone however and this way
we still have a chance to see error messages.

Signed-off-by: Sascha Silbe 
---
 tests/docker/docker.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index efb2bf4..b85c165 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -239,8 +239,9 @@ class BuildCommand(SubCommand):
 # Is there a .pre file to run in the build context?
 docker_pre = os.path.splitext(args.dockerfile)[0]+".pre"
 if os.path.exists(docker_pre):
+stdout = DEVNULL if args.quiet else None
 rc = subprocess.call(os.path.realpath(docker_pre),
- cwd=docker_dir)
+ cwd=docker_dir, stdout=stdout)
 if rc == 3:
 print "Skip"
 return 0
-- 
1.9.1

[Qemu-devel] [Patch v3 08/30] s390x/cpumodel: register defined CPU models as subclasses

[David Hildenbrand]

This patch adds the CPU model definitions that are known on s390x -
like z900, zBC12 or z13. For each definition, introduce two CPU models:

1. Base model (e.g. z13-base): Minimum feature set we expect to be around
   on all z13 systems. These models are migration-safe and will never
   change.
2. Flexible models (e.g. z13): Models that can change between QEMU versions
   and will be extended over time as we implement further features that
   are already part of such a model in real hardware of certain
   configurations.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 target-s390x/cpu-qom.h|   2 +
 target-s390x/cpu_models.c | 113 ++
 target-s390x/cpu_models.h |  36 +++
 3 files changed, 151 insertions(+)
 create mode 100644 target-s390x/cpu_models.h

diff --git a/target-s390x/cpu-qom.h b/target-s390x/cpu-qom.h
index bb993d4..4e936e7 100644
--- a/target-s390x/cpu-qom.h
+++ b/target-s390x/cpu-qom.h
@@ -21,6 +21,7 @@
 #define QEMU_S390_CPU_QOM_H
 
 #include "qom/cpu.h"
+#include "cpu_models.h"
 
 #define TYPE_S390_CPU "s390-cpu"
 
@@ -45,6 +46,7 @@ typedef struct S390CPUClass {
 /*< private >*/
 CPUClass parent_class;
 /*< public >*/
+const S390CPUDef *cpu_def;
 bool kvm_required;
 bool is_static;
 bool is_migration_safe;
diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index 7b887f7..b59765c 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -12,11 +12,66 @@
 
 #include "qemu/osdep.h"
 #include "cpu.h"
+#include "gen-features.h"
 #include "qapi/error.h"
 #ifndef CONFIG_USER_ONLY
 #include "sysemu/arch_init.h"
 #endif
 
+#define CPUDEF_INIT(_type, _gen, _ec_ga, _mha_pow, _hmfai, _name, _desc) \
+{\
+.name = _name,   \
+.type = _type,   \
+.gen = _gen, \
+.ec_ga = _ec_ga, \
+.mha_pow = _mha_pow, \
+.hmfai = _hmfai, \
+.desc = _desc,   \
+.base_feat = { S390_FEAT_LIST_GEN ## _gen ## _GA ## _ec_ga ## _BASE }, 
 \
+.default_feat = { S390_FEAT_LIST_GEN ## _gen ## _GA ## _ec_ga ## 
_DEFAULT },  \
+.full_feat = { S390_FEAT_LIST_GEN ## _gen ## _GA ## _ec_ga ## _FULL }, 
 \
+}
+
+/*
+ * CPU definiton list in order of release. For now, base features of a
+ * following release are always a subset of base features of the previous
+ * release. Same is correct for the other feature sets.
+ * A BC release always follows the corresponding EC release.
+ */
+static const S390CPUDef s390_cpu_defs[] = {
+CPUDEF_INIT(0x2064, 7, 1, 38, 0xU, "z900", "IBM zSeries 900 GA1"),
+CPUDEF_INIT(0x2064, 7, 2, 38, 0xU, "z900.2", "IBM zSeries 900 
GA2"),
+CPUDEF_INIT(0x2064, 7, 3, 38, 0xU, "z900.3", "IBM zSeries 900 
GA3"),
+CPUDEF_INIT(0x2066, 7, 3, 38, 0xU, "z800", "IBM zSeries 800 GA1"),
+CPUDEF_INIT(0x2084, 8, 1, 38, 0xU, "z990", "IBM zSeries 990 GA1"),
+CPUDEF_INIT(0x2084, 8, 2, 38, 0xU, "z990.2", "IBM zSeries 990 
GA2"),
+CPUDEF_INIT(0x2084, 8, 3, 38, 0xU, "z990.3", "IBM zSeries 990 
GA3"),
+CPUDEF_INIT(0x2086, 8, 3, 38, 0xU, "z890", "IBM zSeries 880 GA1"),
+CPUDEF_INIT(0x2084, 8, 4, 38, 0xU, "z990.4", "IBM zSeries 990 
GA4"),
+CPUDEF_INIT(0x2086, 8, 4, 38, 0xU, "z890.2", "IBM zSeries 880 
GA2"),
+CPUDEF_INIT(0x2084, 8, 5, 38, 0xU, "z990.5", "IBM zSeries 990 
GA5"),
+CPUDEF_INIT(0x2086, 8, 5, 38, 0xU, "z890.3", "IBM zSeries 880 
GA3"),
+CPUDEF_INIT(0x2094, 9, 1, 40, 0xU, "z9EC", "IBM System z9 EC GA1"),
+CPUDEF_INIT(0x2094, 9, 2, 40, 0xU, "z9EC.2", "IBM System z9 EC 
GA2"),
+CPUDEF_INIT(0x2096, 9, 2, 40, 0xU, "z9BC", "IBM System z9 BC GA1"),
+CPUDEF_INIT(0x2094, 9, 3, 40, 0xU, "z9EC.3", "IBM System z9 EC 
GA3"),
+CPUDEF_INIT(0x2096, 9, 3, 40, 0xU, "z9BC.2", "IBM System z9 BC 
GA2"),
+CPUDEF_INIT(0x2097, 10, 1, 43, 0xU, "z10EC", "IBM System z10 EC 
GA1"),
+CPUDEF_INIT(0x2097, 10, 2, 43, 0xU, "z10EC.2", "IBM System z10 EC 
GA2"),
+CPUDEF_INIT(0x2098, 10, 2, 43, 0xU, "z10BC", "IBM System z10 BC 
GA1"),
+CPUDEF_INIT(0x2097, 10, 3, 43, 0xU, "z10EC.3", "IBM System z10 EC 
GA3"),
+CPUDEF_INIT(0x2098, 10, 3, 43, 0xU, "z10BC.2", "IBM System z10 BC 
GA2"),
+CPUDEF_INIT(0x2817, 11, 1, 44, 0x0800U, "z196", "IBM zEnterprise 196 
GA1"),
+CPUDEF_INIT(0x2817, 11, 2, 44, 0x0800U, "z196.2", "IBM 

[Qemu-devel] [PATCH v2 0/7] docker tests fixes

[Sascha Silbe]

A couple of fixes for issues encountered while trying out the new
docker test support. As of v2 building the debian-debootstrap image
now works on my laptop, too.

Thanks for the docker test support, BTW. The centos6 test came in
rather handy today for testing the glib < 2.30 compatibility code.

v1→v2:
  - found a good place to stick the warning about EXECUTABLE
  - additional fixes for the debian-debootstrap image

Sascha Silbe (7):
  docker.py: don't hang on large docker output
  docker: avoid dependency on 'realpath' package
  docker: debian-bootstrap.pre: print helpful message if
DEB_ARCH/DEB_TYPE unset
  docker: print warning if EXECUTABLE is not set when building
debootstrap image
  docker: make sure debootstrap is at least 1.0.67
  docker: build debootstrap after cloning
  docker: silence debootstrap when --quiet is given

 tests/docker/Makefile.include |  5 -
 tests/docker/docker.py| 12 
 tests/docker/dockerfiles/debian-bootstrap.pre | 23 +++
 3 files changed, 35 insertions(+), 5 deletions(-)

-- 
1.9.1

[Qemu-devel] [Patch v3 10/30] s390x/cpumodel: expose features and feature groups as properties

[David Hildenbrand]

Let's add all features and feature groups as properties to all CPU models.
If the "host" CPU model is unknown, we can neither query nor change
features. KVM will just continue to work like it did until now.

We will not allow to enable features that were not part of the original
CPU model, because that could collide with the IBC in KVM.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 target-s390x/cpu.c|   1 +
 target-s390x/cpu.h|   1 +
 target-s390x/cpu_models.c | 149 ++
 3 files changed, 151 insertions(+)

diff --git a/target-s390x/cpu.c b/target-s390x/cpu.c
index d7d0b62..2f3c8e2 100644
--- a/target-s390x/cpu.c
+++ b/target-s390x/cpu.c
@@ -291,6 +291,7 @@ static void s390_cpu_initfn(Object *obj)
 cs->exception_index = EXCP_HLT;
 object_property_add(OBJECT(cpu), "id", "int64_t", s390x_cpu_get_id,
 s390x_cpu_set_id, NULL, NULL, NULL);
+s390_cpu_model_register_props(obj);
 #if !defined(CONFIG_USER_ONLY)
 qemu_get_timedate(, 0);
 env->tod_offset = TOD_UNIX_EPOCH +
diff --git a/target-s390x/cpu.h b/target-s390x/cpu.h
index 3b76654..d03f0f1 100644
--- a/target-s390x/cpu.h
+++ b/target-s390x/cpu.h
@@ -632,6 +632,7 @@ extern void subsystem_reset(void);
 
 void s390_cpu_list(FILE *f, fprintf_function cpu_fprintf);
 #define cpu_list s390_cpu_list
+void s390_cpu_model_register_props(Object *obj);
 void s390_cpu_model_class_register_props(ObjectClass *oc);
 void s390_realize_cpu_model(CPUState *cs, Error **errp);
 ObjectClass *s390_cpu_class_by_name(const char *name);
diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index aaf53f6..75c3099 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -14,6 +14,7 @@
 #include "cpu.h"
 #include "gen-features.h"
 #include "qapi/error.h"
+#include "qapi/visitor.h"
 #ifndef CONFIG_USER_ONLY
 #include "sysemu/arch_init.h"
 #endif
@@ -103,8 +104,24 @@ void s390_cpu_list(FILE *f, fprintf_function print)
 f = f,
 print = print,
 };
+S390FeatGroup group;
+S390Feat feat;
 
 object_class_foreach(print_cpu_model_list, TYPE_S390_CPU, false, );
+
+(*print)(f, "\nRecognized feature flags:\n");
+for (feat = 0; feat < S390_FEAT_MAX; feat++) {
+const S390FeatDef *def = s390_feat_def(feat);
+
+(*print)(f, "%-20s %-50s\n", def->name, def->desc);
+}
+
+(*print)(f, "\nRecognized feature groups:\n");
+for (group = 0; group < S390_FEAT_GROUP_MAX; group++) {
+const S390FeatGroupDef *def = s390_feat_group_def(group);
+
+(*print)(f, "%-20s %-50s\n", def->name, def->desc);
+}
 }
 
 #ifndef CONFIG_USER_ONLY
@@ -151,6 +168,138 @@ void s390_realize_cpu_model(CPUState *cs, Error **errp)
 }
 }
 
+static void get_feature(Object *obj, Visitor *v, const char *name,
+void *opaque, Error **errp)
+{
+S390Feat feat = (S390Feat) opaque;
+S390CPU *cpu = S390_CPU(obj);
+bool value;
+
+if (!cpu->model) {
+error_setg(errp, "Details about the host CPU model are not available, "
+ "features cannot be queried.");
+return;
+}
+
+value = test_bit(feat, cpu->model->features);
+visit_type_bool(v, name, , errp);
+}
+
+static void set_feature(Object *obj, Visitor *v, const char *name,
+void *opaque, Error **errp)
+{
+S390Feat feat = (S390Feat) opaque;
+DeviceState *dev = DEVICE(obj);
+S390CPU *cpu = S390_CPU(obj);
+bool value;
+
+if (dev->realized) {
+error_setg(errp, "Attempt to set property '%s' on '%s' after "
+   "it was realized", name, object_get_typename(obj));
+return;
+} else if (!cpu->model) {
+error_setg(errp, "Details about the host CPU model are not available, "
+ "features cannot be changed.");
+return;
+}
+
+visit_type_bool(v, name, , errp);
+if (*errp) {
+return;
+}
+if (value) {
+if (!test_bit(feat, cpu->model->def->full_feat)) {
+error_setg(errp, "Feature '%s' is not available for CPU model 
'%s',"
+   " it was introduced with later models.",
+   name, cpu->model->def->name);
+return;
+}
+set_bit(feat, cpu->model->features);
+} else {
+clear_bit(feat, cpu->model->features);
+}
+}
+
+static void get_feature_group(Object *obj, Visitor *v, const char *name,
+  void *opaque, Error **errp)
+{
+S390FeatGroup group = (S390FeatGroup) opaque;
+const S390FeatGroupDef *def = s390_feat_group_def(group);
+S390CPU *cpu = S390_CPU(obj);
+S390FeatBitmap tmp;
+bool value;
+
+if (!cpu->model) {
+error_setg(errp, "Details about the host CPU model are not available, "
+ "features cannot be queried.");
+

[Qemu-devel] [Patch v3 03/30] s390x/cpumodel: expose CPU class properties

[David Hildenbrand]

Let's expose the description and migration safety and whether a definition
is static, as class properties, this can be helpful in the future.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 target-s390x/cpu.c|  1 +
 target-s390x/cpu.h|  1 +
 target-s390x/cpu_models.c | 25 +
 3 files changed, 27 insertions(+)

diff --git a/target-s390x/cpu.c b/target-s390x/cpu.c
index 44e53ec..d7d0b62 100644
--- a/target-s390x/cpu.c
+++ b/target-s390x/cpu.c
@@ -445,6 +445,7 @@ static void s390_cpu_class_init(ObjectClass *oc, void *data)
  * object_unref().
  */
 dc->cannot_destroy_with_object_finalize_yet = true;
+s390_cpu_model_class_register_props(oc);
 }
 
 static const TypeInfo s390_cpu_type_info = {
diff --git a/target-s390x/cpu.h b/target-s390x/cpu.h
index e34742e..075bb37 100644
--- a/target-s390x/cpu.h
+++ b/target-s390x/cpu.h
@@ -631,6 +631,7 @@ extern void subsystem_reset(void);
 
 void s390_cpu_list(FILE *f, fprintf_function cpu_fprintf);
 #define cpu_list s390_cpu_list
+void s390_cpu_model_class_register_props(ObjectClass *oc);
 void s390_realize_cpu_model(CPUState *cs, Error **errp);
 ObjectClass *s390_cpu_class_by_name(const char *name);
 
diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index 01f442c..7b887f7 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -110,6 +110,31 @@ static void s390_cpu_model_finalize(Object *obj)
 {
 }
 
+static bool get_is_migration_safe(Object *obj, Error **errp)
+{
+return S390_CPU_GET_CLASS(obj)->is_migration_safe;
+}
+
+static bool get_is_static(Object *obj, Error **errp)
+{
+return S390_CPU_GET_CLASS(obj)->is_static;
+}
+
+static char *get_description(Object *obj, Error **errp)
+{
+return g_strdup(S390_CPU_GET_CLASS(obj)->desc);
+}
+
+void s390_cpu_model_class_register_props(ObjectClass *oc)
+{
+object_class_property_add_bool(oc, "migration-safe", get_is_migration_safe,
+   NULL, NULL);
+object_class_property_add_bool(oc, "static", get_is_static,
+   NULL, NULL);
+object_class_property_add_str(oc, "description", get_description, NULL,
+  NULL);
+}
+
 #ifdef CONFIG_KVM
 static void s390_host_cpu_model_class_init(ObjectClass *oc, void *data)
 {
-- 
2.6.6

[Qemu-devel] [PATCH v2 3/7] docker: debian-bootstrap.pre: print helpful message if DEB_ARCH/DEB_TYPE unset

[Sascha Silbe]

The debian-bootstrap image doesn't choose a default architecture and
distribution version, instead the user has to set both DEB_ARCH and
DEB_TYPE in the environment. Print a reasonably helpful message if
either of them isn't set instead of complaining about "qemu-" being
missing or erroring out because we cannot cd to the mirror URL.

Signed-off-by: Sascha Silbe 
---
 tests/docker/dockerfiles/debian-bootstrap.pre | 13 +
 1 file changed, 13 insertions(+)

diff --git a/tests/docker/dockerfiles/debian-bootstrap.pre 
b/tests/docker/dockerfiles/debian-bootstrap.pre
index 5d9c8d5..2ae363f 100755
--- a/tests/docker/dockerfiles/debian-bootstrap.pre
+++ b/tests/docker/dockerfiles/debian-bootstrap.pre
@@ -15,6 +15,19 @@ exit_and_skip()
 if [ -z $FAKEROOT ]; then
 echo "Please install fakeroot to enable bootstraping"
 exit_and_skip
+
+fi
+
+if [ -z "${DEB_ARCH}" ]; then
+echo "Please set DEB_ARCH to choose an architecture (e.g. armhf)"
+exit_and_skip
+
+fi
+
+if [ -z "${DEB_TYPE}" ]; then
+echo "Please set DEB_TYPE to a Debian archive name (e.g. testing)"
+exit_and_skip
+
 fi
 
 # We check in order for
-- 
1.9.1

[Qemu-devel] [Patch v3 25/30] qmp: add QMP interface "query-cpu-model-expansion"

[David Hildenbrand]

Let's provide a standardized interface to expand CPU models. This interface
can be used by tooling to get details about a specific CPU model in a
certain configuration, e.g. about the "host" model.

To take care of all architectures, two detail levels for an expansion
are introduced. Certain architectures might not support all detail levels.
While "full" will expand and indicate all relevant properties/features
of a CPU model, "static" expands to a static base CPU model, that will
never change between QEMU versions and therefore have the same features
when used under different compatibility machines.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 include/sysemu/arch_init.h |  3 ++
 qapi-schema.json   | 92 ++
 qmp-commands.hx|  6 +++
 qmp.c  |  7 +++
 stubs/Makefile.objs|  1 +
 stubs/arch-query-cpu-model-expansion.c | 12 +
 6 files changed, 121 insertions(+)
 create mode 100644 stubs/arch-query-cpu-model-expansion.c

diff --git a/include/sysemu/arch_init.h b/include/sysemu/arch_init.h
index d690dfa..37b2e86 100644
--- a/include/sysemu/arch_init.h
+++ b/include/sysemu/arch_init.h
@@ -35,5 +35,8 @@ int kvm_available(void);
 int xen_available(void);
 
 CpuDefinitionInfoList *arch_query_cpu_definitions(Error **errp);
+CpuModelExpansionInfo *arch_query_cpu_model_expansion(CpuModelExpansionType 
type,
+  CpuModelInfo *mode,
+  Error **errp);
 
 #endif
diff --git a/qapi-schema.json b/qapi-schema.json
index 0d9ae50..94064ff 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3066,6 +3066,98 @@
 ##
 { 'command': 'query-cpu-definitions', 'returns': ['CpuDefinitionInfo'] }
 
+##
+# @CpuModelInfo:
+#
+# Virtual CPU model.
+#
+# A CPU model consists of the name of a CPU definition, to which
+# delta changes are applied (e.g. features added/removed). Most magic values
+# that an architecture might require should be hidden behind the name.
+# However, if required, architectures can expose relevant properties.
+#
+# @name: the name of the CPU definition the model is based on
+# @props: #optional a dictionary of QOM properties to be applied
+#
+# Since: 2.8.0
+##
+{ 'struct': 'CpuModelInfo',
+  'data': { 'name': 'str',
+'*props': 'any' } }
+
+##
+# @CpuModelExpansionType
+#
+# An enumeration of CPU model expansion types.
+#
+# @static: Expand to a static CPU model, a combination of a static base
+#  model name and property delta changes. As the static base model will
+#  never change, the expanded CPU model will be the same, independant 
of
+#  independent of QEMU version, machine type, machine options, and
+#  accelerator options. Therefore, the resulting model can be used by
+#  tooling without having to specify a compatibility machine - e.g. 
when
+#  displaying the "host" model. static CPU models are migration-safe.
+#
+# @full: Expand all properties. The produced model is not guaranteed to be
+#migration-safe, but allows tooling to get an insight and work with
+#model details.
+#
+# Since: 2.8.0
+##
+{ 'enum': 'CpuModelExpansionType',
+  'data': [ 'static', 'full' ] }
+
+
+##
+# @CpuModelExpansionInfo
+#
+# The result of a cpu model expansion.
+#
+# @model: the expanded CpuModelInfo.
+#
+# Since: 2.8.0
+##
+{ 'struct': 'CpuModelExpansionInfo',
+  'data': { 'model': 'CpuModelInfo' } }
+
+
+##
+# @query-cpu-model-expansion:
+#
+# Expands a given CPU model (or a combination of CPU model + additional 
options)
+# to different granularities, allowing tooling to get an understanding what a
+# specific CPU model looks like in QEMU under a certain configuration.
+#
+# This interface can be used to query the "host" CPU model.
+#
+# The data returned by this command may be affected by:
+#
+# * QEMU version: CPU models may look different depending on the QEMU version.
+#   (Except for CPU models reported as "static" in query-cpu-definitions.)
+# * machine-type: CPU model  may look different depending on the machine-type.
+#   (Except for CPU models reported as "static" in query-cpu-definitions.)
+# * machine options (including accelerator): in some architectures, CPU models
+#   may look different depending on machine and accelerator options. (Except 
for
+#   CPU models reported as "static" in query-cpu-definitions.)
+# * "-cpu" arguments and global properties: arguments to the -cpu option and
+#   global properties may affect expansion of CPU models. Using
+#   query-cpu-model-expansion while using these is not advised.
+#
+# Some architectures may not support all expansion types.
+#
+# Returns: a CpuModelExpansionInfo. Returns an error if expanding CPU models is
+#  not supported, if the model cannot be expanded, if the model 

[Qemu-devel] [Patch v3 19/30] linux-headers: update against kvm/next

[David Hildenbrand]

Update against 29b4817d4018 ("Linux 4.8-rc1")

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 include/standard-headers/linux/input-event-codes.h | 32 +
 include/standard-headers/linux/input.h |  1 +
 include/standard-headers/linux/virtio_config.h | 10 +-
 include/standard-headers/linux/virtio_ids.h|  1 +
 include/standard-headers/linux/virtio_net.h|  3 ++
 linux-headers/asm-arm/kvm.h|  4 +--
 linux-headers/asm-arm64/kvm.h  |  2 ++
 linux-headers/asm-s390/kvm.h   | 41 ++
 linux-headers/asm-x86/unistd_x32.h |  4 +--
 linux-headers/linux/kvm.h  | 18 --
 linux-headers/linux/vhost.h| 33 +
 11 files changed, 142 insertions(+), 7 deletions(-)

diff --git a/include/standard-headers/linux/input-event-codes.h 
b/include/standard-headers/linux/input-event-codes.h
index 354f0de..5c10f7e 100644
--- a/include/standard-headers/linux/input-event-codes.h
+++ b/include/standard-headers/linux/input-event-codes.h
@@ -611,6 +611,37 @@
 #define KEY_KBDINPUTASSIST_ACCEPT  0x264
 #define KEY_KBDINPUTASSIST_CANCEL  0x265
 
+/* Diagonal movement keys */
+#define KEY_RIGHT_UP   0x266
+#define KEY_RIGHT_DOWN 0x267
+#define KEY_LEFT_UP0x268
+#define KEY_LEFT_DOWN  0x269
+
+#define KEY_ROOT_MENU  0x26a /* Show Device's Root Menu */
+/* Show Top Menu of the Media (e.g. DVD) */
+#define KEY_MEDIA_TOP_MENU 0x26b
+#define KEY_NUMERIC_11 0x26c
+#define KEY_NUMERIC_12 0x26d
+/*
+ * Toggle Audio Description: refers to an audio service that helps blind and
+ * visually impaired consumers understand the action in a program. Note: in
+ * some countries this is referred to as "Video Description".
+ */
+#define KEY_AUDIO_DESC 0x26e
+#define KEY_3D_MODE0x26f
+#define KEY_NEXT_FAVORITE  0x270
+#define KEY_STOP_RECORD0x271
+#define KEY_PAUSE_RECORD   0x272
+#define KEY_VOD0x273 /* Video on Demand */
+#define KEY_UNMUTE 0x274
+#define KEY_FASTREVERSE0x275
+#define KEY_SLOWREVERSE0x276
+/*
+ * Control a data application associated with the currently viewed channel,
+ * e.g. teletext or data broadcast application (MHEG, MHP, HbbTV, etc.)
+ */
+#define KEY_DATA   0x275
+
 #define BTN_TRIGGER_HAPPY  0x2c0
 #define BTN_TRIGGER_HAPPY1 0x2c0
 #define BTN_TRIGGER_HAPPY2 0x2c1
@@ -749,6 +780,7 @@
 #define SW_ROTATE_LOCK 0x0c  /* set = rotate locked/disabled */
 #define SW_LINEIN_INSERT   0x0d  /* set = inserted */
 #define SW_MUTE_DEVICE 0x0e  /* set = device disabled */
+#define SW_PEN_INSERTED0x0f  /* set = pen inserted */
 #define SW_MAX_0x0f
 #define SW_CNT (SW_MAX_+1)
 
diff --git a/include/standard-headers/linux/input.h 
b/include/standard-headers/linux/input.h
index a52b202..7361a16 100644
--- a/include/standard-headers/linux/input.h
+++ b/include/standard-headers/linux/input.h
@@ -244,6 +244,7 @@ struct input_mask {
 #define BUS_ATARI  0x1B
 #define BUS_SPI0x1C
 #define BUS_RMI0x1D
+#define BUS_CEC0x1E
 
 /*
  * MT_TOOL types
diff --git a/include/standard-headers/linux/virtio_config.h 
b/include/standard-headers/linux/virtio_config.h
index b30d0cb..b777069 100644
--- a/include/standard-headers/linux/virtio_config.h
+++ b/include/standard-headers/linux/virtio_config.h
@@ -49,7 +49,7 @@
  * transport being used (eg. virtio_ring), the rest are per-device feature
  * bits. */
 #define VIRTIO_TRANSPORT_F_START   28
-#define VIRTIO_TRANSPORT_F_END 33
+#define VIRTIO_TRANSPORT_F_END 34
 
 #ifndef VIRTIO_CONFIG_NO_LEGACY
 /* Do we get callbacks when the ring is completely used, even if we've
@@ -63,4 +63,12 @@
 /* v1.0 compliant. */
 #define VIRTIO_F_VERSION_1 32
 
+/*
+ * If clear - device has the IOMMU bypass quirk feature.
+ * If set - use platform tools to detect the IOMMU.
+ *
+ * Note the reverse polarity (compared to most other features),
+ * this is for compatibility with legacy systems.
+ */
+#define VIRTIO_F_IOMMU_PLATFORM33
 #endif /* _LINUX_VIRTIO_CONFIG_H */
diff --git a/include/standard-headers/linux/virtio_ids.h 
b/include/standard-headers/linux/virtio_ids.h
index 77925f5..3228d58 100644
--- a/include/standard-headers/linux/virtio_ids.h
+++ b/include/standard-headers/linux/virtio_ids.h
@@ -41,5 +41,6 @@
 #define VIRTIO_ID_CAIF12 /* Virtio caif */
 #define 

[Qemu-devel] [Patch v3 16/30] s390x/sclp: propagate the ibc val(lowest and unblocked ibc)

[David Hildenbrand]

If we have a lowest ibc, we can indicate the ibc to the guest.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 hw/s390x/sclp.c   |  2 ++
 include/hw/s390x/sclp.h   |  3 ++-
 target-s390x/cpu_models.c | 21 +
 target-s390x/cpu_models.h | 12 
 4 files changed, 37 insertions(+), 1 deletion(-)

diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index 3c126ee..52f8bb9 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -62,6 +62,8 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
 read_info->offset_cpu = cpu_to_be16(offsetof(ReadInfo, entries));
 read_info->highest_cpu = cpu_to_be16(max_cpus);
 
+read_info->ibc_val = cpu_to_be32(s390_get_ibc_val());
+
 /* Configuration Characteristic (Extension) */
 s390_get_feat_block(S390_FEAT_TYPE_SCLP_CONF_CHAR,
  read_info->conf_char);
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index 30a40ea..664be9b 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -121,7 +121,8 @@ typedef struct ReadInfo {
 uint8_t  loadparm[8];   /* 24-31 */
 uint8_t  _reserved3[48 - 32];   /* 32-47 */
 uint64_t facilities;/* 48-55 */
-uint8_t  _reserved0[80 - 56];   /* 56-79 */
+uint8_t  _reserved0[76 - 56];   /* 56-75 */
+uint32_t ibc_val;
 uint8_t  conf_char[96 - 80];/* 80-95 */
 uint8_t  _reserved4[100 - 96];  /* 96-99 */
 uint32_t rnsize2;
diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index 4968fdc..70337b8 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -74,6 +74,27 @@ static const S390CPUDef s390_cpu_defs[] = {
 CPUDEF_INIT(0x2965, 13, 2, 47, 0x0800U, "z13s", "IBM z13s GA1"),
 };
 
+uint32_t s390_get_ibc_val(void)
+{
+uint16_t unblocked_ibc, lowest_ibc;
+static S390CPU *cpu;
+
+if (!cpu) {
+cpu = S390_CPU(qemu_get_cpu(0));
+}
+
+if (!cpu || !cpu->model) {
+return 0;
+}
+unblocked_ibc = s390_ibc_from_cpu_model(cpu->model);
+lowest_ibc = cpu->model->lowest_ibc;
+/* the lowest_ibc always has to be <= unblocked_ibc */
+if (!lowest_ibc || lowest_ibc > unblocked_ibc) {
+return 0;
+}
+return ((uint32_t) lowest_ibc << 16) | unblocked_ibc;
+}
+
 void s390_get_feat_block(S390FeatType type, uint8_t *data)
 {
 static S390CPU *cpu;
diff --git a/target-s390x/cpu_models.h b/target-s390x/cpu_models.h
index 04c47c6..bbb85ac 100644
--- a/target-s390x/cpu_models.h
+++ b/target-s390x/cpu_models.h
@@ -43,6 +43,18 @@ typedef struct S390CPUModel {
 uint8_t cpu_ver;/* CPU version, usually "ff" for kvm */
 } S390CPUModel;
 
+#define S390_GEN_Z10 0xa
+
+uint32_t s390_get_ibc_val(void);
+static inline uint16_t s390_ibc_from_cpu_model(const S390CPUModel *model)
+{
+uint16_t ibc = 0;
+
+if (model->def->gen >= S390_GEN_Z10) {
+ibc = ((model->def->gen - S390_GEN_Z10) << 4) + model->def->ec_ga;
+}
+return ibc;
+}
 void s390_get_feat_block(S390FeatType type, uint8_t *data);
 bool s390_has_feat(S390Feat feat);
 
-- 
2.6.6

Re: [Qemu-devel] [PATCH] 9pfs: disallow / in path components

[Michael S. Tsirkin]

On Wed, Aug 24, 2016 at 06:41:45PM +0200, Greg Kurz wrote:
> On Wed, 24 Aug 2016 18:46:10 +0300
> "Michael S. Tsirkin"  wrote:
> 
> > On Wed, Aug 24, 2016 at 04:00:24PM +0100, Peter Maydell wrote:
> > > On 24 August 2016 at 15:29, Greg Kurz  wrote:  
> > > > At various places in 9pfs, full paths are created by concatenating a 
> > > > guest
> > > > originated string to the export path. A malicious guest could forge a
> > > > relative path and access files outside the export path.
> > > >
> > > > A tentative fix was sent recently by Prasad J Pandit, but it was only
> > > > focused on the local backend and did not get a positive review. This 
> > > > patch
> > > > tries to address the issue more globally, based on the official 9P spec.
> > > >
> > > > The walk request described in the 9P spec [1] clearly shows that the 
> > > > client
> > > > is supposed to send individual path components: the official linux 
> > > > client
> > > > never sends portions of path containing the / character for example.
> > > >
> > > > Moreover, the 9P spec [2] also states that a system can decide to 
> > > > restrict
> > > > the set of supported characters used in path components, with an 
> > > > explicit
> > > > mention "to remove slashes from name components".
> > > >
> > > > This patch introduces a new name_has_illegal_characters() helper that 
> > > > looks
> > > > for such unwanted characters in strings sent by the client. Since 9pfs 
> > > > is
> > > > only supported on linux hosts, only the / character is checked at the
> > > > moment. When support for other hosts (AKA. win32) is added, other chars
> > > > may need to be blacklisted as well.  
> > > 
> > > Do we also need ".." and "." to be illegal names (for at least most
> > > operations)?
> > > 
> > > thanks
> > > -- PMM  
> > 
> > I agree, and I think this implies name_is_legal would be a better function 
> > name.
> > 
> 
> No I think this is a different issue that calls for a followup patch (see my
> other mail).

OK, that's fine.

-- 
MST

[Qemu-devel] [Patch v3 18/30] s390x/sclp: propagate hmfai

[David Hildenbrand]

hmfai is provided on CPU models >= z196. Let's propagate it properly.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 hw/s390x/sclp.c   |  1 +
 include/hw/s390x/sclp.h   |  3 ++-
 target-s390x/cpu_models.c | 14 ++
 target-s390x/cpu_models.h |  1 +
 4 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index 5ffcf51..883592c 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -106,6 +106,7 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
 read_info->facilities |= cpu_to_be64(SCLP_FC_ASSIGN_ATTACH_READ_STOR);
 }
 read_info->mha_pow = s390_get_mha_pow();
+read_info->hmfai = cpu_to_be32(s390_get_hmfai());
 
 rnsize = 1 << (sclp->increment_size - 20);
 if (rnsize <= 128) {
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index dab3c0f..3008a51 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -131,7 +131,8 @@ typedef struct ReadInfo {
 uint8_t  _reserved6[116 - 112]; /* 112-115 */
 uint8_t  conf_char_ext[120 - 116];   /* 116-119 */
 uint16_t highest_cpu;
-uint8_t  _reserved5[128 - 122]; /* 122-127 */
+uint8_t  _reserved5[124 - 122]; /* 122-123 */
+uint32_t hmfai;
 struct CPUEntry entries[0];
 } QEMU_PACKED ReadInfo;
 
diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index bda67a5..d794f02 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -74,6 +74,20 @@ static const S390CPUDef s390_cpu_defs[] = {
 CPUDEF_INIT(0x2965, 13, 2, 47, 0x0800U, "z13s", "IBM z13s GA1"),
 };
 
+uint32_t s390_get_hmfai(void)
+{
+static S390CPU *cpu;
+
+if (!cpu) {
+cpu = S390_CPU(qemu_get_cpu(0));
+}
+
+if (!cpu || !cpu->model) {
+return 0;
+}
+return cpu->model->def->hmfai;
+}
+
 uint8_t s390_get_mha_pow(void)
 {
 static S390CPU *cpu;
diff --git a/target-s390x/cpu_models.h b/target-s390x/cpu_models.h
index ee019b4..986f7cb 100644
--- a/target-s390x/cpu_models.h
+++ b/target-s390x/cpu_models.h
@@ -45,6 +45,7 @@ typedef struct S390CPUModel {
 
 #define S390_GEN_Z10 0xa
 
+uint32_t s390_get_hmfai(void);
 uint8_t s390_get_mha_pow(void);
 uint32_t s390_get_ibc_val(void);
 static inline uint16_t s390_ibc_from_cpu_model(const S390CPUModel *model)
-- 
2.6.6

[Qemu-devel] [Patch v3 13/30] s390x/sclp: factor out preparation of cpu entries

[David Hildenbrand]

Let's factor out the common code of "read cpu info" and "read scp
info". This will make the introduction of new cpu entry fields easier.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 hw/s390x/sclp.c | 22 --
 1 file changed, 12 insertions(+), 10 deletions(-)

diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index fca37f5..15d7114 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -29,6 +29,16 @@ static inline SCLPDevice *get_sclp_device(void)
 return SCLP(object_resolve_path_type("", TYPE_SCLP, NULL));
 }
 
+static void prepare_cpu_entries(SCLPDevice *sclp, CPUEntry *entry, int count)
+{
+int i;
+
+for (i = 0; i < count; i++) {
+entry[i].address = i;
+entry[i].type = 0;
+}
+}
+
 /* Provide information about the configuration, CPUs and storage */
 static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
 {
@@ -37,7 +47,6 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
 sclpMemoryHotplugDev *mhd = get_sclp_memory_hotplug_dev();
 CPUState *cpu;
 int cpu_count = 0;
-int i = 0;
 int rnsize, rnmax;
 int slots = MIN(machine->ram_slots, s390_get_memslot_count(kvm_state));
 
@@ -50,10 +59,7 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
 read_info->offset_cpu = cpu_to_be16(offsetof(ReadInfo, entries));
 read_info->highest_cpu = cpu_to_be16(max_cpus);
 
-for (i = 0; i < cpu_count; i++) {
-read_info->entries[i].address = i;
-read_info->entries[i].type = 0;
-}
+prepare_cpu_entries(sclp, read_info->entries, cpu_count);
 
 read_info->facilities = cpu_to_be64(SCLP_HAS_CPU_INFO |
 SCLP_HAS_PCI_RECONFIG);
@@ -304,7 +310,6 @@ static void sclp_read_cpu_info(SCLPDevice *sclp, SCCB *sccb)
 ReadCpuInfo *cpu_info = (ReadCpuInfo *) sccb;
 CPUState *cpu;
 int cpu_count = 0;
-int i = 0;
 
 CPU_FOREACH(cpu) {
 cpu_count++;
@@ -318,10 +323,7 @@ static void sclp_read_cpu_info(SCLPDevice *sclp, SCCB 
*sccb)
 cpu_info->offset_standby = cpu_to_be16(cpu_info->offset_configured
 + cpu_info->nr_configured*sizeof(CPUEntry));
 
-for (i = 0; i < cpu_count; i++) {
-cpu_info->entries[i].address = i;
-cpu_info->entries[i].type = 0;
-}
+prepare_cpu_entries(sclp, cpu_info->entries, cpu_count);
 
 sccb->h.response_code = cpu_to_be16(SCLP_RC_NORMAL_READ_COMPLETION);
 }
-- 
2.6.6

[Qemu-devel] [Patch v3 11/30] s390x/cpumodel: let the CPU model handle feature checks

[David Hildenbrand]

If we have certain features enabled, we have to migrate additional state
(e.g. vector registers or runtime-instrumentation registers). Let the
CPU model control that unless we have no "host" CPU model in the KVM
case. This will later on be the case for compatibility machines, so
migration from QEMU versions without the CPU model will still work.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 target-s390x/cpu_models.c | 25 +
 target-s390x/cpu_models.h |  2 ++
 target-s390x/kvm.c|  4 ++--
 target-s390x/machine.c| 14 ++
 4 files changed, 31 insertions(+), 14 deletions(-)

diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index 75c3099..946b565 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -73,6 +73,31 @@ static const S390CPUDef s390_cpu_defs[] = {
 CPUDEF_INIT(0x2965, 13, 2, 47, 0x0800U, "z13s", "IBM z13s GA1"),
 };
 
+bool s390_has_feat(S390Feat feat)
+{
+static S390CPU *cpu;
+
+if (!cpu) {
+cpu = S390_CPU(qemu_get_cpu(0));
+}
+
+if (!cpu || !cpu->model) {
+#ifdef CONFIG_KVM
+if (kvm_enabled()) {
+if (feat == S390_FEAT_VECTOR) {
+return kvm_check_extension(kvm_state,
+   KVM_CAP_S390_VECTOR_REGISTERS);
+}
+if (feat == S390_FEAT_RUNTIME_INSTRUMENTATION) {
+return kvm_s390_get_ri();
+}
+}
+#endif
+return 0;
+}
+return test_bit(feat, cpu->model->features);
+}
+
 struct S390PrintCpuListInfo {
 FILE *f;
 fprintf_function print;
diff --git a/target-s390x/cpu_models.h b/target-s390x/cpu_models.h
index 244256b..fe988cc 100644
--- a/target-s390x/cpu_models.h
+++ b/target-s390x/cpu_models.h
@@ -43,4 +43,6 @@ typedef struct S390CPUModel {
 uint8_t cpu_ver;/* CPU version, usually "ff" for kvm */
 } S390CPUModel;
 
+bool s390_has_feat(S390Feat feat);
+
 #endif /* TARGET_S390X_CPU_MODELS_H */
diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c
index 80ac621..55bf6d7 100644
--- a/target-s390x/kvm.c
+++ b/target-s390x/kvm.c
@@ -1517,7 +1517,7 @@ static void sigp_store_adtl_status(void *arg)
 {
 SigpInfo *si = arg;
 
-if (!kvm_check_extension(kvm_state, KVM_CAP_S390_VECTOR_REGISTERS)) {
+if (!s390_has_feat(S390_FEAT_VECTOR)) {
 set_sigp_status(si, SIGP_STAT_INVALID_ORDER);
 return;
 }
@@ -2089,7 +2089,7 @@ static uint64_t build_channel_report_mcic(void)
 MCIC_VB_WP | MCIC_VB_MS | MCIC_VB_PM | MCIC_VB_IA | MCIC_VB_FP |
 MCIC_VB_GR | MCIC_VB_CR | MCIC_VB_ST | MCIC_VB_AR | MCIC_VB_PR |
 MCIC_VB_FC | MCIC_VB_CT | MCIC_VB_CC;
-if (kvm_check_extension(kvm_state, KVM_CAP_S390_VECTOR_REGISTERS)) {
+if (s390_has_feat(S390_FEAT_VECTOR)) {
 mcic |= MCIC_VB_VR;
 }
 return mcic;
diff --git a/target-s390x/machine.c b/target-s390x/machine.c
index aa39e5d..edc3a47 100644
--- a/target-s390x/machine.c
+++ b/target-s390x/machine.c
@@ -78,12 +78,7 @@ static const VMStateDescription vmstate_fpu = {
 
 static bool vregs_needed(void *opaque)
 {
-#ifdef CONFIG_KVM
-if (kvm_enabled()) {
-return kvm_check_extension(kvm_state, KVM_CAP_S390_VECTOR_REGISTERS);
-}
-#endif
-return 0;
+return s390_has_feat(S390_FEAT_VECTOR);
 }
 
 static const VMStateDescription vmstate_vregs = {
@@ -147,12 +142,7 @@ static const VMStateDescription vmstate_vregs = {
 
 static bool riccb_needed(void *opaque)
 {
-#ifdef CONFIG_KVM
-if (kvm_enabled()) {
-return kvm_s390_get_ri();
-}
-#endif
-return 0;
+return s390_has_feat(S390_FEAT_RUNTIME_INSTRUMENTATION);
 }
 
 const VMStateDescription vmstate_riccb = {
-- 
2.6.6

[Qemu-devel] [Patch v3 26/30] qmp: add QMP interface "query-cpu-model-comparison"

[David Hildenbrand]

Let's provide a standardized interface to compare two CPU models.
"query-cpu-model-compare" takes two models and returns  how they compare
in a specific configuration.

The result will give guarantees about runnability. E.g. if a CPU model A
is a subset of CPU model B, model A is guaranteed to run in configurations
where model B runs, but not the other way around (might or might not run).

Usually, CPU features or CPU generations are used to calculate the result.
If a model is not guaranteed to run in a certain environment (e.g.
incompatible), a  compatible one can be created by "baselining" both models
(follow up patch).

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 include/sysemu/arch_init.h  |  3 ++
 qapi-schema.json| 86 +
 qmp-commands.hx |  6 +++
 qmp.c   |  7 +++
 stubs/Makefile.objs |  1 +
 stubs/arch-query-cpu-model-comparison.c | 12 +
 6 files changed, 115 insertions(+)
 create mode 100644 stubs/arch-query-cpu-model-comparison.c

diff --git a/include/sysemu/arch_init.h b/include/sysemu/arch_init.h
index 37b2e86..96d47c0 100644
--- a/include/sysemu/arch_init.h
+++ b/include/sysemu/arch_init.h
@@ -38,5 +38,8 @@ CpuDefinitionInfoList *arch_query_cpu_definitions(Error 
**errp);
 CpuModelExpansionInfo *arch_query_cpu_model_expansion(CpuModelExpansionType 
type,
   CpuModelInfo *mode,
   Error **errp);
+CpuModelCompareInfo *arch_query_cpu_model_comparison(CpuModelInfo *modela,
+ CpuModelInfo *modelb,
+ Error **errp);
 
 #endif
diff --git a/qapi-schema.json b/qapi-schema.json
index 94064ff..4f1b9ec 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3158,6 +3158,92 @@
 'model': 'CpuModelInfo' },
   'returns': 'CpuModelExpansionInfo' }
 
+##
+# @CpuModelCompareResult:
+#
+# An enumeration of CPU model comparation results. The result is usually
+# calcualted using e.g. CPU features or CPU generations.
+#
+# @incompatible: If model A is incompatible to model B, model A is not
+#guaranteed to run where model B runs and the other way around.
+#
+# @identical: If model A is identical to model B, model A is guaranteed to run
+# where model B runs and the other way around.
+#
+# @superset: If model A is a superset of model B, model B is guaranteed to run
+#where model A runs. There are no guarantees about the other way.
+#
+# @subset: If model A is a subset of model B, model A is guaranteed to run
+#  where model B runs. There are no guarantees about the other way.
+#
+# Since: 2.8.0
+##
+{ 'enum': 'CpuModelCompareResult',
+  'data': [ 'incompatible', 'identical', 'superset', 'subset' ] }
+
+##
+# @CpuModelCompareInfo
+#
+# The result of a CPU model comparison.
+#
+# @result: The result of the compare operation.
+# @responsible-properties: List of properties that led to the comparison result
+#  not being identical.
+#
+# @responsible-properties is a list of QOM property names that led to
+# both CPUs not being detected as identical. For identical models, this
+# list is empty.
+# If a QOM property is read-only, that means there's no known way to make the
+# CPU models identical. If the special property name "type" is included, the
+# models are by definition not identical and cannot be made identical.
+#
+# Since: 2.8.0
+##
+{ 'struct': 'CpuModelCompareInfo',
+  'data': {'result': 'CpuModelCompareResult',
+   'responsible-properties': ['str']
+  }
+}
+
+##
+# @query-cpu-model-comparison:
+#
+# Compares two CPU models, returning how they compare in a specific
+# configuration. The results indicates how both models compare regarding
+# runnability. This result can be used by tooling to make decisions if a
+# certain CPU model will run in a certain configuration or if a compatible
+# CPU model has to be created by baselining.
+#
+# Usually, a CPU model is compared against the maximum possible CPU model
+# of a ceratin configuration (e.g. the "host" model for KVM). If that CPU
+# model is identical or a subset, it will run in that configuration.
+#
+# The result returned by this command may be affected by:
+#
+# * QEMU version: CPU models may look different depending on the QEMU version.
+#   (Except for CPU models reported as "static" in query-cpu-definitions.)
+# * machine-type: CPU model  may look different depending on the machine-type.
+#   (Except for CPU models reported as "static" in query-cpu-definitions.)
+# * machine options (including accelerator): in some architectures, CPU models
+#   may look different depending on machine and accelerator options. (Except 
for
+#   CPU models 

[Qemu-devel] [Patch v3 28/30] s390x/cpumodel: implement QMP interface "query-cpu-model-expansion"

[David Hildenbrand]

In order to expand CPU models, we create temporary cpus that handle the
feature/group parsing. Only CPU feature properties are expanded.

When converting the data structure back, we always fall back to the
static base CPU model, which is by definition migration-safe.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 qapi-schema.json  |   3 +-
 target-s390x/cpu_models.c | 147 ++
 2 files changed, 149 insertions(+), 1 deletion(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index 7f469d0..93d2dc2 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3143,7 +3143,8 @@
 #   global properties may affect expansion of CPU models. Using
 #   query-cpu-model-expansion while using these is not advised.
 #
-# Some architectures may not support all expansion types.
+# Some architectures may not support all expansion types. s390x supports
+# "full" and "static".
 #
 # Returns: a CpuModelExpansionInfo. Returns an error if expanding CPU models is
 #  not supported, if the model cannot be expanded, if the model 
contains
diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index 7419bd0..ee52ac0 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -16,6 +16,9 @@
 #include "qapi/error.h"
 #include "qapi/visitor.h"
 #include "qemu/error-report.h"
+#include "qapi/qmp/qerror.h"
+#include "qapi/qmp-input-visitor.h"
+#include "qapi/qmp/qbool.h"
 #ifndef CONFIG_USER_ONLY
 #include "sysemu/arch_init.h"
 #endif
@@ -303,6 +306,150 @@ CpuDefinitionInfoList *arch_query_cpu_definitions(Error 
**errp)
 
 return list;
 }
+
+static void cpu_model_from_info(S390CPUModel *model, const CpuModelInfo *info,
+Error **errp)
+{
+const QDict *qdict = NULL;
+const QDictEntry *e;
+Visitor *visitor;
+ObjectClass *oc;
+S390CPU *cpu;
+Object *obj;
+
+if (info->props) {
+qdict = qobject_to_qdict(info->props);
+if (!qdict) {
+error_setg(errp, QERR_INVALID_PARAMETER_TYPE, "props", "dict");
+return;
+}
+}
+
+oc = cpu_class_by_name(TYPE_S390_CPU, info->name);
+if (!oc) {
+error_setg(errp, "The CPU definition \'%s\' is unknown.", info->name);
+return;
+}
+if (S390_CPU_CLASS(oc)->kvm_required && !kvm_enabled()) {
+error_setg(errp, "The CPU definition '%s' requires KVM", info->name);
+return;
+}
+obj = object_new(object_class_get_name(oc));
+cpu = S390_CPU(obj);
+
+if (!cpu->model) {
+error_setg(errp, "Details about the host CPU model are not available, "
+ "it cannot be used.");
+object_unref(obj);
+return;
+}
+
+if (qdict) {
+visitor = qmp_input_visitor_new(info->props, true);
+visit_start_struct(visitor, NULL, NULL, 0, errp);
+if (*errp) {
+object_unref(obj);
+return;
+}
+for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
+object_property_set(obj, visitor, e->key, errp);
+if (*errp) {
+break;
+}
+}
+if (!*errp) {
+visit_check_struct(visitor, errp);
+}
+visit_end_struct(visitor, NULL);
+visit_free(visitor);
+if (*errp) {
+object_unref(obj);
+return;
+}
+}
+
+/* copy the model and throw the cpu away */
+memcpy(model, cpu->model, sizeof(*model));
+object_unref(obj);
+}
+
+static void qdict_add_disabled_feat(const char *name, void *opaque)
+{
+qdict_put((QDict *) opaque, name, qbool_from_bool(false));
+}
+
+static void qdict_add_enabled_feat(const char *name, void *opaque)
+{
+qdict_put((QDict *) opaque, name, qbool_from_bool(true));
+}
+
+/* convert S390CPUDef into a static CpuModelInfo */
+static void cpu_info_from_model(CpuModelInfo *info, const S390CPUModel *model,
+bool delta_changes)
+{
+QDict *qdict = qdict_new();
+S390FeatBitmap bitmap;
+
+/* always fallback to the static base model */
+info->name = g_strdup_printf("%s-base", model->def->name);
+
+if (delta_changes) {
+/* features deleted from the base feature set */
+bitmap_andnot(bitmap, model->def->base_feat, model->features,
+  S390_FEAT_MAX);
+if (!bitmap_empty(bitmap, S390_FEAT_MAX)) {
+s390_feat_bitmap_to_ascii(bitmap, qdict, qdict_add_disabled_feat);
+}
+
+/* features added to the base feature set */
+bitmap_andnot(bitmap, model->features, model->def->base_feat,
+  S390_FEAT_MAX);
+if (!bitmap_empty(bitmap, S390_FEAT_MAX)) {
+s390_feat_bitmap_to_ascii(bitmap, qdict, qdict_add_enabled_feat);
+}
+} else {
+/* expand all features */
+

[Qemu-devel] [Patch v3 12/30] s390x/cpumodel: check and apply the CPU model

[David Hildenbrand]

We have to test if a configured CPU model is runnable in the current
configuration, and if not report why that is the case. This is done by
comparing it to the maximum supported model (host for KVM or z900 for TCG).
Also, we want to do some base sanity checking for a configured CPU model.

We'll cache the maximum model and the applied model (for performance
reasons and because KVM can only be configured before any VCPU is created).

For unavailable "host" model, we have to make sure that we inform KVM,
so it can do some compatibility stuff (enable CMMA later on to be precise).

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 target-s390x/cpu_models.c | 153 ++
 1 file changed, 153 insertions(+)

diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index 946b565..e226a2f 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -15,6 +15,7 @@
 #include "gen-features.h"
 #include "qapi/error.h"
 #include "qapi/visitor.h"
+#include "qemu/error-report.h"
 #ifndef CONFIG_USER_ONLY
 #include "sysemu/arch_init.h"
 #endif
@@ -183,14 +184,166 @@ CpuDefinitionInfoList *arch_query_cpu_definitions(Error 
**errp)
 }
 #endif
 
+static void check_consistency(const S390CPUModel *model)
+{
+static int dep[][2] = {
+{ S390_FEAT_IPTE_RANGE, S390_FEAT_DAT_ENH },
+{ S390_FEAT_IDTE_SEGMENT, S390_FEAT_DAT_ENH },
+{ S390_FEAT_IDTE_REGION, S390_FEAT_DAT_ENH },
+{ S390_FEAT_IDTE_REGION, S390_FEAT_IDTE_SEGMENT },
+{ S390_FEAT_LOCAL_TLB_CLEARING, S390_FEAT_DAT_ENH},
+{ S390_FEAT_LONG_DISPLACEMENT_FAST, S390_FEAT_LONG_DISPLACEMENT },
+{ S390_FEAT_DFP_FAST, S390_FEAT_DFP },
+{ S390_FEAT_TRANSACTIONAL_EXE, S390_FEAT_STFLE_49 },
+{ S390_FEAT_EDAT_2, S390_FEAT_EDAT},
+{ S390_FEAT_MSA_EXT_5, S390_FEAT_KIMD_SHA_512 },
+{ S390_FEAT_MSA_EXT_5, S390_FEAT_KLMD_SHA_512 },
+{ S390_FEAT_MSA_EXT_4, S390_FEAT_MSA_EXT_3 },
+{ S390_FEAT_SIE_CMMA, S390_FEAT_CMM },
+{ S390_FEAT_SIE_CMMA, S390_FEAT_SIE_GSLS },
+{ S390_FEAT_SIE_PFMFI, S390_FEAT_EDAT },
+};
+int i;
+
+for (i = 0; i < ARRAY_SIZE(dep); i++) {
+if (test_bit(dep[i][0], model->features) &&
+!test_bit(dep[i][1], model->features)) {
+error_report("Warning: \'%s\' requires \'%s\'.",
+ s390_feat_def(dep[i][0])->name,
+ s390_feat_def(dep[i][1])->name);
+}
+}
+}
+
+static void error_prepend_missing_feat(const char *name, void *opaque)
+{
+error_prepend((Error **) opaque, "%s ", name);
+}
+
+static void check_compatibility(const S390CPUModel *max_model,
+const S390CPUModel *model, Error **errp)
+{
+S390FeatBitmap missing;
+
+if (model->def->gen > max_model->def->gen) {
+error_setg(errp, "Selected CPU generation is too new. Maximum "
+   "supported model in the configuration: \'%s\'",
+   max_model->def->name);
+return;
+} else if (model->def->gen == max_model->def->gen &&
+   model->def->ec_ga > max_model->def->ec_ga) {
+error_setg(errp, "Selected CPU GA level is too new. Maximum "
+   "supported model in the configuration: \'%s\'",
+   max_model->def->name);
+return;
+}
+
+/* detect the missing features to properly report them */
+bitmap_andnot(missing, model->features, max_model->features, 
S390_FEAT_MAX);
+if (bitmap_empty(missing, S390_FEAT_MAX)) {
+return;
+}
+
+error_setg(errp, " ");
+s390_feat_bitmap_to_ascii(missing, errp, error_prepend_missing_feat);
+error_prepend(errp, "Some features requested in the CPU model are not "
+  "available in the configuration: ");
+}
+
+static S390CPUModel *get_max_cpu_model(Error **errp)
+{
+#ifndef CONFIG_USER_ONLY
+static S390CPUModel max_model;
+static bool cached;
+
+if (cached) {
+return _model;
+}
+
+if (kvm_enabled()) {
+error_setg(errp, "KVM does not support CPU models.");
+} else {
+/* TCG enulates a z900 */
+max_model.def = _cpu_defs[0];
+bitmap_copy(max_model.features, max_model.def->default_feat,
+S390_FEAT_MAX);
+}
+if (!*errp) {
+cached = true;
+return _model;
+}
+#endif
+return NULL;
+}
+
+static inline void apply_cpu_model(const S390CPUModel *model, Error **errp)
+{
+#ifndef CONFIG_USER_ONLY
+static S390CPUModel applied_model;
+static bool applied;
+
+/*
+ * We have the same model for all VCPUs. KVM can only be configured before
+ * any VCPUs are defined in KVM.
+ */
+if (applied) {
+if (model && memcmp(_model, model, sizeof(S390CPUModel))) {
+error_setg(errp, "Mixed CPU models are not supported on s390x.");
+  

[Qemu-devel] [Patch v3 21/30] s390x/kvm: implement CPU model support

[David Hildenbrand]

Let's implement our two hooks so we can support CPU models.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 target-s390x/cpu_models.c |  75 +++-
 target-s390x/cpu_models.h |  50 
 target-s390x/kvm.c| 295 ++
 3 files changed, 417 insertions(+), 3 deletions(-)

diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index d794f02..abe35e1 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -162,6 +162,61 @@ bool s390_has_feat(S390Feat feat)
 return test_bit(feat, cpu->model->features);
 }
 
+uint8_t s390_get_gen_for_cpu_type(uint16_t type)
+{
+int i;
+
+for (i = 0; i < ARRAY_SIZE(s390_cpu_defs); i++) {
+if (s390_cpu_defs[i].type == type) {
+return s390_cpu_defs[i].gen;
+}
+}
+return 0;
+}
+
+const S390CPUDef *s390_find_cpu_def(uint16_t type, uint8_t gen, uint8_t ec_ga,
+S390FeatBitmap features)
+{
+const S390CPUDef *last_compatible = NULL;
+int i;
+
+if (!gen) {
+ec_ga = 0;
+}
+if (!gen && type) {
+gen = s390_get_gen_for_cpu_type(type);
+}
+
+for (i = 0; i < ARRAY_SIZE(s390_cpu_defs); i++) {
+const S390CPUDef *def = _cpu_defs[i];
+S390FeatBitmap missing;
+
+/* don't even try newer generations if we know the generation */
+if (gen) {
+if (def->gen > gen) {
+break;
+} else if (def->gen == gen && ec_ga && def->ec_ga > ec_ga) {
+break;
+}
+}
+
+if (features) {
+/* see if the model satisfies the minimum features */
+bitmap_andnot(missing, def->base_feat, features, S390_FEAT_MAX);
+if (!bitmap_empty(missing, S390_FEAT_MAX)) {
+break;
+}
+}
+
+/* stop the search if we found the exact model */
+if (def->type == type && def->ec_ga == ec_ga) {
+return def;
+}
+last_compatible = def;
+}
+return last_compatible;
+}
+
 struct S390PrintCpuListInfo {
 FILE *f;
 fprintf_function print;
@@ -324,7 +379,7 @@ static S390CPUModel *get_max_cpu_model(Error **errp)
 }
 
 if (kvm_enabled()) {
-error_setg(errp, "KVM does not support CPU models.");
+kvm_s390_get_host_cpu_model(_model, errp);
 } else {
 /* TCG enulates a z900 */
 max_model.def = _cpu_defs[0];
@@ -357,8 +412,7 @@ static inline void apply_cpu_model(const S390CPUModel 
*model, Error **errp)
 }
 
 if (kvm_enabled()) {
-/* FIXME KVM */
-error_setg(errp, "KVM doesn't support CPU models.");
+kvm_s390_apply_cpu_model(model, errp);
 } else if (model) {
 /* FIXME TCG - use data for stdip/stfl */
 }
@@ -563,6 +617,21 @@ static void s390_cpu_model_initfn(Object *obj)
 #ifdef CONFIG_KVM
 static void s390_host_cpu_model_initfn(Object *obj)
 {
+S390CPU *cpu = S390_CPU(obj);
+Error *err = NULL;
+
+if (!kvm_enabled() || !kvm_s390_cpu_models_supported()) {
+return;
+}
+
+cpu->model = g_malloc0(sizeof(*cpu->model));
+kvm_s390_get_host_cpu_model(cpu->model, );
+if (err) {
+error_report_err(err);
+g_free(cpu->model);
+/* fallback to unsupported cpu models */
+cpu->model = NULL;
+}
 }
 #endif
 
diff --git a/target-s390x/cpu_models.h b/target-s390x/cpu_models.h
index 986f7cb..a1ee3d6 100644
--- a/target-s390x/cpu_models.h
+++ b/target-s390x/cpu_models.h
@@ -43,7 +43,25 @@ typedef struct S390CPUModel {
 uint8_t cpu_ver;/* CPU version, usually "ff" for kvm */
 } S390CPUModel;
 
+/*
+ * CPU ID
+ *
+ * bits 0-7: Zeroes (ff for kvm)
+ * bits 8-31: CPU ID (serial number)
+ * bits 32-48: Machine type
+ * bits 48-63: Zeroes
+ */
+#define cpuid_type(x) (((x) >> 16) & 0x)
+#define cpuid_id(x)   (((x) >> 32) & 0xff)
+#define cpuid_ver(x)  (((x) >> 56) & 0xff)
+
+#define lowest_ibc(x) (((uint32_t)(x) >> 16) & 0xfff)
+#define unblocked_ibc(x)  ((uint32_t)(x) & 0xfff)
+#define has_ibc(x)(lowest_ibc(x) != 0)
+
 #define S390_GEN_Z10 0xa
+#define ibc_gen(x)(x == 0 ? 0 : ((x >> 4) + S390_GEN_Z10))
+#define ibc_ec_ga(x)  (x & 0xf)
 
 uint32_t s390_get_hmfai(void);
 uint8_t s390_get_mha_pow(void);
@@ -59,5 +77,37 @@ static inline uint16_t s390_ibc_from_cpu_model(const 
S390CPUModel *model)
 }
 void s390_get_feat_block(S390FeatType type, uint8_t *data);
 bool s390_has_feat(S390Feat feat);
+uint8_t s390_get_gen_for_cpu_type(uint16_t type);
+static inline bool s390_known_cpu_type(uint16_t type)
+{
+return s390_get_gen_for_cpu_type(type) != 0;
+}
+static inline uint64_t s390_cpuid_from_cpu_model(const S390CPUModel *model)
+{
+return ((uint64_t)model->cpu_ver << 56) |
+   ((uint64_t)model->cpu_id << 32) |
+   ((uint64_t)model->def->type << 

[Qemu-devel] [Patch v3 30/30] s390x/cpumodel: implement QMP interface "query-cpu-model-baseline"

[David Hildenbrand]

Let's implement that interface by reusing our conversion code and
lookup code for CPU definitions.

In order to find a compatible CPU model, we first detect the maximum
possible CPU generation and then try to find a maximum model, satisfying
all base features (not exceeding the maximum generation).

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 qapi-schema.json  |  3 ++-
 target-s390x/cpu_models.c | 55 +++
 2 files changed, 57 insertions(+), 1 deletion(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index ddf921b..6673e89 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3283,7 +3283,8 @@
 #   global properties may affect expansion of CPU models. Using
 #   query-cpu-model-expansion while using these is not advised.
 #
-# Some architectures may not support baselining CPU models.
+# Some architectures may not support baselining CPU models. s390x supports
+# baselining CPU models.
 #
 # Returns: a CpuModelBaselineInfo. Returns an error if baselining CPU models is
 #  not supported, if a model cannot be used, if a model contains
diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index cd37bb1..755d35a 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -534,6 +534,61 @@ CpuModelCompareInfo 
*arch_query_cpu_model_comparison(CpuModelInfo *infoa,
 }
 return compare_info;
 }
+
+CpuModelBaselineInfo *arch_query_cpu_model_baseline(CpuModelInfo *infoa,
+CpuModelInfo *infob,
+Error **errp)
+{
+CpuModelBaselineInfo *baseline_info;
+S390CPUModel modela, modelb, model;
+uint16_t cpu_type;
+uint8_t max_gen_ga;
+uint8_t max_gen;
+
+/* convert both models to our internal representation */
+cpu_model_from_info(, infoa, errp);
+if (*errp) {
+return NULL;
+}
+
+cpu_model_from_info(, infob, errp);
+if (*errp) {
+return NULL;
+}
+
+/* features both models support */
+bitmap_and(model.features, modela.features, modelb.features, 
S390_FEAT_MAX);
+
+/* detect the maximum model not regarding features */
+if (modela.def->gen == modelb.def->gen) {
+if (modela.def->type == modelb.def->type) {
+cpu_type = modela.def->type;
+} else {
+cpu_type = 0;
+}
+max_gen = modela.def->gen;
+max_gen_ga = MIN(modela.def->ec_ga, modelb.def->ec_ga);
+} else if (modela.def->gen > modelb.def->gen) {
+cpu_type = modelb.def->type;
+max_gen = modelb.def->gen;
+max_gen_ga = modelb.def->ec_ga;
+} else {
+cpu_type = modela.def->type;
+max_gen = modela.def->gen;
+max_gen_ga = modela.def->ec_ga;
+}
+
+model.def = s390_find_cpu_def(cpu_type, max_gen, max_gen_ga,
+  model.features);
+/* strip off features not part of the max model */
+bitmap_and(model.features, model.features, model.def->full_feat,
+   S390_FEAT_MAX);
+
+baseline_info = g_malloc0(sizeof(*baseline_info));
+baseline_info->model = g_malloc0(sizeof(*baseline_info->model));
+cpu_info_from_model(baseline_info->model, , true);
+return baseline_info;
+}
 #endif
 
 static void check_consistency(const S390CPUModel *model)
-- 
2.6.6

[Qemu-devel] [Patch v3 29/30] s390x/cpumodel: implement QMP interface "query-cpu-model-comparison"

[David Hildenbrand]

Let's implement that interface by reusing our convertion code implemented
for expansion.

We use CPU generations and CPU features to calculate the result. This
means, that a zEC12 cannot simply be converted into a z13 by stripping
of features. This is required, as other magic values (e.g. maximum
address sizes) belong to a CPU generation and cannot simply be
emulated by an older generation.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 qapi-schema.json  |  3 +-
 target-s390x/cpu_models.c | 84 +++
 2 files changed, 86 insertions(+), 1 deletion(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index 93d2dc2..ddf921b 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3232,7 +3232,8 @@
 #   global properties may affect expansion of CPU models. Using
 #   query-cpu-model-expansion while using these is not advised.
 #
-# Some architectures may not support comparing CPU models.
+# Some architectures may not support comparing CPU models. s390x supports
+# comparing CPU models.
 #
 # Returns: a CpuModelBaselineInfo. Returns an error if comparing CPU models is
 #  not supported, if a model cannot be used, if a model contains
diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index ee52ac0..cd37bb1 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -450,6 +450,90 @@ CpuModelExpansionInfo 
*arch_query_cpu_model_expansion(CpuModelExpansionType type
 cpu_info_from_model(expansion_info->model, _model, delta_changes);
 return expansion_info;
 }
+
+static void list_add_feat(const char *name, void *opaque)
+{
+strList **last = (strList **) opaque;
+strList *entry;
+
+entry = g_malloc0(sizeof(*entry));
+entry->value = g_strdup(name);
+entry->next = *last;
+*last = entry;
+}
+
+CpuModelCompareInfo *arch_query_cpu_model_comparison(CpuModelInfo *infoa,
+ CpuModelInfo *infob,
+ Error **errp)
+{
+CpuModelCompareResult feat_result, gen_result;
+CpuModelCompareInfo *compare_info;
+S390FeatBitmap missing, added;
+S390CPUModel modela, modelb;
+
+/* convert both models to our internal representation */
+cpu_model_from_info(, infoa, errp);
+if (*errp) {
+return NULL;
+}
+cpu_model_from_info(, infob, errp);
+if (*errp) {
+return NULL;
+}
+compare_info = g_malloc0(sizeof(*compare_info));
+
+/* check the cpu generation and ga level */
+if (modela.def->gen == modelb.def->gen) {
+if (modela.def->ec_ga == modelb.def->ec_ga) {
+/* ec and corresponding bc are identical */
+gen_result = CPU_MODEL_COMPARE_RESULT_IDENTICAL;
+} else if (modela.def->ec_ga < modelb.def->ec_ga) {
+gen_result = CPU_MODEL_COMPARE_RESULT_SUBSET;
+} else {
+gen_result = CPU_MODEL_COMPARE_RESULT_SUPERSET;
+}
+} else if (modela.def->gen < modelb.def->gen) {
+gen_result = CPU_MODEL_COMPARE_RESULT_SUBSET;
+} else {
+gen_result = CPU_MODEL_COMPARE_RESULT_SUPERSET;
+}
+if (gen_result != CPU_MODEL_COMPARE_RESULT_IDENTICAL) {
+/* both models cannot be made identical */
+list_add_feat("type", _info->responsible_properties);
+}
+
+/* check the feature set */
+if (bitmap_equal(modela.features, modelb.features, S390_FEAT_MAX)) {
+feat_result = CPU_MODEL_COMPARE_RESULT_IDENTICAL;
+} else {
+bitmap_andnot(missing, modela.features, modelb.features, 
S390_FEAT_MAX);
+s390_feat_bitmap_to_ascii(missing,
+  _info->responsible_properties,
+  list_add_feat);
+bitmap_andnot(added, modelb.features, modela.features, S390_FEAT_MAX);
+s390_feat_bitmap_to_ascii(added, _info->responsible_properties,
+  list_add_feat);
+if (bitmap_empty(missing, S390_FEAT_MAX)) {
+feat_result = CPU_MODEL_COMPARE_RESULT_SUBSET;
+} else if (bitmap_empty(added, S390_FEAT_MAX)) {
+feat_result = CPU_MODEL_COMPARE_RESULT_SUPERSET;
+} else {
+feat_result = CPU_MODEL_COMPARE_RESULT_INCOMPATIBLE;
+}
+}
+
+/* combine the results */
+if (gen_result == feat_result) {
+compare_info->result = gen_result;
+} else if (feat_result == CPU_MODEL_COMPARE_RESULT_IDENTICAL) {
+compare_info->result = gen_result;
+} else if (gen_result == CPU_MODEL_COMPARE_RESULT_IDENTICAL) {
+compare_info->result = feat_result;
+} else {
+compare_info->result = CPU_MODEL_COMPARE_RESULT_INCOMPATIBLE;
+}
+return compare_info;
+}
 #endif
 
 static void check_consistency(const S390CPUModel *model)
-- 
2.6.6

[Qemu-devel] [Patch v3 07/30] s390x/cpumodel: introduce CPU feature group definitions

[David Hildenbrand]

Let's use the generated groups to create feature group representations for
the user. These groups can later be used to enable/disable multiple
features in one shot and will be used to reduce the amount of reported
features to the user if all subfeatures are in place.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 target-s390x/cpu_features.c | 44 +++-
 target-s390x/cpu_features.h | 23 +++
 2 files changed, 66 insertions(+), 1 deletion(-)

diff --git a/target-s390x/cpu_features.c b/target-s390x/cpu_features.c
index dd14e78..2519506 100644
--- a/target-s390x/cpu_features.c
+++ b/target-s390x/cpu_features.c
@@ -12,6 +12,7 @@
 
 #include "qemu/osdep.h"
 #include "cpu_features.h"
+#include "gen-features.h"
 
 #define FEAT_INIT(_name, _type, _bit, _desc) \
 {\
@@ -320,14 +321,55 @@ void s390_add_from_feat_block(S390FeatBitmap features, 
S390FeatType type,
 }
 }
 
-void s390_feat_bitmap_to_ascii(const S390FeatBitmap bitmap, void *opaque,
+void s390_feat_bitmap_to_ascii(const S390FeatBitmap features, void *opaque,
void (*fn)(const char *name, void *opaque))
 {
+S390FeatBitmap bitmap, tmp;
+S390FeatGroup group;
 S390Feat feat;
 
+bitmap_copy(bitmap, features, S390_FEAT_MAX);
+
+/* process whole groups first */
+for (group = 0; group < S390_FEAT_GROUP_MAX; group++) {
+const S390FeatGroupDef *def = s390_feat_group_def(group);
+
+bitmap_and(tmp, bitmap, def->feat, S390_FEAT_MAX);
+if (bitmap_equal(tmp, def->feat, S390_FEAT_MAX)) {
+bitmap_andnot(bitmap, bitmap, def->feat, S390_FEAT_MAX);
+fn(def->name, opaque);
+}
+}
+
+/* report leftovers as separate features */
 feat = find_first_bit(bitmap, S390_FEAT_MAX);
 while (feat < S390_FEAT_MAX) {
 fn(s390_feat_def(feat)->name, opaque);
 feat = find_next_bit(bitmap, S390_FEAT_MAX, feat + 1);
 };
 }
+
+#define FEAT_GROUP_INIT(_name, _group, _desc)\
+{\
+.name = _name,   \
+.desc = _desc,   \
+.feat = { S390_FEAT_GROUP_LIST_ ## _group }, \
+}
+
+/* indexed by feature group number for easy lookup */
+static const S390FeatGroupDef s390_feature_groups[] = {
+FEAT_GROUP_INIT("plo", PLO, "Perform-locked-operation facility"),
+FEAT_GROUP_INIT("tods", TOD_CLOCK_STEERING, "Tod-clock-steering facility"),
+FEAT_GROUP_INIT("gen13ptff", GEN13_PTFF, "PTFF enhancements introduced 
with z13"),
+FEAT_GROUP_INIT("msa", MSA, "Message-security-assist facility"),
+FEAT_GROUP_INIT("msa1", MSA_EXT_1, "Message-security-assist-extension 1 
facility"),
+FEAT_GROUP_INIT("msa2", MSA_EXT_2, "Message-security-assist-extension 2 
facility"),
+FEAT_GROUP_INIT("msa3", MSA_EXT_3, "Message-security-assist-extension 3 
facility"),
+FEAT_GROUP_INIT("msa4", MSA_EXT_4, "Message-security-assist-extension 4 
facility"),
+FEAT_GROUP_INIT("msa5", MSA_EXT_5, "Message-security-assist-extension 5 
facility"),
+};
+
+const S390FeatGroupDef *s390_feat_group_def(S390FeatGroup group)
+{
+return _feature_groups[group];
+}
diff --git a/target-s390x/cpu_features.h b/target-s390x/cpu_features.h
index ed278df..9e295ad 100644
--- a/target-s390x/cpu_features.h
+++ b/target-s390x/cpu_features.h
@@ -272,6 +272,29 @@ void s390_add_from_feat_block(S390FeatBitmap features, 
S390FeatType type,
 void s390_feat_bitmap_to_ascii(const S390FeatBitmap features, void *opaque,
void (*fn)(const char *name, void *opaque));
 
+/* static groups that will never change */
+typedef enum {
+S390_FEAT_GROUP_PLO,
+S390_FEAT_GROUP_TOD_CLOCK_STEERING,
+S390_FEAT_GROUP_GEN13_PTFF_ENH,
+S390_FEAT_GROUP_MSA,
+S390_FEAT_GROUP_MSA_EXT_1,
+S390_FEAT_GROUP_MSA_EXT_2,
+S390_FEAT_GROUP_MSA_EXT_3,
+S390_FEAT_GROUP_MSA_EXT_4,
+S390_FEAT_GROUP_MSA_EXT_5,
+S390_FEAT_GROUP_MAX,
+} S390FeatGroup;
+
+/* Definition of a CPU feature group */
+typedef struct {
+const char *name;   /* name exposed to the user */
+const char *desc;   /* description exposed to the user */
+S390FeatBitmap feat;/* features contained in the group */
+} S390FeatGroupDef;
+
+const S390FeatGroupDef *s390_feat_group_def(S390FeatGroup group);
+
 #define BE_BIT_NR(BIT) (BIT ^ (BITS_PER_LONG - 1))
 #define BE_BIT(BIT) (1ULL < BE_BIT_NR(BIT))
 
-- 
2.6.6

[Qemu-devel] [Patch v3 00/30] s390x CPU models: exposing features

[David Hildenbrand]

Only minor changes this version. I think this is ready to go. If we
find  model inconsitencies or bad feature names, we can still fix them
until the next release.

Latest version can be found on branch:
github.com/cohuck/qemu cpumodel-s390x-v3

Concept

This is our second attempt to implement CPU models for s390x. We realized
that we also want to have features exposed via the CPU model. While doing
that we realized that we want to have a better interface for libvirt.

Unfortunately, CPU models on s390x are special and we have to take care of:
- A CPU like z13 looks differently in various environments (under different
  LPAR versions, under different z/VM versions, under different KVM
  versions, export regulation) - we have _a lot_ of feature variability.
- We still have certain features that are not published but might be
  implemented/introduced in the future. As they are a theoretical part
  of a CPU already, we have to find a way to model these future changes.
- We still have certain features that are already published, but not
  implemented. Implementation might be added in the future in KVM.
- We heavily rely on KVM to tell us which features it can actually
  virtualize - user space queries like "STFL(e)" give no guarantees.
- Certain "subfeatures" were introduced in one run. In practice, they are
  always around, but in theory, subfeatures could be dropped in the future.
- Just because two CPU models have the same features doesn't mean they
  are equal - some internal numbers might be different. E.g. we won't allow
  running a z13 under a zBC12 just by turning off features.
- We cannot blindly enable all possible features for a CPU generation,
  the IBC "Instruction Blocking Control" in KVM will try to block
  instructions introduced with certain features. So a CPU generation always
  has some maximum feature set that is guaranteed to work.

It all boils down to a specific released CPU to have.
a) A minimum feature set that we expect it to be have on every hypervisor.
b) A variable part that depends on the hypervisor and that could be
   extended in the future (adding not yet implemented features) that we
   always want to enable later on.
c) A variable part that we want to enable only if requested - nested
   virtualization ("vsie") and assists are one example.

As we want our users to always make use of most features, e.g. when using
the "z13" CPU model, we will have to update our CPU models between QEMU
releases, to include the latest feature additions we implemented/unlocked.
We're badically trying to be able at one point in the future to really look
like a maximum "z13" CPU in QEMU. However, that means that a "z13" CPU
looks differently on different QEMU versions. We will make sure using
compatibility machines, that migration stays safe.

However, if the feature set of a CPU model is bound to a compatibility
machine, how can it be of any help when used without a compatibility
machine? E.g. when indicating the host CPU model in "virsh capabilities",
simply saying "z13" is not really clear. Also when baselining and
comparing CPU models, we usually have no compatibility machine "context"
at hand. For this reason we introduce "static" CPU models, which will
never change, so everybody knows without a compatibility machine, what we
are talking about.

CPU definitions/models can be categorized:
1. migratable: all features _can_ be identified + properly migrated.
--> With migratable=off on x86, "host" cannot be migrated properly.
2. migration-safe: in combination with a QEMU machine, migration will work.
--> No CPU features are lost/added during migration
3. static: not bound to a QEMU machine - featureset will never* change.
--> Everybody knows which features are part of it, not affected by a compat
machine. "virsh capabilities" can show this as "host" model.
*except for trivial bugfixes, especially on very old models nobody used in
production. Should be avoided.

We are currently planning to also support a "migratable=off" on s390x
for the "-cpu host" model, in order to enable all features, including not
recognized ones.

So we have on s390x:
a) A static CPU model for each released CPU that will never change and
   maps to the minimum feature set we expect to be around on all
   hypervisors. e.g. "z13-base" or "z10EC.2-base".
b) A "default" CPU model for each released CPU, that can change between
   QEMU versions and that will always include the features we expect to
   be around in our currently supported environments and will contain only
   features we expect to be stable. E.g. nested virtualization will not be
   contained in these models  for now. "z13" or "z10EC.2" are examples.
c) An internal "maximum" CPU model for each generation that tells us which
   features were supported as a maximum back when the hardware was
   released. This will not be exposed but used internally.

To not have to replicate all 

[Qemu-devel] [Patch v3 24/30] s390x/kvm: don't enable key wrapping if msa3 is disabled

[David Hildenbrand]

As the CPU model now controls msa3, trying to set wrapping keys without
msa3 being around/enable in the kernel will produce misleading errors.

So let's simply not configure key wrapping if msa3 is not enabled and
make compat machines with disabled CPU model work correctly.

Signed-off-by: David Hildenbrand 
---
 target-s390x/cpu_models.c | 3 +++
 target-s390x/kvm.c| 6 --
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index abe35e1..7419bd0 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -155,6 +155,9 @@ bool s390_has_feat(S390Feat feat)
 if (feat == S390_FEAT_RUNTIME_INSTRUMENTATION) {
 return kvm_s390_get_ri();
 }
+if (feat == S390_FEAT_MSA_EXT_3) {
+return true;
+}
 }
 #endif
 return 0;
diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c
index 5de4490..8fa50f0 100644
--- a/target-s390x/kvm.c
+++ b/target-s390x/kvm.c
@@ -259,8 +259,10 @@ static void kvm_s390_init_dea_kw(void)
 
 void kvm_s390_crypto_reset(void)
 {
-kvm_s390_init_aes_kw();
-kvm_s390_init_dea_kw();
+if (s390_has_feat(S390_FEAT_MSA_EXT_3)) {
+kvm_s390_init_aes_kw();
+kvm_s390_init_dea_kw();
+}
 }
 
 int kvm_arch_init(MachineState *ms, KVMState *s)
-- 
2.6.6

[Qemu-devel] [Patch v3 02/30] s390x/cpumodel: "host" and "qemu" as CPU subclasses

[David Hildenbrand]

This patch introduces two CPU models, "host" and "qemu".
"qemu" is used as default when running under TCG. "host" is used
as default when running under KVM. "host" cannot be used without KVM.
"host" is not migration-safe. They both inherit from the base s390x CPU,
which is turned into an abstract class.

This patch also changes CPU creation to take care of the passed CPU string
and reuses common code parse_features() function for that purpose. Unknown
CPU definitions are now reported. The "-cpu ?" and "query-cpu-definition"
commands are changed to list all CPU subclasses automatically, including
migration-safety and whether static.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 hw/s390x/s390-virtio.c |   6 +-
 target-s390x/Makefile.objs |   2 +-
 target-s390x/cpu-qom.h |   4 +
 target-s390x/cpu.c |  33 ++---
 target-s390x/cpu.h |   2 +
 target-s390x/cpu_models.c  | 177 +
 target-s390x/helper.c  |  33 -
 7 files changed, 229 insertions(+), 28 deletions(-)
 create mode 100644 target-s390x/cpu_models.c

diff --git a/hw/s390x/s390-virtio.c b/hw/s390x/s390-virtio.c
index 544c616..0a96347 100644
--- a/hw/s390x/s390-virtio.c
+++ b/hw/s390x/s390-virtio.c
@@ -101,7 +101,11 @@ void s390_init_cpus(MachineState *machine)
 gchar *name;
 
 if (machine->cpu_model == NULL) {
-machine->cpu_model = "host";
+if (kvm_enabled()) {
+machine->cpu_model = "host";
+} else {
+machine->cpu_model = "qemu";
+}
 }
 
 cpu_states = g_new0(S390CPU *, max_cpus);
diff --git a/target-s390x/Makefile.objs b/target-s390x/Makefile.objs
index dd62cbd..34bd693 100644
--- a/target-s390x/Makefile.objs
+++ b/target-s390x/Makefile.objs
@@ -1,5 +1,5 @@
 obj-y += translate.o helper.o cpu.o interrupt.o
 obj-y += int_helper.o fpu_helper.o cc_helper.o mem_helper.o misc_helper.o
-obj-y += gdbstub.o
+obj-y += gdbstub.o cpu_models.o
 obj-$(CONFIG_SOFTMMU) += machine.o ioinst.o arch_dump.o mmu_helper.o
 obj-$(CONFIG_KVM) += kvm.o
diff --git a/target-s390x/cpu-qom.h b/target-s390x/cpu-qom.h
index 66b5d18..bb993d4 100644
--- a/target-s390x/cpu-qom.h
+++ b/target-s390x/cpu-qom.h
@@ -45,6 +45,10 @@ typedef struct S390CPUClass {
 /*< private >*/
 CPUClass parent_class;
 /*< public >*/
+bool kvm_required;
+bool is_static;
+bool is_migration_safe;
+const char *desc;
 
 int64_t next_cpu_id;
 
diff --git a/target-s390x/cpu.c b/target-s390x/cpu.c
index e43e2d6..44e53ec 100644
--- a/target-s390x/cpu.c
+++ b/target-s390x/cpu.c
@@ -44,30 +44,6 @@
 #define CR0_RESET   0xE0UL
 #define CR14_RESET  0xC200UL;
 
-/* generate CPU information for cpu -? */
-void s390_cpu_list(FILE *f, fprintf_function cpu_fprintf)
-{
-#ifdef CONFIG_KVM
-(*cpu_fprintf)(f, "s390 %16s\n", "host");
-#endif
-}
-
-#ifndef CONFIG_USER_ONLY
-CpuDefinitionInfoList *arch_query_cpu_definitions(Error **errp)
-{
-CpuDefinitionInfoList *entry;
-CpuDefinitionInfo *info;
-
-info = g_malloc0(sizeof(*info));
-info->name = g_strdup("host");
-
-entry = g_malloc0(sizeof(*entry));
-entry->value = info;
-
-return entry;
-}
-#endif
-
 static void s390_cpu_set_pc(CPUState *cs, vaddr value)
 {
 S390CPU *cpu = S390_CPU(cs);
@@ -206,6 +182,12 @@ static void s390_cpu_realizefn(DeviceState *dev, Error 
**errp)
 CPUS390XState *env = >env;
 Error *err = NULL;
 
+/* the model has to be realized before qemu_init_vcpu() due to kvm */
+s390_realize_cpu_model(cs, );
+if (err) {
+goto out;
+}
+
 #if !defined(CONFIG_USER_ONLY)
 if (cpu->id >= max_cpus) {
 error_setg(, "Unable to add CPU: %" PRIi64
@@ -435,6 +417,7 @@ static void s390_cpu_class_init(ObjectClass *oc, void *data)
 scc->cpu_reset = s390_cpu_reset;
 scc->initial_cpu_reset = s390_cpu_initial_reset;
 cc->reset = s390_cpu_full_reset;
+cc->class_by_name = s390_cpu_class_by_name,
 cc->has_work = s390_cpu_has_work;
 cc->do_interrupt = s390_cpu_do_interrupt;
 cc->dump_state = s390_cpu_dump_state;
@@ -470,7 +453,7 @@ static const TypeInfo s390_cpu_type_info = {
 .instance_size = sizeof(S390CPU),
 .instance_init = s390_cpu_initfn,
 .instance_finalize = s390_cpu_finalize,
-.abstract = false,
+.abstract = true,
 .class_size = sizeof(S390CPUClass),
 .class_init = s390_cpu_class_init,
 };
diff --git a/target-s390x/cpu.h b/target-s390x/cpu.h
index c216bda..e34742e 100644
--- a/target-s390x/cpu.h
+++ b/target-s390x/cpu.h
@@ -631,6 +631,8 @@ extern void subsystem_reset(void);
 
 void s390_cpu_list(FILE *f, fprintf_function cpu_fprintf);
 #define cpu_list s390_cpu_list
+void s390_realize_cpu_model(CPUState *cs, Error **errp);
+ObjectClass *s390_cpu_class_by_name(const char *name);
 
 #define EXCP_EXT 1 /* external interrupt */
 #define EXCP_SVC 2 /* supervisor call (syscall) */
diff --git 

[Qemu-devel] [Patch v3 14/30] s390x/sclp: introduce sclp feature blocks

[David Hildenbrand]

The sclp "read cpu info" and "read scp info" commands can include
features for the cpu info and configuration characteristics (extended),
decribing some advanced features available in the configuration.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 include/hw/s390x/sclp.h | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index ba28d1d..30a40ea 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -98,11 +98,14 @@ typedef struct SCCBHeader {
 } QEMU_PACKED SCCBHeader;
 
 #define SCCB_DATA_LEN (SCCB_SIZE - sizeof(SCCBHeader))
+#define SCCB_CPU_FEATURE_LEN 6
 
 /* CPU information */
 typedef struct CPUEntry {
 uint8_t address;
-uint8_t reserved0[13];
+uint8_t reserved0;
+uint8_t features[SCCB_CPU_FEATURE_LEN];
+uint8_t reserved2[6];
 uint8_t type;
 uint8_t reserved1;
 } QEMU_PACKED CPUEntry;
@@ -118,10 +121,13 @@ typedef struct ReadInfo {
 uint8_t  loadparm[8];   /* 24-31 */
 uint8_t  _reserved3[48 - 32];   /* 32-47 */
 uint64_t facilities;/* 48-55 */
-uint8_t  _reserved0[100 - 56];
+uint8_t  _reserved0[80 - 56];   /* 56-79 */
+uint8_t  conf_char[96 - 80];/* 80-95 */
+uint8_t  _reserved4[100 - 96];  /* 96-99 */
 uint32_t rnsize2;
 uint64_t rnmax2;
-uint8_t  _reserved4[120-112];   /* 112-119 */
+uint8_t  _reserved6[116 - 112]; /* 112-115 */
+uint8_t  conf_char_ext[120 - 116];   /* 116-119 */
 uint16_t highest_cpu;
 uint8_t  _reserved5[128 - 122]; /* 122-127 */
 struct CPUEntry entries[0];
-- 
2.6.6

[Qemu-devel] [Patch v3 04/30] s390x/cpumodel: introduce CPU features

[David Hildenbrand]

From: Michael Mueller 

The patch introduces s390x CPU features (most of them refered to as
facilities) along with their discription and some functions that will be
helpful when working with the features later on.

Please note that we don't introduce all known CPU features, only the
ones currently supported by KVM + QEMU. We don't want to enable later
on blindly any facilities, for which we don't know yet if we need QEMU
support to properly support them (e.g. migrate additional state when
active). We can update QEMU later on.

Acked-by: Cornelia Huck 
Signed-off-by: Michael Mueller 
Signed-off-by: David Hildenbrand 
[reworked to include non-stfle features, added definitions]
---
 target-s390x/Makefile.objs  |   2 +-
 target-s390x/cpu_features.c | 333 
 target-s390x/cpu_features.h | 278 
 3 files changed, 612 insertions(+), 1 deletion(-)
 create mode 100644 target-s390x/cpu_features.c
 create mode 100644 target-s390x/cpu_features.h

diff --git a/target-s390x/Makefile.objs b/target-s390x/Makefile.objs
index 34bd693..4266c87 100644
--- a/target-s390x/Makefile.objs
+++ b/target-s390x/Makefile.objs
@@ -1,5 +1,5 @@
 obj-y += translate.o helper.o cpu.o interrupt.o
 obj-y += int_helper.o fpu_helper.o cc_helper.o mem_helper.o misc_helper.o
-obj-y += gdbstub.o cpu_models.o
+obj-y += gdbstub.o cpu_models.o cpu_features.o
 obj-$(CONFIG_SOFTMMU) += machine.o ioinst.o arch_dump.o mmu_helper.o
 obj-$(CONFIG_KVM) += kvm.o
diff --git a/target-s390x/cpu_features.c b/target-s390x/cpu_features.c
new file mode 100644
index 000..dd14e78
--- /dev/null
+++ b/target-s390x/cpu_features.c
@@ -0,0 +1,333 @@
+/*
+ * CPU features/facilities for s390x
+ *
+ * Copyright 2016 IBM Corp.
+ *
+ * Author(s): David Hildenbrand 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#include "qemu/osdep.h"
+#include "cpu_features.h"
+
+#define FEAT_INIT(_name, _type, _bit, _desc) \
+{\
+.name = _name,   \
+.type = _type,   \
+.bit = _bit, \
+.desc = _desc,   \
+}
+
+/* indexed by feature number for easy lookup */
+static const S390FeatDef s390_features[] = {
+FEAT_INIT("esan3", S390_FEAT_TYPE_STFL, 0, "Instructions marked as n3"),
+FEAT_INIT("zarch", S390_FEAT_TYPE_STFL, 1, "z/Architecture architectural 
mode"),
+FEAT_INIT("dateh", S390_FEAT_TYPE_STFL, 3, "DAT-enhancement facility"),
+FEAT_INIT("idtes", S390_FEAT_TYPE_STFL, 4, "IDTE selective TLB 
segment-table clearing"),
+FEAT_INIT("idter", S390_FEAT_TYPE_STFL, 5, "IDTE selective TLB 
region-table clearing"),
+FEAT_INIT("asnlxr", S390_FEAT_TYPE_STFL, 6, "ASN-and-LX reuse facility"),
+FEAT_INIT("stfle", S390_FEAT_TYPE_STFL, 7, "Store-facility-list-extended 
facility"),
+FEAT_INIT("edat", S390_FEAT_TYPE_STFL, 8, "Enhanced-DAT facility"),
+FEAT_INIT("srs", S390_FEAT_TYPE_STFL, 9, "Sense-running-status facility"),
+FEAT_INIT("csske", S390_FEAT_TYPE_STFL, 10, "Conditional-SSKE facility"),
+FEAT_INIT("ctop", S390_FEAT_TYPE_STFL, 11, "Configuration-topology 
facility"),
+FEAT_INIT("ipter", S390_FEAT_TYPE_STFL, 13, "IPTE-range facility"),
+FEAT_INIT("nonqks", S390_FEAT_TYPE_STFL, 14, "Nonquiescing key-setting 
facility"),
+FEAT_INIT("etf2", S390_FEAT_TYPE_STFL, 16, "Extended-translation facility 
2"),
+FEAT_INIT("msa-base", S390_FEAT_TYPE_STFL, 17, "Message-security-assist 
facility (excluding subfunctions)"),
+FEAT_INIT("ldisp", S390_FEAT_TYPE_STFL, 18, "Long-displacement facility"),
+FEAT_INIT("ldisphp", S390_FEAT_TYPE_STFL, 19, "Long-displacement facility 
has high performance"),
+FEAT_INIT("hfpm", S390_FEAT_TYPE_STFL, 20, "HFP-multiply-add/subtract 
facility"),
+FEAT_INIT("eimm", S390_FEAT_TYPE_STFL, 21, "Extended-immediate facility"),
+FEAT_INIT("etf3", S390_FEAT_TYPE_STFL, 22, "Extended-translation facility 
3"),
+FEAT_INIT("hfpue", S390_FEAT_TYPE_STFL, 23, "HFP-unnormalized-extension 
facility"),
+FEAT_INIT("etf2eh", S390_FEAT_TYPE_STFL, 24, "ETF2-enhancement facility"),
+FEAT_INIT("stckf", S390_FEAT_TYPE_STFL, 25, "Store-clock-fast facility"),
+FEAT_INIT("parseh", S390_FEAT_TYPE_STFL, 26, "Parsing-enhancement 
facility"),
+FEAT_INIT("mvcos", S390_FEAT_TYPE_STFL, 27, 
"Move-with-optional-specification facility"),
+FEAT_INIT("tods-base", S390_FEAT_TYPE_STFL, 28, "TOD-clock-steering 
facility (excluding subfunctions)"),
+FEAT_INIT("etf3eh", S390_FEAT_TYPE_STFL, 30, "ETF3-enhancement facility"),
+FEAT_INIT("ectg", S390_FEAT_TYPE_STFL, 31, "Extract-CPU-time 

[Qemu-devel] [Patch v3 05/30] s390x/cpumodel: generate CPU feature lists for CPU models

[David Hildenbrand]

From: Michael Mueller 

This patch introduces the helper "gen-features" which allows to generate
feature list definitions at compile time. Its flexibility is better and the
error-proneness is lower when compared to static programming time added
statements.

The helper includes "target-s390x/cpu_features.h" to be able to use named
facility bits instead of numbers. The generated defines will be used for
the definition of CPU models.

We generate feature lists for each HW generation and GA for EC models. BC
models are always based on a EC version and have no separate definitions.

Base features: Features we expect to be always available in sane setups.
Migration safe - will never change. Can be seen as "minimum features
required for a CPU model".

Default features: Features we expect to be stable and around in latest
setups (e.g. having KVM support) - not migration safe.

Max features: All supported features that are theoretically allowed for a
CPU model. Exceeding these features could otherwise produce problems with
IBC (instruction blocking controls) in KVM.

Acked-by: Cornelia Huck 
Signed-off-by: Michael Mueller 
Signed-off-by: David Hildenbrand 
[generate base, default and models. renaming and cleanup]
---
 Makefile.target |   2 +-
 rules.mak   |   1 +
 target-s390x/Makefile.objs  |  20 ++
 target-s390x/gen-features.c | 507 
 4 files changed, 529 insertions(+), 1 deletion(-)
 create mode 100644 target-s390x/gen-features.c

diff --git a/Makefile.target b/Makefile.target
index a440bcb..8c7a072 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -212,7 +212,7 @@ hmp-commands-info.h: $(SRC_PATH)/hmp-commands-info.hx 
$(SRC_PATH)/scripts/hxtool
 qmp-commands-old.h: $(SRC_PATH)/qmp-commands.hx $(SRC_PATH)/scripts/hxtool
$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -h < $< > $@,"  GEN  
 $(TARGET_DIR)$@")
 
-clean:
+clean: clean-target
rm -f *.a *~ $(PROGS)
rm -f $(shell find . -name '*.[od]')
rm -f hmp-commands.h qmp-commands-old.h gdbstub-xml.c
diff --git a/rules.mak b/rules.mak
index 99cd0b3..55b0121 100644
--- a/rules.mak
+++ b/rules.mak
@@ -14,6 +14,7 @@ MAKEFLAGS += -rR
 %.cpp:
 %.m:
 %.mak:
+clean-target:
 
 # Flags for C++ compilation
 QEMU_CXXFLAGS = -D__STDC_LIMIT_MACROS $(filter-out -Wstrict-prototypes 
-Wmissing-prototypes -Wnested-externs -Wold-style-declaration 
-Wold-style-definition -Wredundant-decls, $(QEMU_CFLAGS))
diff --git a/target-s390x/Makefile.objs b/target-s390x/Makefile.objs
index 4266c87..745d501 100644
--- a/target-s390x/Makefile.objs
+++ b/target-s390x/Makefile.objs
@@ -3,3 +3,23 @@ obj-y += int_helper.o fpu_helper.o cc_helper.o mem_helper.o 
misc_helper.o
 obj-y += gdbstub.o cpu_models.o cpu_features.o
 obj-$(CONFIG_SOFTMMU) += machine.o ioinst.o arch_dump.o mmu_helper.o
 obj-$(CONFIG_KVM) += kvm.o
+
+# build and run feature list generator
+feat-src = $(SRC_PATH)/target-$(TARGET_BASE_ARCH)/
+feat-dst = $(BUILD_DIR)/$(TARGET_DIR)
+ifneq ($(MAKECMDGOALS),clean)
+GENERATED_HEADERS += $(feat-dst)gen-features.h
+endif
+
+$(feat-dst)gen-features.h: $(feat-dst)gen-features.h-timestamp
+   @cmp $< $@ >/dev/null 2>&1 || cp $< $@
+$(feat-dst)gen-features.h-timestamp: $(feat-dst)gen-features
+   $(call quiet-command,$< >$@,"  GEN   $(TARGET_DIR)gen-features.h")
+
+$(feat-dst)gen-features: $(feat-src)gen-features.c config-target.h
+   $(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(CFLAGS) -o 
$@ $<,"  CC$(TARGET_DIR)gen-features")
+
+clean-target:
+   rm -f gen-features.h-timestamp
+   rm -f gen-features.h
+   rm -f gen-features
diff --git a/target-s390x/gen-features.c b/target-s390x/gen-features.c
new file mode 100644
index 000..db63389
--- /dev/null
+++ b/target-s390x/gen-features.c
@@ -0,0 +1,507 @@
+/*
+ * S390 feature list generator
+ *
+ * Copyright 2016 IBM Corp.
+ *
+ * Author(s): Michael Mueller 
+ *David Hildenbrand 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "cpu_features.h"
+
+/* BEGIN FEATURE DEFS */
+
+#define S390_FEAT_GROUP_PLO \
+S390_FEAT_PLO_CL, \
+S390_FEAT_PLO_CLG, \
+S390_FEAT_PLO_CLGR, \
+S390_FEAT_PLO_CLX, \
+S390_FEAT_PLO_CS, \
+S390_FEAT_PLO_CSG, \
+S390_FEAT_PLO_CSGR, \
+S390_FEAT_PLO_CSX, \
+S390_FEAT_PLO_DCS, \
+S390_FEAT_PLO_DCSG, \
+S390_FEAT_PLO_DCSGR, \
+S390_FEAT_PLO_DCSX, \
+S390_FEAT_PLO_CSST, \
+S390_FEAT_PLO_CSSTG, \
+S390_FEAT_PLO_CSSTGR, \
+S390_FEAT_PLO_CSSTX, \
+S390_FEAT_PLO_CSDST, \
+S390_FEAT_PLO_CSDSTG, \
+S390_FEAT_PLO_CSDSTGR, \
+S390_FEAT_PLO_CSDSTX, \
+

[Qemu-devel] [Patch v3 23/30] s390x/kvm: let the CPU model control CMM(A)

[David Hildenbrand]

Starting with recent kernels, if the cmma attributes are available, we
actually have hardware support. Enabling CMMA then means providing the
guest VCPU with CMM, therefore enabling its CMM facility.

Let's not blindly enable CMM anymore but let's control it using CPU models.
For disabled CPU models, CMMA will continue to always get enabled.

Also enable it in the applicable default models.

Please note that CMM doesn't work with hugetlbfs, therefore we will
warn the user and keep it disabled. Migrating from/to a hugetlbfs
configuration works, as it will be disabled on both sides.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 target-s390x/gen-features.c |  1 +
 target-s390x/kvm.c  | 47 ++---
 2 files changed, 37 insertions(+), 11 deletions(-)

diff --git a/target-s390x/gen-features.c b/target-s390x/gen-features.c
index 2ae4405..349a047 100644
--- a/target-s390x/gen-features.c
+++ b/target-s390x/gen-features.c
@@ -372,6 +372,7 @@ static uint16_t full_GEN13_GA1[] = {
 static uint16_t default_GEN9_GA1[] = {
 S390_FEAT_STORE_HYPERVISOR_INFO,
 S390_FEAT_GROUP_MSA_EXT_1,
+S390_FEAT_CMM,
 };
 #define default_GEN9_GA2 EmptyFeat
 #define default_GEN9_GA3 EmptyFeat
diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c
index 3c0929b..5de4490 100644
--- a/target-s390x/kvm.c
+++ b/target-s390x/kvm.c
@@ -174,6 +174,18 @@ int kvm_s390_set_mem_limit(KVMState *s, uint64_t 
new_limit, uint64_t *hw_limit)
 return kvm_vm_ioctl(s, KVM_SET_DEVICE_ATTR, );
 }
 
+static bool kvm_s390_cmma_available(void)
+{
+static bool initialized, value;
+
+if (!initialized) {
+initialized = true;
+value = kvm_vm_check_mem_attr(kvm_state, KVM_S390_VM_MEM_ENABLE_CMMA) 
&&
+kvm_vm_check_mem_attr(kvm_state, KVM_S390_VM_MEM_CLR_CMMA);
+}
+return value;
+}
+
 void kvm_s390_cmma_reset(void)
 {
 int rc;
@@ -182,11 +194,15 @@ void kvm_s390_cmma_reset(void)
 .attr = KVM_S390_VM_MEM_CLR_CMMA,
 };
 
+if (!mem_path || !kvm_s390_cmma_available()) {
+return;
+}
+
 rc = kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, );
 trace_kvm_clear_cmma(rc);
 }
 
-static void kvm_s390_enable_cmma(KVMState *s)
+static void kvm_s390_enable_cmma(void)
 {
 int rc;
 struct kvm_device_attr attr = {
@@ -194,12 +210,7 @@ static void kvm_s390_enable_cmma(KVMState *s)
 .attr = KVM_S390_VM_MEM_ENABLE_CMMA,
 };
 
-if (!kvm_vm_check_mem_attr(s, KVM_S390_VM_MEM_ENABLE_CMMA) ||
-!kvm_vm_check_mem_attr(s, KVM_S390_VM_MEM_CLR_CMMA)) {
-return;
-}
-
-rc = kvm_vm_ioctl(s, KVM_SET_DEVICE_ATTR, );
+rc = kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, );
 trace_kvm_enable_cmma(rc);
 }
 
@@ -259,10 +270,6 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
 cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP);
 cap_s390_irq = kvm_check_extension(s, KVM_CAP_S390_INJECT_IRQ);
 
-if (!mem_path) {
-kvm_s390_enable_cmma(s);
-}
-
 if (!kvm_check_extension(s, KVM_CAP_S390_GMAP)
 || !kvm_check_extension(s, KVM_CAP_S390_COW)) {
 phys_mem_set_alloc(legacy_s390_alloc);
@@ -2520,6 +2527,11 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, 
Error **errp)
 return;
 }
 
+/* with cpu model support, CMM is only indicated if really available */
+if (kvm_s390_cmma_available()) {
+set_bit(S390_FEAT_CMM, model->features);
+}
+
 if (s390_known_cpu_type(cpu_type)) {
 /* we want the exact model, even if some features are missing */
 model->def = s390_find_cpu_def(cpu_type, ibc_gen(unblocked_ibc),
@@ -2552,6 +2564,10 @@ void kvm_s390_apply_cpu_model(const S390CPUModel *model, 
Error **errp)
 int rc;
 
 if (!model) {
+/* compatibility handling if cpu models are disabled */
+if (kvm_s390_cmma_available() && !mem_path) {
+kvm_s390_enable_cmma();
+}
 return;
 }
 if (!kvm_s390_cpu_models_supported()) {
@@ -2580,4 +2596,13 @@ void kvm_s390_apply_cpu_model(const S390CPUModel *model, 
Error **errp)
 error_setg(errp, "KVM: Error configuring CPU subfunctions: %d", rc);
 return;
 }
+/* enable CMM via CMMA - disable on hugetlbfs */
+if (test_bit(S390_FEAT_CMM, model->features)) {
+if (mem_path) {
+error_report("Warning: CMM will not be enabled because it is not "
+ "compatible to hugetlbfs.");
+} else {
+kvm_s390_enable_cmma();
+}
+}
 }
-- 
2.6.6

[Qemu-devel] [Patch v3 01/30] qmp: details about CPU definitions in query-cpu-definitions

[David Hildenbrand]

It might be of interest for tooling whether a CPU definition can be safely
used when migrating, or if e.g. CPU features might get lost during
migration when migrationg from/to a different QEMU version or host, even if
the same compatibility machine is used.

Also, we want to know if a CPU definition is static and will never change.
Beause these definitions can then be used independantly of a compatibility
machine and will always have the same feature set, they can e.g. be used
to indicate the "host" model in libvirt later on.

Let's add two return values to query-cpu-definitions, stating for each
returned CPU definition, if it is migration-safe and if it is static.

While "migration-safe" is optional, "static" will be set to "false"
automatically by all implementing architectures. If a model really was
static all the time and will be in the future, this can simply be changed
later.

Signed-off-by: David Hildenbrand 
---
 qapi-schema.json | 14 +-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index 5658723..0d9ae50 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3038,10 +3038,22 @@
 #
 # @name: the name of the CPU definition
 #
+# @migration-safe: #optional whether a CPU definition can be safely used for
+#  migration in combination with a QEMU compatibility machine
+#  when migrating between different QMU versions and between
+#  hosts  with different sets of (hardware or software)
+#  capabilities. If not provided, information is not available
+#  and callers should not assume the CPU definition to be
+#  migration-safe.
+#
+# @static: whether a CPU definition is static and will not change depending on
+#  QEMU version, machine type, machine options and accelerator options.
+#  A static model is always migration-safe.
+#
 # Since: 1.2.0
 ##
 { 'struct': 'CpuDefinitionInfo',
-  'data': { 'name': 'str' } }
+  'data': { 'name': 'str', '*migration-safe' : 'bool', 'static' : 'bool' } }
 
 ##
 # @query-cpu-definitions:
-- 
2.6.6

[Qemu-devel] [Patch v3 15/30] s390x/sclp: indicate sclp features

[David Hildenbrand]

We have three different blocks in the SCLP read-SCP information response
that indicate sclp features. Let's prepare propagation.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 hw/s390x/sclp.c   |  9 +
 target-s390x/cpu_models.c | 14 ++
 target-s390x/cpu_models.h |  1 +
 3 files changed, 24 insertions(+)

diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index 15d7114..3c126ee 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -31,11 +31,14 @@ static inline SCLPDevice *get_sclp_device(void)
 
 static void prepare_cpu_entries(SCLPDevice *sclp, CPUEntry *entry, int count)
 {
+uint8_t features[SCCB_CPU_FEATURE_LEN] = { 0 };
 int i;
 
+s390_get_feat_block(S390_FEAT_TYPE_SCLP_CPU, features);
 for (i = 0; i < count; i++) {
 entry[i].address = i;
 entry[i].type = 0;
+memcpy(entry[i].features, features, sizeof(entry[i].features));
 }
 }
 
@@ -59,6 +62,12 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
 read_info->offset_cpu = cpu_to_be16(offsetof(ReadInfo, entries));
 read_info->highest_cpu = cpu_to_be16(max_cpus);
 
+/* Configuration Characteristic (Extension) */
+s390_get_feat_block(S390_FEAT_TYPE_SCLP_CONF_CHAR,
+ read_info->conf_char);
+s390_get_feat_block(S390_FEAT_TYPE_SCLP_CONF_CHAR_EXT,
+ read_info->conf_char_ext);
+
 prepare_cpu_entries(sclp, read_info->entries, cpu_count);
 
 read_info->facilities = cpu_to_be64(SCLP_HAS_CPU_INFO |
diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index e226a2f..4968fdc 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -74,6 +74,20 @@ static const S390CPUDef s390_cpu_defs[] = {
 CPUDEF_INIT(0x2965, 13, 2, 47, 0x0800U, "z13s", "IBM z13s GA1"),
 };
 
+void s390_get_feat_block(S390FeatType type, uint8_t *data)
+{
+static S390CPU *cpu;
+
+if (!cpu) {
+cpu = S390_CPU(qemu_get_cpu(0));
+}
+
+if (!cpu || !cpu->model) {
+return;
+}
+s390_fill_feat_block(cpu->model->features, type, data);
+}
+
 bool s390_has_feat(S390Feat feat)
 {
 static S390CPU *cpu;
diff --git a/target-s390x/cpu_models.h b/target-s390x/cpu_models.h
index fe988cc..04c47c6 100644
--- a/target-s390x/cpu_models.h
+++ b/target-s390x/cpu_models.h
@@ -43,6 +43,7 @@ typedef struct S390CPUModel {
 uint8_t cpu_ver;/* CPU version, usually "ff" for kvm */
 } S390CPUModel;
 
+void s390_get_feat_block(S390FeatType type, uint8_t *data);
 bool s390_has_feat(S390Feat feat);
 
 #endif /* TARGET_S390X_CPU_MODELS_H */
-- 
2.6.6

[Qemu-devel] [Patch v3 20/30] s390x/kvm: allow runtime-instrumentation for "none" machine

[David Hildenbrand]

To be able to query the correct host model for the "none" machine,
let's allow runtime-instrumentation for that machine.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 hw/s390x/s390-virtio-ccw.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 91d9cef..f7dd2c8 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -249,6 +249,11 @@ bool ri_allowed(void)
 
 return s390mc->ri_allowed;
 }
+/*
+ * Make sure the "none" machine can have ri, otherwise it won't * be
+ * unlocked in KVM and therefore the host CPU model might be wrong.
+ */
+return true;
 }
 return 0;
 }
-- 
2.6.6

[Qemu-devel] [Patch v3 06/30] s390x/cpumodel: generate CPU feature group lists

[David Hildenbrand]

Feature groups will be very helpful to reduce the amount of features
typically available in sane configurations. E.g. the MSA facilities
introduced loads of subfunctions, which could - in theory - go away
in the future, but we want to avoid reporting hundrets of features to
the user if usually all of them are in place.

Groups only contain features that were introduced in one shot, not just
random features. Therefore, groups can never change. This is an important
property regarding migration.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 target-s390x/gen-features.c | 80 +
 1 file changed, 80 insertions(+)

diff --git a/target-s390x/gen-features.c b/target-s390x/gen-features.c
index db63389..2ae4405 100644
--- a/target-s390x/gen-features.c
+++ b/target-s390x/gen-features.c
@@ -178,6 +178,35 @@
 S390_FEAT_MSA_EXT_5, \
 S390_FEAT_PPNO_SHA_512_DRNG
 
+/* cpu feature groups */
+static uint16_t group_PLO[] = {
+S390_FEAT_GROUP_PLO,
+};
+static uint16_t group_TOD_CLOCK_STEERING[] = {
+S390_FEAT_GROUP_TOD_CLOCK_STEERING,
+};
+static uint16_t group_GEN13_PTFF[] = {
+S390_FEAT_GROUP_GEN13_PTFF,
+};
+static uint16_t group_MSA[] = {
+S390_FEAT_GROUP_MSA,
+};
+static uint16_t group_MSA_EXT_1[] = {
+S390_FEAT_GROUP_MSA_EXT_1,
+};
+static uint16_t group_MSA_EXT_2[] = {
+S390_FEAT_GROUP_MSA_EXT_2,
+};
+static uint16_t group_MSA_EXT_3[] = {
+S390_FEAT_GROUP_MSA_EXT_3,
+};
+static uint16_t group_MSA_EXT_4[] = {
+S390_FEAT_GROUP_MSA_EXT_4,
+};
+static uint16_t group_MSA_EXT_5[] = {
+S390_FEAT_GROUP_MSA_EXT_5,
+};
+
 /* base features in order of release */
 static uint16_t base_GEN7_GA1[] = {
 S390_FEAT_GROUP_PLO,
@@ -431,6 +460,34 @@ static CpuFeatDefSpec CpuFeatDef[] = {
 CPU_FEAT_INITIALIZER(GEN13_GA2),
 };
 
+#define FEAT_GROUP_INITIALIZER(_name)  \
+{  \
+.name = "S390_FEAT_GROUP_LIST_" #_name,\
+.bits =\
+{ .data = group_##_name,   \
+  .len = ARRAY_SIZE(group_##_name) },  \
+}
+
+typedef struct {
+const char *name;
+BitSpec bits;
+} FeatGroupDefSpec;
+
+/***
+ * feature groups
+ ***/
+static FeatGroupDefSpec FeatGroupDef[] = {
+FEAT_GROUP_INITIALIZER(PLO),
+FEAT_GROUP_INITIALIZER(TOD_CLOCK_STEERING),
+FEAT_GROUP_INITIALIZER(GEN13_PTFF),
+FEAT_GROUP_INITIALIZER(MSA),
+FEAT_GROUP_INITIALIZER(MSA_EXT_1),
+FEAT_GROUP_INITIALIZER(MSA_EXT_2),
+FEAT_GROUP_INITIALIZER(MSA_EXT_3),
+FEAT_GROUP_INITIALIZER(MSA_EXT_4),
+FEAT_GROUP_INITIALIZER(MSA_EXT_5),
+};
+
 static void set_bits(uint64_t list[], BitSpec bits)
 {
 uint32_t i;
@@ -488,6 +545,28 @@ static void print_feature_defs(void)
 }
 }
 
+static void print_feature_group_defs(void)
+{
+int i, j;
+
+printf("\n/* CPU feature group list data */\n");
+
+for (i = 0; i < ARRAY_SIZE(FeatGroupDef); i++) {
+uint64_t feat[S390_FEAT_MAX / 64 + 1] = {};
+
+set_bits(feat, FeatGroupDef[i].bits);
+printf("#define %s\t", FeatGroupDef[i].name);
+for (j = 0; j < ARRAY_SIZE(feat); j++) {
+printf("0x%016"PRIx64"UL", feat[j]);
+if (j < ARRAY_SIZE(feat) - 1) {
+printf(",");
+} else {
+printf("\n");
+}
+}
+}
+}
+
 int main(int argc, char *argv[])
 {
 printf("/*\n"
@@ -502,6 +581,7 @@ int main(int argc, char *argv[])
" */\n\n"
"#ifndef %s\n#define %s\n", __FILE__, _YEARS, _NAME_H, _NAME_H);
 print_feature_defs();
+print_feature_group_defs();
 printf("\n#endif\n");
 return 0;
 }
-- 
2.6.6

[Qemu-devel] [Patch v3 22/30] s390x/kvm: disable host model for problematic compat machines

[David Hildenbrand]

Compatibility machines that touch runtime-instrumentation should not
be used with the CPU model. Otherwise the host model will look different,
depending on the QEMU machine QEMU has been started with.

So let's simply disable the host model for existing compatibility machines
that all disable ri. This, in return, disables the CPU model for these
compat machines completely.

Acked-by: Cornelia Huck 
Signed-off-by: David Hildenbrand 
---
 target-s390x/kvm.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c
index 2cc0357..3c0929b 100644
--- a/target-s390x/kvm.c
+++ b/target-s390x/kvm.c
@@ -2451,6 +2451,10 @@ static int configure_cpu_feat(const S390FeatBitmap 
features)
 
 bool kvm_s390_cpu_models_supported(void)
 {
+if (!ri_allowed()) {
+/* compatibility machines interfere with the cpu model */
+return false;
+}
 return kvm_vm_check_attr(kvm_state, KVM_S390_VM_CPU_MODEL,
  KVM_S390_VM_CPU_MACHINE) &&
kvm_vm_check_attr(kvm_state, KVM_S390_VM_CPU_MODEL,
-- 
2.6.6

[Qemu-devel] [PATCH v2 6/8] cutils: Rewrite x86 buffer zero checking

[Richard Henderson]

Use unaligned load operations.
Add versions for avx1 and sse4.1.

Cc: liang.z...@intel.com
Signed-off-by: Richard Henderson 
---
 util/bufferiszero.c | 169 
 1 file changed, 145 insertions(+), 24 deletions(-)

diff --git a/util/bufferiszero.c b/util/bufferiszero.c
index 264598b..e5e4459 100644
--- a/util/bufferiszero.c
+++ b/util/bufferiszero.c
@@ -131,21 +131,127 @@ static bool select_accel_fn(const void *buf, size_t len)
 return select_accel_int(buf, len);
 }
 
-#elif defined(CONFIG_AVX2_OPT)
+#elif defined(CONFIG_AVX2_OPT) || defined(__SSE2__)
 #include 
 #include 
 
+#ifdef CONFIG_AVX2_OPT
 #pragma GCC push_options
 #pragma GCC target("avx2")
-#define AVX2_NONZERO(X)  !_mm256_testz_si256((X), (X))
-ACCEL_BUFFER_ZERO(buffer_zero_avx2, 128, __m256i, AVX2_NONZERO)
+
+static bool __attribute__((noinline))
+buffer_zero_avx2(const void *buf, size_t len)
+{
+const __m256i *p = buf;
+const __m256i *end = buf + len;
+__m256i t;
+
+do {
+p += 4;
+/* Note that most AVX insns handle unaligned operands by
+   default; we only need take care for the initial load.  */
+__asm("prefetcht0 (%1)\n\t"
+  "vmovdqu -0x80(%1),%0\n\t"
+  "vpor -0x60(%1),%0,%0\n\t"
+  "vpor -0x40(%1),%0,%0\n\t"
+  "vpor -0x20(%1),%0,%0"
+  : "=x"(t) : "r"(p));
+if (unlikely(!_mm256_testz_si256(t, t))) {
+return false;
+}
+} while (p < end);
+return true;
+}
+
+#pragma GCC pop_options
+#pragma GCC push_options
+#pragma GCC target("avx")
+
+static bool __attribute__((noinline))
+buffer_zero_avx(const void *buf, size_t len)
+{
+const __m128i *p = buf;
+const __m128i *end = buf + len;
+__m128i t;
+
+do {
+p += 4;
+/* Note that most AVX insns handle unaligned operands by
+   default; we only need take care for the initial load.  */
+__asm("prefetcht0 (%1)\n\t"
+  "vmovdqu -0x40(%1),%0\n\t"
+  "vpor -0x20(%1),%0,%0\n\t"
+  "vpor -0x20(%1),%0,%0\n\t"
+  "vpor -0x10(%1),%0,%0"
+  : "=x"(t) : "r"(p));
+if (unlikely(!_mm_testz_si128(t, t))) {
+return false;
+}
+} while (p < end);
+return true;
+}
+
 #pragma GCC pop_options
+#pragma GCC push_options
+#pragma GCC target("sse4")
+
+static bool __attribute__((noinline))
+buffer_zero_sse4(const void *buf, size_t len)
+{
+const __m128i *p = buf;
+const __m128i *end = buf + len;
+__m128i t0, t1, t2, t3;
+
+do {
+p += 4;
+__asm("prefetcht0 (%4)\n\t"
+  "movdqu -0x40(%4),%0\n\t"
+  "movdqu -0x20(%4),%1\n\t"
+  "movdqu -0x20(%4),%2\n\t"
+  "movdqu -0x10(%4),%3\n\t"
+  "por %1,%0\n\t"
+  "por %3,%2\n\t"
+  "por %2,%0"
+  : "=x"(t0), "=x"(t1), "=x"(t2), "=x"(t3) : "r"(p));
+if (unlikely(!_mm_testz_si128(t0, t0))) {
+return false;
+}
+} while (p < end);
+return true;
+}
 
+#pragma GCC pop_options
 #pragma GCC push_options
 #pragma GCC target("sse2")
-#define SSE2_NONZERO(X) \
-(_mm_movemask_epi8(_mm_cmpeq_epi8((X), _mm_setzero_si128())) != 0x)
-ACCEL_BUFFER_ZERO(buffer_zero_sse2, 64, __m128i, SSE2_NONZERO)
+#endif /* CONFIG_AVX2_OPT */
+
+static bool __attribute__((noinline))
+buffer_zero_sse2(const void *buf, size_t len)
+{
+const __m128i *p = buf;
+const __m128i *end = buf + len;
+__m128i zero = _mm_setzero_si128();
+__m128i t0, t1, t2, t3;
+
+do {
+p += 4;
+__asm("prefetcht0 (%4)\n\t"
+  "movdqu -0x40(%4),%0\n\t"
+  "movdqu -0x20(%4),%1\n\t"
+  "movdqu -0x20(%4),%2\n\t"
+  "movdqu -0x10(%4),%3\n\t"
+  "por %1,%0\n\t"
+  "por %3,%2\n\t"
+  "por %2,%0"
+  : "=x"(t0), "=x"(t1), "=x"(t2), "=x"(t3) : "r"(p));
+if (unlikely(_mm_movemask_epi8(_mm_cmpeq_epi8(t0, zero)) != 0x)) {
+return false;
+}
+} while (p < end);
+return true;
+}
+
+#ifdef CONFIG_AVX2_OPT
 #pragma GCC pop_options
 
 #define CACHE_SSE21
@@ -186,32 +292,47 @@ static void __attribute__((constructor)) 
init_cpuid_cache(void)
 }
 cpuid_cache = cache;
 }
+#endif /* CONFIG_AVX2_OPT */
 
 static bool select_accel_fn(const void *buf, size_t len)
 {
-uintptr_t ibuf = (uintptr_t)buf;
-if (len % 128 == 0 && ibuf % 32 == 0 && (cpuid_cache & CACHE_AVX2)) {
+#ifdef CONFIG_AVX2_OPT
+int cache = cpuid_cache;
+
+/* Force bits that the compiler tells us must be there.
+   This allows the compiler to optimize subsequent tests.  */
+#ifdef __AVX2__
+cache |= CACHE_AVX2;
+#endif
+#ifdef __AVX__
+cache |= CACHE_AVX1;
+#endif
+#ifdef __SSE4_1__
+cache |= CACHE_SSE4;
+#endif
+#ifdef __SSE2__
+cache |= CACHE_SSE2;
+#endif
+
+if (len 

[Qemu-devel] [PATCH v2 8/8] cutils: Rewrite ppc buffer zero checking

[Richard Henderson]

GCC versions through 6 do a poor job with the indexed addressing,
and (for ppc64le) issues unnecessary xxswapd insns.

Cc: qemu-...@nongnu.org
Cc: David Gibson 
Signed-off-by: Richard Henderson 
---
 util/bufferiszero.c | 40 ++--
 1 file changed, 38 insertions(+), 2 deletions(-)

diff --git a/util/bufferiszero.c b/util/bufferiszero.c
index 28a1419..d580b57 100644
--- a/util/bufferiszero.c
+++ b/util/bufferiszero.c
@@ -119,8 +119,44 @@ static bool select_accel_int(const void *buf, size_t len)
 #undef pixel
 #undef bool
 #define bool _Bool
-#define DO_NONZERO(X)  vec_any_ne(X, (__vector unsigned char){ 0 })
-ACCEL_BUFFER_ZERO(buffer_zero_ppc, 128, __vector unsigned char, DO_NONZERO)
+
+static bool __attribute__((noinline))
+buffer_zero_ppc(const void *buf, size_t len)
+{
+typedef unsigned char vec __attribute__((vector_size(16)));
+const vec *p = buf;
+const vec *end = buf + len;
+vec t0, t1, t2, t3, zero = (vec){ 0 };
+
+do {
+p += 8;
+__builtin_prefetch(p);
+barrier();
+/* ??? GCC6 does poorly with power64le; extra xxswap.  */
+__asm("lvebx %0,%4,%5\n\t"
+  "lvebx %1,%4,%6\n\t"
+  "lvebx %2,%4,%7\n\t"
+  "lvebx %3,%4,%8\n\t"
+  "vor %0,%0,%1\n\t"
+  "vor %1,%2,%3\n\t"
+  "lvebx %2,%4,%9\n\t"
+  "lvebx %3,%4,%10\n\t"
+  "vor %0,%0,%1\n\t"
+  "vor %1,%2,%3\n\t"
+  "lvebx %2,%4,%11\n\t"
+  "lvebx %3,%4,%12\n\t"
+  "vor %0,%0,%1\n\t"
+  "vor %1,%2,%3\n\t"
+  "vor %0,%0,%1"
+  : "=v"(t0), "=v"(t1), "=v"(t2), "=v"(t3)
+  : "b"(p), "b"(-8 * 16), "b"(-7 * 16), "b"(-6 * 16), "b"(-5 * 16),
+"b"(-4 * 16), "b"(-3 * 16), "b"(-2 * 16), "b"(-1 * 16));
+if (unlikely(vec_any_ne(t0, zero))) {
+return false;
+}
+} while (p < end);
+return true;
+}
 
 static bool select_accel_fn(const void *buf, size_t len)
 {
-- 
2.7.4

[Qemu-devel] [PATCH v2 1/8] cutils: Move buffer_is_zero and subroutines to a new file

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 util/Makefile.objs  |   1 +
 util/bufferiszero.c | 272 
 util/cutils.c   | 244 --
 3 files changed, 273 insertions(+), 244 deletions(-)
 create mode 100644 util/bufferiszero.c

diff --git a/util/Makefile.objs b/util/Makefile.objs
index 96cb1e0..ffca8f3 100644
--- a/util/Makefile.objs
+++ b/util/Makefile.objs
@@ -1,4 +1,5 @@
 util-obj-y = osdep.o cutils.o unicode.o qemu-timer-common.o
+util-obj-y += bufferiszero.o
 util-obj-$(CONFIG_POSIX) += compatfd.o
 util-obj-$(CONFIG_POSIX) += event_notifier-posix.o
 util-obj-$(CONFIG_POSIX) += mmap-alloc.o
diff --git a/util/bufferiszero.c b/util/bufferiszero.c
new file mode 100644
index 000..9bb1ae5
--- /dev/null
+++ b/util/bufferiszero.c
@@ -0,0 +1,272 @@
+/*
+ * Simple C functions to supplement the C library
+ *
+ * Copyright (c) 2006 Fabrice Bellard
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to 
deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "qemu/cutils.h"
+
+
+/* vector definitions */
+#ifdef __ALTIVEC__
+#include 
+/* The altivec.h header says we're allowed to undef these for
+ * C++ compatibility.  Here we don't care about C++, but we
+ * undef them anyway to avoid namespace pollution.
+ */
+#undef vector
+#undef pixel
+#undef bool
+#define VECTYPE__vector unsigned char
+#define SPLAT(p)   vec_splat(vec_ld(0, p), 0)
+#define ALL_EQ(v1, v2) vec_all_eq(v1, v2)
+#define VEC_OR(v1, v2) ((v1) | (v2))
+/* altivec.h may redefine the bool macro as vector type.
+ * Reset it to POSIX semantics. */
+#define bool _Bool
+#elif defined __SSE2__
+#include 
+#define VECTYPE__m128i
+#define SPLAT(p)   _mm_set1_epi8(*(p))
+#define ALL_EQ(v1, v2) (_mm_movemask_epi8(_mm_cmpeq_epi8(v1, v2)) == 0x)
+#define VEC_OR(v1, v2) (_mm_or_si128(v1, v2))
+#elif defined(__aarch64__)
+#include "arm_neon.h"
+#define VECTYPEuint64x2_t
+#define ALL_EQ(v1, v2) \
+((vgetq_lane_u64(v1, 0) == vgetq_lane_u64(v2, 0)) && \
+ (vgetq_lane_u64(v1, 1) == vgetq_lane_u64(v2, 1)))
+#define VEC_OR(v1, v2) ((v1) | (v2))
+#else
+#define VECTYPEunsigned long
+#define SPLAT(p)   (*(p) * (~0UL / 255))
+#define ALL_EQ(v1, v2) ((v1) == (v2))
+#define VEC_OR(v1, v2) ((v1) | (v2))
+#endif
+
+#define BUFFER_FIND_NONZERO_OFFSET_UNROLL_FACTOR 8
+
+static bool
+can_use_buffer_find_nonzero_offset_inner(const void *buf, size_t len)
+{
+return (len % (BUFFER_FIND_NONZERO_OFFSET_UNROLL_FACTOR
+   * sizeof(VECTYPE)) == 0
+&& ((uintptr_t) buf) % sizeof(VECTYPE) == 0);
+}
+
+/*
+ * Searches for an area with non-zero content in a buffer
+ *
+ * Attention! The len must be a multiple of
+ * BUFFER_FIND_NONZERO_OFFSET_UNROLL_FACTOR * sizeof(VECTYPE)
+ * and addr must be a multiple of sizeof(VECTYPE) due to
+ * restriction of optimizations in this function.
+ *
+ * can_use_buffer_find_nonzero_offset_inner() can be used to
+ * check these requirements.
+ *
+ * The return value is the offset of the non-zero area rounded
+ * down to a multiple of sizeof(VECTYPE) for the first
+ * BUFFER_FIND_NONZERO_OFFSET_UNROLL_FACTOR chunks and down to
+ * BUFFER_FIND_NONZERO_OFFSET_UNROLL_FACTOR * sizeof(VECTYPE)
+ * afterwards.
+ *
+ * If the buffer is all zero the return value is equal to len.
+ */
+
+static size_t buffer_find_nonzero_offset_inner(const void *buf, size_t len)
+{
+const VECTYPE *p = buf;
+const VECTYPE zero = (VECTYPE){0};
+size_t i;
+
+assert(can_use_buffer_find_nonzero_offset_inner(buf, len));
+
+if (!len) {
+return 0;
+}
+
+for (i = 0; i < BUFFER_FIND_NONZERO_OFFSET_UNROLL_FACTOR; i++) {
+if (!ALL_EQ(p[i], zero)) {
+return i * sizeof(VECTYPE);
+}
+}
+
+for (i = BUFFER_FIND_NONZERO_OFFSET_UNROLL_FACTOR;
+ i < len / sizeof(VECTYPE);
+ i += 

[Qemu-devel] [PATCH v2 5/8] cutils: Add generic prefetch

[Richard Henderson]

There's no real knowledge of the cacheline size,
just prefetching one loop ahead.

Signed-off-by: Richard Henderson 
---
 util/bufferiszero.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/util/bufferiszero.c b/util/bufferiszero.c
index 5246c5b..264598b 100644
--- a/util/bufferiszero.c
+++ b/util/bufferiszero.c
@@ -38,6 +38,8 @@ NAME(const void *buf, size_t len) 
  \
 do {\
 const VECTYPE *p = buf; \
 VECTYPE t;  \
+__builtin_prefetch(buf + SIZE); \
+barrier();  \
 if (SIZE == sizeof(VECTYPE) * 4) {  \
 t = (p[0] | p[1]) | (p[2] | p[3]);  \
 } else if (SIZE == sizeof(VECTYPE) * 8) {   \
@@ -241,6 +243,9 @@ bool buffer_is_zero(const void *buf, size_t len)
 return true;
 }
 
+/* Fetch the beginning of the buffer while we select the accelerator.  */
+__builtin_prefetch(buf);
+
 /* Use an optimized zero check if possible.  Note that this also
includes a check for an unrolled loop over longs, as well as
the unsized, unaligned fallback to buffer_zero_base.  */
-- 
2.7.4

[Qemu-devel] [PATCH v2 4/8] cutils: Rearrange buffer_is_zero acceleration

[Richard Henderson]

Allow selection of several acceleration functions
based on the size and alignment of the buffer.
Do not require ifunc support for AVX2 acceleration.

Signed-off-by: Richard Henderson 
---
 configure   |  21 +---
 util/bufferiszero.c | 352 +---
 2 files changed, 172 insertions(+), 201 deletions(-)

diff --git a/configure b/configure
index 4b808f9..9f3d1fa 100755
--- a/configure
+++ b/configure
@@ -1788,28 +1788,19 @@ fi
 ##
 # avx2 optimization requirement check
 
-
-if test "$static" = "no" ; then
-  cat > $TMPC << EOF
+cat > $TMPC << EOF
 #pragma GCC push_options
 #pragma GCC target("avx2")
 #include 
 #include 
-
 static int bar(void *a) {
-return _mm256_movemask_epi8(_mm256_cmpeq_epi8(*(__m256i *)a, 
(__m256i){0}));
+__m256i x = *(__m256i *)a;
+return _mm256_testz_si256(x, x);
 }
-static void *bar_ifunc(void) {return (void*) bar;}
-int foo(void *a) __attribute__((ifunc("bar_ifunc")));
-int main(int argc, char *argv[]) { return foo(argv[0]);}
+int main(int argc, char *argv[]) { return bar(argv[0]); }
 EOF
-  if compile_object "" ; then
-  if has readelf; then
-  if readelf --syms $TMPO 2>/dev/null |grep -q "IFUNC.*foo"; then
-  avx2_opt="yes"
-  fi
-  fi
-  fi
+if compile_object "" ; then
+  avx2_opt="yes"
 fi
 
 #
diff --git a/util/bufferiszero.c b/util/bufferiszero.c
index 0cf8b6e..5246c5b 100644
--- a/util/bufferiszero.c
+++ b/util/bufferiszero.c
@@ -27,242 +27,222 @@
 
 
 /* vector definitions */
-#ifdef __ALTIVEC__
-#include 
-/* The altivec.h header says we're allowed to undef these for
- * C++ compatibility.  Here we don't care about C++, but we
- * undef them anyway to avoid namespace pollution.
- */
-#undef vector
-#undef pixel
-#undef bool
-#define VECTYPE__vector unsigned char
-#define ALL_EQ(v1, v2) vec_all_eq(v1, v2)
-#define VEC_OR(v1, v2) ((v1) | (v2))
-/* altivec.h may redefine the bool macro as vector type.
- * Reset it to POSIX semantics. */
-#define bool _Bool
-#elif defined __SSE2__
-#include 
-#define VECTYPE__m128i
-#define ALL_EQ(v1, v2) (_mm_movemask_epi8(_mm_cmpeq_epi8(v1, v2)) == 0x)
-#define VEC_OR(v1, v2) (_mm_or_si128(v1, v2))
-#elif defined(__aarch64__)
-#include "arm_neon.h"
-#define VECTYPEuint64x2_t
-#define ALL_EQ(v1, v2) \
-((vgetq_lane_u64(v1, 0) == vgetq_lane_u64(v2, 0)) && \
- (vgetq_lane_u64(v1, 1) == vgetq_lane_u64(v2, 1)))
-#define VEC_OR(v1, v2) ((v1) | (v2))
-#else
-#define VECTYPEunsigned long
-#define ALL_EQ(v1, v2) ((v1) == (v2))
-#define VEC_OR(v1, v2) ((v1) | (v2))
-#endif
 
-#define BUFFER_FIND_NONZERO_OFFSET_UNROLL_FACTOR 8
-
-static bool
-can_use_buffer_find_nonzero_offset_inner(const void *buf, size_t len)
-{
-return (len % (BUFFER_FIND_NONZERO_OFFSET_UNROLL_FACTOR
-   * sizeof(VECTYPE)) == 0
-&& ((uintptr_t) buf) % sizeof(VECTYPE) == 0);
+extern void link_error(void);
+
+#define ACCEL_BUFFER_ZERO(NAME, SIZE, VECTYPE, NONZERO) \
+static bool __attribute__((noinline))   \
+NAME(const void *buf, size_t len)   \
+{   \
+const void *end = buf + len;\
+do {\
+const VECTYPE *p = buf; \
+VECTYPE t;  \
+if (SIZE == sizeof(VECTYPE) * 4) {  \
+t = (p[0] | p[1]) | (p[2] | p[3]);  \
+} else if (SIZE == sizeof(VECTYPE) * 8) {   \
+t  = p[0] | p[1];   \
+t |= p[2] | p[3];   \
+t |= p[4] | p[5];   \
+t |= p[6] | p[7];   \
+} else {\
+link_error();   \
+}   \
+if (unlikely(NONZERO(t))) { \
+return false;   \
+}   \
+buf += SIZE;\
+} while (buf < end);\
+return true;\
 }
 
-/*
- * Searches for an area with non-zero content in a buffer
- *
- * Attention! The len must be a multiple of
- * BUFFER_FIND_NONZERO_OFFSET_UNROLL_FACTOR * sizeof(VECTYPE)
- * and addr must be a multiple of sizeof(VECTYPE) due to
- * restriction of optimizations in this function.
- *
- * 

[Qemu-devel] [PATCH v2 7/8] cutils: Rewrite aarch64 buffer zero checking

[Richard Henderson]

Provide 64-byte and 128-byte versions.
Use dczid_el0 as a proxy for the cacheline size.

Cc: qemu-...@nongnu.org
Cc: vijay.kil...@gmail.com
Signed-off-by: Richard Henderson 
---
 util/bufferiszero.c | 28 +---
 1 file changed, 25 insertions(+), 3 deletions(-)

diff --git a/util/bufferiszero.c b/util/bufferiszero.c
index e5e4459..28a1419 100644
--- a/util/bufferiszero.c
+++ b/util/bufferiszero.c
@@ -340,13 +340,35 @@ static bool select_accel_fn(const void *buf, size_t len)
 #include "arm_neon.h"
 
 #define DO_NONZERO(X)  (vgetq_lane_u64((X), 0) | vgetq_lane_u64((X), 1))
-ACCEL_BUFFER_ZERO(buffer_zero_neon, 128, uint64x2_t, DO_NONZERO)
+ACCEL_BUFFER_ZERO(buffer_zero_neon_64, 64, uint64x2_t, DO_NONZERO)
+ACCEL_BUFFER_ZERO(buffer_zero_neon_128, 128, uint64x2_t, DO_NONZERO)
+
+static uint32_t buffer_zero_line_mask;
+static accel_zero_fn buffer_zero_accel;
+
+static void __attribute__((constructor)) init_buffer_zero_accel(void)
+{
+uint64_t t;
+
+/* Use the DZP block size as a proxy for the cacheline size,
+   since the later is not available to userspace.  This seems
+   to work in practice for existing implementations.  */
+asm("mrs %0, dczid_el0" : "=r"(t));
+if ((t & 15) * 16 >= 128) {
+buffer_zero_line_mask = 128 - 1;
+buffer_zero_accel = buffer_zero_neon_128;
+} else {
+buffer_zero_line_mask = 64 - 1;
+buffer_zero_accel = buffer_zero_neon_64;
+}
+}
 
 static bool select_accel_fn(const void *buf, size_t len)
 {
 uintptr_t ibuf = (uintptr_t)buf;
-if (len % 128 == 0 && ibuf % sizeof(uint64x2_t) == 0) {
-return buffer_zero_neon(buf, len);
+if (likely(ibuf % sizeof(uint64_t) == 0)
+&& (len & buffer_zero_line_mask) == 0) {
+return buffer_zero_accel(buf, len);
 }
 return select_accel_int(buf, len);
 }
-- 
2.7.4

[Qemu-devel] [PATCH v2 0/8] Improve buffer_is_zero

[Richard Henderson]

Patches 1-4 remove the use of ifunc from the implementation.

Patch 6 adjusts the x86 implementation a bit more to take
advantage of ptest (in sse4.1) and unaligned accesses (in avx1).

Patches 3 and 7 are the result of my conversation with Vijaya
Kumar with respect to ThunderX.

Patch 8 is the result of seeing some really really horrible code
produced for ppc64le (gcc 4.9 and mainline).

This has had limited testing.  What I don't know is the best way
to benchmark this -- the only way I know to trigger this is via
the console, by hand, which doesn't make for reasonable timing.

Changes v1-v2:
  * Add patch 1, moving everything to a new file.
  * Fix a typo or two, which had the wrong sense of zero test.
These had mostly beed fixed in the intermediate patches,
but it wouldn't have helped bisection.


r~


Richard Henderson (8):
  cutils: Move buffer_is_zero and subroutines to a new file
  cutils: Remove SPLAT macro
  cutils: Export only buffer_is_zero
  cutils: Rearrange buffer_is_zero acceleration
  cutils: Add generic prefetch
  cutils: Rewrite x86 buffer zero checking
  cutils: Rewrite aarch64 buffer zero checking
  cutils: Rewrite ppc buffer zero checking

 configure |  21 +--
 include/qemu/cutils.h |   2 -
 migration/ram.c   |   2 +-
 migration/rdma.c  |   5 +-
 util/Makefile.objs|   1 +
 util/bufferiszero.c   | 432 ++
 util/cutils.c | 244 
 7 files changed, 441 insertions(+), 266 deletions(-)
 create mode 100644 util/bufferiszero.c

-- 
2.7.4

[Qemu-devel] [PATCH v2 3/8] cutils: Export only buffer_is_zero

[Richard Henderson]

Since the two users don't make use of the returned offset,
beyond ensuring that the entire buffer is zero, consider the
can_use_buffer_find_nonzero_offset and buffer_find_nonzero_offset
functions internal.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Richard Henderson 
---
 include/qemu/cutils.h | 2 --
 migration/ram.c   | 2 +-
 migration/rdma.c  | 5 +
 util/bufferiszero.c   | 8 
 4 files changed, 6 insertions(+), 11 deletions(-)

diff --git a/include/qemu/cutils.h b/include/qemu/cutils.h
index 3e4ea23..ca58577 100644
--- a/include/qemu/cutils.h
+++ b/include/qemu/cutils.h
@@ -168,8 +168,6 @@ int64_t qemu_strtosz_suffix_unit(const char *nptr, char 
**end,
 /* used to print char* safely */
 #define STR_OR_NULL(str) ((str) ? (str) : "null")
 
-bool can_use_buffer_find_nonzero_offset(const void *buf, size_t len);
-size_t buffer_find_nonzero_offset(const void *buf, size_t len);
 bool buffer_is_zero(const void *buf, size_t len);
 
 /*
diff --git a/migration/ram.c b/migration/ram.c
index a3d70c4..a6e1c63 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -73,7 +73,7 @@ static const uint8_t ZERO_TARGET_PAGE[TARGET_PAGE_SIZE];
 
 static inline bool is_zero_range(uint8_t *p, uint64_t size)
 {
-return buffer_find_nonzero_offset(p, size) == size;
+return buffer_is_zero(p, size);
 }
 
 /* struct contains XBZRLE cache and a static page
diff --git a/migration/rdma.c b/migration/rdma.c
index 5110ec8..88bdb64 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -1934,10 +1934,7 @@ retry:
  * memset() + madvise() the entire chunk without RDMA.
  */
 
-if (can_use_buffer_find_nonzero_offset((void *)(uintptr_t)sge.addr,
-   length)
-   && buffer_find_nonzero_offset((void *)(uintptr_t)sge.addr,
-length) == length) {
+if (buffer_is_zero((void *)(uintptr_t)sge.addr, length)) {
 RDMACompress comp = {
 .offset = current_addr,
 .value = 0,
diff --git a/util/bufferiszero.c b/util/bufferiszero.c
index 067d08f..0cf8b6e 100644
--- a/util/bufferiszero.c
+++ b/util/bufferiszero.c
@@ -192,9 +192,9 @@ static bool avx2_support(void)
 return b & bit_AVX2;
 }
 
-bool can_use_buffer_find_nonzero_offset(const void *buf, size_t len) \
+static bool can_use_buffer_find_nonzero_offset(const void *buf, size_t len) \
  __attribute__ ((ifunc("can_use_buffer_find_nonzero_offset_ifunc")));
-size_t buffer_find_nonzero_offset(const void *buf, size_t len) \
+static size_t buffer_find_nonzero_offset(const void *buf, size_t len) \
  __attribute__ ((ifunc("buffer_find_nonzero_offset_ifunc")));
 
 static void *buffer_find_nonzero_offset_ifunc(void)
@@ -215,12 +215,12 @@ static void 
*can_use_buffer_find_nonzero_offset_ifunc(void)
 }
 #pragma GCC pop_options
 #else
-bool can_use_buffer_find_nonzero_offset(const void *buf, size_t len)
+static bool can_use_buffer_find_nonzero_offset(const void *buf, size_t len)
 {
 return can_use_buffer_find_nonzero_offset_inner(buf, len);
 }
 
-size_t buffer_find_nonzero_offset(const void *buf, size_t len)
+static size_t buffer_find_nonzero_offset(const void *buf, size_t len)
 {
 return buffer_find_nonzero_offset_inner(buf, len);
 }
-- 
2.7.4

[Qemu-devel] [PATCH] Postcopy vs xbzrle: Don't send xbzrle pages once in postcopy [for 2.8]

[Dr. David Alan Gilbert (git)]

From: "Dr. David Alan Gilbert" 

xbzrle relies on reading pages that have already been sent
to the destination and then applying the modifications; we can't
do that in postcopy because the destination may well have
modified the page already or the page has been discarded.

I already didn't allow reception of xbzrle pages, but I
forgot to add the test to stop them being sent.

Enabling both xbzrle and postcopy can make some sense;
if you think that your migration might finish if you
have xbzrle, then when it doesn't complete you flick
over to postcopy and stop xbzrle'ing.

This corresponds to RH bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1368422

Symptom is:

Unknown combination of migration flags: 0x60 (postcopy mode)
(either 0x60 or 0x40)

Signed-off-by: Dr. David Alan Gilbert 
---
 migration/ram.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/migration/ram.c b/migration/ram.c
index a3d70c4..79fdc94 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -771,7 +771,9 @@ static int ram_save_page(QEMUFile *f, PageSearchStatus *pss,
  * page would be stale
  */
 xbzrle_cache_zero_page(current_addr);
-} else if (!ram_bulk_stage && migrate_use_xbzrle()) {
+} else if (!ram_bulk_stage &&
+   !migration_in_postcopy(migrate_get_current()) &&
+   migrate_use_xbzrle()) {
 pages = save_xbzrle_page(f, , current_addr, block,
  offset, last_stage, bytes_transferred);
 if (!last_stage) {
-- 
2.7.4

Re: [Qemu-devel] [PATCH] 9pfs: disallow / in path components

[Greg Kurz]

On Wed, 24 Aug 2016 18:46:10 +0300
"Michael S. Tsirkin"  wrote:

> On Wed, Aug 24, 2016 at 04:00:24PM +0100, Peter Maydell wrote:
> > On 24 August 2016 at 15:29, Greg Kurz  wrote:  
> > > At various places in 9pfs, full paths are created by concatenating a guest
> > > originated string to the export path. A malicious guest could forge a
> > > relative path and access files outside the export path.
> > >
> > > A tentative fix was sent recently by Prasad J Pandit, but it was only
> > > focused on the local backend and did not get a positive review. This patch
> > > tries to address the issue more globally, based on the official 9P spec.
> > >
> > > The walk request described in the 9P spec [1] clearly shows that the 
> > > client
> > > is supposed to send individual path components: the official linux client
> > > never sends portions of path containing the / character for example.
> > >
> > > Moreover, the 9P spec [2] also states that a system can decide to restrict
> > > the set of supported characters used in path components, with an explicit
> > > mention "to remove slashes from name components".
> > >
> > > This patch introduces a new name_has_illegal_characters() helper that 
> > > looks
> > > for such unwanted characters in strings sent by the client. Since 9pfs is
> > > only supported on linux hosts, only the / character is checked at the
> > > moment. When support for other hosts (AKA. win32) is added, other chars
> > > may need to be blacklisted as well.  
> > 
> > Do we also need ".." and "." to be illegal names (for at least most
> > operations)?
> > 
> > thanks
> > -- PMM  
> 
> I agree, and I think this implies name_is_legal would be a better function 
> name.
> 

No I think this is a different issue that calls for a followup patch (see my
other mail).

Re: [Qemu-devel] [PATCH] 9pfs: disallow / in path components

[Greg Kurz]

On Wed, 24 Aug 2016 16:00:24 +0100
Peter Maydell  wrote:

> On 24 August 2016 at 15:29, Greg Kurz  wrote:
> > At various places in 9pfs, full paths are created by concatenating a guest
> > originated string to the export path. A malicious guest could forge a
> > relative path and access files outside the export path.
> >
> > A tentative fix was sent recently by Prasad J Pandit, but it was only
> > focused on the local backend and did not get a positive review. This patch
> > tries to address the issue more globally, based on the official 9P spec.
> >
> > The walk request described in the 9P spec [1] clearly shows that the client
> > is supposed to send individual path components: the official linux client
> > never sends portions of path containing the / character for example.
> >
> > Moreover, the 9P spec [2] also states that a system can decide to restrict
> > the set of supported characters used in path components, with an explicit
> > mention "to remove slashes from name components".
> >
> > This patch introduces a new name_has_illegal_characters() helper that looks
> > for such unwanted characters in strings sent by the client. Since 9pfs is
> > only supported on linux hosts, only the / character is checked at the
> > moment. When support for other hosts (AKA. win32) is added, other chars
> > may need to be blacklisted as well.  
> 
> Do we also need ".." and "." to be illegal names (for at least most
> operations)?
> 
> thanks
> -- PMM

I understand how ".." could be an issue, but I don't for "."... can you
please elaborate ?

The spec says:

  Directories are created by create with DMDIR set in the per-
  missions argument (see stat(5)). The members of a directory
  can be found with read(5). All directories must support
  walks to the directory .. (dot-dot) meaning parent direc-
  tory, although by convention directories contain no explicit
  entry for .. or . (dot).  The parent of the root directory
  of a server's tree is itself.

So I don't think we should boldly make ".." an illegal name, but
rather ignore it. Pretty much like doing chdir("..") when the current
directory is /.

All operations except walk take an existing fid and a single path component.
A possible fix would be to convert ".." to "" when the fid points to the root
of the export path. Makes sense ?

Since the walk operation takes an array of path components, we would need
to do extend the above check to each component.

Cheers.

--
Greg

Re: [Qemu-devel] [PATCH] commit: get the overlay node before manipulating the backing chain

[Max Reitz]

On 2016-08-21 at 23:36, Alberto Garcia wrote:

The 'block-commit' command has a 'top' parameter to specify the
topmost node from which the data is going to be copied.

   [E] <- [D] <- [C] <- [B] <- [A]

In this case if [C] is the top node then this is the result:

   [E] <- [B] <- [A]

[B] must be modified so its backing image string points to [E] instead
of [C]. commit_start() takes care of reopening [B] in read-write
mode, and commit_complete() puts it back in read-only mode once the
operation has finished.

In order to find [B] (the overlay node) we look for the node that has
[C] (the top node) as its backing image. However in commit_complete()
we're doing it after [C] has been removed from the chain, so [B] is
never found and remains in read-write mode.


That's... Unfortunate.


This patch gets the overlay node before the backing chain is
manipulated.

Signed-off-by: Alberto Garcia 
---
 block/commit.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)


Thanks, applied to my block-next tree:

https://github.com/XanClic/qemu/commits/block-next

Max

Re: [Qemu-devel] [PATCH] MAINTAINERS: Add Fam and Jsnow for Bitmap support

[Max Reitz]

On 2016-08-04 at 14:18, John Snow wrote:

These files are currently unmaintained.

I'm proposing that Fam and I co-maintain them; under the model that
whomever between us isn't authoring a given series will be responsible
for reviewing it.

Signed-off-by: John Snow 
---
 MAINTAINERS | 14 ++
 1 file changed, 14 insertions(+)


Acked-by: Max Reitz